A final, singular, PRE-INSTALLED & OFFICIAL, Add-on Manager to rule them all
BAM / Blender Add-on Manager
I'm new. I may be wrong. But this is my personal experience after one year of jumping into Blender, in terms of managing add-ons:
Add-on maintenance is a mess.
And since add-ons play such a fundamental role in Blender, I think the time has come for a unified, pre-packaged, standard solution that addresses two important key tasks:
- Notifying when installed add-ons have new versions.
- Updating outdated add-ons from within Blender itself*.
* Whenever/if that's technically possible. (I don't know how or if that'd work, but if not possible, then a companion executable might be an option).
There are existing efforts out there, such as
CG Cookie's
add-on updater and a plethora of seemingly defunct scripts on places like Github. CG Cookie's solution seems to come closest to a workable solution, but in my mind, it's pretty limited and dare I say (no harm intended) ineffective, by nature of how it works (duplicated code in each add-on using it; multiple online requests from many add-ons at startup, potentially borking down performance if there are many instances using it, compared to the single request in my proposal). It would be better if there was a public .json catalog where developers could list their add-ons, including name and current version; where this proposed manager would be able to poll every installed version against the .json file and perform updates manually (default) or automatically at program start or intervals. For this to work, developers would need to expose a parameter, let's call it
release_url
in their add-ons. The advantage here, is that this add-on manager would not be restricted to only add-ons available on places like Github, GitLab or BitBucket, but other repositories and (hopefully) even publically shared .zip-file urls and .py ditto would work. Developers benefit from more users actually using their latest versions (thus, among other things, potentially getting rid of 1-star reviews whenever something doesn't work including bad rap comments online, where in fact the latest release has the reported grievances fixed).
23 Comments
Good idea, but how should we manage this database? Where does the server space come from, and who decides what goes into that database? Python scripts could be used for malicious purposes, Blender Addons are no exception! So someone would have to review this, before letting this into that database. And if I can't use an addon, that's not in that database, it would destroy the purpose of the python interface.
My suggestion: In BL_INFO, there should be a property called update_url or something (optional at first), which has to be the URL to a git repository. Then, the version could be looked up using Git, so it doesn't matter, on which platform it's uploaded. For Paid add-ons, there should be another property for a token, used to access the repository (which is, logically, not available to the public).
There's a lot of things to think of before implementing something like this, communication with the internet also means a security risk, since bugs could possibly be exploited to hack into a computer.
In theory, your idea is good, and maybe someone could develop an addon to do this, but delivering this natively with Blender is a whole different thing since 16 Million Blender Users would be forced to accept this feature, with an addon, maybe a few thousand people would really download and use this on their own risk.
Greetings,
Blender Defender
* @Blender Defender * Great feedback and valid points, Blender Defender. I honestly just wanted a ball to start rolling on this topic, because in any case, I think this is the next step and a big one for maturing the app even further. I think there’s too many loose parts and a bit of a chore to keep track of add-on updates... As for your notions on security. For starters, this could be a disabled add-on by default. The database could perhaps be a Github repo, however I’m not technically adept to say what’s ideal. But as for malicious code being spread, I don’t see how things would get worse compared to any other package managers out there, eg. npm, Conda, pip, etc. Having someone check the integrity of add-on code is of course likely to be out of the picture, but like with any online-distributed code, there’s a natural inherent risk involved. Now, if that’s something that prevents moving forward with the idea, there’s always precautionary steps to consider, such as adding a ’trusted’ tier among add-ons, based on user input. Not sure if that one in particular is great or not, my point is, that this is something worthy of discussing but I don’t think it should stop the ball rolling just because of the security factor. I’m also quite keen to place my bet, if you were to poll every Blender user on the planet (including add-on devs), whether they’d accept and use such an add-on manager, that a majority would be positive.
In case someone feel this is worth continuing, I’d be interested gathering a group of thinkers/software architects to discuss feature and functionality and shape a spec around it.
EDIT1: To clarify, the database would never include code. It would be like an updated list of primarily name, version, repourl. Then of course, the issue of someone potentially tampering with the url is the hot potato, but there has to be some way around it!?
EDIT2: I think I already have a proposed solution. It’s deceptively simple but pretty damn solid: Just ditch the repourl altogether! Simple as that. Then, users would keep doing their first-time downloads, just as before, from whatever sources they use. AND, here’s the thing, add-on devs would just start chucking in the repourls in their new releases, then update the database accordingly (easy process) and we have a working system! I don’t see any issues with that. However there might be something I miss and that’s why I’d love to scramble together and work out a proper prototype/draft.
* @Henrik Cederblad * repo url shoud have too a url for report issue as bug, then from addon lists can be forwared info about version of addon and blender or OS what can support develoepr - somethin like on blender when we reporting bug
* @Henrik Cederblad * "But as for malicious code being spread, I don’t see how things would get worse compared to any other package managers out there, eg. npm, Conda, pip, etc." The point I tried to make there was, that if there's an official database where add-ons are stored, people trust it to be safe, and someone would have to manage it (especially, if it's a Github repo) to make sure, it's really safe. I think that is not possible, as there are thousands of addon devs (like me) out there, whose add-ons all would have to be managed in this one database if you want this feature to apply to all add-ons.
Therefore, my proposal was to have a decentralized way of doing this: No official database, instead, each developer would have to provide a URL to a Git repository. All Git repositories have the same structure: In /.git/refs/tags/[TAGNAMES], the Tags are stored as Files with the commit it's pointing to inside (e.g. 1.0 and 1.1). Comparing the addon version number with the tag numbers, we can find out, when there's a new version. That would work, independently from the platform, where it's uploaded.
But now that I've offered a solution for the database, there's still a question to answer: The difference to other package managers: As far as I understood you right, Blender would communicate with the internet, in the background. Automatically. Communication with the internet can possibly be abused, and the more programs communicate with the internet, the higher the chances are, that a hacker can find an exploit, a 0-Day Bug to hack a victim's PC. So security is a point to be discussed, maybe, there could be an operator to check for updates (for all addons), so communication to the internet happens only, if the user requests it.
* @Blender Defender * So how Visual Studio Code and MS solved this? Sure.. can be trust issues.. but bether to stay tuned around public community, with the rates, blender artsis, etc. All this can be on own risk.. who want use, then use.. who not.. then not. Always we can't accept the update. In fact these plugins are not to much advanced as with .dll or other.. and easy to see when something is wrong. Maybe it's isse for blender to checking some call around http, named pipes, tpc from addons?
* @Blender Defender * and the important is RULE of OFFICIAL or trusted.. we can develop these "tag" to build more trust..
* @ManBlender * Aside from not knowing, what exactly you want to tell me (bad grammar and many typos), I've already offered a solution for the database problem. Twice. VS Code has a bigger team of developers and more frequent updates. When they find a vulnerability, they can patch it and publish a fix-release almost immediately. I'm not sure, but I guess they are paying or have paid pentesters (in the past). So you can't compare Blender to VS Code. The problem that still remains, is that Blender might have a 0-Day Exploit hidden in the code, which could become vulnerable when connecting to the internet in the background. I've * also * come up with a solution for that: Instead of connecting to the internet in the background, there could be an Operator to check for updates. (Connects to the internet only one time)
Greetings,
Blender Defender
* @Blender Defender * First off, let's keep this discussion free of potential negativism! No remarks on grammar and typos for the sake of constructivism, please. :-) Secondly, I don't know if my idea was clear: NO actual add-on code would be downloaded from the DB. The database would only act as an updated (json) list with addon-names and current versions. (The addons themselves specify where to download from.) The add-on I'm proposing should merely check installed versions against that database and download from the ORIGINAL SOURCE, as defined in the add-ons by the original developers. That way, there's no middleman. Same security level as we currently have! That's my thinking. Good? Bad? Let us cut the "can't be done" talk and just focus around "making this happen", please. The whole issue for the Blender community is that there's really no central standard way around addon updates. That's what we need to solve here.
* @Henrik Cederblad * I agree, the Blender Community should be positive. I think we should stay realistic though, focused on looking for solutions. I don't want to focus on that grammar thing, but if I can't understand someone, because of that bad grammar, I point that out directly, to avoid a nonsense discussion, based on misunderstandings. A database without code is easier to manage, so that's something to think of (Now there are two solutions: Your solution and my solution). So now I'd like to know, what others think about these solutions .
I've also offered a solution for the potential abuse of Blender making connections in the background: Just make a connection to the internet once, when the user requests it.
So now, I don't see any obstacles to this proposal. But I'm also just a human, so there may remain unexpected problems. So please, talk about this. Search for problems, to come up with solutions. Vote this post up. Ask devs, what they would do. Get this feature into Blender, in the best way possible.
Greetings,
Blender Defender :)
* @Blender Defender * Great to see everyone onboard with the idea! Personally, I don't mind whichever approach gets chosen. People will benefit from having a choice to let their add-ons come freshly baked out the oven, and that's the main importance right now.
In the hopes to see this unborn idea in clear daylight, I think we should take the opportunity and hereby dub it …
BAM / Blender Add-on Manager
* @ManBlender * what about addons that aren't in a repo? what about addons that are only on gumroad or blendermarket? what about addons that are distributed through forums like blenderartists.org?
* @ManBlender * I like the sound of that! Absolutely, great idea! However I think that needs to go in the backburner, since the most important thing is to first come up with a solid update solution. Then build on top of that. (For instance, I know I'd like to see a search feature! Where you could search by feature/tag/name etc. And an organizer feature, where things like N-panel apparance etc. is possible. But to prevent this from not becoming anything at all, I think it's super important to focus on first things first.)
* @testure * * @testure * As for the first and last of your questions, that's not going to be an issue (fingers crossed) if it's possible to provide a link to a .zip and have the add-on replace current add-on with the new .zip? (My idea was that add-ons would expose a
release_urlparameter that points to either a repo, a .zip or a .py script.) As for proprietary distribution methods, that's a harder nut to crack. If technical restrictions would lock them out of this add-on, then so be it, I'd say. (It's up to those services to keep their customers updated on new releases anyway.)* @testure *
* @Blender Defender * what is not clear for You?
Tag: OFFICIAL, TEST, COMMUNITY, BLENDERARTIS
VSC have many addons and are not controled by MS/VSC thats what I trying explain and show. And.. show me.. how to crack me by Blender plugin.. most of are pure python without .dll etc.. There is paranoic.. if you don't want autoupdate, then turn off and do this manualy. We can suggest for blender eg. checking if addon try call http, www, tcp as internet connection. HVR.. some of use this connection, eg. external material libs etc. Soo.. byt his you must allow for this type action.
There another case.. what if downloaded version have bug, ant want back to previous : ) We should have option after back to "download only new" github.com/dfki-ric/phobos/releases.atom
Also able to manage which addons to appear on N-panel and ability to categorise them. Currently they clutter the space in N-panel.
How would it be (and is it possible to do it) if we used python's package manager to install blender plugins, and if blender could support installing that way it would be cool.. We could then use an existing package management system that can handle updates, versioning, dependency check, commentary, cli, etc.. I mean the only differences I see are:
I'm late to the party but I have just decided to create an 100% full addon manager with all the features I can pack into it. That's because I have MANY addons and getting tired of spending 2-3 hours with every new Blender software default install. I'll keep everyone posted.
Take for example the PortableApps Platform, it downloads and installs .paf.exe installers for portable Applications, and there's no (or little) security involved, just the download.
And it's Open Source, so it would be possible to inspect and discover how they did it.
Blender should behave almost the same way, downloading AddOns and installing them, then, (in my mind), install required pip packages.
Updating should work the same way.
And yes, why not make some kind of AddOn DataBase to rate and comment them?
Paid AddOns could use PayPal for getting paid, so we should neither worry about secure transactions.
And, finally, this way it would also be possible to update AddOns to the latest version easily.
Regards.
IMHO, all AddOns should only be available through a manager, with the ability to comment and rate them, and to install required Python packages through PIP, if needed (I tried to add PyGame to Blender, it's a pain if you use a portable package, not every user could made it). The manager should also have a search function, and it should keep track of acquired AddOns using your account (maybe a free BlenderCloud account?)
There is a idea around Autoupdate addons from repo and blender updates
* @ManBlender * Hi ManBlender! I think that solution presents some security issues by letting anyone update the repo with any sort of code. I propose keeping the repo-url embedded within the add-ons instead. That way, when the "Add-on Manager" I'm looking for checks the public JSON file and finds updates, it effectively grabs new version (from specified url within the addon) and replaces the old. In essense, this "Add-on Manager" would perform TWO tasks only: 1.) Inform of available updates. 2.) Install updates. => Really, it does exactly those things we currently do manually, by hand, in a matter of seconds and a few mouse clicks!