Image Texture Auto-Save should take into account whether the image has users
Background
Currently, Blender keeps track of whether a datablock is in use via its user/fake user system, and only the used datablocks (or ones that have a fake user - i.e. that the artist marks as wanted) are saved.
At present, there is ongoing work to introduce auto-saving to image textures in projects.blender.org/blender/blender/issues/134518 , which is important because it helps prevent people from losing work and brings the saving behaviour of images in line with the rest of the data created in Blender, but so far it is blanket saving all the images, rather than only those that are in use or marked as wanted.
Proposal
When auto-saving images, only the images that are in use (or marked as wanted via a fake user) should be saved.
Reasoning
When working with Blender, a lot of unwanted cruft can build up because of trying and discarding lots of different things. With the user/fake user system, the unwanted cruft is automatically cleared out every time Blender is closed. This behaviour is largely consistent across Blender (and extremely helpful), so the same behaviour should be applied to the automatic saving of images as well.
Potential Drawbacks
Artists may not know how the user/fake user system works and might not know they need to mark images they want to keep that aren't in use, so these images could be lost. However, this drawback applies to all datablocks and should likely be addressed in the future with a universal solution (possibly some sort of viewer for unused datablocks that can be shown on exit, but this is beyond the scope of this proposal).
Open Questions
-
Should an option to also save unused images be added?
- I'm leaning towards no on this because it's likely that any images the artist cares about will be in use by image texture nodes. UPDATE: after discussion in the comments it may be better to move towards a system where auto-cleaning of unused data is optional, so I'm no longer leaning towards no on this question.
-
Should render results be saved regardless?
- Render results are unlikely to ever have users, so they will almost never be included in what's auto-saved; however, I'm also leaning towards no on this because render results would almost exclusively be used outside of Blender, so it is already natural for the artist to save them out (or they are already saved out automatically when rendering an animation).
Estimated Project Size
Small.
The user/fake user system is already present, so it is probably only a matter of checking whether the image datablock has a user and skipping it if it doesn't.
Suggested Implementer(s)
Those working on projects.blender.org/blender/blender/issues/134518
Additional Context/Notes
This was brought up initially to Sean Kim in the User Interface Module channel of chat.blender.org in regard to projects.blender.org/blender/blender/pulls/155636
I have used the term "artist" to mean the person operating Blender to prevent any confusion with the user/fake user system.
10 Comments
Honestly, what's the worse scenario:
It's not the 90s anymore. People have more storage than ever , and they know to clean up when needed. And the fact is, there are a plethora of requests that have been posted here and on other platforms to do away with this fake user system altogether and just never delete anything the user didn't ask.
It is not about storage size, it is about the ability to distinguish garbage data from useful data.
If it was just "the ability to distinguish garbage data from useful data", then it would not delete anything behind your back. It would tell you " hey there is this bunch of data that doesn't seem used, maybe you should check it up ". In fact, that's what the Outliner in Orphan Data mode does very well.
But instead it assumes anything that doesn't have a user, or a fake user, must be deleted, and you learn it the hard way because it's not even an established behavior we commonly see in other softwares. People have and still do lose useful data because of this. And i see no reason to force this unless you are worried about memory. Which was a valid concern in Blender's beginings decades ago.
This isn't about storage space, that never even crossed my mind when writing this proposal. This is about auto-cleaning, which I find beneficial to my workflow.
Consider an automatic vacuum cleaner, it goes around and vacuums up anything in its path. regardless of whether it is dust or one of the gears from your very expensive Rolex watch that accidentally got knocked onto the floor while you had it apart for maintenance. Yes, there are downsides with any automated system, but if used carefully it can be quite beneficial.
No one wants people to lose data and I agree that the system could use more guardrails. If you check, you'll see that I explicitly mention this in the "Potential Drawbacks" section of my proposal. To expand on that example a little more, you could have a warning about orphan data on quit with an option to pop up a window with the Outliner in Orphan Data mode so that users are simultaneously informed that Blender doesn't save orphan data and given a chance to review the orphan data before it is discarded. That being said, the downside to this is that all you see are the names, so if you have 20 Cube.xxx and 10 untitled images it isn't all that helpful (though once you are aware of the system, you're probably not going to need to review it all that often -- I almost never view Orphan Data -- or you'll probably just look for named assets that you forgot to mark with a fake user, e.g. material variants).
Also, don't dismiss the difficulty people have throwing things out. It's pretty well known now that people can have a hard time deleting stuff, so it's much easier to have someone or something else do it for you, rather than having to make the decision yourself (e.g.: zoho.com/workdrive/digest/psychology-of-file-hoarding.html ).
I would prefer neither scenario (assuming that the data it's saving that I didn't ask to be deleted is unused) and I see no reason why there can't be a solution for both.
To clarify, I am talking about Blender's current orphan cleanup design, I guess I don't understand the point of keeping building up on its shaky foundations.
Honestly I 100% agree that it would be ideal if we could have a form of autocleaning tool IF:
But it's not what we have. And everybody on other softwares that have neither this or Blender's current autocleaning are fine with it. But they have manual tools to help you cleanup broken or unused things, which leave the user in total power of what gets cleaned up and when. It's manual, but it's also way safer because of it: you run it when and how you want, you can save your file before it, see what happens, confirm the changes or revert it. I would rather keep building on something like that, than on what we currently have, it that makes sense.
Thank you for the clarification. I actually think it's a pretty solid foundation that just needs a few tweaks. Hopefully, why I think this will be made clearer with this reply.
How do you propose the program determine what data is data you need? Currently, Blender ensures any data that is used, or has been marked by the user as wanted, is saved. What else do you think should be done in an auto-cleaning system to differentiate needed data from unneeded data? (To be clear, that is a real question)
I agree this is very important, and that Blender needs improvement here.
Blender partially does this, it has an option to manually purge orphan data, but it doesn't have an option to turn the automatic cleaning off. It might make sense to provide such an option, but if how Blender handles auto-cleaning is made more visible and some additional guardrails are added, I would prefer not to (for the sake of collaboration and team projects).
A point of clarification, just to make sure we're on the same page because there is some important nuance here. Blender doesn't auto clean by deleting data (unless you manually choose to purge it); what it does is it skips over orphan data when saving. For the most part, this is very predictable and reliable (at least now, previously it wasn't as much); however, it doesn't account for non-standard shutdowns of Blender. So an additional guardrail that would make sense would be to save orphan data in auto-saves, so you can completely recover when there's a crash or power outage.
True, manual is safer, but it's also not as convenient, and convenience matters. Humans often take the path of least resistance and often won't (or can't) put forth the extra time and effort to do manual cleanups, so if you have a good, reliable system that can handle some of the cleanup for you, I think that is beneficial.
I often save many incremental versions of a project (sometimes dozens in a single session), and it's unrealistic for me to break my flow and manually review and purge orphan data for each save in order to keep them clean and have a high signal to noise ratio. So if everything was manual, I would probably end up either just purging the data without looking at it (which gives the same result as what we have currently, but takes more work) or not purging it and dealing with it growing and making it harder to find things until I got fed up enough that I purged it without review anyway (but this doesn't help if I have to link or append something from a previous blend file that still has all the orphan data).
I personally don't think this would happen that often. You can see exactly what data will get purged in the Outliner beforehand, so there is no need to actually perform the purge to check what gets deleted (and most of it is out of sight, so you're not going to notice a difference until you go looking for a specific thing, which you can do before purging).
Wanting to be in control makes sense, and I agree that Blender currently doesn't completely provide that feeling, I just think that it can be accomplished by modifying the current system and that this would end up being more beneficial.
That's a thing I do not know. Anytime i try to think of something I instantly find a case where it's incorrect. And in the meantime it's there and hurting.
And that's my issue with the current "forced and can't choose to have it or not" situation: it will make you lose data. Even in the case of a user knowing about the system perfectly well and doing all the things in the world to be safe with it: data loss can still happen. It's not even an "if", it's just a "when".
And when that happens on professional work, all the benefits of having an automatic vacuum cleaner go out the window.
That's fair. So is it also fair to say you're not necessarily against there being an auto-cleaning system, but that you're against an auto-cleaning system being the only option?
Yes. To me this should be optional. And it being prone to data-loss by-design should be justification to make it disabled by default.
I think I would want to see how the auto-cleaning system with proper guardrails functions before agreeing that it should be off by default, but this is getting into final details that are still a fair bit ahead.
Thank you for the feedback, I have updated the proposal to no longer be against having an option to save everything/turn auto cleaning off.