ASCII version for .blend (readable and git managable)
I strongly believe that blender needs an ASCII version for the save file.
Currently .blend is a binary file. This means that the changes done on the file is encrypted and cannot be understood externally.
If we can save them as a readable text file, we will benefit in many ways. Examples are...
- We can manage our project with the sophisticated git system.
- Blender files can be parsed with a simple script.
Any ideas?
20 Comments
I coudn't agree more. In production, the people from pipeline can add tags and automate tasks from scripts, but the ascii file must be accesible to read/write. It was one of the things Blender is lacking that prevented us from migrating to Blender (we were seriously considering it).
This would be highly useful in game development. From a programmer's perspective, being able to diff a newly broken asset could save a lot of troubleshooting time.
@graehu: Unless yur using BG game logic isnt a part that should be inside a .blend file, ea armory exported logic should be a seperate file then youor blend shape resource file, blends can e binary compact.
@peter.boos: Yeah, I'm not talking about logic. I'm talking about missing textures and the like.
Will you see your scrollbar at all? It will be less than a pixel!
Scrollbar size is completely irrelevant.
I mean this sincerely: Can you spot the difference between a technical hurdle to be crossed by anyone attempting this (2 above), versus personal preferences that don't apply generally (1, 3, 4, and scrollbar, etc)?
I agree with jd.knox, moreover:
I work in mobile game development, where models are few hundred polygons at most. As serializing as text would be optional, I would only turn it on on small models
I'd rather see time spent on the IO UI to be more native to the OS.
I heartily agree, Allosteric. Since moving to Windows, what I really miss is the kind of control I had over my environment with FreeBSD where every config file was ASCII. Easy to back up, easy to understand, easy to replace if they got nuked.
If I had a third thumb, it would be up, too.
Sounds like a very useful idea. I support this. Just imagine the possibilities of bulk editing properties, attributes etc. from just a text editor. You could use text editing scripts that are not Blender-specific to alter the contents of your Blends.
I am hoping that the move to USD format export, and Import would benefit from an ASCII format. It is easily parsed, scripted and read. Search and replace, etc. I currently have a bug in which the Blend file references graphics cards I no longer have on my station??? No idea how to purge these. Upvote this option.
blender.stackexchange.com/questions/112844/creating-a-copy-of-a-bpy-prop-collection-instance
this may be a good starting point for addon developers.
While having more granular version control on .blend files would be desirable, full ASCII probably isn't necessary to achieve that. Furthermore, there are already a couple ways to access .blend files from a script (see this question on the Blender StackExchange ).
@Jason van Gumster: thanks for your comment. I'm not quite sure what shouldn't be in ASCII when fbx can be in ASCII and when blender has its DNA structure. So, any examples? As for the link, they seem extremely helpful for parsing thorough .blend files. However, they still cannot resolve the problem that .blend cannot be controlled under the git system, which I crave for when I develop my Armory3d projects.
@Allosteric: I have a bit of experience with this as I use version control on all of my 3D projects (Mercurial, FWIW). The difficulty with a lot of text-based formats is that it's very easy to lose coherence from one save to the next. DXF is a perfect horrible example of this. Each time you save, it's possible for the data to be organized and ordered in a completely different way. So a diff with this kind of format is almost as meaningless as a binary diff.
The problem with Git is that it sucks at binary compression. Mercurial is marginally better. But at some point, you're going to have to accept using binary data as input and just use a different tool for your diffs (if that's what you want). I find that I don't need diffs nearly as much as I value having versions with notes about what changed in each version.
That said, I do see a lot of value in having more granular versioning in a project. I had an add-on written a while back that would take any .blend file and break it up into smaller chunks. It would disassemble each datablock (image, texture, text, mesh, object, etc.) into a separate file and use a JSON file as glue (referencing all of the parts and holding extra numeric and text-based values and attributes). You could version control the disassembled file. Then, when you wanted to work, you would load that JSON file and it would assemble a .blend for you from those parts. At the time, I stopped developing it because it would be really slow for large files (disk IO was the bottleneck)... but with modern SSDs, that's likely less of an issue.
@Jason van Gumster: I get your point. I agree that some data may produce errors when it is externally manipulated. As for them, maybe we can keep them single lined or something so that a git manipulation won't be dirty. The macro structure shouldn't be the case, I think.
@Allosteric: Moving it to a single line just kinda shuffles the problem further down the pipe, doesn't it? Also, it would probably make diffs even more difficult to read and verify.
@Jason van Gumster: maybe so. I just meant; I prefer them kept in a single file, and I prefer them to be a readable text even when they are not quite fit for an external editting.
@Allosteric: Maybe I just have a different expectation for what "readable" means. :)
The text representation should have a defined order the datablocks has to be saved. For Textures and other binary parts there should be a external relative link instead of using base64 or somthing else. At least each datablock must start on a new line but it would be better if a single datablock is split over multiple lines. It isn't necessary for a human to read the whole file, the format should be designed with git usage in mind.