I need to design an asset management system - what are some things I really want to make sure I have from the start, to avoid issues down the line?
I'm thinking things like, include the serialization version at the top of each file, make sure each object has a guid, etc.
I'm thinking things like, include the serialization version at the top of each file, make sure each object has a guid, etc.
Comments
Some checksum algorithms are more costly (in terms of compute and/or space) than others, ranging from very fast and lightweight (crc32, md5, sha1) to more robust but expensive (sha256, sha512).
https://www.sidefx.com/docs/hdk/_h_d_k__geometry__intro.html
https://help.autodesk.com/view/MAYAUL/2025/ENU/?guid=GUID-238914C9-4129-454C-99D7-B8C57AF423DB
https://openusd.org/release/index.html
For example, if objects saved to a scene in unity had consistent IDs available at runtime, it'd make life a lot less hellish
While it was nice to finding references even after files got moved/renamed I eventually got rid of it and reverted back to paths.
While GUIDs cause issues transparent to the user e.g. when they duplicate assets in another file Explorer.
Instead I now offer a robust asset loading system that prompts the user for...
So far I fared way better with this than with GUIDs.
TL;DR GUIDs are not as nice as they may sound at first, I suggest regular old filepaths
In the future, you can extend the format by adding new chunk types, and old versions can ignore unknown chunks. You can also add a “required” flag, so you error instead of ignoring them!
Some clients/users are a bit strange.
If its for personal usage, I would say tag search system, version history, automatic backup tools, for 2D I would also add a tileset creation tool.
I liked the ULID spec.
A path backup may not be a bad idea, but I would consider not keeping it in the same file? Not sure..
https://youtu.be/wTQeMkYRMcw?si=8mZTnrVLgefEFXjq
- you’ll need ability to store metadata associated w/ assets; Best if VCS-friendly text format. XML, TOML, YAML
- save the import settings for each source asset
- support asset import setting presets
- support editing multiple files at a time
Also worth having a "subtype mask" stored in the metadata/asset index, allowing for some diversity under a single asset type (e.g. whether a texture is a normal map/HDR env/gradient etc.) for quick filtering.
Also really think out how you want versioning to work. One GUID per version? One GUID per asset with a bunch of version references? How do you find a given version?
I manually dispatch the rendering calls, instead of having an enabled camera in the scene
JSON or YAML would be good candidates I feel?
always end up putting the display rules next to the code
(half-edge component key)
https://help.autodesk.com/view/ARNOL/ENU/?guid=arnold_user_guide_ac_scene_source_html
when sharing of data matters: combine files that fit together into an (uncompressed) zip. Kinda like modern docx/xlsx etc.
When 'being a normal file but it keeps meta for sure, locally' matters, I use 'alternate data streams' instead.
I'm thinking something along the lines of the way vscode does it, where you can edit files in isolation, or, edit within a project
While GUIDs may seem like a good choice, paths are usually more friendly to the end user. At least that's my experience from making game engines. Perhaps there are less problems with GUIDs in a 3D modelling tool!
Users need to be very aware of the GUID system or it can cause more problems than it solves.
File paths are intuitive and users are usually aware that they break when the move things.
Granted, in my case it is not a game engine. Asset dependency trees are not insanely complex, more like acyclic directed graphs. Also user tech-savyness varies greatly in my user base.
Just saying that I had less support calls once I switched away from GUIDs
I have considered using both though, a guid + a path hint as fallback or even just for error logging
For packed assets it can make debugging easier. E.g.: find "TEXTURE " instead of guessing what bytes are a version.