08.03Rethinking Asset Control - by Dan Goodman
Many of the available source control solutions out there are great if you are a programmer. Both Subversion and Perforce adequately handle the storing of assets, but neither is very friendly to creative types. How often do “bad checkins” happen because some new and obscure file created on the user’s machine didn’t get added? Or maybe the user didn’t get latest, merge the data, build the game and test it one last time before checking everything in.
Team sizes are increasing. So are the assets, themselves. The more users stressing the system, the more fragile it becomes.
NxN had the right idea with Alienbrain but never really got anywhere due to serious technical issues with their back-end. It’s been a few years since I used it last, so they may have fixed a lot of those problems. Anyway, it also had some very nice features you don’t get in other source control solutions. You could easily redesign the whole interface (it was mostly html and javascript as I recall), and they included a feature that was very art-centric. Previews.
You could generate previews of assets and view them right in the Alienbrain interface. It was a very slick feature and a selling point of the software. Finally, a user could see a preview of a model or texture (and many other asset types) without doing a get and opening the files in Maya or Photoshop, etc. That’s a real time-saver if you don’t remember the filename that was used for a specific asset. You have the chance to browse all the assets of that type and find the one you want pretty easily.
Like I said, though, NxN had its share of troubles. Still, I believe we can do better than the source control status-quo. I imagine an asset database solution that integrates with every asset generating tool, as well as the build process, generates a preview for each asset (even if it’s a bitmap that says “Preview Not Available”), and is searchable by its meta-data, including tags, creator, last modified, and so on.
The classic view of assets as a collection of files inside of folders, with users having to know exactly what files need to be checked in and out of source control when changes are made seems a little antiquated. Instead of searching through folders ten layers deep, how about using a tag cloud to find assets instead?
I imagine being able to open a web-based interface, searching for an animated character from an old project and clicking a button to copy it to a new project, including all of it’s vertex, texture and animation data and using it as the starting point for a brand new character, or maybe just as a placeholder until a new character is created. How many walk cycles does one studio need to recreate every time a new project is started, anyway? Why not take something you have and modify it to fit a new character in a completely different game?
I really beieve that asset databases are the wave of the future for game development. When the Xbox360 and PS3 came along, team sizes doubled, and assets got bigger and more complex. What’ll happen next time there’s a hardware revolution? We need to streamline the way we manage assets, or else, it’s going to bite us in the ass… even more.
I totally agree that the “traditional” way of handling assets by putting them in SVN/Perforce/etc. repositories just doesn’t cut it.
At my studio, it has turned into quite the battle to get everything synced up properly. Often, several people try to add new files with the same names which causes a lot of problems in addition to the usual bad check-ins. Locking/corruption issues are also much more frequent than I’d like them to be, and none of these problems tend to surface in the source tree.
It’s also fairly obvious that SVN in particular really wasn’t made to handle big repositories with gigabytes of data, as it tends to grind to a halt when dealing with them. Even when located on the same intranet as the server, a fresh check out of a 5-6GB repository commonly takes several hours (at least here). And 5-6GB really isn’t much these days.
We’re always on a look out for a better solution, but have yet to find any. Alienbrain looks interesting, but for us at least it’s a major downside that the server doesn’t run on *NX platforms since that’s what our dev servers are running. We might try the evaluation at some point though to see if it’s worth it.
August 11th, 2009 at 6:27 am
There’s nothing wrong with the using standard version control on your assets – just make sure you have a user friendly (and as automated as possible) front end on top.
I think your mistake is trying to manage all your data in one chunk – rarely does anyone have any need to work with all assets at once. Although I agree that none of the code repository software handles large binary data well (at least SVN and Perforce)
I actually recently blogged a bit about this here:
http://systematicgaming.wordpress.com/2009/06/24/asset-management/
(although on a quick re-read it’s a pretty meandering article)
August 21st, 2009 at 3:49 am