The Tech Behind the Tools of Insomniac Games

Thanks to everyone that was able to show up to my talk at GDC. I thought it went very well, and I got a lot of good questions and feedback after the lecture.

The focal points of the lecture were:

  • Indirection of file locations using unique IDs instead of string paths to allow files to be moved around efficiently
  • Organizing data within asset definitions using a modular attribute system, and classifying hard engine types based on those attributes
  • Our flexible data-driven properties systems (Nocturnal’s Inspect) coupled with C++ reflection (Nocturnal’s Reflect) providing extremely flexible and highly extensible property editing
  • Sharing general processed data results using signature-based cache files to remove processed data from revision control
  • Perforce for code and assets storage, branching for milestones, and the virtues of less live assets coupled with our continuous integration system for keeping users working reliably through an increment of content creation

The slides are available from GDC’s website here.  If anyone has any questions I will be happy to handle them as best I can in the comments.

Happy hunting (bug hunting), folks :)

GDC Times Solidified

We’re just one week away from GDC and the times for most of the talks have been solidified. The links in my previous post will be able to tell you where and when to catch some tools related talks. This includes our official Tools SIG Round Table, which I will be moderating. Note that, while the time on the site is correct, the round table was originally scheduled at a different time, so the time in the program may be wrong.

REPEAT! THE TIME IN THE GDC PROGRAM FOR THE TOOLS SIG ROUND TABLE MAY BE WRONG!!! And will be corrected in the addendum. THE TIME ON THE WEB SITE IS CORRECT!!!

I encourage all of our readers that will be attending GDC to attend this round table. The discussions we have there will (hopefully) set the tone for SIG business for the coming year and get us all on the “same page” in terms of where we want the SIG to go, what projects we want to do, and how we plan to move those projects forward. This includes new blog authors and features. If you are unable to make the roundtable, but would like to suggest a topic of discussion or volunteer to be part of a project, please feel free to comment.

As a note, I will be sure to post notes of the round table (and other tools related talks) on our wiki page after GDC.

Hope to see everyone there!

Election Results

The votes have been counted and the results are in for the new IGDA Tools SIG chair, which is me! Late last week, Raquel let everyone know the results of the election on the mailing list and asked if I had a few words to share with the membership. Below are my comments sent to the list which hopefully explains my positions on where I see the SIG going over the next year. I mention it in the post, but I want to reiterate that I cannot do this alone. If the Tools SIG is going to succeed, we need to foster the collaboration of all of its members. I think this blog is a good start for that, but we have a long way to go.

I’d like to thank everyone who voted for me, and I’m looking forward to talking on the responsibilities as SIG Chair.  I am positive that this year will be full of new advancements and initiatives for the SIG.  Over the next few years, I’m hoping that we’ll be able to leverage the experience of our members into many tangible benefits for the community.  Whether this takes the form of books, white papers, surveys, open source tools and frameworks, or even just increased awareness of the benefits of a good tools team, I want the SIG to be on the forefront of driving the state of tools and tools development forward in the industry.

But this is obviously not something I can tackle alone.  One of the first things I’d like to do is create a SIG leadership: a set of individuals who are interested in taking on leadership roles, heading up small project teams and initiatives, and being evangelists for the SIG itself.  In my mind, my position as SIG chair will only be to help these people really get organized and working, and keep the daily politics and maintenance of the SIG out of their way.  If this sounds like a good position to you, please let me know, and what projects you might want to see take off.

All of this, however, is open for discussion.  If you have any ideas on how the SIG can improve, please don’t keep silent.  If you can, please try to make it out to the GDC Round Table so we can start some face to face discussion about the direction of the SIG over the coming year, and find some volunteers who will be interested in tackling the hard projects.

Engine Survey Results

A few weeks ago, I posted a survey on game middleware from Mark DeLoura . Well, the survey is done, and Mark has been nice enough to share the results and some analysis on Gamasutra. All of the analysis here is very interesting, but I find the concerns over rapid prototyping tools and asset turn-around time the most interesting. From the article:

Following up, what has been the impact of these rising development costs and a dwindling economy? What concerns have increased most for developers in recent years? […]Here we find interesting new news. Rapid prototyping enables a developer to more quickly draft and test game concepts for fun in the early stages of a project, and also use the prototype to acquire funding. Rapid iteration gives one the ability to quickly try out many ideas during development, improving the game through frequent experimentation and fine-tuning.

If rapid prototyping and rapid iteration are weighing heavily on people’s minds, what are they using now? And how many studios have live preview on the target platform in their current content pipelines?

It looks like they probably create one-off C++ applications, sketch things out on paper, or use Flash or Lua. I had suspected that more developers would be using C#/XNA due to the ease of quickly knocking out quick test applications with it, but only 5% of the responders said they are using this for prototype development. (However, 76% of developers are using C#/XNA for tool building.)

If rapid iteration is also a growing concern in game development, how many developers currently have the ability to do live preview on the target platform for their content developers (artists and designers)? According to the results, 62.5% currently have this capability. Several responders noted that they preview on a PC version of their engine and that this is good enough for most work. Certainly using the actual target platform would be even more valuable though!

This is one of many reasons why I think that increased effort in the Tools SIG will be important in the coming years. We should not only be helping people create prototyping systems (or building on those already available), but pushing for sharing of information that would allow things like live preview on all platforms, cutting turn-around time for all studios, and thus cutting costs.

As I said in the previous post, the Tools SIG will probably be running its own, similar survey soon, which will hopefully give us some ideas on where to focus our attention to make the biggest impact in tools development.

Best of Comments: Sweat and Development

Again, here I am late with the Best of Comments post, but, hey, better late than never right?

My post about Darius’s ad-hoc level editor by far created the most traffic and comments so far, and a lot of people made some excellent observations about the simple, yet effective tools. First, sjml reports on an interesting 2D level editor used by EA for prototyping in 2D:

[W]e set it up so that we could use Photoshop as the level creation tool.

Each layer or color could be assigned meaning in an accompanying text file — you could say that a red pixel on layer 2 was an enemy, and a green pixel on layer 5 was a poison trap, for example.

I’m a big fan of leveraging existing tools, especially for quick and dirty solutions. It turns out that Photoshop is already a really polished, stable program that most game developers already know how to use. Some relatively simple image parsing code, and we didn’t have to worry about bugs in the tool, teaching designers a new program, etc. Saved a bunch of time and let us focus on actually making levels and learning about our game.

Leveraging tools that developers already understand is always better than developing something they have to learn. This was a little bit more complicated (I’m sure) than Darius’s quick solution, but functional none the less.

Casey O’Donnell also posts about how easy it is to over-think simple tasks:

It is also frighteningly simple to over-think tools sometimes. Once I had a massive CSV file that I needed to convert into XML for importing into a database. At first I started writing a little command line tool to do it. As I was staring at the file in Excel to make sure I was doing it right, it dawned on me to just generate the XML tags using formulas. Even selectively excluding tags when data wasn’t present. What was going to take me a bit was done in less than 30 minutes.

Lastly (on this particular topic), Duncan talks about why this tool was fine for what it was, but why it shouldn’t be looked at for more than it was: a simple tool for an embedded platform:

Working with a real tool chain for data structures intended to be used in a Windows (or any sufficiently sophisticated programming environment) is much different than designing something for a micro environment.

I’ve spent the last 4 years programming 8-bit micros in assembler. I’ve used Excel to create all sorts of pre-baked data tables. Most of it only has to be done once, and the data has to be formatted in such a way that it can then be popped right into the code. It doesn’t change, it doesn’t need to be dynamic, and writing a tool just to do it would take longer than needed.

The smaller you constraints are, and the tighter the data, the less you need fancy tools to create it.

In addition, we had some great comments on Dan’s post about A New Way Of Developing Tools. Joseph Young points out the major hurdle for getting programmers to accept foreign tools:

One of the biggest challenges we face is the “not built here” mentality that many studios have. Even though the tools do not contribute to the ROI of a game, many of the developers would rather build the tool themselves than hire another company to do this. We see this as detrimental in the long run if you don’t have a dedicated team to work on tools. In my past development experience, the person writing to tools was often the game programmer who wrote a system in the game and now needed to provide an interface for designers. These tools are often thrown together with duck tape and barely work. But then these tools get used for the next 3 games, with more features taped on the side. This is where it really starts to hurt the game developers.

How would you get around the problem of the “not built here” mentality that faces so many studios?

This was my major argument against ad-hoc tools teams. Jeff Massung, however, responds with an actual answer to the question:

Certainly the “not made here” mentality is prevalent. That mentality exists everywhere – both in- and out- side the game industry. Most importantly, the reasons for that mentality are universal, and usually misguided. It’s important to attack each of these concerns head on. Point out the flaws in the misguided ones, and then show how Robotic Arm Software handles the real ones.

This post is way too long to repost here, so I suggest you read if for yourself. It’s definitely worth the read (maybe we can convince Jeff to formalize it and guest post it eh? ;) ).

Scheduling Tools

Scheduling tools is tricky business.  Most tools aren’t even started until they’re needed, unless you’ve got some very forward thinking management (an unlikely scenario at many game companies).  The time constraints are impossible since they’re in sync with an equally impossible game production schedule, and budget constraints can be equally constricting.

So, how do we get tools into the developer’s hands as quickly and cheaply as possible, while still delivering high quality software?  Well, if you’ve heard of the quality-speed-cost triangle, you know that if you have to have high quality, and you need it fast, then it’s going to be expensive.  Most game companies choose cheap, fast and low quality when it comes to tools, but I believe we can wrangle the schedule a bit to get pretty fast, pretty cheap and very high quality.  Unfortunately, it takes a great deal of upfront planning to get there – something game developers aren’t very inclined to do.

First of all, schedule in design.  At least 25% of your total time should be devoted to designing the tool.  If you have a week, schedule an entire day (or two).  If you have a month, schedule a week.  If it’s a large project with a GUI, include a mock-up of the the UI.  The design should also include a list of features for each of the other milestones in as much detail as possible as well as goals for each one.  These go beyond features to explain what all the features are meant to achieve.  A milestone should meet these goals, and not just the features, to be considered complete.

After the design is finished, the next major milestone (there may be smaller milestones in-between) comes halfway through the development process and is what I’m calling the “first usable version”.  This is similar to the “vertical slice” almost every game has to go through these days, and includes core functionality only.  The goal is to deliver a tool that the developers can use to get their work done, and isn’t meant to be pretty or user-friendly.  Subsequent milestones will add usability features to the tool.  It take a great deal of planning to pull this off without creating work that will get thrown away later, which should be carefully considered in the design phase.

The alpha and beta milestones deliver the final promises on quality.  It’s useful to sit with the representative users and get feedback on remaining issues. Formal usability studies can be performed if time permits and those problems addressed.  The final deliverable should include documentation on using the tool, something that has been sorely lacking in the game industry.  Too many times, frustrated users turn to the developers for help on common problems which could be easily documented, saving everyone a great deal of time.  Tutorials are also important, as they can give users a chance to see how the developers intended the tools to be used. 

 As you can see, we’re able to extended the speed side of the triangle to reduce cost and improve quality, and add a stop-gap in the middle to get users up and running quickly and cheaply.  When these components are included in the schedule, then we are not only able to get something to the users in a timely fashion, but we can also (eventually) deliver something of high quality, as long as management doesn’t see the first usable version as finished.  They must be informed that the first usable is something to build on, and not at all final, and that increasing the usability of the tool beyond basic functionality will improve the development process greatly.

Of course, this is just one aspect of scheduling tools, but any more detail would too long for this blog.  Maybe we’ll talk about the other issues in an upcoming post, or you could leave a comment describing your own process.

2009 Game Development Tools Survey

For those that don’t already know, my company has put together a game development tools survey.  This survey has been up since the end of last year.  It’s different from  Mark DeLoura’s survey, as ours focuses more on the attitudes and issues related to internally developed tools as well as middleware.  I believe that once we know the issues that developers are facing and make them public, we’ll be able to make improvements.  Our current plan is to end this survey in April and release the results.  What we’ve seen so far is pretty telling, but I won’t give away our preliminary results yet.  We still need more game devloper opinions, so please  take the game development tools survey here:

http://www.surveygizmo.com/s/73762/2009-game-development-tools-survey

Thanks for your help improving the way people make games.