If you are free at 6:00 PM on Thursday come out for some drinks during GDC. We will be at The Thirsty Bear Brewing Company.

The Toolsmiths now have a twitter feed here (thetoolsmiths). Blog posts should be tweeted there, and we will drop some tweets about happenings at GDC.

First off, our official Tools SIG gathering at GDC is on Thursday 1:30 to 2:30 at the IGDA booth.

Everyone should also show up to at least one of John Walker’s excellent roundtables. I find them useful to see what approaches people are using in their pipelines and share lessons learned from the past year.

Technical Issues in Tools Development
John Walker (Applied Signal Technology)
Thursday 4:30- 5:30 Room 120, North Hall
Friday 1:30- 2:30 Room 120, North Hall
Saturday 9:00-10:00 Room 120, North Hall

Lectures:

Data is a Four-Letter Word
Paul Du Bois (Double Fine) and Henry Goffin (Double Fine)
Thursday 10:30-11:30 Room 131, North Hall

Building Blocks: Artist Driven Procedural Buildings
James Golding (Epic Games)
Thursday 10:30-11:30 Room 130, North Hall

The Asset Pipeline for Just Cause 2: Lessons Learned
Mathias Westerdahl (Avalanche Studios)
Thursday 3:00- 4:00 Room 131, North Hall

Introduction to Maya API Programming and Custom UI
Kristine Middlemiss (Autodesk Inc.)
Thursday 4:30- 5:30 Room 300, South Hall

Collada – Featuring WebGL
Mark Barnes (Khronos Group)
Friday 1:30- 2:30 Room 123, North Hall

The Role of Middleware in Game Development: Today and Tomorrow
Martin Walker (Artificial Mind & Movement), Kevin Scharff (Spark Unlimited), Matthew Shaw (Electronic Arts, Mythic Entertainment Studio) and Mark Stevens (Autodesk)
Friday 4:30- 5:30 Room 123, North Hall

One of its weak points is the project file property editor. While it does wrangle compile and link flags pretty well, it tends to break when organizing include and lib paths in large codebases. The knee jerk thing to do is to laundry list each include and lib path that every project needs in each project file. This can be a real pain when moving to a different version of an SDK or library (since you have to update it in every project where it is referenced).

The built-in solution to this problem is Visual Studio Property Sheets. Property Sheets are separate files of config data that each of your Visual Studio projects reference (or inherit from since they can be nested). Property Sheets let you define your own macros that can be expanded using the standard $(Macro) syntax. Using property sheet macros to path to the include and lib folders of your libraries and SDKs allows for updating all your project’s SDK dependencies by editing one shared property sheet file.

Insomniac’s property sheets are organized like this:
Root.vsprops
> SDK.vsprops
>> Windows.vsprops

Root.vsprops defines for top level macros, SDK.vsprops defines macros that path to versioned SDKs and libraries checked into revision control, and Windows.vsprops defines macros and build settings for projects that target the Windows platform. We also have a simple perl script (getsdk.pl) that syncs SDKs specified by SDK.vsprops by reading that file in directly (vsprops files are really XML files).

Another tricky bit of dealing with Visual Studio comes from its lack of a real compiler plugin API. Each of the major build system integrations on the market (Incredibuild, SN Systems ProDG VSI) have to essentially hack your Visual Studio installation to function as a replacement for stock features. When using these plugins Visual Studio still thinks its calling cl.exe and link.exe but really its calling a replacement program that either does the plugin’s bidding or falls back to the real stock program.

This creates an interesting situation with SN System’s ProDG VSI because it needs to know which version of the PS3 compiler to use when you go to build a project. The VSI plugin looks to environment variables of Visual Studio’s devenv.exe to find the compiler to call from the replacement cl.exe and link.exe. Since most of us like to branch our code, defining which PS3 SDK to use as a global environment variable is a pain because you have to update that environment variable on every machine that builds code every time you change SDKs.

Sadly one cannot use property sheet macros to solve this problem. Even when the “Set in build environment” option is checked the user macros are never sourced into devenv.exe environment and will not be defined when the VSI goes to build your PS3 project. VSI will then error out that its unable to find the PS3 compiler. Presumably this option only causes the macro to be in the environment when Visual Studio calls out to external build tools when it is running its own builds (and not the builds of a hacked-in plugin like Incredibuild or VSI). If Visual Studio had a real plugin architecture for alternate platforms then this would probably “just work” since the proper integration would be able to use the Visual Studio build system to call out to 3rd party compilers without unfortunate file swapping hacks.

A handy solution to this problem is to use Workspace Whiz’s Solution Build Environment to read environments variable from a text file in the same folder as the solution file. The text file just needs to use a basename that matches the solution file (Foo.sln would have a corresponding Foo.slnenv file). This will allow you to specify the proper environment variables and have them be loaded into devenv.exe’s environment and be ready for VSI to utilize when it needs to build your PS3 projects. Since this file can be checked into revision control you don’t have to ever worry about setting a machine-wide environment variable for VSI to do its job.

The elections are very important, especially this year. In the past, the IGDA has had image problems, in no small part because of the actions of certain board members. It is important that we get board members that are actually extensions of our own voices, so that the IGDA can actually represent us as developers. If you are a member of the IGDA (which I really hope you are) please go vote.

So please take a moment to visit the site and respond to the poll in this post to let us know when you’d most like to gather, and we’ll see you at GDC!

The properties set by the designers were stored internally as an XML document, since XML in C# is incredibly easy to manipulate, and since the data would be different for each level design pattern.  It was easy enough to save out the raw XML data as it was stored internally, but the problem was how to transform this data into the format that would be read into the game. Enter XSLT.

C# has the functionality to transform data internally with an external XSLT file.  It’s easy enough to associate a XSLT file with a specific pattern, apply it at run time, and then simply write out the result.  At this point, it’s simply a matter of providing the designers with the data files necessary to generate their data.

The advantages here are great.  Putting the complex structure and syntax in an external file, and separating it from the designer’s view, allows them to concentrate on what’s  actually important.  The downside is that someone has to craft these XSLT files, which is a format that is a bit obtuse for someone used to functional languages.  Additionaly, XSLT has little to no debugging capability.

Now, more recently, I’ve had the challenge of writing an exporter from OpenOffice Calc, which also uses XSLT as filters for importing and exporting.  All OpenOffice documents are are stored as XML.  In fact, if you simply rename an .ODS file to a .ZIP file, you can explore the format, which is spread over several files inside the .ZIP, the most interesting of which is called content.xml.  This is the data that needs to be transformed in order to get the information out of OpenOffice into your own format.

What we wanted was a way to store properies of all enemies in a single table, so designers could tweak values easily.  The data then needed to be exported into a proprietary text format (not XML).  This is also relatively easy with XSLT, but as with anything, exporting from Calc with XSLT had its pitfalls.

It’s easy enough to access individual cells in a row in Calc.  Unfortunately, for some reason, they decided that if adjacent cells in a single row have the same value, it would only be stored in the first cell and an attribute on that cell would indicate the value repeats for n-number of cells.

With XSLT, there are no variables that can change their values, only constants within a single template (the XSLT equivalent of a function).  Consequently, there are no “for loops” in the traditional sense.  The way to get around this with XSLT is to use recursion.  In my script, I basically have every cell recursively calling the template on the next cell in the row, or on itself in the case of repeated cells, and track the cell heading (the first row of the table) separately.  Just thinking about it gives me nightmares.

At any rate, more and more data is being stored as XML, and consequently XSLT is probably here to stay.  It’s another tool that should be available in the game developer’s toolbox, but must be used with care.  To learn more, there are tons of resources on the web, but your best bet for a good introduction is the W3 schools site at http://www.w3schools.com/xsl/.

I really wasn’t sure of the answers to these questions until recently, and it made me think a lot about… well… you’ll see.

The Fundamental Trait

During my thinking, I think I identified the fundamental trait that makes a good tools programmer, or at least a programmer that would rather be in tools than any other field of game development: the desire to fix things.

This is different from most programmers who enter the game industry.  Most game programmers want to create things.  They want to point at a portion of a game and say “You see that?  I did that.  I made that happen.”  As tools programmers, we can’t really say that.  I worked on Oblivion and Fallout 3, on the game team, but there’s nothing I can specifically point to in the game that I really created.  The closest I can get to is that I can point at the models and say “You see that model?  I made the tool that made that model more memory efficient on the 360.”

Game programmers want to create, and, in my experience, when you talk to people both inside and outside the industry and say you’re a game programmer, the first thing they’ll ask is “So what did you do on the game?” expecting that you’ll be able to point to something specific.  Gameplay programmers can say “I made that system work.” AI programmers can say “I made that person move.”  Graphics programmers can say “I made that thing render and look good.”

Tools programmers can really only point at the team and say “I made that work better.”  And honestly that’s how we want it.  We take joy in oiling the machine.

The problem with the fundamental trait is that it’s not the trait that makes people enter the game industry.  A programmer that takes joy in oiling the machine can do so in almost any industry, and, unfortunately, the game industry offers some of the worst hours, worst pay, and worst return on emotional investment of any programming industry.  So, the only people you’re going to get looking to be tools programmers are going to be those that are attracted to the game industry in the first place, and the problems that it stands to solve.  In my experience, this means creative people.

The Fundamental Dichotomy

Because you’re working with creative people, game industry tools programmers not only want to oil the machine, they also want to take part in its construction.  Like most good programmers, good tools programmers want to constantly try new things, learn new technologies, and generally broaden their knowledge and hopefully find a way to improve your tool chain as a result.

The dichotomy here is that, at some point, you need to stop these creative people from being creative, and earlier than you would the rest of the game team.  They need to stabilize their tools for use by the team, and therefore they enter support for code a lot sooner than the rest of the team, and support is the one thing creative individuals dread.  It means the end of creative solutions, and the start of debugging drudgery.  Keeping a balance of both is tough, but necessary, to keep the best tools programmers on your team.

Achieving the Balance

In my opinion, there are a few things you can do to help keep this fundamental balance between oiling the machine and constructing the machine.  It boils down to the fundamental roles of your tools team during the different phases of a game’s production.

During pre-production, give your tools programmers some time to be in full construction mode.  Allow them to try new things, new pipelines, new technologies.  Let them look into converting the build system to Erlang, rewrite your Win32 tool in C#, or improve the interface on many tools.  This is a prototyping phase, and should be labeled as such.  Things made during this time do not necessarily need to be working, just prove that something will be more efficient than what is already available.

During code production, you should select the prototypes that make the most sense to implement and implement them.  Here, your tools programmers are working very closely with the other teams to make sure all of their needs are going to be handled.  They’re constructing the tools and letting out creative energy, but the tools now have clients (the team) and the client’s needs have to be met.

As the game enters production, the tools team needs to transition to support and minor improvements.  At this point you’re mostly looking at bug fixes, but all of your tools programmers should be researching how well tools are doing “in the field” and look for ways to improve interfaces and save other team members time, either now with minor changes or during the next phase.  When code lock down occurs, this should include tools, and the team has entered support mode.

However, once code lockdown occurs, the tools programmers, provided they have time, should be already entering pre-production on their new tools.  Now is the best time.  The problems of their technologies are fresh in their mind, and if they can fix them now, when more of the core team moves on to pre-production they can start being productive immediately with new tools.

This cycle helps keep your tools programmers creative, and helps them really point to the whole process and say “I made that work better,” which is exactly what they want.

What do others think?

As we approach the start of 2010, it is time for us to begin the process of electing new representatives to our IGDA Board of Directors.  These are the people who will be adding their voices to the decision making aspects of your association and choosing the direction for the organization to follow for years to come.  Each of these individuals gives of their time and energy to move the association forward in the direction they feel most represents the will of the membership and the good of the association.

The link to nominate Board members is here: http://www.igda.org/elections/nomination-packet Note that you have to be an IGDA member to nominate a Board member, and you must be nominating another IGDA member.  (We recommend you be one anyway, and please note in your membership profile that you are a member of the Tools SIG)

It’s important to remember that the Board affects how the IGDA is run, and in many ways, it is the public face of the IGDA.  In some people’s eyes, the IGDA had a huge problem with public relations last year, partially because of the positions and actions of some of its board members.  This is your opportunity to make sure that developers that you respect get on the board and represent your interests, so please do not skip out.

A few months ago, I gave a talk at GameX about improving builds and build process, and I’ve finally gotten around to posting the slides on my website, here.

There are a few things I wish I’d hit in the talk that just didn’t make it in, including talking about ways to distribute asset optimization and best practices for version control, but much of that is in flux for me right now, especially with my new found fascination with Mercurial and distributed version control (and it’s very real lack of binary / large file support).  Even without those concerns, I’ve yet to see anyone really tackle best practices in distributed asset optimization, including best practices in file composing (taking multiple files that make up a level, and composing so that they load faster), so it wasn’t something I was prepared to address.

What about from the readers?  What would you have liked to see in this talk that never got mentioned?  What would have rather I’d spent more time on?