XSLT as a Development Tool

Some time ago, I decided to write a small C# “wizard” tool for enemy encounters and other level design patterns.  The idea was to create pattern types, that designers could define with a small amount of data (different for each type of pattern) that could be exported into a much more complex xml format that could describe when to spawn what enemies, what behaviors and properties to give them, etc. in order to make each instance of a pattern unique. 

The properties set by the designers were stored internally as an XML document, since XML in C# is incredibly easy to manipulate, and since the data would be different for each level design pattern.  It was easy enough to save out the raw XML data as it was stored internally, but the problem was how to transform this data into the format that would be read into the game. Enter XSLT.

C# has the functionality to transform data internally with an external XSLT file.  It’s easy enough to associate a XSLT file with a specific pattern, apply it at run time, and then simply write out the result.  At this point, it’s simply a matter of providing the designers with the data files necessary to generate their data.

The advantages here are great.  Putting the complex structure and syntax in an external file, and separating it from the designer’s view, allows them to concentrate on what’s  actually important.  The downside is that someone has to craft these XSLT files, which is a format that is a bit obtuse for someone used to functional languages.  Additionaly, XSLT has little to no debugging capability.

Now, more recently, I’ve had the challenge of writing an exporter from OpenOffice Calc, which also uses XSLT as filters for importing and exporting.  All OpenOffice documents are are stored as XML.  In fact, if you simply rename an .ODS file to a .ZIP file, you can explore the format, which is spread over several files inside the .ZIP, the most interesting of which is called content.xml.  This is the data that needs to be transformed in order to get the information out of OpenOffice into your own format.

What we wanted was a way to store properies of all enemies in a single table, so designers could tweak values easily.  The data then needed to be exported into a proprietary text format (not XML).  This is also relatively easy with XSLT, but as with anything, exporting from Calc with XSLT had its pitfalls.

It’s easy enough to access individual cells in a row in Calc.  Unfortunately, for some reason, they decided that if adjacent cells in a single row have the same value, it would only be stored in the first cell and an attribute on that cell would indicate the value repeats for n-number of cells.

With XSLT, there are no variables that can change their values, only constants within a single template (the XSLT equivalent of a function).  Consequently, there are no “for loops” in the traditional sense.  The way to get around this with XSLT is to use recursion.  In my script, I basically have every cell recursively calling the template on the next cell in the row, or on itself in the case of repeated cells, and track the cell heading (the first row of the table) separately.  Just thinking about it gives me nightmares.

At any rate, more and more data is being stored as XML, and consequently XSLT is probably here to stay.  It’s another tool that should be available in the game developer’s toolbox, but must be used with care.  To learn more, there are tons of resources on the web, but your best bet for a good introduction is the W3 schools site at http://www.w3schools.com/xsl/.

Tools Programmer Fundamentals

A while back, after a talk I gave at our local IGDA chapter meeting, I got an email from a recruiter at a local game development company.  He was looking to fill a position for a tools engineer and wanted to know what he needed to look for.  I never got back to him (sorry!) mostly because I didn’t know what to tell him.  What makes a good tools engineer?  More importantly, what’s going to make someone want to take a position in tools development over a position in, say, game play programming, or AI programming, or graphics programming?  Is there a distinct difference?

I really wasn’t sure of the answers to these questions until recently, and it made me think a lot about… well… you’ll see.

The Fundamental Trait

During my thinking, I think I identified the fundamental trait that makes a good tools programmer, or at least a programmer that would rather be in tools than any other field of game development: the desire to fix things.

This is different from most programmers who enter the game industry.  Most game programmers want to create things.  They want to point at a portion of a game and say “You see that?  I did that.  I made that happen.”  As tools programmers, we can’t really say that.  I worked on Oblivion and Fallout 3, on the game team, but there’s nothing I can specifically point to in the game that I really created.  The closest I can get to is that I can point at the models and say “You see that model?  I made the tool that made that model more memory efficient on the 360.”

Game programmers want to create, and, in my experience, when you talk to people both inside and outside the industry and say you’re a game programmer, the first thing they’ll ask is “So what did you do on the game?” expecting that you’ll be able to point to something specific.  Gameplay programmers can say “I made that system work.” AI programmers can say “I made that person move.”  Graphics programmers can say “I made that thing render and look good.”

Tools programmers can really only point at the team and say “I made that work better.”  And honestly that’s how we want it.  We take joy in oiling the machine.

The problem with the fundamental trait is that it’s not the trait that makes people enter the game industry.  A programmer that takes joy in oiling the machine can do so in almost any industry, and, unfortunately, the game industry offers some of the worst hours, worst pay, and worst return on emotional investment of any programming industry.  So, the only people you’re going to get looking to be tools programmers are going to be those that are attracted to the game industry in the first place, and the problems that it stands to solve.  In my experience, this means creative people.

The Fundamental Dichotomy

Because you’re working with creative people, game industry tools programmers not only want to oil the machine, they also want to take part in its construction.  Like most good programmers, good tools programmers want to constantly try new things, learn new technologies, and generally broaden their knowledge and hopefully find a way to improve your tool chain as a result.

The dichotomy here is that, at some point, you need to stop these creative people from being creative, and earlier than you would the rest of the game team.  They need to stabilize their tools for use by the team, and therefore they enter support for code a lot sooner than the rest of the team, and support is the one thing creative individuals dread.  It means the end of creative solutions, and the start of debugging drudgery.  Keeping a balance of both is tough, but necessary, to keep the best tools programmers on your team.

Achieving the Balance

In my opinion, there are a few things you can do to help keep this fundamental balance between oiling the machine and constructing the machine.  It boils down to the fundamental roles of your tools team during the different phases of a game’s production.

During pre-production, give your tools programmers some time to be in full construction mode.  Allow them to try new things, new pipelines, new technologies.  Let them look into converting the build system to Erlang, rewrite your Win32 tool in C#, or improve the interface on many tools.  This is a prototyping phase, and should be labeled as such.  Things made during this time do not necessarily need to be working, just prove that something will be more efficient than what is already available.

During code production, you should select the prototypes that make the most sense to implement and implement them.  Here, your tools programmers are working very closely with the other teams to make sure all of their needs are going to be handled.  They’re constructing the tools and letting out creative energy, but the tools now have clients (the team) and the client’s needs have to be met.

As the game enters production, the tools team needs to transition to support and minor improvements.  At this point you’re mostly looking at bug fixes, but all of your tools programmers should be researching how well tools are doing “in the field” and look for ways to improve interfaces and save other team members time, either now with minor changes or during the next phase.  When code lock down occurs, this should include tools, and the team has entered support mode.

However, once code lockdown occurs, the tools programmers, provided they have time, should be already entering pre-production on their new tools.  Now is the best time.  The problems of their technologies are fresh in their mind, and if they can fix them now, when more of the core team moves on to pre-production they can start being productive immediately with new tools.

This cycle helps keep your tools programmers creative, and helps them really point to the whole process and say “I made that work better,” which is exactly what they want.

What do others think?