The properties set by the designers were stored internally as an XML document, since XML in C# is incredibly easy to manipulate, and since the data would be different for each level design pattern.  It was easy enough to save out the raw XML data as it was stored internally, but the problem was how to transform this data into the format that would be read into the game. Enter XSLT.

C# has the functionality to transform data internally with an external XSLT file.  It’s easy enough to associate a XSLT file with a specific pattern, apply it at run time, and then simply write out the result.  At this point, it’s simply a matter of providing the designers with the data files necessary to generate their data.

The advantages here are great.  Putting the complex structure and syntax in an external file, and separating it from the designer’s view, allows them to concentrate on what’s  actually important.  The downside is that someone has to craft these XSLT files, which is a format that is a bit obtuse for someone used to functional languages.  Additionaly, XSLT has little to no debugging capability.

Now, more recently, I’ve had the challenge of writing an exporter from OpenOffice Calc, which also uses XSLT as filters for importing and exporting.  All OpenOffice documents are are stored as XML.  In fact, if you simply rename an .ODS file to a .ZIP file, you can explore the format, which is spread over several files inside the .ZIP, the most interesting of which is called content.xml.  This is the data that needs to be transformed in order to get the information out of OpenOffice into your own format.

What we wanted was a way to store properies of all enemies in a single table, so designers could tweak values easily.  The data then needed to be exported into a proprietary text format (not XML).  This is also relatively easy with XSLT, but as with anything, exporting from Calc with XSLT had its pitfalls.

It’s easy enough to access individual cells in a row in Calc.  Unfortunately, for some reason, they decided that if adjacent cells in a single row have the same value, it would only be stored in the first cell and an attribute on that cell would indicate the value repeats for n-number of cells.

With XSLT, there are no variables that can change their values, only constants within a single template (the XSLT equivalent of a function).  Consequently, there are no “for loops” in the traditional sense.  The way to get around this with XSLT is to use recursion.  In my script, I basically have every cell recursively calling the template on the next cell in the row, or on itself in the case of repeated cells, and track the cell heading (the first row of the table) separately.  Just thinking about it gives me nightmares.

At any rate, more and more data is being stored as XML, and consequently XSLT is probably here to stay.  It’s another tool that should be available in the game developer’s toolbox, but must be used with care.  To learn more, there are tons of resources on the web, but your best bet for a good introduction is the W3 schools site at http://www.w3schools.com/xsl/.

In my efforts to put the tools and tool-chain wiki together (another one of many projects that was sidelined during the summer), I stumbled upon the (now not-so) recent blog post of 2D Boy, announcing the release of their rapid protyping framework (“BoyLib“). I don’t like having design students get bogged down in technical issues right away, so I am always looking at a variety of different tools for testing and thinking through game mechanics. While evaluating the BoyLib framework, Jeff Ward suggested that I compare it to Angel and AngelXNA [Jeff on AngelXNA], both of which were new to me. This is also read through the lens of my current efforts to port IndieLib to Mac OS X. Thus I spiraled further down the rabbit hole of rapid prototyping.

One of the primary advantages of AngelXNA that I’ve found is its linkages to XNA and the Game Studio asset pipelines. While rapid prototyping, it is nice to not give a second thought to how your assets will be imported into the game. Of course this isn’t entirely the case, sounds must still be in particular formats and in particular places for XNA to properly locate them. BoyLib uses irrKlang and Angel uses FMOD for sound importing, which buys you some flexibility greater than XNA. IndieLib uses DevIL for its image reading and TinyXML for XML. BoyLib also uses TinyXML for XML. Angel makes liberal usage of .ini configuration files and Python for scripting. Angel uses Box2D, BoyLib doesn’t seem to include a physics engine of any kind, so you’ll be coding that up from scratch. AngelXNA of course implements the underlying physics in an XNA friendly manner. IndyLib really excells in the 2D realm. The developers have clearly thought through the issues that most often affect 2D game development. Things like layers are an underlying construct, so end up being elements you pay attention to at the outset rather than thinking about them after the fact.

Ultimately, what I found interesting about each engine was its approach to pipelines. Getting art assets and design data into an engine becomes the crucial component. Image libraries, text file parsing, and supported scripting languages quickly become the focus for engineering problems on these engines. All except AngelXNA that is. The real plusses and minuses for each engine seem to come down to file support, preferences toward coding conventions, and supported platforms. While AngelXNA is great for prototyping and does an excellent job of using XNA’s asset pipelines, it creates Windows only binaries, and while it may allow you to prototype on your Xbox, taking it other platforms will provide not an insignificant amount of work.

Of course Angel has has had a bit more time to mature that BoyLib, and AngleXNA benefits from this history. IndyLib, though quite promising, offers the least amount of support for various file types and means by which to get things moving. Not to mention that Angel works on Windows, Mac OS X, and AngelXNA on Windows/Xbox 360 Community. BoyLib and IndieLib are currently stuck without a Mac port, which is something I’ve spent several days now really investigating. Combined with Angel’s overall better support for various file types, it seems to have the overall edge for a robust protyping tool and overall indy “engine.” That isn’t to say that IndyLib doesn’t have a great deal going for it. Indeed its focus on the 2D may very well make it a more desirable tool in some cases.

I was also intrigued by the near ubiquitous support for some sort of USB controller for these engines. This seems to indicate the for many game developers, the likely idea seems to be to really use these to develop prototypes for consoles rather than as “engines” in the more traditional sense. These are frameworks used to test out mechanics and user interactions quickly and relatively inexpensively. Personally, I’ve found that a USB Xbox 360 Controller can really provide an identifiable and productive gaming experience more quickly than the mouse or keyboard when trying out simple game mechanics and user interaction methods. Thus having controller support right away is a treat for most of these libraries.

This post is also an invitation, I’d like to get a sense of what frameworks readers are using. What works for you? What doesn’t work? What Open Source frameworks do you use? What closed source tools/frameworks do you use (assuming you can say)? As I gather all of these bits and pieces together, you become my greatest assets, so let me know what tools and tool-chain bits you want to see on the wiki.

[youtube=http://www.youtube.com/watch?v=DPIA2g8T6Hw]

His integration with Uni-Verse is definitely an interesting take on what life as a content creator could be like without having to deal with load/save/export via files on disk. Verse seems to disconnect where data is stored and how its managed from the user. It comes across to me as fitting the client/server paradigm of the web into content creation tools for games.

In my mind it would be awesome if there was a container library like C++ STL that supported:

• Allocator objects as instance API as well as static API
• Usage of heap handles (instead of pointers) that would allow memory relocation
• Creating and synchronizing threads in a platform-agnostic way
• Lockable containers that are safely accessible from more than one thread
• Sleeping threads on empty containers (with semaphores or events)
• Basic archiving of data to / from flat files

Here are some solutions I have found:

• Intel’s Thread Building Blocks address the locking and sleeping issues.
• EASTL conquers the instance allocator problem, but it is closed source.
• C++0x could potentially address some of these (with native support for threads) but I would expect that support or C++0x is still years away (especially for game console compilers).

What other solutions are out there?  Are there any resources out there that put all of these pieces together?  It seems to me that a container library that solved these problems in symphony could be very powerful for working with code that needs to work on both workstations and consoles (E.G. portable code that abides by strict memory requirements).

Has anyone solved any of these problems interfacing directly with C++ STL?  If not, does the games industry need something besides STL that could suit these needs better?

* The .NET Forms Designer was very unstable for us. Editing code that gets parsed back into the forms designer lead to forms that could no longer be edited in the designer. Cryptic internal exceptions when trying to edit forms were difficult / impossible to troubleshoot. Sometimes the form would appear but silently remove user controls placed within it. We ended up having to hand code all our UIs because the forms designer would inevitably break our layouts. This was especially true with user controls and controls loaded from assemblies outside Windows::Forms.

* Windows::Forms is buggy and lacking features (Multi-select tree view anyone?). We encountered bugs that we had to workaround that compromised our user experience because Windows::Forms is closed source (why is that anyway? we couldn’t think of a single good reason for Microsoft to keep that source close to their chest). wxWidgets comes with full source and we have made fixes and added features that have been picked up by the main line. wxFormBuilder is a superior solution to this problem and its fast and extremely stable.

* We like to tightly integrate code between our game runtime and tools, and we see this as key moving forward with the next generation. This means we have native C++ code that we wanted to call from .NET (and sometimes *god forbid* vice versa!). This lead to managed C++ (C++/CLI) wrapper classes that ended up creating more work for us. Using .NET made it feel like we had less choices rather than more choices when it came to organizing our code (you must use DLLs), integrating code (especially with intricate data structures), and deploying our tools (.exe.manifest and .exe.config madness just to use an assembly in another folder). All of these things are easier with wxWidgets and native code and we are able to spend less time troubleshooting .NET idiocy and more time creating what our users want. Also, mixed-mode debugging is no fun and gathering callstacks from a crashing tool across the managed/unmanaged barrier isn’t really possible or supported by .NET.

* We already had a good C++ reflection system and data-driven property system (they are in Nocturnal). This gave us type checking, serialization, automatic type conversion, and property sheets without using .NET. These were the killer features to us for using .NET, and having our own solutions to these features that gave us the freedom to ditch .NET for our level and asset editor in about 1 man month.

* Embedding technology based on .NET as a plugin in another product (like Maya) is impossible and/or dangerous and/or difficult. This goes back to what I said about having more choices rather than less choices. wxWidgets integrates with Maya very easily.

* Getting support from Microsoft is a joke. The community is a better source of support than contacting Microsoft directly. Several of our tickets we closed in their tracking system (without a real resolution that we could use) because they were fixed in ‘their internal builds’. They just hung us out to dry on a lot of things. If we are going to be paying for a development environment and UI toolkit then it either needs killer support or it needs to be open source so we can fix it ourselves. Getting stuck at the mercy of a company that won’t release service packs to fix your bugs and makes you pay for them in the next product cycle that contains little in the way of other benefits is immensely maddening. Also, the public hotfix directory is a joke… They need to make fully tested service release so we can get on with our lives!

* wxWidgets is cross platform. I like to believe that someday we (as a studio) won’t be as dependent on Windows as we are now, and wxWidgets gives us strategic room to grow in this regard. The Linux and Mac ecosystems have a lot of things that are desirable compared to Vista/Windows7. Not having to reboot your server farm every 2 weeks due to kernel memory leaks in the 32 bit emulation layer is one example

Ultimately switching to wxWidgets simplifies everything in our tools pipeline, makes refactoring and making major changes to key subsystems easier, and makes our tools easier to deploy to our users. It was an easy choice to make, and we haven’t looked back since. We still have several small tools that use .NET and going back to work on them and fix bugs is no fun after switching to wx.

One of the coolest things about working for a company like Insomniac Games is being able to open source technology that our tools team creates.  Nocturnal is our open source project, and one of the most useful things included with the ToolsFramework project is its c++ event and delegate system.

Any programmer that has used Microsoft’s .NET framework will tell you that the events and delegates implementation in .NET is incredibly useful. It pervades the entire Windows Forms framework. UI element callbacks, plug-in APIs, and application automation are all made easier by this design pattern. Unfortunately jumping into the .NET Framework doesn’t go without significant trade-offs.

When Insomniac ported its tools from .NET to wxWidgets we definitely wanted something as slick as .NET events and delegates. This is what we came up with, and we have been iterating on it for several years now. Most of our key libraries and applications lean on it heavily.

This implementation supports a lot of tricky cases that aren’t easily handled by function (or member function) pointers alone. Delegates can be added and removed within delegates of the same event and objects that own events can be deleted from within delegates of the same event.

To get started, just typedef the Signature template with your return and parameter types. Then use the Event class (a member class of Signature) to make a multi-cast event, or Delegate to make a single-cast event.

EventDemo.zip

Geoff Evans
Insomniac Games

From the site:

Angel was originally made by a group of employees at Electronic Arts Los Angeles for use in a GameJam they were planning for April of 2008. The source was opened in January 2009.

Angel provides:

• Actors (game objects with color, shape, responses, attributes, etc.)
• Texturing with Transparency
• “Animations” (texture swapping at defined intervals)
• Rigid-Body Physics
  • A clever programmer can do soft-body physics with it
• Sound (.wav only)
• Text Rendering with multiple fonts
• Particle Systems
• Some basic AI (state machine and pathfinding)
• Config File Processing
• Input from a mouse, keyboard, or XBox 360 controller
  • Binding inputs from a config file
• Tuning Variables that write out to a config file
• In-Game Console
• Logging

It should be noted that Angel is not a framework for beginning programmers.  Again, from the site:

Angel is designed for experienced engineers. That doesn’t mean that it uses all sorts of crazy programming techniques or is difficult to use (quite the opposite: see #1), but nor does it hold the developer’s hand very much. It’s expected that a developer has at least some experience exploring a codebase to see how it works.

I think this is fine for a prototyping framework, as they’re made to get “out of the way” so that a programmer can do exactly what they want quickly.

I look at prototyping frameworks as one of the key tools a company can provide to programmers. It allows them to quickly and easily investigate a new system in your game without having to go through the complicated mess of your engine (and, yes, regardless of how awesome your engine is, it’s still a complicated mess). In my mind, this is a big thing, and I hope it continues to evolve over the next few years.