02.10A New Way of Developing Tools - by Dan Goodman
In these trying economic times, companies are finding ways to cut back on development costs. This often means job losses. When one or more games need to be finished and the tools are relatively stable, it’s hard to sell cutting anyone on gameplay or content. The tools team often bears the brunt of layoffs, especially among the programming staff. This, of course, leads to another type of loss, a loss of expertise.
Often, tools are created and maintained by a very small group, so any loss to the tools department can cause widespread problems down the road when new features are needed or bugs are found. Often, the person responsible for a certain aspect of the tool chain is the only one who fully understands it, and the implications that may arise from making certain changes.
The problem is that (most) game companies don’t make money from tools, and the ROI of keeping tools developers around is less apparent than keeping the developers actively working on game projects. There are also times when the need for tools development wanes, and maintenance is all that is required. It is tempting during those times to reallocate tools developers to game teams, but this rarely works, as the crossover knowledge between the two is minimal.
So what is the solution? The key is to know exactly how many tools developers are needed when tools development is at an ebb. That’s the number required for basic maintenance and bug fixing. Additional development can be farmed out to a contract company that specializes in game development tools.
Such a company can take the weight off the shoulders of developers so they can concentrate on making games. Maintenance and full documentation (design docs, user docs and source code documentation) can even be included in the contract. Expertise is maintained, and even expanded as the tools developer gets a birds-eye view of the industry by working with multiple game companies. An influx of new ideas and problem solving techniques is injected into every new contract.
A company that specializes in tools can become an invaluable partner in achieving success for your future projects. Cultivating such a relationship becomes the contractor’s motivation to deliver high quality work that exceeds the demands of the tool’s end users, while your own company reaps the rewards of happier, more productive employees.
For more information on the benefits of using a tools development contractor, read my whitepaper entitled Build or Buy? Finding the Right Tools Solution for Your Game Company.
Often, developers need a tool right now to help with the game they are working on. Then, they move on to another project and the tool may be lost. However a contractor can anticipate the tools evolution with regard to game play, competitive requirements and technology progress.
Yet I find it is not easy to sell this idea. There is still some resistance to “outside influence”. But the reality of the market may change things. Games are no easier to make in these times. With a reduced workforce, developers can benefit from a new approach to tool development.
February 10th, 2009 at 9:00 pm
Hi Dan,
I think there’s great value for game studios to contract out their game tool development. PixelActive is focused on creating tools for game and virtual world developers creating urban environments.
One of the biggest challenges we face is the “not built here” mentality that many studios have. Even though the tools do not contribute to the ROI of a game, many of the developers would rather build the tool themselves than hire another company to do this. We see this as detrimental in the long run if you don’t have a dedicated team to work on tools. In my past development experience, the person writing to tools was often the game programmer who wrote a system in the game and now needed to provide an interface for designers. These tools are often thrown together with duck tape and barely work. But then these tools get used for the next 3 games, with more features taped on the side. This is where it really starts to hurt the game developers.
How would you get around the problem of the “not built here” mentality that faces so many studios?
February 16th, 2009 at 12:20 pm
This is just out of pure speculation so don’t take my word for it but I think the issue with studios outsourcing their tools development is that it would entail them to divulge part of their game internals to people outside the studio. Unlike middlewares such as Havok, Renderman, etc. they provide an interface and you connect to it (so the game code can keep its “anonymity”) but in the case of tools, you either 1) provide an interface or 2) show where the can connect and they connect to it; in any case, you’ll be exposing part of your games to outsiders. So I think that unless the studio plans to make the game moddable to the public like Neverwinter, I think they’ll have some paranoia exposing their game internals.
February 16th, 2009 at 5:25 pm
Confidentiality and IP control is certainly an issue. And this is where independent tool makers and developers relationship can grow. By respecting the developers agreement and NDA tool makers can become important partners to game studios. Maybe the Tool SIG can educate developers and tool makers on this matter by defining some standards and guidelines…
February 17th, 2009 at 5:19 pm
Re: Joseph Young’s comment…
Certainly the “not made here” mentality is prevalent. That mentality exists everywhere – both in- and out- side the game industry. Most importantly, the reasons for that mentality are universal, and usually misguided. It’s important to attack each of these concerns head on. Point out the flaws in the misguided ones, and then show how Robotic Arm Software handles the real ones…
Cost (misguided).
“We already have a programming staff. Why can’t they do it? Certainly cheaper than paying X.”
I’ve seen this from anything including a simple updater tool all the way to production scheduling software. There are many reasons that this is very wrong and misguided.
1. Third-party tools have likely been developed with many man-years of effort.
2. The tools are made by people whose sole job it is to continue iterating and solving the same problem They’ve likely gotten very good at it.
3. They’ve dealt with many companies and their software. This is analagous to a teacher encouraging questions in class; if you have a question, usually another student has the same one. The third-party vendor is already aware of many problems you’ll need solved before you are, because other companies they’ve worked with had the same ones.
4. Development of a tool has a much larger time investment than Revision One.
We can do better (misguided).
Like Management’s “Cost” concern, programmers have a long history of grossly under-estimating the volume of work required even for a simple tool. Even the most senior engineers have difficulty estimating the time commitment to create a customer-driven tool. Requirements change, and those changes are never planned for. This is root cause of almost all crunch.
Having a contracted party working with you and your changing requirements means your team isn’t taking the brunt of much unpaid overtime. Happier employees make better games.
Ownership (semi-valid).
I’ve worked for two different game studios now that both considered an engine like Renderware, or even a simple middle-ware library like FMOD, but ended up deciding against it for fear of the third-party being bought out by a competitor.
Robotic Arm Software has a very good stance in response to this concern. Your company owns the software that is created for you. The concern now slowly changes from a tool’s developer being bought to worrying about Robotic Arm Software’s long-term viability to finish the tool and continue support once Revision One is complete. The best solution here is to provide long-term support for the tools created with copious documentation. In addition, offering to periodically work on-site with designers, artists, and other programmers shares the knowledge.
Robotic Arm Software’s long-term goal should be a happy customer who will return to them on their next project for more work (and recommend them to others). That goal is in direct opposition to creating obfuscated libraries and tools that are hard to maintain and extend. The game industry is a very small industry. A solid reputation is worth more than gold.
Tool/middle-ware X doesn’t do what we need (valid).
Every single project is different. Levels stream but the middle-ware does its own random disk access. Data needs to run on SPUs, but the middle-ware is riddled with virtual methods and abstract classes. Pooled memory allocation schemes, but the third-party code doesn’t provide allocator overrides. There are many reasons why a tool or middle-ware would work for 1 project and not work for 99 others.
Robotic Arm’s greatest strength will lie in their ability to work with developers. Years of experience on many platforms, and having worked on many different titles that had to solve many different problems (networking, streaming, multi-threading, in-game debugging tools, …) are what will set them apart. Being able to sit down with a company’s programming staff and not only understand the problems and requests, but intelligently bring questions to the table showing that there may be other concerns that should be thought of as well before work begins.
If Robotic Arm Software can meet these challenges head on, I see it having a very bright future ahead of it.
Good luck, Dan!
February 20th, 2009 at 4:38 pm
Hi Jeff,
The explanation brought up some good points that PixelActive could use when speaking with customers. I appreciate the well thought out response.
Cheers,
Joseph
February 20th, 2009 at 8:14 pm
Many of the concerns are valid.
I have seen an example of a studio getting burnt by having an outsider build a tool. The person was later not available to update/support the tool. Plus the code was poorly written and undocumented. It’s pretty much the same situation as building something in-house and the programmer leaving.
I think the model that works is middleware, ie. game companies not getting the ownership of the tool. There really is potential for getting outstanding value for money, because the tool is licensed to several companies. Also there is often several years of accumulated man years of work in these tools/libraries, which you often don’t really have to pay for, because competition forces them to keep prices low. (Although the price of the top game engines is still ridiculously high, there is some excellent value for money middleware out there.)
There are also some valid concerns about middleware, and it does sometimes make good sense to build things yourself.
Unfortunately companies also seem to refuse to bring in contractors for programming in cases where it would make sense.
The model you propose might work in a few cases. Eg. I can imagine such a provider who is an expert in eg. 3ds max plugin development. For a game company it might not make sense to keep a 3ds max plugin developer on a payroll, because they use eg. Collada for exporting. But they could suddenly need a plugin to do some very specific thing.
March 1st, 2009 at 1:19 am
[...] addition, we had some great comments on Dan’s post about A New Way Of Developing Tools. Joseph Young points out the major hurdle for getting programmers to accept foreign tools: One of [...]
March 3rd, 2009 at 10:11 am