Category Archives: Making the Case

Doing The Math

Posted on by
3

In response to Dan’s post on when to rewrite vs. refactor existing tools, I wanted to point out what I felt was a key section:

Now comes the real decision point though.  Does a rewrite make sense for the current project or should it be put off for a later time? If you’re in beta, rewriting a tool now isn’t going to help you get your game done.  Consider how long a rewrite will take in man-days and calendar days.  If you can get the new and improved tool into the hands of your developers fast enough to save them more time than it took to develop it, then I say, go ahead.

The key point here, is the suggestion that you “do the math’ on the tool: figure out how much time it will take to rewriting versus refactor, and balance that against the time saved by the number of developers that use the tool.

But doing the math should be a key concept when you’re trying to figure out anything tools related, including trying to convince higher ups that you really need a dedicated tools team or process team. What you need to take to them is real data that shows that you save more money with a tools team, or a tools refactor, than without one. So the question is, how do you accomplish this?

To answer this question, you need to know three pieces of data:

  1. How many developers use your tool?
  2. How much time does each developer waste because of poor design or poor implementation, OR how much time would be saved if a new tool was implemented?
  3. How much does each developer cost?

Number 1 and number 3 are easy to know. Just take a quick head count, and then compare their levels to the average salary for their field and the experience level using the published data from the Game Developer Salary Survey. Estimates here are usually fine. Using averages across the board (about $45k per year per developer, which comes up to about $22 an hour) here’s the numbers you’re going to come up with.

Number of
Developers		 Hours Lost
/developer / day		 Cost / day Cost / year
1 1 $21.63 $5,625.00
2 1 43.27 11,250.00
3 1 $64.90 $16,875.00
4 1 $86.54 $22,500.00
5 1 $108.17 $28,125.00
1 2 $43.27 $11,250.00
2 2 $86.54 $22,500.00
3 2 $129.81 $33,750.00
4 2 $173.08 $45,000.00
5 2 $216.35 $56,250.00

You’ll notice that at about 4 developers loosing 2 hours per day, you’ve basically paid for another developer. Even if you have 10 developers loosing 30 minutes per day, you can afford an intern to fix that problem.

With that said, hours lost per day, or hours of productivity gained is always going to be a best guess, and if you’re trying to sell this concept to higher ups, you’re going to have to make sure that you get that number right, or can somehow convince them that you got the number right. Now, the best way to do this is by having your tools gather metrics concerning how often they crash, time between key actions, build times, cook times, and turnaround times, but that only helps if you already have a team and are just looking to expand it. Otherwise, you have to rely on hearsay, but here are some techniques that should help:

  • Ask other developers how much time they think they lose on a given day because of bad tool design or performance and average those numbers. Ask for comments about how tools could improve.
  • Show time lost from other developers who are only spending half of their time (or less) working on tools. If you have a bug tracker, you can use those numbers to show amount of time spent on tools bugs. Combine these with well known metrics concerning hours lost in task switching to show real cost for these support requests.
  • Show an unfilled developer need. If you hear people having trouble with a specific issue, ask them how much time they think they could save (on average) if a tool were made to help them. Show that it would cost less to hire a tools developer than to leave the problem unfixed.

Of course, once you’ve convinced higher ups to create a tools team, don’t stop there. Show them it was worth it. Too many people stop once they have what they want and don’t show the real and tangible benefits. These are not always obvious, especially to people who aren’t in the “pits” (meaning doing actual development), especially when some developers may not be vocal about their increased productivity, only their frustrations with a new tool. Show the amount of productivity gained, and amount of money saved. That will help prove to you and your boss the value that a tools team can bring.

The Problem With Ad-Hoc Tools Teams

Posted on by
9

Regardless of what they think and regardless of whether it’s formalized, I think every game company has a tools team in some form or another. Every field of game development needs tools, and every field very frequently needs custom tools to get their jobs done more efficiently. Certainly, designers, programmers, and artists have the most obvious needs, but QA, and production frequently need ways to track changes and bug fixes beyond what a bug tracker will provide, or ways to automate test and bug reports. The thing is, when people don’t have these tools, they tend to go out and make them themselves, forming your “ad-hoc” tools team.

Ad-hoc tools teams cause many problems, which, in some cases, form the groundwork for the 6 reasons Dan posted earlier. So, how does this happen, and what problems does it cause?

The way any ad-hoc tools team starts is with a need. When someone as a problem they need a solution to, their first inclination is to fix it, and when someone needs a tool to solve a problem they have *right now* they’re going to rush to put that tool together to help them solve their problem. By definition, Ad-hoc tools are always rushed, and as a result these tools are usually buggy and badly designed. In addition to being rushed, anyone making an
Ad-hoc tool designs for themselves, not for general use. Since the person making the tool very frequently believes he or she is making a “one off” tool, the tool’s interface is made in such a way that only the creator understands it.

The way an ad-hoc tools team evolves is when others have the same need. Here, one of two things will happen, either others will know about the ad-hoc tool or they won’t, which results in either developers duplicate the work of others by recreating the same ad-hoc tool, which is obviously bad because it simply results in lost man hours, or ad-hoc tools enter general use without support or documentation, which is bad because now you have many people using a tool which is almost always far from what’s needed, but better than the alternative. What’s worse is when a combination of both problems occurs and multiple tools serving the same need enter the workflow, serving complementary needs but supported by two different groups. Here, you have lost man hours AND tools that don’t actually serve anyone’s needs properly.

Finally, your ad-hoc tools team devolves when the developers that made the tools get pulled away on “more important” tasks, leaving those using these ad-hoc tools in a state of limbo: unable to use the tools because they’re too buggy or badly designed, but unable to not use them because they do actually solve a legitimate problem. In this case, your developers are only slightly more productive than they were without the tools, but can’t complain because there’s really no one to complain to. The tool’s developer is long gone on other tasks.

This whole cycle creates a whole lot of problems that you might not notice unless you took a really close look at how your developers spend their day.

  1. Tools made by ad-hoc tools teams are rarely made available or announced to the rest of the team, and thus potential performance improvements are lost.
  2. Tools made by ad-hoc tools teams have no clear support chain, and often no way to report bugs or request new features. As a result, most users don’t think of this as an option available to them.
  3. Since ad-hoc tools team members are constantly in flux, there is no central location to go when you want to request a new tool or feature. This leads to lunch room conversations that start “wouldn’t it be nice if…” but go nowhere.
  4. Since no member of an ad-hoc tools team things of himself as a part of a tools team, they take no time or initiative to fix potential problems with the tools, and no time learning actual use-cases.

As a member of an ad-hoc tools team in the past, I can tell you I’ve witnessed this all first hand, and I’m sure many of you that work in locations without tools teams can say the same. Interestingly, you can actually make the case for a tools team occasionally by showing management the tools that are being created without a tools team, and the frustrations that are occurring because of them. It doesn’t always make the full case, but it can be a start.