07.22Doing The Math - by Jeff Ward
In response to Dan’s post on when to rewrite vs. refactor existing tools, I wanted to point out what I felt was a key section:
Now comes the real decision point though. Does a rewrite make sense for the current project or should it be put off for a later time? If you’re in beta, rewriting a tool now isn’t going to help you get your game done. Consider how long a rewrite will take in man-days and calendar days. If you can get the new and improved tool into the hands of your developers fast enough to save them more time than it took to develop it, then I say, go ahead.
The key point here, is the suggestion that you “do the math’ on the tool: figure out how much time it will take to rewriting versus refactor, and balance that against the time saved by the number of developers that use the tool.
But doing the math should be a key concept when you’re trying to figure out anything tools related, including trying to convince higher ups that you really need a dedicated tools team or process team. What you need to take to them is real data that shows that you save more money with a tools team, or a tools refactor, than without one. So the question is, how do you accomplish this?
To answer this question, you need to know three pieces of data:
- How many developers use your tool?
- How much time does each developer waste because of poor design or poor implementation, OR how much time would be saved if a new tool was implemented?
- How much does each developer cost?
Number 1 and number 3 are easy to know. Just take a quick head count, and then compare their levels to the average salary for their field and the experience level using the published data from the Game Developer Salary Survey. Estimates here are usually fine. Using averages across the board (about $45k per year per developer, which comes up to about $22 an hour) here’s the numbers you’re going to come up with.
| Number of Developers |
Hours Lost /developer / day |
Cost / day | Cost / year |
|---|---|---|---|
| 1 | 1 | $21.63 | $5,625.00 |
| 2 | 1 | 43.27 | 11,250.00 |
| 3 | 1 | $64.90 | $16,875.00 |
| 4 | 1 | $86.54 | $22,500.00 |
| 5 | 1 | $108.17 | $28,125.00 |
| 1 | 2 | $43.27 | $11,250.00 |
| 2 | 2 | $86.54 | $22,500.00 |
| 3 | 2 | $129.81 | $33,750.00 |
| 4 | 2 | $173.08 | $45,000.00 |
| 5 | 2 | $216.35 | $56,250.00 |
You’ll notice that at about 4 developers loosing 2 hours per day, you’ve basically paid for another developer. Even if you have 10 developers loosing 30 minutes per day, you can afford an intern to fix that problem.
With that said, hours lost per day, or hours of productivity gained is always going to be a best guess, and if you’re trying to sell this concept to higher ups, you’re going to have to make sure that you get that number right, or can somehow convince them that you got the number right. Now, the best way to do this is by having your tools gather metrics concerning how often they crash, time between key actions, build times, cook times, and turnaround times, but that only helps if you already have a team and are just looking to expand it. Otherwise, you have to rely on hearsay, but here are some techniques that should help:
- Ask other developers how much time they think they lose on a given day because of bad tool design or performance and average those numbers. Ask for comments about how tools could improve.
- Show time lost from other developers who are only spending half of their time (or less) working on tools. If you have a bug tracker, you can use those numbers to show amount of time spent on tools bugs. Combine these with well known metrics concerning hours lost in task switching to show real cost for these support requests.
- Show an unfilled developer need. If you hear people having trouble with a specific issue, ask them how much time they think they could save (on average) if a tool were made to help them. Show that it would cost less to hire a tools developer than to leave the problem unfixed.
Of course, once you’ve convinced higher ups to create a tools team, don’t stop there. Show them it was worth it. Too many people stop once they have what they want and don’t show the real and tangible benefits. These are not always obvious, especially to people who aren’t in the “pits” (meaning doing actual development), especially when some developers may not be vocal about their increased productivity, only their frustrations with a new tool. Show the amount of productivity gained, and amount of money saved. That will help prove to you and your boss the value that a tools team can bring.
I believe that your estimates might be conservative. When talking about total cost of employee time, you have to take into account all of the other elements of their compensation. Benefits and other overhead are not a part of the Game Developer survey. You studio should be able to provide you with your organization’s burn rate which you can then use to compute the costs within your organization.
There is a tangental point here also. While it is possible to perform rewrites on tools, if those tools are large and critical path, there are many other costs to consider:
1.) Ramp up time on new tool.
2.) Missing functionality in new tool.
3.) New bugs introduced in new tool.
4.) Lost work due to data migration.
5.) Downtime during transition.
This is really just the tip of the iceberg in terms of things you are likely to encounter and need to be prepared for if you or your team undertakes a rewrite of a major tool while in production on your product. While it may be more expensive to do so, refactors should almost always be the correct solution to major tools and tech work being done during development. You need to be able to ensure uptime for your users, and you need them to provide validation on the work being done.
But, I don’t disagree with anything posted on this topic so far.
July 23rd, 2009 at 6:45 am
I agree that the estimates are conservative, mostly to keep the math simple, though in some cases, by averaging the cost of a single employee, you can iron out the additional costs incurred by having to deal with ramp up time or missed bugs.
If you were really going to make this case, yes, you’re right, you have to take into account the total cost of a new employee, not just his salary (benefits, taxes, new computer, ramp up time). However, if you’re looking at the long term productivity of the team, the initial costs are outweighed by the length of the new employee’s stay, and, again, averaging, or taking a large top number for the cost of a new Tools programmer, will iron out taxes / benefits. I’ll probably do a more involved post at some point.
Those last 2 points, by the way, are issues that I think need to be address on this blog. How to avoid lost work due to data migration, and how to avoid transition downtime. This comes into the larger issue of how to branch off builds, and how to automate a data transition.
July 23rd, 2009 at 7:05 am
Yeah, I have to say that the first points were really just a transition into the uptime discussion. Tell you what, I will get in touch about trying to put something together for the blog.
July 24th, 2009 at 6:40 am