07.26Sharing code with p4share - by Geoff Evans
Recently Insomniac Games has expanded to include a second studio in Durham, NC. Durham has their own Perforce server instance to support engineering and asset production for their titles. While the Core Team (engine and tools engineering) is still located in Burbank, Durham has a small group that add features and improvements to help get their games done. Until recently we got by okay with Durham taking code drops from Burbank, but we needed something better. We needed a way to share code bidirectionally.
Unfortunately Perforce was not designed as a distributed revision control system, so we needed to come up with our own solution. We needed to allow sharing code across Perforce server instances. p4share is a Perl script I wrote to help solve this problem without involving a huge list of complicated manual steps.
To get the job done it does a lot of deleting, syncing, and copying of files on the local client… nothing too exciting. I was however able streamline the process in a interesting way given Perforce’s ability to open a file for edit at the client have revision (as opposed to the head revision). When you open a file for edit at the client have revision then all of the changes that have been made in subsequent revisions must be resolved into your edits before you submit your changes. This resolve step is only necessary when your have revision does not equal the head revision when checking out a file, or the file in question allows for multiple checkout and someone edited and submit changes before you can submit.
Given the ability of Perforce to open a historical revision for edit, I was able to make p4share less likely to loose edits on files that have changed on both servers. p4share uses a label to store the revision at which each file was last shared. When sharing happens again in the future, the client is synced back to the revision that was submitted the last time files were shared. The files are then opened for edit at that historical revision and overwritten with files from the other server. In this state any file that had changed on both sides will require resolution to submit, but the resolving mechanics of Perforce has all the information it needs to do automatic resolution (two versions of a file and a base revision… which in this case is the revision labeled during the last share session).
Using this technique is a win because it removes the possibility of stomping files and losing changes on either side when manually merging changes from both servers.
You can find p4share on Nocturnal Initiative‘s Perforce server: nocturnal.insomniacgames.com:1666 at //Source/Trunk/p4share/p4share.pl, and via p4web here.
I’m not sure whether I’m reading your post correctly, but it sounds to me that you describe a situation that a depot ‘mirroring’ technique might be more applicable?
The basic idea is that you have a ‘mirror’ branch in both Studio A and B’s depots. In Studio A, when you want to create a code drop, you branch that drop into the second branch (Adrop). Then you take these changes and apply them directly into a mirror branch in B’s depot (Bdrop). Next you branch the changes from Bdrop into Studio B’s normal development branch. In this way you can take full advantage of Perforce’s integration tools – maintaining history along the way, and the best bit is that the whole process works two ways. Studio B can send changes back to Studio A with exactly the same process but in reverse.
Studio A Server Studio B Server
———————— ————————
Studio A Production Code Studio B Production Code
/\ /\
¦ ¦
¦ ¦
\/ \/
Studio A Mirror Drop Branch Studio B Mirror Drop Branch
Would there be any difficulties in you using this approach?
July 27th, 2009 at 1:33 am
My ASCII art seems to be formatted a little wrong, here’s another try:
Studio.A.Server………………………………..Studio.B.Server
————————………………………..————————
Studio.A.Production.Code………………………..Studio.B.Production.Code
……/\…………………………………………../\
……¦…………………………………………….¦
……¦…………………………………………….¦
……\/…………………………………………..\/
Studio.A.Mirror.Drop.Branch…Studio.B.Mirror.Drop.Branch
July 27th, 2009 at 1:39 am
That definitely works, but you need two more branches and two more integrates to get it done.
Sorry, I didn’t mean to imply that my way is the only way
July 27th, 2009 at 7:41 am
Hi Geoff,
An interesting solution. I’m assuming you came across the Perforce Proxy system (http://www.perforce.com/perforce/doc.091/manuals/p4sag/09_p4p.html) before deciding to write your own solution.
Why did you choose to go with p4share instead of using Perforce’s solution?
July 28th, 2009 at 5:34 am
Since we are on different coasts and separated by 3 time zones we wanted different downtime windows for each office, and we also wanted complete independence to prevent unscheduled downtime in Burbank from stopping production in Durham.
July 28th, 2009 at 7:46 am
Interesting. We used P4 Proxy for the same problem, having proxies at montreal (3 time zones difference) and the Netherlands (8 I think). While we definitely faced the issue of unscheduled downtime causing trouble, it never actually happened. The only real issue that ever occurred was political rather than technical (Studio X wanting things different than Studio Y using the same proxied server). It ended up being a very cheap and effective solution over the course of the projects that were using them.
August 19th, 2009 at 1:18 pm