King Arthur: The swallow may fly south with the sun or the house martin or the plover may seek warmer climes in winter, yet these are not strangers to our land?
1st soldier with a keen interest in birds: Are you suggesting coconuts migrate?
-Monty Python and the Quest for the Holy Grail
Unfortunately, source code, like coconuts, needs help to migrate from on repository to another. When we started, we only wanted to change where our Smalltalk source was stored. We had lived with multiple repositories and were tired of ensuring that dependent changes in each of the repositories got into the right build, or was correctly backed out when there were problems. Our original goal was only to replace the ENVY repository with CVS, or any other version control system, and continue to use all the existing browsers.
To get to that goal, we needed a way to export our Smalltalk code to flat files and a way to load from the files back into the image. The obvious answer is Smalltalk's existing file-format, the "chunk" format. Chunk format, while fine for computers, leaves something to be desired when it comes to human consumption. Various other file-formats for Smalltalk have been development but none of them felt quite right for us.
Our solution is something we call FileBasedSmalltalk (FBS), which has become on of the core components of STDT. FBS provides a meta-model to map classes and methods back to their files so that the source can be kept on disk, not in memory. The model maps FbApplications, FbSubApplications, FbClassDefinitions, FbExtensions and FbMethods onto their "normal" ENVY model objects. The FbLoader is responsible for reading the files in and creating these model objects. To provide name resolution, the FbResolver was created and it can be thought of as the Smalltalk dictionary for FBS.
I can hear you thinking, "You have a meta-model... you still haven't actually installed any classes in the image" and you're right. The EmImageBuilder is a fantastically complex piece of code that does a great job of atomically loading classes, applications, etc. We definitely didn't want to rewrite it, so we did the next best thing - we subclassed it. FbImageBuilder is the new image builder that can force the FBS model objects into the right shape, allowing them to be loaded and installed as classes. Getting this correct was a lot of fun as mistakes often meant starting with a new image.
Now, I've explained how the Smalltalk source gets loaded into the image, but what does this have to do with repository migration? At the same time we were getting the FBS figured out and working, we had started to mirror all of our code from ENVY into CVS each as part of every build. Source code without history - no change comments and no ability to see why things changed - makes it very difficult to determine if that nonsensical code snippet is there for some vastly important reason or can be safely removed. Today there is over a full year's worth of history in CVS for all of our Smalltalk source.
As part of each build, our Smalltalk tools were run twice, once from ENVY and once using FBS. The output from the two were compared to ensure they were identical. It was now safe to only run the tools using FBS, although ENVY was still the "master" for the source and the mirroring to CVS continued. And that was the migration, a gradual switch once confidence had been built up in the new tools.
Turned out that trying to keep the ENVY browsers and make them work on files was more work than was practical. Instead, the STDT tools were born.[Read More]