Mechanics of moving an other-repo to the monorepo

Once a Bazel monorepo has been setup, a next task is to slowly consume other projects, one-at-a-time. We'll walk through the details of how you perform that operation.

This post assumes you've already done some preliminary work:

  • You have a monorepo with a folder structure and "governance": a set of policies and a group that enforces them.
  • The other-repo has a similar Bazel setup, so that it can merge into the monorepo Bazel workspace. Alternatively, you can add the new folder to .bazelignore and "Bazelify" it after performing the migration.

Preparation

Make sure the monorepo is not a regression for the other-repo developers. It should have a good developer experience, and feature-parity for common workflows.

Communicate this process with the developers in the other-repo. For example, open PRs in that repo should be landed before the move, or else they'll have to follow a similar process to rebase those changes into the monorepo.

Create a merge commit

It is always recommended to preserve the git history of the other-repo, so that we can use the blame layer for future forensics about who changed what. Fortunately this is easy to do.

First, install a git plugin: https://github.com/newren/git-filter-repo

Now you can run these commands:

$ cd other-repo

# Make sure you are carrying the current history of the default branch, don't lose anything!
other-repo$ git fetch; git reset --hard origin/main

# Rewrite the history as if it always had been authored under the other/ folder
other-repo$ git filter-repo --to-subdirectory=other

# make sure you're at HEAD of monorepo as well
$ cd ../monorepo
$ git fetch; git reset --hard origin/main

# Make the other-repo history visible to the monorepo
monorepo$ git remote add other ../other-repo
monorepo$ git fetch other

# Make a "merge" commit that has one parent at monorepo HEAD and another at other-repo rewritten HEAD
mono-repo$ git merge other/main --allow-unrelated-histories

At this point you have some local commits, and you can test as usual and send the merge commit as a PR. At this point, you CANNOT REBASE. If you need to make changes, for example because monorepo HEAD has new commits, just do these steps again. In a busy monorepo, you may need to wait until outside working hours, or even freeze commits landing on the monorepo, in order to perform the operations without new commits being added.

Make the move

Now it's time to merge onto main.

Note that you might have to change the monorepo branch protection to allow merge commits, if you normally permit only linear history.

As soon as you land the merge commit to the monorepo, you're in danger of the code diverging between the two copies. You should use whatever means you have to avoid new commits landing in other-repo, as you'll need another recipe to rewrite-and-sync those commits and they might not merge cleanly. As quickly as possible, get the owning team to verify their CI and release workflows, and archive the other-repo. It's easier to un-archive it again later if needed than to deal with consequences from it being a "fork" of the code which diverges as new commits land there rather than in the monorepo.

Re-train developers

Now that the other-repo is archived, developers who worked there need to be oriented to working in the monorepo. They should be aware of "trunk-based development" - if they work on a library which was previously versioned through a release process, they aren't used to their changes being "live" as soon as they land them.

Setup CI/CD workflows

Bazel is supposed to make your Build and Test fast. However it has setup steps (Repository rule execution and Analysis phase over the target graph) and these always run before any cache hits can be looked up. Therefore a cold Bazel worker may be slower than your legacy build.

We suggest looking at our Aspect Workflows product and read our other blog posts about making Bazel fast on CI.