Google copybara: moving code between repositories

A tool for transforming and moving code between repositories. Copybara is a tool used internally at Google. It transforms and moves code between repositories. Often, source code needs to exist in multiple repositories, and Copybara allows you to transform and move source code between these repositories. A common case is a project that involves maintaining a confidential repository and a public repository in sync. Copybara requires you to choose one of the repositories to be the authoritative repository, so that there is always one source of truth. However, the tool allows contributions to any repository, and any repository can be used to cut a release. The most common use case involves repetitive movement of code from one repository to another. Copybara can also be used for moving code once to a new repository. Importing sections of code from a confidential repository to a public repository. Importing code from a public repository to a confidential repository. Importing a change from a non-authoritative repository into the authoritative repository. When a change is made in the non-authoritative repository (for example, a contributor in the public repository), Copybara transforms and moves that change into the appropriate place in the authoritative repository. Any merge conflicts are dealt with in the same way as an out-of-date change within the authoritative repository. One of the main features of Copybara is that it is stateless, or more specifically, that it stores the state in the destination repository (As a label in the commit message). This allows several users (or a service) to use Copybara for the same config/repositories and get the same result. Currently, the only supported type of repository is Git. Copybara is also able to read from Mercurial repositories, but the feature is still experimental. The extensible architecture allows adding bespoke origins and destinations for almost any use case. Official support for other repositories types will be added in the future. core.workflow( name = "default", origin = git.github_origin( url = "https://github.com/google/copybara.git", ref = "master", ), destination = git.destination( url = "file:///tmp/foo", ), # Copy everything but don't remove a README_INTERNAL.txt file if it exists. destination_files = glob(["third_party/copybara/**"], exclude = ["README_INTERNAL.txt"]), authoring = authoring.pass_thru("Default email "), transformations = [ core.replace( before = "//third_party/bazel/bashunit", after = "//another/path:bashunit", paths = glob(["**/BUILD"])), core.move("", "third_party/copybara") ], ) Run: $ (mkdir /tmp/foo ; cd /tmp/foo ; git init --bare) $ copybara copy.bara.sky Getting Started using Copybara The easiest way to start is with weekly "snapshot" releases, that include pre-built a binary. Note that these are released automatically without any manual testing, version compatibility or correctness guarantees. Choose a release from https://github.com/google/copybara/releases. To use an unreleased version of copybara, so you need to compile from HEAD. In order to do that, you need to do the following: These packages can be installed using the appropriate package manager for your system. aur/copybara-git Using Intellij with Bazel plugin If you use Intellij and the Bazel plugin, use this project configuration: directories: copybara/integration java/com/google/copybara javatests/com/google/copybara third_party targets: //copybara/integration/... //java/com/google/copybara/... //javatests/com/google/copybara/... //third_party/... Note: configuration files can be stored in any place, even in a local folder. We recommend using a VCS (like git) to store them; treat them as source code. If using a weekly snapshot release, install Copybara as follows: http_jar( name = "com_github_google_copybara", # Fill in from https://github.com/google/copybara/releases/download/[version]/copybara_deploy.jar.sha256 # sha256 = "", urls = ["https://github.com/google/copybara/releases/download/[version]/copybara_deploy.jar"], ) In any BUILD file (perhaps /tools/BUILD.bazel) declare the java_binary: load("@rules_java//java:java_binary.bzl", "java_binary") java_binary( name = "copybara", main_class = "com.google.copybara.Main", runtime_deps = ["@com_github_google_copybara//jar"], ) Use that target with bazel run, for example bazel run //tools:copybara -- migrate copy.bara.sky Building Copybara from Source as an external Bazel repository There are convenience macros defined for all of Copybara's dependencies. Add the following code to your WORKSPACE file, replacing {{ sha256sum }} and {{ commit }} as necessary. http_archive( name = "com_github_google_copybara", sha256 = "{{ sha256sum }}", strip_prefix = "copybara-{{ commit }}", url = "https://github.com/google/copybara/archive/{{ commit }}.zip", ) load("@com_github_google_copybara//:repositories.bzl", "copybara_repositories") copybara_repositories() load("@com_github_google_copybara//:r

Google copybara: moving code between repositories

Key takeaways

More in computer-science