When doing continuous integration, you do not want your build to fail because a a tool invoked during the build has been updated or some environmental conditions have changed. Because Bazel is designed for reproducible builds and keeps track of almost every dependency of your project, Bazel is a great tool for use inside a CI system. Bazel also caches results of previous build, including test results and will not re-run unchanged tests, speeding up each build.

Running Bazel on virtual or physical machines.

For ci.bazel.build, we use Google Compute Engine virtual machine for our Linux build and a physical Mac mini for our Mac build. Apart from Bazel tests that are run using the ./compile.sh script, we also run some projects to validate Bazel binaries against: the Bazel Tutorial here, re2 here, protobuf here, and TensorFlow here.

Bazel is reinstalled each time we run the tutorial or TensorFlow, but the Bazel cache is maintained across installs. The setup for those jobs is the following:

set -e

# Fetch the Bazel installer
URL=https://github.com/bazelbuild/bazel/releases/download/${BAZEL_VERSION}/bazel-${BAZEL_VERSION}-installer-${INSTALLER_PLATFORM}.sh
export BAZEL_INSTALLER=${PWD}/bazel-installer/install.sh
curl -L -o ${BAZEL_INSTALLER} ${URL}
BASE="${PWD}/bazel-install"

# Install bazel inside ${BASE}
bash "${BAZEL_INSTALLER}" \
  --base="${BASE}" \
  --bazelrc="${BASE}/bin/bazel.bazelrc" \
  --bin="${BASE}/binary"

# Run the build
BAZEL="${BASE}/binary/bazel --bazelrc=${BASE}/bin/bazel.bazelrc"
${BAZEL} test //...

This tests installing a specific version of Bazel each time. Of course, if Bazel is installed on the path, one can simply bazel test //.... However, even with reinstalling all the time, Bazel caching simply works.

Running Bazel inside a Docker container

Several people want to use Bazel in a Docker container. First of all, Bazel has some feature that are incompatibles with Docker:

• Bazel runs by default in client/server mode using UNIX domain sockets, so if you cannot mount the socket inside the Docker container, then you must disable client-server communication by running Bazel in batch mode with the --batch flag.
• Bazel sandboxes all actions on linux by default and this needs special privileges in the Docker container (enabled by --privilege=true. If you cannot enable the namespace sandbox, you can deactivate it in Bazel with the --genrule_strategy=standalone --spawn_strategy=standalone flags.

So the last step of the previous script would look like:

# Run the build
BAZEL="${BASE}/binary/bazel --bazelrc=${BASE}/bin/bazel.bazelrc --batch"
${BAZEL} test --genrule_strategy=standalone --spawn_strategy=standalone \
    //...

This build will however be slower because the server has to restart for every build and the cache will be lost when the Docker container is destroyed.

To prevent the loss of the cache, it is better to mount a persistent volume for ~/.cache/bazel (where the Bazel cache is stored).

Return code and XML output

A final consideration when setting up a continuous integration system is getting the result from the build. Bazel has the following interesting exit codes when using test and build commands:

These return codes can be used to determine the reason for a failure (in ci.bazel.build, we mark builds that have exited with exit code 3 as unstable, and other non zero code as failed).

You can also control how much information about test results Bazel prints out with the --test_output flag. Generally, printing the output of test that fails with --test_output=errors is a good setting for a CI system.

Finally, Bazel's built-in JUnit test runner generates Ant-style XML output file (in bazel-testlogs/pkg/target/test.xml) that summarizes the results of your tests. This test runner can be activated with the --nolegacy_bazel_java_test flag (this will soon be the default). Other tests also get a basic XML output file that contains only the result of the test (success or failure).