Bazel Blog

Protocol Buffers in Bazel

Bazel currently provides built-in rules for Java, JavaLite and C++.

proto_library is a language-agnostic rule that describes relations between .proto files.

java_proto_library, java_lite_proto_library and cc_proto_library are rules that "attach" to proto_library and generate language-specific bindings.

By making a java_library (resp. cc_library) depend on java_proto_library (resp. cc_proto_library) your code gains access to the generated code.

TL;DR - Usage example

TIP: contains a buildable example.

NOTE: Bazel 0.4.4 lacks some features the example uses - you'll need to build Bazel from head. The easiest is to install Bazel, download Bazel's source code, build it (bazel build //src:bazel) and copy it somewhere (e.g., cp bazel-bin/src/bazel ~/bazel)


Bazel's proto rules implicitly depend on the distribution (described below, in "Implicit Dependencies and Proto Toolchains"). The following satisfies these dependencies:

TIP: This is a shortened version of

# proto_library rules implicitly depend on @com_google_protobuf//:protoc,
# which is the proto-compiler.
# This statement defines the @com_google_protobuf repo.
    name = "com_google_protobuf",
    urls = [""],

# cc_proto_library rules implicitly depend on @com_google_protobuf_cc//:cc_toolchain,
# which is the C++ proto runtime (base classes and common utilities).
    name = "com_google_protobuf_cc",
    urls = [""],

# java_proto_library rules implicitly depend on @com_google_protobuf_java//:java_toolchain,
# which is the Java proto runtime (base classes and common utilities).
    name = "com_google_protobuf_java",
    urls = [""],

BUILD files

TIP: This is a shortened version of

    name = "person_java_proto",
    deps = [":person_proto"],

    name = "person_cc_proto",
    deps = [":person_proto"],
    name = "person_proto",
    srcs = ["person.proto"],
    deps = [":address_proto"],

    name = "address_proto",
    srcs = ["address.proto"],
    deps = [":zip_code_proto"],

    name = "zip_code_proto",
    srcs = ["zip_code.proto"],

This file yields the following dependency graph:

proto_library dependency graph

Notice how the proto_library provide structure for both Java and C++ code generators, and how there's only one java_proto_library even though there multiple .proto files.


... in comparison with a macro that's responsible for compiling all .proto files in a project.

  1. Caching + incrementality: changing a single .proto only causes the rebuilding of dependant .proto files. This includes not only regenerating code, but also recompiling it. For large proto graphs this could be significant.
  2. Depend on pieces of a proto graph from multiple places: in the example above, one can add a cc_proto_library that deps on zip_code_proto, and including it together with //src:person_cc_proto in the same project. Though they both transitively depend on zip_code_proto, there won't be a linking error.

Recommended Code Organization

  1. One proto_library rule per .proto file.
  2. A file named foo.proto will be in a rule named foo_proto, which is located in the same package.
  3. A X_proto_library that wraps a proto_library named foo_proto should be called foo_X_proto, and be located in the same package.


Q: I already have rules named java_proto_library and cc_proto_library. Will there be a problem?
A: No. Since Skylark extensions imported through load statements take precedence over native rules with the same name, the new rule should not affect existing usage of the java_proto_library macro.

Q: How do I use gRPC with these rules?
A: The Bazel rules do not generate RPC code since protobuf is independent of any RPC system. We will work with the gRPC team to create Skylark extensions to do so. (C++ Issue, Java Issue)

Q: Do you plan to release additional languages?
A: We can relatively easily create py_proto_library. Our end goal is to improve Skylark to the point where these rules can be written in Skylark, making them independent of Bazel.

Q: How does one use well-known types? (e.g., any.proto, descriptor.proto)
A: Once is resolved, the following should be added to a .proto file: import google/protobuf/any.proto and the following: @com_google_protobuf//:well_known_types_protos to one's proto_library rule.

Q: Any tips for writing my own such rules?
A: First, make sure you're able to register actions that compile your target language. (as far as I know, Bazel Python actions are not exposed to Skylark, for example).
Second, take extra care to generate unique symbol names and unique filenames. There's an implicit assumption that different proto rules with different options, generate different symbols. For example, if you write a new rule foo_java_proto_library, it must not generate symbols that java_proto_library might. The risk is that a binary will contain both, leading to a one-definition rule violation (e.g., linking errors). The downside is that the binary might bloat, as it must contain multiple generated code for the same proto. We're working on a Skylark version of java_lite_proto_library which should provide a good example.

Implementation Details

Implicit Dependencies and Proto Toolchains

The proto_library rule implicitly depends on @com_google_protobuf//:protoc, which is the protocol buffer compiler. It must be a binary rule (in protobuf, it's a cc_binary). The rule can be overridden using the --proto_compiler command-line flag.

X_proto_library rules implicitly depend on @com_google_protobuf_X//:X_toolchain, which is a proto_lang_toolchain rule. These rules can be overridden using the --proto_toolchain_for_X command-line flags.

A proto_lang_toolchain rule describes how to call the protocol compiler, and what is the library (if any) that the resulting generated code needs to compile against. See an example in the protobuf repository.

Bazel Aspects

The X_proto_library rules are implemented using Bazel Aspects to have the best of two worlds -

  1. Only need a single X_proto_library rule for an arbitrarily-large proto graph.
  2. Incrementality, caching and no linking errors.

Conceptually, an X_proto_library rule creates a shadow graph of the proto_library it depends on, and each shadow node calls protocol-compiler and then compiles the generated code. This way, if there are multiple paths from a rule to a proto_library through X_proto_library, they all share the same node.

Descriptor Sets

When compiled on the command-line, a proto_library creates a descriptor set for the messages it srcs. The file is a serialized FileDescriptorSet, which is described in

One use case for the descriptor set is generating code without having to parse .proto files. ( tracks this ability in the protobuf compiler)

The aforementioned file only contains information about the .proto files directly mentioned by a proto_library rule; the collection of transitive descriptor sets is available through the 'proto.transitivedescriptorsets' Skylark provider. See documentation in ProtoSourcesProvider.

By Carmi Grushko

Invalidation of repository rules

Remote repositories are the way to use dependencies from "outside" of the Bazel world in Bazel. Using them, you can download binaries from the internet or use some from your own host. You can even use Skylark to define your own repository rules to depend on a custom package manager or to implement auto-configuration rules.

This post explains when Skylark repositories are invalidated and hence when they are executed.


The implementation attribute of the repository_rule defines a function (the fetch operation) that is executed inside a Skyframe function. This function is executed when one of its dependencies change.

For repository that are declared local (set local = True in the call to the repository_rule function), the fetch operation is performed on every call of the Skyframe function.

Since a lot of dependencies can trigger this execution (if any part of the WORKSPACE file change for instance), a supplemental mechanism ensure that we re-execute the fetch operation only when stricly needed for non-local repository rules (see the design doc for more details).

After is released, Bazel will re-perform the fetch operation if and only if any of the following dependencies change:

  • Skylark files needed to define the repository rule.
  • Declaration of the repository rule in the WORKSPACE file.
  • Value of any environment variable declared with the environ attribute of the repository_rule function. The value of those environment variable can be enforced from the command line with the --action_env flag (but this flag will invalidate every action of the build).
  • Content of any file used and referred using a label (e.g., //mypkg:label.txt not mypkg/label.txt).

Good practices regarding refetching

Declare your repository as local very carefully

First and foremost, declaring a repository local should be done only for rule that needs to be eagerly invalidated and are fast to update. For native rule, this is used only for local_repository and new_local_repository.

Put all slow operation at the end, resolve dependencies first

Since a dependency might be unresolved when asked for, the function will be executed up to where the dependency is requested and all that part will be replayed if the dependency is not resolved. Put those file dependencies at the top, for instance prefer

def _impl(repository_ctx):
   repository_ctx.file("BUILD", repository_ctx.attr.build_file)"BIGFILE", sha256 = "...")

myrepo = repository_rule(_impl, attrs = {"build_file": attr.label()})


def _impl(repository_ctx):"BIGFILE")
   repository_ctx.file("BUILD", repository_ctx.attr.build_file)

myrepo = repository_rule(_impl, attrs = {"build_file": attr.label()})

(in the later example, the download operation will be re-executed if build_file is not resolved when executing the fetch operation).

Declare your environment variables

To avoid spurious refetch of repository rules (and the impossibility of tracking all usages of environmnent variables), only environment variables that have been declared through the environ attribute of the repository_rule function are invalidating the repositories.

Therefore, if you think you should re-run if an environment variable changes (like for auto-configuration rules), you should declare those dependencies, or your user will have to do bazel clean --expunge each time they change their environment.

We are now!

As you might have seen either in our 0.4 announcement or simply by going to our website, we have recently switched over to the domain name.

We decided to switch over to the new .build top-level domain, which reflects what Bazel is for: building!

Our old domain,, will redirect to for the forseenable future.

Bazel 0.4.0 Released

We are delighted to announce the 0.4.0 release of Bazel. This release marks major improvements in support for Windows, sandboxing and performance.

We are also moving also moving to a new domain. is already up and running and we are slowly moving every reference to point to that new domain..

Improvements from our roadmap

Java workers are now used by default

Java workers reduce the time of java compilation by keeping a hot JVM in the background. It improves java compilation speed by 5x and we decided to make it the default.

Sandboxing is now working also on OS X

With our Beta release, we added sandboxing of action to Linux. This feature ensured that the rule set does not access undeclared input, allowing correct execution of actions. We leveraged the sandbox-exec command to generate a similar sandbox on OS X.

Other major changes since 0.3.0

We provide Bazel binaries for Windows

As announced in our Bazel on Windows blog post, we are now providing binary distribution of Bazel for Windows. A chocolatey package was contributed by Peter Mounce so you can just do choco install bazel to get Bazel installed on Windows, big thanks to Peter Mounce! This release also marks a big step for us: TensorFlow PIP package can now be built on Windows with Bazel!

Skylark implementation of repository rules

We now have implementation of two repository rules (gitrepository and mavenjar) in Skylark and we recommend using them instead of the native ones, to do so simply adds the following lines at the top of your WORKSPACE file:

load("@bazel_tools//tools/build_defs/repo:maven_rules.bzl", "maven_jar")

And various more

  • The --watchfs flag is ready to be turned on by default. It improves performance of Bazel, try it out!
  • The Linux sandbox got revamped for better performance and usability: no performance hit should be perceptible and accessing system tools should be possible.

For changes since 0.3.2 (the minor version before 0.4.0), see the release notes for changes.

Future plans

Looking ahead to 0.5.0:

  • With the help of your feedback, we will resolve the last issue to make our Windows port work seamlessly for Java, C++ and Python.
  • The new distributed execution API will be stabilized.


A big thank you to our community for your continued support. Particular shout-outs to the following contributors:

Thank you all, keep the discussion and bug reports coming!

IntelliJ and Android Studio support

We've recently open-sourced Bazel plugins for IntelliJ and Android Studio.

Key Features

  • Import a project directly from a BUILD file.
  • BUILD file integration: syntax highlighting, refactoring, find usages, code completion, etc. Skylark extensions are fully supported.
  • Compile your project and get navigatable Bazel compile errors in the IDE.
  • Buildifier integration
  • Support for Bazel run configurations for certain rule classes.
  • Run/debug tests directly through Bazel by right-clicking on methods/classes/BUILD targets.

How do I get started?

To try them out, you can install them directly from within the IDE (Settings > Plugins > Browse repositories), download them from the JetBrains plugin repository, or build directly from source.

Detailed docs are available here.

The plugins are currently Linux-only, with plans for Mac support in the future.

Bazel on Windows

We first announced experimental Windows support in 0.3.0. Since then, we've implemented support for building, running and testing C++, Java and Python, as well as improved performance and stability. Starting with Bazel version 0.3.2, we are making prebuilt Bazel Windows binaries available as part of our releases (installation instructions).

In addition to bootstrapping Bazel itself, we're also able to build significant parts of TensorFlow with Bazel on Windows (pull request). Bazel on Windows currently requires msys2 and still has a number of issues. Some of the more important ones are:

Our GitHub issue tracker has a full list of known issues.

Now, we need your help! Please try building your Bazel project on Windows, and let us know what works or what doesn't work yet, and what we can do better.

We are looking forward to what you build (on Windows)!

IDE support

One of Bazel’s longest-standing feature requests is integration with IDEs. With the 0.3 release, we finally have all machinery in place that allows implementing integration with Bazel in IDEs. Simultaneous with that Bazel release we are also making public two IDE plugins:

  • Tulsi: Bazel support for Xcode.
  • e4b: a sample Bazel plugin for Eclipse.

In this post, we will look into how Bazel enables IDE integration and how an IDE plugin integrating with Bazel can be implemented.

Principles of Bazel IDE support

Bazel BUILD files provide a description of a project’s source code: what source files are part of the project, what artifacts (targets) should be built from those files, what the dependencies between those files are, etc. Bazel uses this information to perform a build, that is, it figures out the set of actions needed to produce the artifacts (such as running a compiler or linker) and executes those actions. Bazel accomplishes this by constructing a dependency graph between targets and visiting this graph to collect those actions.

IDEs (as well as other tools working with source code) also need the same information about the set of sources and their roles; but instead of building the artifacts, IDEs use it to provide code navigation, autocompletion and other code-aware features.

In the 0.3.0 Bazel release, we are adding a new concept to Bazel - aspects. Aspects allow augmenting build dependency graphs with additional information and actions. Applying an aspect to a build target creates a "shadow dependency graph" reflecting all transitive dependencies of that target, and the aspect's implementation determines the actions that Bazel executes while traversing that graph. The documentation on aspects explains this in more detail.

Architecture of a Bazel IDE plugin.

As an example of how aspects are useful for IDE integration, we will take a look at a sample Eclipse plugin for Bazel support, e4b.

e4b includes an aspect, defined in a file e4b_aspect.bzl, that when applied to a particular target, generates a small JSON file with information about that target relevant to Eclipse. Those JSON files are then consumed by the e4b plugin inside Eclipse to build Eclipse's representation of a project, IClasspathContainer:

e4bazel workflow

Through the e4b plugin UI, the user specifies an initial set of targets (typically a java or android binary, a selection of tests, all targets in certain packages, etc). E4b plugin then invokes bazel as follows:

bazel build //java/com/company/example:main \
--aspects e4b_aspect.bzl%e4b_aspect \
--output_groups ide-info

(some details are omitted for clarity; see e4b source for complete invocation)

The --aspects flag directs Bazel to apply e4b_aspect, exported from e4bazel.bzl Skylark extension, to target //java/com/company/example:main.

The aspect is then applied transitively to the dependencies of the specified targets, producing .e4b-build.json files for each target in the transitive closure of dependencies. The e4b plugin reads those outputs and provides a Classpath for Eclipse core to consume. If the input BUILD files change so that a project model needs to be re-synced, the plugin still invokes the exact same command: Bazel will rebuild only those files that are affected by the change, so the plugin need only reexamine only those newly built .e4b-build.json files. ide-info is an output group defined by e4b_aspect; the --output_groups flag ensures that only the artifacts belonging to that group (and hence only to the aspect) are built, and therefore that no unnecessary build steps are performed.

The aspect uses the java provider on the targets it applies to to access a variety of information about Java targets.

Bazel 0.3.0 Released

We are delighted to announce the 0.3.0 release of Bazel. This milestone is marked by support for IDE integration but also major features such as remote caching of build artifacts and experimental Windows support.

Improvements from our roadmap

IDE support

In this release, we made it possible to generate information for IDEs from Bazel build files using Skylark aspects.

Simultaneous with Bazel 0.3 release, we are announcing the availability of two projects integrating Bazel with different IDEs:

  • Tulsi is an Xcode plugin for Bazel. This is the same plugin that teams inside Google use for developing iOS applications.
  • e4b is an experimental Eclipse plugin for Bazel. It was made to illustrate the use of Skylark aspects for IDE integration. This is an experimental plugin but we welcome any contributions to it.

Windows support

Bazel can now bootstrap on Windows without admin privilege and can use the Microsoft Visual C++ toolchain. Windows support is still highly experimental and we have identified several issues and their solutions. We are dedicated to a good native experience on Windows.

Remote caching of distributed artifacts

Alpha Lam has contributed experimental support for distributed caching and execution. This is an ongoing area of development and several engineers from Google are working with Alpha to enhance that support.

Skylark remote repositories

Remote repository rules can now be created using Skylark. This can be used to support your custom protocols, interfacing with new packaging systems or even do auto-configuration to use a toolchain on your local disk. We use it especially to have a better out-of-the-box experience with C++ toolchains.

Other major changes since 0.2.0

  • We just open-sourced our BUILD file formatter, buildifier.
  • We now provide a Debian APT repository for installing bazel, see the installation guide on how to use it.
  • Our JUnit test runner for Java tests (--nolegacy_bazel_java_test) is now the default.

For changes since 0.2.3 (the minor version before 0.3.0), see the release notes for changes.

Future plans

Looking ahead to 0.4.0:

  • The last blockers for --strategy=worker=Javac will be resolved, making Java builds faster.
  • Yue has made great progress in making Ulf's prototype of sandboxing for OS X real.


A big thank you to our community for your continued support. Particular shout-outs to the following contributors:

  • Justine Tunney - for developing and maintaining the Closure JS rules of Bazel.
  • Alpha Lam - for implementing remote caching/execution and following up on these features.
  • David Chen - for going above and beyond, far more than a standard 20% contribution: improving our documentation, creating the Skylark documentation generator, fixing bugs and contributing features in Bazel and helping out TensorFlow with their use of Bazel.

Thank you all, keep the discussion and bug reports coming!

Using Skylark remote repositories to auto-detect the C++ toolchain.

Skylark remote repositories let you create custom external repositories using Skylark. This not only enables creating rules for custom package systems such as PyPi but also generating a repository to reflect the toolchain installed on the workstation Bazel is running on. We explain here how we implemented auto-configuration for the C++ toolchain.


C++ toolchain: the set of binaries and libraries required to build C++ code. Crosstool: a compiler capable of building for a certain architecture, which can be different from the host architecture (e.g., gcc running on Linux and building for Raspberry Pi).

C++ toolchains are configured in Bazel using a crosstool target and a CROSSTOOL file.

This crosstool target (:default_toolchain) is the first step in moving the contents of the CROSSTOOL file entirely into BUILD file rules. The CROSSTOOL file defines where to find the C++ compiler, its include directories and also the various flag to use at each compilation step.

When your C++ compiler is not in the standard location, then this static CROSSTOOL file cannot find it. To cope with the variety of installation out there, we created a cc_configure Skylark repository rule that will generates a @local_config_cc//tools/cpp package containing a generated CROSSTOOL file based on the information we gathered from the operating system.


The cc_configure rule is actually a macro wrapping the cc_autoconf enforcing the local_config_cc name for the repository. The implementation of the cc_autoconf rule does the following step:

So using the function provided by repository_ctx, we can discover the binaries on the system, what version they are, and which options they support, then generate a configuration to match the local C++ toolchain.

Creating your own repository rules

When creating a Skylark remote repository, a few things should be taken in considerations:

  • The Skylark implementation of a remote repository is run during the loading phase of the repository, which means that unless the rule definition is changed in the WORKSPACE file or the implementation fails, it will not be re-run unless the user does a bazel clean --expunge. We are thinking of further command to force re-run that loading phase for a specific remote repository (#974).
  • Skylark remote repository can do a lot of non hermetic operation, it is recommended to check as many things as possible to ensure hermeticity (and overall, we recommend using a vendored toolchain instead of using auto-detected one if reproducibility is important). For example, it is recommended to use the sha256 argument of the method.
  • Naming a rule can be complex and we recommend to not use standard suffix of classical rules for remote repositories (e.g. *_library or *_binary). If you create a package rule, a good name would probably be xxx_package (e.g., pypi_package). If you create an autoconfiguration rule, xxx_configure is probably the best name (e.g. cc_configure).

Easier Debugging of Sandbox Failures

We have often heard that debugging failed execution due to issues with the sandbox is difficult and requires knowledge of the sandboxing code of Bazel to actually understand what’s happening. With these changes, we hope that you will be able to solve common issues easier on your own and make debugging easier.

If you don’t know much about Bazel sandbox, you might want to read this blog post

What we did:

  • When using --verbose_failures and --sandbox_debug, Bazel now shows the detailed command that it ran when your build failed, including the part that sets up the sandbox.
  • When you copy & paste the shown command into a terminal, the failed command is rerun - but when it fails this time, we provide access to a shell inside a new sandbox which is the same as the old sandbox we made before, so that you can explore the sandbox from the inside and find out why the command failed.

How to use it:

Let’s say you wrote a Skylark rule and forgot to add your compiler to the input files. Before this change, when you ran bazel build, you would get several error messages like this:

ERROR: path/to/your/project/BUILD:1:1: compilation of rule '//path/to/your/project:all' failed: No such file or directory.
ERROR: /your/project/BUILD:x:1: Executing genrule //project/dir:genrule failed: bash failed: error executing command /path/to/your/compiler some command

But you probably have no idea what to do, because the error message is not detailed enough and you have everything you needed in your system.

With this new feature, you will get an error message like this instead:

ERROR: path/to/your/project/BUILD:1:1: compilation of rule '//path/to/your/project:all' failed:

Sandboxed execution failed, which may be legitimate (e.g. a compiler error), or due to missing dependencies. To enter the sandbox environment for easier debugging, run the following command in parentheses. On command failure, a bash shell running inside the sandbox will then automatically be spawned

namespace-sandbox failed: error executing command
  (cd /some/path && \
  exec env - \
    LANG=en_US \
    PATH=/some/path/bin:/bin:/usr/bin \
    PYTHONPATH=/usr/local/some/path \
  /some/path/namespace-sandbox @/sandbox/root/path/this-sandbox-name.params -- /some/path/to/your/some-compiler --some-params some-target)

Then you can simply copy & paste the command above in parentheses into a new terminal:

(cd /some/path && \
  exec env - \
    LANG=en_US \
    PATH=/some/path/bin:/bin:/usr/bin \
    PYTHONPATH=/usr/local/some/path \
  /some/path/namespace-sandbox @/sandbox/root/path/this-sandbox-name.params -- /some/path/to/your/some-compiler --some-params some-target)

There will be the same error message about not finding your compiler, but after that error message, you will find yourself in a bash shell inside a new sandbox. You can now debug the failure, e.g. you can explore the sandbox: look for any missing file, check for possible errors in your BUILD files, run your compiler again manually, or even use strace.

For this example, we run our compiler in the sandbox again manually and the error message shows No command ‘some-compiler’ found - looking around, you notice that the compiler binary is missing. This means it was not part of the action inputs, because Bazel always mounts all action inputs into the sandbox - so you check out your Skylark rule and notice that this is indeed the case. Adding your compiler to the input files in your Skylark rule should thus fix the error.

Next time you run bazel build, it should mount your compiler into the sandbox and thus find it correctly. If you get a different error, you could repeat the steps above.