Bazel Blog

A glimpse of the design of Skylark

This blog post describes the design of Skylark, the language used to specify builds in Bazel.

A brief history

Many years ago, code at Google was built using Makefiles. As other people noticed, Makefiles don't scale well with a large code base. A temporary solution was to generate Makefiles using Python scripts, where the description of the build was stored in BUILD files containing calls to the Python functions. But this solution was way too slow, and the bottleneck was Make.

The project Blaze (later open-sourced as Bazel) was started in 2006. It used a simple parser to read the BUILD files (supporting only function calls, list comprehensions and variable assignments). When Blaze could not directly parse a BUILD file, it used a preprocessing step that ran the Python interpreter on the user BUILD file to generate a simplified BUILD file. The output was used by Blaze.

This approach was simple and allowed developers to create their own macros. But again, this led to lots of problems in terms of maintenance, performance, and safety. It also made any kind of tooling more complicated, as Blaze was not able to parse the BUILD files itself.

In the current iteration of Bazel, we've made the system saner by removing the Python preprocessing step. We kept the Python syntax, though, in order to migrate our codebase. This seems to be a good idea anyway: Many people like the syntax of our BUILD files and other build tools (e.g. Buck, Pants, and Please) have adopted it.

Design requirements

We decided to separate description of the build from the extensions (macros and rules). The description of the build resides in BUILD files and the extensions reside in .bzl files, although they are all evaluated with the same interpreter. We want the code to be easy to read and maintain. We designed Bazel to be used by thousands of engineers. Most of them are not familiar with build systems internals and most of them don't want to spend time learning a new language. BUILD files need to be simple and declarative, so that we can build tools to manipulate them.

The language also needed to:

  • Run on the JVM. Bazel is written in Java. The data structures should be shared between Bazel and the language (due to memory requirements in large builds).

  • Use a Python syntax, to preserve our codebase.

  • Be deterministic and hermetic. We have to guarantee that the execution of the code will always yield the same results. For example, we forbid access to I/O and date and time, and ensure deterministic iteration order of dictionaries.

  • Be thread-safe. We need to evaluate a lot of BUILD files in parallel. Execution of the code needs to be thread-safe in order to guarantee determinism.

Finally, we have performance concerns. A typical BUILD file is simple and can be executed quickly. In most cases, evaluating the code directly is faster than compiling it first.

Parallelism and imports

One special feature of Skylark is how it handles parallelism. In Bazel, a large build require the evaluation of hundreds of BUILD files, so we have to load them in parallel. Each BUILD file may use any number of extensions, and those extensions might need other files as well. This means that we end up with a graph of dependencies.

Bazel first evaluates the leaves of this graph (i.e. the files that have no dependencies) in parallel. It will load the other files as soon as their dependencies have been loaded, which means the evaluation of BUILD and .bzl files is interleaved. This also means that the order of the load statements doesn't matter at all.

Each file is loaded at most once. Once it has been evaluated, its definitions (the global variables and functions) are cached. Any other file can access the symbols through the cache.

Since multiple threads can access a variable at the same time, we need a restriction on side-effects to guarantee thread-safety. The solution is simple: when we cache the definitions of a file, we "freeze" them. We make them read-only, i.e. you can iterate on an array, but not modify its elements. You may create a copy and modify it, though.

In a future blog post, we'll take a look at the other features of the language.

By Laurent Le Brun

Skylark and Java rules interoperability

As of Bazel 0.4.4, Java compilation is possible from a Skylark rule. This facilitates the Skylark and Java interoperability and allows creating what we call Java sandwiches in Bazel.

What is a Bazel Java sandwich?

A Java sandwich refers to custom rules written in Skylark being able to depend on Bazel native rules (e.g. java_library) and the other way around. A typical Java sandwich in Bazel could be illustrated like this:

java_library(name = "top", ...)
java_skylark_library(name = "middle", deps = [":top", ...], ...)
java_library(name = "bottom", deps = [":middle", ...], ...)

Built-in support for Java

In Skylark, an interface to built-in Java functionality is available via the java_common module. The full API can be found in the documentation.


Compiles Java source files/jars from the implementation of a Skylark rule and returns a java_common.provider that encapsulates the compilation details.


Merges the given providers into a single java_common.provider.


To allow other Java rules (native or custom) to depend on a Skylark rule, the Skylark rule should return a java_common.provider. All native Java rules return java_common.provider by default, which makes it possible for any Java related Skylark rule to depend on them.

For now, there are 3 ways of creating a java_common.provider:

  1. The result of java_common.compile.
  2. Fetching it from a Java dependency.
  3. Merging multiple java_common.provider instances using java_common.merge.

Using the Java sandwich with compilation example

This example illustrates the typical Java sandwich described above, that will make use of Java compilation:

java_library(name = "top", ...)
java_skylark_library(name = "middle", deps = [":top", ...], ...)
java_library(name = "bottom", deps = [":middle", ...], ...)

In the BUILD file we load the Skylark rule and have the rules:

load(':java_skylark_library.bzl', 'java_skylark_library')

  name = "top",
  srcs = [""],
  deps = [":middle"]

  name = "middle",
  srcs = [""],
  deps = [":bottom"]

  name = "bottom",
  srcs = [""]

The implementation of java_skylark_library rule does the following:

  1. Collects all the java_common.providers from its dependencies and merges them using java_common.merge.
  2. Creates an artifact that will be the output jar of the Java compilation.
  3. Compiles the specified Java source files using java_common.compile, passing as dependencies the collected java_common.providers.
  4. Returns the output jar and the java_common.provider resulting from the compilation.
def _impl(ctx):
  deps = []
  for dep in ctx.attr.deps:
    if java_common.provider in dep:

  output_jar = ctx.new_file("lib" + + ".jar")

  compilation_provider = java_common.compile(
    source_files = ctx.files.srcs,
    output = output_jar,
    javac_opts = [],
    deps = deps,
    strict_deps = "ERROR",
    java_toolchain = ctx.attr._java_toolchain,
    host_javabase = ctx.attr._host_javabase
  return struct(
    files = set([output_jar]),
    providers = [compilation_provider]

java_skylark_library = rule(
  implementation = _impl,
  attrs = {
    "srcs": attr.label_list(allow_files=True),
    "deps": attr.label_list(),
    "_java_toolchain": attr.label(default = Label("@bazel_tools//tools/jdk:toolchain")),
    "_host_javabase": attr.label(default = Label("//tools/defaults:jdk"))
  fragments = ["java"]

Just passing around information about Java rules example

In some use cases there is no need for Java compilation, but rather just passing information about Java rules around. A Skylark rule can have some other (irrelevant here) purpose, but if it is placed somewhere between two Java rules it should not lose information from bottom to top.

In this example we have the same Bazel sandwich as above:

java_library(name = "top", ...)
java_skylark_library(name = "middle", deps = [":top", ...], ...)
java_library(name = "bottom", deps = [":middle", ...], ...)

only that java_skylark_library won't make use of Java compilation, but will make sure that all the Java information encapsulated by the Java library bottom will be passed on to the Java library top.

The BUILD file is identical to the one from the previous example.

The implementation of java_skylark_library rule does the following:

  1. Collects all the java_common.providers from its dependencies
  2. Returns the java_common.provider that resulted from merging the collected dependencies.
def _impl(ctx):
  deps = []
  for dep in ctx.attr.deps:
    if java_common.provider in dep:
  deps_provider = java_common.merge(deps)
  return struct(
    providers = [deps_provider]

java_skylark_library = rule(
  implementation = _impl,
  attrs = {
    "srcs": attr.label_list(allow_files=True),
    "deps": attr.label_list(),
    "_java_toolchain": attr.label(default = Label("@bazel_tools//tools/jdk:toolchain")),
    "_host_javabase": attr.label(default = Label("//tools/defaults:jdk"))
  fragments = ["java"]

More to come

Right now there is no way of creating a java_common.provider that encapsulates compiled code (and its transitive dependencies), other than java_common.compile. For example one may want to create a provider from a .jar file produced by some other means.

Soon there will be support for use cases like this. Stay tuned!

If you are interested in tracking the progress on Bazel Java sandwich you can subscribe to this Github issue.

By Irina Iancu, on behalf of the Bazel Java team

A Google Summer of Code with Bazel

I'm happy to announce that Bazel has been accepted as a mentor organization for the Google Summer of Code 2017. If you are a student and interested in working on Bazel this summer, please read on.

Take a look at our ideas page: it is not exhaustive and we may extend it over time, but it should give you a rough idea of what you could work on. Feel free to come up with your new ideas or suggest variations on our proposals. Not all projects on the page will be taken: we expect to accept up to three students. Students will not work with a single mentor, you can expect to interact with multiple people from the Bazel team (although there will be a main contact point). This will ensure you'll get timely responses and assistance, even if one of us goes on vacation.

This is the first time we participate in Google Summer of Code, please bear with us if you miss some information. We will update our ideas page to answer the most frequent questions.

If you have any question, please contact us on

I'm looking forward to hearing from you,

By Laurent Le Brun, on behalf of the Bazel mentors.

Protocol Buffers in Bazel

Bazel currently provides built-in rules for Java, JavaLite and C++.

proto_library is a language-agnostic rule that describes relations between .proto files.

java_proto_library, java_lite_proto_library and cc_proto_library are rules that "attach" to proto_library and generate language-specific bindings.

By making a java_library (resp. cc_library) depend on java_proto_library (resp. cc_proto_library) your code gains access to the generated code.

TL;DR - Usage example

TIP: contains a buildable example.

NOTE: Bazel 0.4.4 lacks some features the example uses - you'll need to build Bazel from head. The easiest is to install Bazel, download Bazel's source code, build it (bazel build //src:bazel) and copy it somewhere (e.g., cp bazel-bin/src/bazel ~/bazel)


Bazel's proto rules implicitly depend on the distribution (described below, in "Implicit Dependencies and Proto Toolchains"). The following satisfies these dependencies:

TIP: This is a shortened version of

# proto_library rules implicitly depend on @com_google_protobuf//:protoc,
# which is the proto-compiler.
# This statement defines the @com_google_protobuf repo.
    name = "com_google_protobuf",
    urls = [""],

# cc_proto_library rules implicitly depend on @com_google_protobuf_cc//:cc_toolchain,
# which is the C++ proto runtime (base classes and common utilities).
    name = "com_google_protobuf_cc",
    urls = [""],

# java_proto_library rules implicitly depend on @com_google_protobuf_java//:java_toolchain,
# which is the Java proto runtime (base classes and common utilities).
    name = "com_google_protobuf_java",
    urls = [""],

BUILD files

TIP: This is a shortened version of

    name = "person_java_proto",
    deps = [":person_proto"],

    name = "person_cc_proto",
    deps = [":person_proto"],
    name = "person_proto",
    srcs = ["person.proto"],
    deps = [":address_proto"],

    name = "address_proto",
    srcs = ["address.proto"],
    deps = [":zip_code_proto"],

    name = "zip_code_proto",
    srcs = ["zip_code.proto"],

This file yields the following dependency graph:

proto_library dependency graph

Notice how the proto_library provide structure for both Java and C++ code generators, and how there's only one java_proto_library even though there multiple .proto files.


... in comparison with a macro that's responsible for compiling all .proto files in a project.

  1. Caching + incrementality: changing a single .proto only causes the rebuilding of dependant .proto files. This includes not only regenerating code, but also recompiling it. For large proto graphs this could be significant.
  2. Depend on pieces of a proto graph from multiple places: in the example above, one can add a cc_proto_library that deps on zip_code_proto, and including it together with //src:person_cc_proto in the same project. Though they both transitively depend on zip_code_proto, there won't be a linking error.

Recommended Code Organization

  1. One proto_library rule per .proto file.
  2. A file named foo.proto will be in a rule named foo_proto, which is located in the same package.
  3. A X_proto_library that wraps a proto_library named foo_proto should be called foo_X_proto, and be located in the same package.


Q: I already have rules named java_proto_library and cc_proto_library. Will there be a problem?
A: No. Since Skylark extensions imported through load statements take precedence over native rules with the same name, the new rule should not affect existing usage of the java_proto_library macro.

Q: How do I use gRPC with these rules?
A: The Bazel rules do not generate RPC code since protobuf is independent of any RPC system. We will work with the gRPC team to create Skylark extensions to do so. (C++ Issue, Java Issue)

Q: Do you plan to release additional languages?
A: We can relatively easily create py_proto_library. Our end goal is to improve Skylark to the point where these rules can be written in Skylark, making them independent of Bazel.

Q: How does one use well-known types? (e.g., any.proto, descriptor.proto)
A: Once is resolved, the following should be added to a .proto file: import google/protobuf/any.proto and the following: @com_google_protobuf//:well_known_types_protos to one's proto_library rule.

Q: Any tips for writing my own such rules?
A: First, make sure you're able to register actions that compile your target language. (as far as I know, Bazel Python actions are not exposed to Skylark, for example).
Second, take extra care to generate unique symbol names and unique filenames. There's an implicit assumption that different proto rules with different options, generate different symbols. For example, if you write a new rule foo_java_proto_library, it must not generate symbols that java_proto_library might. The risk is that a binary will contain both, leading to a one-definition rule violation (e.g., linking errors). The downside is that the binary might bloat, as it must contain multiple generated code for the same proto. We're working on a Skylark version of java_lite_proto_library which should provide a good example.

Implementation Details

Implicit Dependencies and Proto Toolchains

The proto_library rule implicitly depends on @com_google_protobuf//:protoc, which is the protocol buffer compiler. It must be a binary rule (in protobuf, it's a cc_binary). The rule can be overridden using the --proto_compiler command-line flag.

X_proto_library rules implicitly depend on @com_google_protobuf_X//:X_toolchain, which is a proto_lang_toolchain rule. These rules can be overridden using the --proto_toolchain_for_X command-line flags.

A proto_lang_toolchain rule describes how to call the protocol compiler, and what is the library (if any) that the resulting generated code needs to compile against. See an example in the protobuf repository.

Bazel Aspects

The X_proto_library rules are implemented using Bazel Aspects to have the best of two worlds -

  1. Only need a single X_proto_library rule for an arbitrarily-large proto graph.
  2. Incrementality, caching and no linking errors.

Conceptually, an X_proto_library rule creates a shadow graph of the proto_library it depends on, and each shadow node calls protocol-compiler and then compiles the generated code. This way, if there are multiple paths from a rule to a proto_library through X_proto_library, they all share the same node.

Descriptor Sets

When compiled on the command-line, a proto_library creates a descriptor set for the messages it srcs. The file is a serialized FileDescriptorSet, which is described in

One use case for the descriptor set is generating code without having to parse .proto files. ( tracks this ability in the protobuf compiler)

The aforementioned file only contains information about the .proto files directly mentioned by a proto_library rule; the collection of transitive descriptor sets is available through the proto.transitive_descriptor_sets Skylark provider. See documentation in ProtoSourcesProvider.

By Carmi Grushko

Invalidation of repository rules

Remote repositories are the way to use dependencies from "outside" of the Bazel world in Bazel. Using them, you can download binaries from the internet or use some from your own host. You can even use Skylark to define your own repository rules to depend on a custom package manager or to implement auto-configuration rules.

This post explains when Skylark repositories are invalidated and hence when they are executed.


The implementation attribute of the repository_rule defines a function (the fetch operation) that is executed inside a Skyframe function. This function is executed when one of its dependencies change.

For repository that are declared local (set local = True in the call to the repository_rule function), the fetch operation is performed on every call of the Skyframe function.

Since a lot of dependencies can trigger this execution (if any part of the WORKSPACE file change for instance), a supplemental mechanism ensure that we re-execute the fetch operation only when stricly needed for non-local repository rules (see the design doc for more details).

After is released, Bazel will re-perform the fetch operation if and only if any of the following dependencies change:

  • Skylark files needed to define the repository rule.
  • Declaration of the repository rule in the WORKSPACE file.
  • Value of any environment variable declared with the environ attribute of the repository_rule function. The value of those environment variable can be enforced from the command line with the --action_env flag (but this flag will invalidate every action of the build).
  • Content of any file used and referred using a label (e.g., //mypkg:label.txt not mypkg/label.txt).

Good practices regarding refetching

Declare your repository as local very carefully

First and foremost, declaring a repository local should be done only for rule that needs to be eagerly invalidated and are fast to update. For native rule, this is used only for local_repository and new_local_repository.

Put all slow operation at the end, resolve dependencies first

Since a dependency might be unresolved when asked for, the function will be executed up to where the dependency is requested and all that part will be replayed if the dependency is not resolved. Put those file dependencies at the top, for instance prefer

def _impl(repository_ctx):
   repository_ctx.file("BUILD", repository_ctx.attr.build_file)"BIGFILE", sha256 = "...")

myrepo = repository_rule(_impl, attrs = {"build_file": attr.label()})


def _impl(repository_ctx):"BIGFILE")
   repository_ctx.file("BUILD", repository_ctx.attr.build_file)

myrepo = repository_rule(_impl, attrs = {"build_file": attr.label()})

(in the later example, the download operation will be re-executed if build_file is not resolved when executing the fetch operation).

Declare your environment variables

To avoid spurious refetch of repository rules (and the impossibility of tracking all usages of environmnent variables), only environment variables that have been declared through the environ attribute of the repository_rule function are invalidating the repositories.

Therefore, if you think you should re-run if an environment variable changes (like for auto-configuration rules), you should declare those dependencies, or your user will have to do bazel clean --expunge each time they change their environment.

By Damien Martin-Guillerez

We are now!

As you might have seen either in our 0.4 announcement or simply by going to our website, we have recently switched over to the domain name.

We decided to switch over to the new .build top-level domain, which reflects what Bazel is for: building!

Our old domain,, will redirect to for the forseenable future.

By Damien Martin-Guillerez

Bazel 0.4.0 Released

We are delighted to announce the 0.4.0 release of Bazel. This release marks major improvements in support for Windows, sandboxing and performance.

We are also moving also moving to a new domain. is already up and running and we are slowly moving every reference to point to that new domain..

Improvements from our roadmap

Java workers are now used by default

Java workers reduce the time of java compilation by keeping a hot JVM in the background. It improves java compilation speed by 5x and we decided to make it the default.

Sandboxing is now working also on OS X

With our Beta release, we added sandboxing of action to Linux. This feature ensured that the rule set does not access undeclared input, allowing correct execution of actions. We leveraged the sandbox-exec command to generate a similar sandbox on OS X.

Other major changes since 0.3.0

We provide Bazel binaries for Windows

As announced in our Bazel on Windows blog post, we are now providing binary distribution of Bazel for Windows. A chocolatey package was contributed by Peter Mounce so you can just do choco install bazel to get Bazel installed on Windows, big thanks to Peter Mounce! This release also marks a big step for us: TensorFlow PIP package can now be built on Windows with Bazel!

Skylark implementation of repository rules

We now have implementation of two repository rules (gitrepository and mavenjar) in Skylark and we recommend using them instead of the native ones, to do so simply adds the following lines at the top of your WORKSPACE file:

load("@bazel_tools//tools/build_defs/repo:maven_rules.bzl", "maven_jar")

And various more

  • The --watchfs flag is ready to be turned on by default. It improves performance of Bazel, try it out!
  • The Linux sandbox got revamped for better performance and usability: no performance hit should be perceptible and accessing system tools should be possible.

For changes since 0.3.2 (the minor version before 0.4.0), see the release notes for changes.

Future plans

Looking ahead to 0.5.0:

  • With the help of your feedback, we will resolve the last issue to make our Windows port work seamlessly for Java, C++ and Python.
  • The new distributed execution API will be stabilized.


A big thank you to our community for your continued support. Particular shout-outs to the following contributors:

Thank you all, keep the discussion and bug reports coming!

By Helen Altshuler

IntelliJ and Android Studio support

We've recently open-sourced Bazel plugins for IntelliJ and Android Studio.

Key Features

  • Import a project directly from a BUILD file.
  • BUILD file integration: syntax highlighting, refactoring, find usages, code completion, etc. Skylark extensions are fully supported.
  • Compile your project and get navigatable Bazel compile errors in the IDE.
  • Buildifier integration
  • Support for Bazel run configurations for certain rule classes.
  • Run/debug tests directly through Bazel by right-clicking on methods/classes/BUILD targets.

How do I get started?

To try them out, you can install them directly from within the IDE (Settings > Plugins > Browse repositories), download them from the JetBrains plugin repository, or build directly from source.

Detailed docs are available here.

The plugins are currently Linux-only, with plans for Mac support in the future.

By Brendan Douglas

Bazel on Windows

We first announced experimental Windows support in 0.3.0. Since then, we've implemented support for building, running and testing C++, Java and Python, as well as improved performance and stability. Starting with Bazel version 0.3.2, we are making prebuilt Bazel Windows binaries available as part of our releases (installation instructions).

In addition to bootstrapping Bazel itself, we're also able to build significant parts of TensorFlow with Bazel on Windows (pull request). Bazel on Windows currently requires msys2 and still has a number of issues. Some of the more important ones are:

Our GitHub issue tracker has a full list of known issues.

Now, we need your help! Please try building your Bazel project on Windows, and let us know what works or what doesn't work yet, and what we can do better.

We are looking forward to what you build (on Windows)!

By Dmitry Lomov

IDE support

One of Bazel’s longest-standing feature requests is integration with IDEs. With the 0.3 release, we finally have all machinery in place that allows implementing integration with Bazel in IDEs. Simultaneous with that Bazel release we are also making public two IDE plugins:

  • Tulsi: Bazel support for Xcode.
  • e4b: a sample Bazel plugin for Eclipse.

In this post, we will look into how Bazel enables IDE integration and how an IDE plugin integrating with Bazel can be implemented.

Principles of Bazel IDE support

Bazel BUILD files provide a description of a project’s source code: what source files are part of the project, what artifacts (targets) should be built from those files, what the dependencies between those files are, etc. Bazel uses this information to perform a build, that is, it figures out the set of actions needed to produce the artifacts (such as running a compiler or linker) and executes those actions. Bazel accomplishes this by constructing a dependency graph between targets and visiting this graph to collect those actions.

IDEs (as well as other tools working with source code) also need the same information about the set of sources and their roles; but instead of building the artifacts, IDEs use it to provide code navigation, autocompletion and other code-aware features.

In the 0.3.0 Bazel release, we are adding a new concept to Bazel - aspects. Aspects allow augmenting build dependency graphs with additional information and actions. Applying an aspect to a build target creates a "shadow dependency graph" reflecting all transitive dependencies of that target, and the aspect's implementation determines the actions that Bazel executes while traversing that graph. The documentation on aspects explains this in more detail.

Architecture of a Bazel IDE plugin.

As an example of how aspects are useful for IDE integration, we will take a look at a sample Eclipse plugin for Bazel support, e4b.

e4b includes an aspect, defined in a file e4b_aspect.bzl, that when applied to a particular target, generates a small JSON file with information about that target relevant to Eclipse. Those JSON files are then consumed by the e4b plugin inside Eclipse to build Eclipse's representation of a project, IClasspathContainer:

e4bazel workflow

Through the e4b plugin UI, the user specifies an initial set of targets (typically a java or android binary, a selection of tests, all targets in certain packages, etc). E4b plugin then invokes bazel as follows:

bazel build //java/com/company/example:main \
--aspects e4b_aspect.bzl%e4b_aspect \
--output_groups ide-info

(some details are omitted for clarity; see e4b source for complete invocation)

The --aspects flag directs Bazel to apply e4b_aspect, exported from e4bazel.bzl Skylark extension, to target //java/com/company/example:main.

The aspect is then applied transitively to the dependencies of the specified targets, producing .e4b-build.json files for each target in the transitive closure of dependencies. The e4b plugin reads those outputs and provides a Classpath for Eclipse core to consume. If the input BUILD files change so that a project model needs to be re-synced, the plugin still invokes the exact same command: Bazel will rebuild only those files that are affected by the change, so the plugin need only reexamine only those newly built .e4b-build.json files. ide-info is an output group defined by e4b_aspect; the --output_groups flag ensures that only the artifacts belonging to that group (and hence only to the aspect) are built, and therefore that no unnecessary build steps are performed.

The aspect uses the java provider on the targets it applies to to access a variety of information about Java targets.

By Dmitry Lomov