The GraalVM compiler is a replacement to HotSpot’s server-side JIT compiler, widely known as the C2 compiler. It is written in Java with the goal of better performance compared to the C2 compiler. New changes, starting with Java 9, mean that we can now plug in our own hand-written C2 compiler into the JVM thanks to JVMCI. The researchers and engineers at Oracle Labs have created a variant of JDK8 with JVMCI enabled, which can be used to build the GraalVM compiler. The GraalVM compiler is open source and is available on GitHub (along with the HotSpot JVMCI sources needed to build the GraalVM compiler). This gives us the ability to fork/clone it and build our own version of the GraalVM compiler.
In this post, we are going to build the GraalVM compiler with JDK8 on CircleCI. The resulting artifacts are going to be:
Note: We are not covering how to build the GraalVM suite in this post; that can be done in another post. Although these scripts can be used to do that, and there exists a branch that contains the rest of the steps.
Continuous integration (CI) and continuous deployment (CD) tools have many benefits. One of the greatest is the ability to check the health of a code-base. Seeing why your builds are failing provides you with an opportunity to make a fix faster. For this project, it is important that we are able to verify and validate the scripts required to build the GraalVM compiler for Linux and macOS both locally and in a Docker container. A CI/CD tool lets us add automated tests to ensure that we get the desired outcome from our scripts when every PR is merged. In addition to ensuring that our new code does not introduce a breaking change, another great feature of CI/CD tools is that we can automate the creation of binaries. The automatic deployment of those binaries makes them available for open source distribution.
During the process of researching CircleCI as a CI/CD solution to build the GraalVM compiler, I learned that we could run builds via two different approaches, namely:
We will now go through the two approaches mentioned above and see the pros and cons of both of them.
For this approach, CircleCI requires a Docker image that is available in Docker Hub or another public/private registry it has access to. We will have to install the necessary dependencies in this available environment in order for a successful build. We expect the build to run longer the first time, and, depending on the levels of caching, it will speed up.
To understand how this is done, we will be going through the CircleCI configuration file section-by-section (stored in .circleci/circle.yml). See config.yml in .circleci for the full listing; see commit df28ee7 for the source changes.
The below lines in the configuration file will ensure that our installed applications are cached (referring to the two specific directories), so that we don’t have to reinstall the dependencies each time a build occurs:
dependencies:
cache_directories:
- "vendor/apt"
- "vendor/apt/archives"We will be referring to the Docker image by its full name (as available on http://hub.docker.com under the account name used — adoptopenjdk). In this case, it is a standard docker image containing JDK8 made available by the good folks behind the Adopt OpenJDK build farm. In theory, we can use any image as long as it supports the build process. It will act as the base layer on which we will install the necessary dependencies:
docker:
- image: adoptopenjdk/openjdk8:jdk8u152-b16 Next, in the pre-Install Os dependencies step, we will restore the cache; if it already exists, this may look a bit odd, but for unique key labels, the below implementation is recommended by the docs:
- restore_cache:
keys:
- os-deps-{{ arch }}-{{ .Branch }}-{{ .Environment.CIRCLE_SHA1 }}
- os-deps-{{ arch }}-{{ .Branch }}
Then, in the Install Os dependencies step, we run the respective Shell script to install the dependencies needed. We have set this step to timeout if the operation takes longer than two minutes to complete. (See docs for timeout):
- run:
name: Install Os dependencies
command: ./build/x86_64/linux_macos/osDependencies.sh
timeout: 2m
Then, in then post-Install Os dependencies step, we save the results of the previous step — the layer from the above run step. (The key name is formatted to ensure uniqueness, and the specific paths to save are included):
- save_cache:
key: os-deps-{{ arch }}-{{ .Branch }}-{{ .Environment.CIRCLE_SHA1 }}
paths:
- vendor/apt
- vendor/apt/archives
Then, in the pre-Build and install make via script step, we restore the cache if one already exists:
- restore_cache:
keys:
- make-382-{{ arch }}-{{ .Branch }}-{{ .Environment.CIRCLE_SHA1 }}
- make-382-{{ arch }}-{{ .Branch }}Then, in the Build and install make via script step, we run the Shell script to install a specific version of make; it is set to timeout if step takes longer than one minute to finish:
- run:
name: Build and install make via script
command: ./build/x86_64/linux_macos/installMake.sh
timeout: 1mThen, in the post Build and install make via script step, we save the results of the above action to the cache:
- save_cache:
key: make-382-{{ arch }}-{{ .Branch }}-{{ .Environment.CIRCLE_SHA1 }}
paths:
- /make-3.82/
- /usr/bin/make
- /usr/local/bin/make
- /usr/share/man/man1/make.1.gz
- /lib/Then, we Define Environment Variables and UpdateJAVA_HOMEandPATHat Runtime. Here, the environment variables are sourced so that we remember them for the next subsequent steps until the end of the build process (please keep this in mind):
- run:
name: Define Environment Variables and update JAVA_HOME and PATH at Runtime
command: |
echo '....' <== a number of echo-es displaying env variable values
source ${BASH_ENV}Then, in the step Display Hardware, Software, Runtime environment, and dependency versions, we display environment-specific information and record it into the logs for posterity. This is also useful during debugging when things go wrong:
- run:
name: Display HW, SW, Runtime env. info and versions of dependencies
command: ./build/x86_64/linux_macos/lib/displayDependencyVersion.shThen, we run the step to set up MX. This is important from the point of view of the GraalVM compiler. (MX is a specialized build system created to facilitate compiling and building Graal/GraalVM and it’s components.)
- run:
name: Setup MX
command: ./build/x86_64/linux_macos/lib/setupMX.sh ${BASEDIR}Then, we run Build JDK JVMCI; we build the JDK with JVMCI enabled here. We timeout if the process takes longer than 15 minutes without any output or if the process takes longer than 20 minutes to finish:
- run:
name: Build JDK JVMCI
command: ./build/x86_64/linux_macos/lib/build_JDK_JVMCI.sh ${BASEDIR} ${MX}
timeout: 20m
no_output_timeout: 15mThen, we run the step Run JDK JVMCI Tests, which runs tests as part of the sanity check after building the JDK JVMCI:
- run:
name: Run JDK JVMCI Tests
command: ./build/x86_64/linux_macos/lib/run_JDK_JVMCI_Tests.sh ${BASEDIR} ${MX}Then, we run Setting up the environment and Build GraalVM Compiler to set up the build environment with the necessary environment variables, which will be used in the following steps:
- run:
name: Setting up environment and Build GraalVM Compiler
command: |
echo ">>>> Currently JAVA_HOME=${JAVA_HOME}"
JDK8_JVMCI_HOME="$(cd ${BASEDIR}/graal-jvmci-8/ && ${MX} --java-home ${JAVA_HOME} jdkhome)"
echo "export JVMCI_VERSION_CHECK='ignore'" >> ${BASH_ENV}
echo "export JAVA_HOME=${JDK8_JVMCI_HOME}" >> ${BASH_ENV}
source ${BASH_ENV}Then, we run the step, Build the GraalVM Compiler and embed it into the JDK (JDK8 with JVMCI enabled), which timeouts if the process takes longer than seven minutes without any output or longer than 10 minutes in total to finish:
- run:
name: Build the GraalVM Compiler and embed it into the JDK (JDK8 with JVMCI enabled)
command: |
echo ">>>> Using JDK8_JVMCI_HOME as JAVA_HOME (${JAVA_HOME})"
./build/x86_64/linux_macos/lib/buildGraalCompiler.sh ${BASEDIR} ${MX} ${BUILD_ARTIFACTS_DIR}
timeout: 10m
no_output_timeout: 7mThen, we run the Simple Sanity check artifacts step to verify the validity of the artifacts created once a build has been completed, just before archiving the artifacts:
- run:
name: Sanity check artifacts
command: |
./build/x86_64/linux_macos/lib/sanityCheckArtifacts.sh ${BASEDIR} ${JDK_GRAAL_FOLDER_NAME}
timeout: 3m
no_output_timeout: 2mThen, we run the step, Archiving artifacts (compressing and copying final artifacts into a separate folder), which timeouts if the process takes longer than two minutes without any output or longer than three minutes in total to finish:
- run:
name: Archiving artifacts
command: |
./build/x86_64/linux_macos/lib/archivingArtifacts.sh ${BASEDIR} ${MX} ${JDK_GRAAL_FOLDER_NAME} ${BUILD_ARTIFACTS_DIR}
timeout: 3m
no_output_timeout: 2mFor posterity and debugging purposes, we capture the generated logs from the various folders and archive them:
- run:
name: Collecting and archiving logs (debug and error logs)
command: |
./build/x86_64/linux_macos/lib/archivingLogs.sh ${BASEDIR}
timeout: 3m
no_output_timeout: 2m
when: always
- store_artifacts:
name: Uploading logs
path: logs/Finally, we store the generated artifacts at a specified location — the following lines will make the location available on the CircleCI interface (we can download the artifacts from here).
- store_artifacts:
name: Uploading artifacts in jdk8-with-graal-local
path: jdk8-with-graal-local/For the second approach, we will be using a pre-built Docker container that has been created and built locally with all necessary dependencies and the docker image saved and then pushed to a remote registry. Then, we will be referencing this Docker image in the CircleCI environment via the configuration file. This saves us time and effort for running all the commands to install the necessary dependencies to create the necessary environment for this approach (see the details steps in the previous section).
We expect the build to run for a shorter time as compared to the previous build; this is a result of the pre-built Docker image that we will see in the Steps to Build the Prebuilt Docker Image section. The additional speed benefit comes from the fact that CircleCI caches the Docker image layers, which in turn results in a quicker startup of the build environment.
We will be going through the CircleCI configuration file (stored in .circleci/circle.yml) section-by-section. For this approach, see config.yml in .circleci for the full listing; see commit e5916f1 for the source changes.
Again, we will be referring to the Docker image by its full name. It is a pre-built Docker image (neomatrix369/graalvm-suite-jdk8) made available by neomatrix369. It was built and uploaded to Docker Hub in advance before the CircleCI build was started. It contains the necessary dependencies for the GraalVM compiler to be built:
docker:
- image: neomatrix369/graal-jdk8:${IMAGE_VERSION:-python-2.7}
steps:
- checkoutAll the sections below do the exact same tasks (and for the same purpose) as in the first approach (see the Explaining Sections of the Config Filesection for more details).
However, in this approach we have removed the following sections, as they are no longer required.
- restore_cache:
keys:
- os-deps-{{ arch }}-{{ .Branch }}-{{ .Environment.CIRCLE_SHA1 }}
- os-deps-{{ arch }}-{{ .Branch }}
- run:
name: Install Os dependencies
command: ./build/x86_64/linux_macos/osDependencies.sh
timeout: 2m
- save_cache:
key: os-deps-{{ arch }}-{{ .Branch }}-{{ .Environment.CIRCLE_SHA1 }}
paths:
- vendor/apt
- vendor/apt/archives
- restore_cache:
keys:
- make-382-{{ arch }}-{{ .Branch }}-{{ .Environment.CIRCLE_SHA1 }}
- make-382-{{ arch }}-{{ .Branch }}
- run:
name: Build and install make via script
command: ./build/x86_64/linux_macos/installMake.sh
timeout: 1m
- save_cache:
key: make-382-{{ arch }}-{{ .Branch }}-{{ .Environment.CIRCLE_SHA1 }}
paths:
- /make-3.82/
- /usr/bin/make
- /usr/local/bin/make
- /usr/share/man/man1/make.1.gz
In the following section, I will go through the steps to show how to build the pre-built Docker image. It will involve running the bash scripts ./build/x86_64/linux_macos/osDependencies.shand ./build/x86_64/linux_macos/installMake.sh to install the necessary dependencies as part of building a Docker image. Finally, we push the image to Docker Hub or any other remote registry of your choice.
Run build-docker-image.sh (see bash script source), which depends on the presence of a Dockerfile (see docker script source). The Dockerfile does all the necessary tasks of running the dependencies inside the container (i.e. it runs the bash scripts ./build/x86_64/linux_macos/osDependencies.shand ./build/x86_64/linux_macos/installMake.sh.
$ ./build-docker-image.shOnce the image has been built successfully, run push-graal-docker-image-to-hub.sh after setting the USER_NAME and IMAGE_NAME (see source code); otherwise, it will use the default values, as set in the bash script:
$ USER_NAME="[your docker hub username]" IMAGE_NAME="[any image name]" \
./push-graal-docker-image-to-hub.sh| Areas of interest | Approach 1 | Approach 2 |
|---|---|---|
| Config file (full source list) | build-on-circleci | build-using-prebuilt-docker-image |
| Commit point (sha) | df28ee7 | e5916f1 |
| Lines of code (loc) | 110 lines | 85 lines |
| Source lines (sloc) | 110 sloc | 85 sloc |
| Steps (steps: section) | 19 | 15 |
| Performance (see Performance section) | Some speedup due to caching, but slower than Approach 2 | Speed-up due to pre-built docker image, and also due to caching at different steps. Faster than Approach 1 Ensure DLC layering is enabled (its a paid feature |
I came across things that wouldn’t work initially but were later fixed with changes to the configuration file or the scripts:
We see the following screen (the last step: updating artifacts enlists where the artifacts have been copied) after a build has been successfully finished:
The artifacts are now placed in the right folder for download. We are mainly concerned about the jdk8-with-graal.tar.gz artifact.
Before writing this post, I ran multiple passes of both the approaches and jotted down the time taken to finish the builds, which can be seen below:
Approach One: standard CircleCI build (caching enabled).
Approach Two: using pre-built docker image (caching enabled, DLC feature unavailable).
Note: Approach Two should show better performance when using a paid tier, as Docker Layer Caching is available as part of this plan.
In order to be sure that by using both the above approaches we have actually built a valid JDK embedded with the GraalVM compiler, we perform the following steps with the created artifact:
First, download the jdk8-with-graal.tar.gz artifact from under the Artifacts tab on the CircleCI dashboard (needs sign-in):
Then, unzip the .tar.gz file and do the following:
tar xvf jdk8-with-graal.tar.gzAfter that, run the below command to check the JDK binary is valid:
cd jdk8-with-graal
./bin/java -versionFinally, check if we get the below output:
openjdk version "1.8.0-internal"
OpenJDK Runtime Environment (build 1.8.0-internal-jenkins_2017_07_27_20_16-b00)
OpenJDK 64-Bit Graal:compiler_ab426fd70e30026d6988d512d5afcd3cc29cd565:compiler_ab426fd70e30026d6988d512d5afcd3cc29cd565 (build 25.71-b01-internal-jvmci-0.46, mixed mode)Similarly, to confirm if the JRE is valid and has the GraalVM compiler built in, we do this:
./bin/jre/java -versionCheck if we get a similar output as the one from the previous code block.
openjdk version "1.8.0-internal"
OpenJDK Runtime Environment (build 1.8.0-internal-jenkins_2017_07_27_20_16-b00)
OpenJDK 64-Bit Graal:compiler_ab426fd70e30026d6988d512d5afcd3cc29cd565:compiler_ab426fd70e30026d6988d512d5afcd3cc29cd565 (build 25.71-b01-internal-jvmci-0.46, mixed mode)With this, we have successfully built JDK8 with the GraalVM compiler embedded into it. We also bundled the Graal and Truffle components in an archive file, both of which are available for download via the CircleCI interface.
Note: you will notice that we do perform sanity checks of the binaries built just before we pack them into compressed archives, as part of the build steps (see bottom section of CircleCI the configuration files).
We all like to show-off; we also like to know the current status of our build jobs. A green-color-build-status icon is a nice indication of success.
We can very easily embed both of these status badges by displaying the build status of our project (branch-specific i.e. master or another branch you have created) built on CircleCI. (See the docs for more information.)
We explored two approaches to build the GraalVM compiler using the CircleCI environment. They were good experiments to compare performance between the two approaches. We also saw a number of things to avoid. Additionally, we saw how useful some of the CircleCI features are.
Once we know the CircleCI environment, it’s pretty easy to use and always gives us consistent behavior every time we run it. We can also set up checks on build-time for every step of the build, and abort a build if the time taken to finish a step surpasses the threshold time-period.
The ability to use pre-built Docker images coupled with Docker Layer Caching on CircleCI can be a major performance boost, as it saves us build-time needed to reinstall any necessary dependencies at every build. Additional performance speedups are available on CircleCI, with caching of the build steps — this again saves build time by not having to re-run the same steps if they haven’t changed.
There are a lot of useful features available on CircleCI with plenty of documentation, and everyone on the community forum is helpful; questions are answered pretty much instantly.
Next, let’s build the same and more on another build environment / build farm — hint, hint, are you think the same as me? Adopt OpenJDK buildfarm? We can give it a try!
Thanks and credits to Ron Powell from CircleCI and Oleg Šelajev from Oracle Labs for proof-reading and giving constructive feedback.
Please do let me know if this is helpful by dropping a line in the comments below or by tweeting at @theNeomatrix369. I would also welcome feedback — see how you can reach me — and above all, please check out the links mentioned above.