Jenkins Continuous Integration (CI) Server

icon NWebster Jenkins CI testing server

Click the above link to access our testing server. This is where you will set up your automated build for your project. We encourage you to start early on setting up your automated build, so that there is enough time to work out any kinks and get help from the TAs if anything is broken in your server setup. Please email your customer TA (and CC the instructor) if you need help.

Setting up a project in Jenkins:

You can set up Jenkins to do an automated build for your CSE 403 project. We have guides for setting up Jenkins to perform the following tasks:

icon Creating a command-line Ant build script for your project:

You probably created your project as an Eclipse project, but you will need to configure it so that it can be fully compiled and built from the command line. Android projects are built on the command line using a utility named Ant, which is like the older well known utility named Make. Ant uses a file named build.xml to manage the compiling of your project. We assume that you are using a Linux or Unix-compatible machine with the Android SDK and Ant installed, and that both of these are in your PATH so that you can run commands like android on the command line. If you don't have Ant, go to the Ant homepage to download and install it, or install it from your Linux package manager (e.g. yum or apt-get).

You can use the Android SDK command-line tools to generate a build.xml file for your project. We will assume that your project has a test-project underneath it in a subdirectory named tests/ containing your JUnit test cases. In a terminal window, cd to the root directory of your project and then type the following commands. The commands contain a dot . indicating the current directory of your project.

android update project -p .
android update test-project -m . -p tests/

After running these commands, you should have a build.xml and some other new files. You can try to do a command-line build now on your machine by typing:

ant clean debug

You should also try to do a command-line run of your unit tests on your machine by typing:

ant debug install test

The files build.xml, ant.properties, and project.properties (for both your main project and your test project) must be checked in to your version control on GitHub. The file local.properties MUST NOT BE checked in to GitHub; it contains local configuration, like the location where your Android SDK is installed. Since this location can be different on i.e. Jenkins build machines, this file should not be checked in to version control. So it is recommended to add "local.properties" to the "Ignored Resources" in Eclipse.

You must also make sure that Jenkins has access to check out files from your Github. If your Github is public, you're set. If it is a private repo, make sure that the course instructor's email has access to your Github.

Now you can type ant target to execute an Ant action. Your build script will have targets in it for every action that you want Jenkins to perform. The auto-generated build.xml file already has the following targets in it:

  • The target named clean removes any .class files.
  • The target named debug often compiles the "debug" version of your code.
  • The target named release often compiles the "release" version of your code.
  • The target named install copies your compiled .apk onto your Android virtual device.
  • The target named test runs your unit tests from your test project.
  • The target named emma runs the Emma tool to check unit test coverage.
  • Later you can add targets such as checkstyle or findbugs, etc.

Any member of your team who checks out your project after this will get the build.xml and related files, but they won't get a local.properties file because that is not in the repo. So if they try to build from the command line, they will get errors about missing symbols that would have been defined in the local properties file, such as, sdk.dir is missing. Make sure to generate local.properties using 'android update project' or to inject it through the ANDROID_HOME environment variable. To correct these errors, that team member should run the following command to get the local.properties on their machine. Note that this file should never be checked into the GitHub.

android update project -p .

icon Creating a Jenkins job for your project on the server:

For this step, we are assuming that your GitHub already has a suitable Ant build.xml file properly checked in as described above. Now go to the Jenkins server at the URL above. Click icon New Job. Under Job name, give your job a name equal to your group's name, such as "Interview Annihilator" (without the quotes). Choose Build a free-style software project. Click OK.

Now you have to configure your job. In the configuration screen that appears, there are many fields that you can just ignore. But make sure you fill out the following fields. Here is a screenshot of an example project configuration.

  • GitHub project: Set this to the URL of your GitHub, such as https://github.com/foobar/InterviewAnnihilator/
  • Source Code Management: Set this to Git. Under Repository URL, put the same GitHub repo URL you just put above under GitHub project.
  • Build Triggers: Check the box Build when a change is pushed to GitHub.
  • Build: Click Add Build Step. From the menu of choices, choose Invoke Ant. Under Targets, type in the Ant build target name(s) of the targets you want to build, separated by spaces. For example, if you want Jenkins to automate a build of your "debug" version APK, you might type:
    clean debug
  • Click Apply then Save.

Go test your build by opening your new job in Jenkins and clicking icon Build Now. If it is successful (icon blue), you did it! If it is unsuccessful (icon red), click on the build and then click icon Console Output to see what went wrong. Check your project configuration or email your customer TA for help.

Once your build is successful, you may want to test a build failure by temporarily checking in to GitHub a line of code that does not compile, then running a build in Jenkins, making sure that it fails (red icon), then checking in a fix for the compiler error, then running the build again, to see that it passes (blue icon).

icon Setting up an automatic daily build:

Open the Jenkins page for your project's job. Click icon Configure. Under Build Triggers, check the box for Build periodically. A text box named "Schedule" appears. In the box, type a string indicating to run your build at least once per day. To understand the syntax of the schedule string, click the help icon icon and read the tutorial text. (It is the same syntax as that used by the unix cron command.) For example, to schedule a build every day at midnight, you'd write:

0 0 * * *

It is generally bad to schedule at an exact time, because if every project tries to build at exactly midnight, it can bog down the server. So instead, use "H" (short for "hash") to pick a random minute or hour. For example, to schedule one daily build at a random time between 3:00 and 3:59 AM, and another daily build at a random time between 6:00 and 6:59 PM, you'd write:

H 3 * * *
H 18 * * *

(You might not want to set this up until you have your build generally running properly and passing. Otherwise your automatic daily build will just run and fail each day.)

Click Apply then Save.

icon Emailing your group when a build breaks:

Open the Jenkins page for your project's job. Click icon Configure. Under Post-build Actions, click Add post-build action. In the list of choices that appears, choose E-mail Notification. In the "Recipients" text box that appears, type the email alias for your project group, such as cse403XYZ@cs.washington.edu .

(You might not want to set this up until you have your build generally running properly and passing. Otherwise your group would get several emails for failed builds while you are setting things up and working out the kinks.)

Click Apply then Save.

icon Automatically running your JUnit tests:

Up to this point we have been able to survive with the default automatically-generated Ant build.xml file. But for JUnit you will need to edit your build.xml and add a new target that runs your JUnit tests. There is already a target called test that does this, but unfortunately in an Android project it does not consider the build to have "failed" if a unit test fails; but we do want a failing unit test to be considered a failed build.

Open your test subproject's build.xml file, the one in the tests/ subdirectory of your overall project. (Please note that this is not the main project build.xml in the project's root directory.) First, find a line near the bottom of the file that looks like this:

<!-- version-tag: 1 -->

Change the line to read:

<!-- version-tag: custom -->

Now we will create our new target. In this tutorial, we will call the new target unittests . Add the following content to the tests/build.xml file on your local machine, near the bottom of the file, just before the </project> tag that ends the file.

    <target name="unittests" depends="debug,install" >
        <property name="unittestlog.file" value="unit_tests_log.txt" />
        <record name="${unittestlog.file}" action="start" />
        <antcall target="test" />
        <record name="${unittestlog.file}" action="stop" />
        <loadfile property="tests.output" srcFile="${unittestlog.file}" />
        <echo>Checking for failures</echo>
        <fail message="Unit tests failed!">
            <condition>
                <contains string="${tests.output}" substring="FAILURES" />
            </condition>
        </fail>
        <echo>Unit tests passed!</echo>
    </target>

Test your new target locally by running ant unittests and make sure that the unit tests actually run and pass. If so, you can check in your test project's modified build.xml into GitHub and proceed. (After running this target on your local machine, it will generate a unit_tests_log.txt output file. You don't want to check this file in to your GitHub.) You might want to try changing the project code temporarily to intentionally cause a unit test to fail, making sure that the ant unittests target fails, and then changing your code back.

Now let's tell Jenkins to run your unit tests as part of your build job. Go to the Jenkins server in the browser and open the Jenkins page for your project's job. Click icon Configure. Under Build Environment, check the box for Run an Android emulator during build.

You want to use a custom Android virtual device with specific settings, so click Run emulator with properties. Set the properties to whatever values you like. You can look at your Android AVD device manager settings for your virtual device and copy those settings into the text boxes here. Here is one example device configuration; you can use it if you like, but we recommend you copy the state from your AVD configuration instead.

  • Android OS version: 4.2
  • Screen density: 240
  • Screen resolution: 400x800
  • Target ABI: armeabi-v7a

Under Common emulator options, you should un-check both of the options, "Show emulator window" and "Use emulator snapshots". You should do this because the server does not have a graphical user interface showing, and the snapshots are not efficient to save and restore on the server. If you get timeouts during Android test runs, you may also want to check "reset emulator state at start up".

Under Build, you should already have an Invoke Ant entry that performs your build. You should add your unittests target to the "Targets" text area so that Ant will also run your unit tests. You don't want to include the debug target in the list because the unittests target already depends on it. For example, the full value of the "Targets" text box might now read:

clean unittests

Now, still in your "Invoke Ant" area, click Advanced.... In the "Build File" text box, write the following value to make Ant use your test project's build XML file:

tests/build.xml

Click Apply then Save. Test to make sure that it's working by clicking icon Build Now.

icon Running Emma (test coverage):

Android SDK already includes support for running Emma from the command line in your Ant build script. You can try it out by going to the command line of your Android project folder and typing ant emma .

To set up Jenkins to run Emma for you and generate a report of your test coverage: Open the Jenkins page for your project's job. Click icon Configure. Under Build, you should already have an Invoke Ant entry that performs your build. You should add your emma target to the "Targets" text area, just after clean, so that Ant will now also run Emma. For example, the full value of the "Targets" text box might now read:

clean emma unittests

It's also nice to get a graphical report of your test coverage over time. To do this, under Post-build Actions, click Add post-build action. In the list of choices that appears, choose Record Emma coverage report. In the "Folders or files containing Emma XML reports" text box, write the following value, assuming that your test project is located in the tests/ subfolder within your overall project:

**/tests/bin/coverage.xml

In the "Health reporting" text boxes, you probably want to change the "% Line" percentage to the percentage required for the current phase of the project (e.g. 50). % Line is the metric we will use, though you can also include other metrics such as % Method or % Decision/Condition.

Click Apply then Save. Test to make sure that it's working by clicking icon Build Now. If it worked, after there have been a few build runs, you should start to see your Emma coverage report on the main page for your project's job.

icon Running Checkstyle:

For this step, you should first make sure that Checkstyle is appropriately integrated into your project and your GitHub. That is, the Checkstyle JAR library should be part of your project in its libs/ subdirectory and linked into your Eclipse project's Build Path. For this tutorial we will assume that your Checkstyle JAR is stored in your project as the file libs/checkstyle.jar .

Checkstyle uses an XML configuration file to guide what kinds of style checking it should perform. We are assuming that you have already acquired or created such an XML file. For this tutorial we will assume that your Checkstyle XML configuration file is called my_project_styles.xml.

Up to this point we have been able to survive with the default automatically-generated Ant build.xml file. But for Checkstyle you will need to edit your build.xml and add a new target that runs Checkstyle. Note that we are editing the build.xml for your main project, not the one in the tests/ subdirectory for your test project.

Open build.xml in your text editor. First, find a line near the bottom of the file that looks like this:

<!-- version-tag: 1 -->

Change the line to read:

<!-- version-tag: custom -->

Now we will create our new target. In this tutorial, we will call the new target checkstyle . (It is possible to connect Checkstyle to your existing targets, like debug or release, but we recommend making a new target that runs Checkstyle on its own.) Edit the main project's build.xml file on your local machine and add the following content, near the bottom of the file, just before the </project> tag that ends the file.

    <taskdef resource="checkstyletask.properties" classpath="${basedir}/libs/checkstyle.jar" />
    <target name="checkstyle">
        <checkstyle config="my_project_styles.xml">
            <fileset dir="src" includes="**/*.java" />
            <formatter type="plain" />
            <formatter type="xml" toFile="checkstyle-result.xml" />
        </checkstyle>
    </target>

Test your new target locally by running ant checkstyle and make sure that Checkstyle actually runs. If so, you can check in your modified build.xml into GitHub and proceed. (After running CheckStyle on your local machine, it will generate a checkstyle-result.xml output file. You don't want to check this file in to your GitHub.)

Now open the Jenkins page for your project's job. Click icon Configure. Under Build, you should already have an Invoke Ant entry that performs your build. You could add your new static analysis target there, but we recommend that you separate it. Click Add build step, and in the list that appears, choose Invoke Ant. In the "Targets" text area, write the name of your new target, checkstyle. (For the first Invoke Ant task, you needed to click the Advanced button and specify the build file as tests/build.xml; but since this second Invoke Ant action is not running the unit tests, you don't need to do the Advanced part here.)

It's also nice to get a graphical report of your Checkstyle results over time. To do this, under Post-build Actions, click Add post-build action. In the list of choices that appears, choose Publish Checkstyle analysis results. You don't need to change any of the values under this new item, though if you like, you can configure the "Health thresholds" under Advanced to give your build a bit of slack if a few small style checks fail.

To finish the process, click Apply then Save. Test to make sure that it's working by clicking icon Build Now.

Running FindBugs:

Unfortunately, getting FindBugs to run is somewhat tricky. First we need to get FindBugs compiling properly as a command-line task on your local computer. First, if you have not already done so, download the FindBugs tool onto your local computer. From the FindBugs downloaded package, find the file lib/findbugs-ant.jar and copy it into the libs directory of your Android project, and check in that JAR to your GitHub.

Now you must tell your project where your FindBugs is located. Either set your computer to have an environment variable named FINDBUGS_HOME that points to the directory where you have FindBugs installed, or edit your machine's local.properties file and add the following line to the end (replacing the italic part with your FindBugs directory):

findbugs.home=/path/to/your/findbugs

Now set up a file to tell FindBugs to exclude certain files in your project from being checked. The most common file to exclude is the R.java resource file auto-generated by Android projects. Create and save a file findbugs-exclude.xml in your main project's root directory with exactly the following contents:

<?xml version="1.0" encoding="UTF-8"?>
<FindBugsFilter>
    <Match>
        <Class name="~.*\.R\$.*" />
        <Bug code="Nm" />
    </Match>
</FindBugsFilter>

Up to this point we have been able to survive with the default automatically-generated Ant build.xml file. But for FindBugs you will need to edit your build.xml and add a new target that runs FindBugs. Note that we are editing the build.xml for your main project, not the one in the tests/ subdirectory for your test project.

Open build.xml in your text editor. First (if you didn't already do this while setting up Checkstyle on Jenkins), find a line near the bottom of the file that looks like this:

<!-- version-tag: 1 -->

Change the line to read:

<!-- version-tag: custom -->

Now we will create our new target. In this tutorial, we will call the new target findbugs . Edit the main project's build.xml file on your local machine and add the following content, near the bottom of the file, just before the </project> tag that ends the file.

    <condition property="findbugs.home" value="${env.FINDBUGS_HOME}">
        <isset property="env.FINDBUGS_HOME" />
    </condition>
    <taskdef name="findbugs" classpath="libs/findbugs-ant.jar" classname="edu.umd.cs.findbugs.anttask.FindBugsTask" />
    <target name="findbugs" depends="debug">
        <gettarget
            androidJarFileOut="project.target.android.jar"
            androidAidlFileOut="project.target.framework.aidl"
            bootClassPathOut="project.target.class.path"
            targetApiOut="project.target.apilevel"
            minSdkVersionOut="project.minSdkVersion" />
        <findbugs home="${findbugs.home}" output="xml" outputFile="findbugs.xml"
                excludeFilter="findbugs-exclude.xml" warningsProperty="findbugsWarning" errorProperty="findbugsError">
            <auxClasspath path="${project.target.android.jar}" />
            <class location="${out.dir}" />
        </findbugs>
        <fail if="findbugsError" message="FindBugs detected at least one error." />
        <fail if="findbugsWarning" message="FindBugs detected at least one warning." />
    </target>

Test your new target locally by running ant findbugs and make sure that FindBugs actually runs. If so, you can check in your modified build.xml into GitHub and proceed. (After running FindBugs on your local machine, it will generate a findbugs.xml output file. You don't want to check this file in to your GitHub.)

Open the Jenkins page for your project's job. Click icon Configure. Under Build, you should already have an Invoke Ant entry that performs your build. You could add your new static analysis target there, but we recommend that you separate it. Click Add build step, and in the list that appears, choose Invoke Ant. In the "Targets" text area, write the name of your new target, findbugs . (Make sure not to write debug in the same list of targets as findbugs, because the findbugs target already indicates that it depends on debug, so if you include both in the target, it is a circular reference.)

If you are also running Checkstyle, you can combine your checkstyle and findbugs into one Ant run if you like, separated by spaces in the text box. (For the first Invoke Ant task, you needed to click the Advanced button and specify the build file as tests/build.xml; but since this second Invoke Ant action is not running the unit tests, you don't need to do the Advanced part here.)

It's also nice to get a graphical report of your FindBugs results over time. To do this, under Post-build Actions, click Add post-build action. In the list of choices that appears, choose Publish FindBugs analysis results. You don't need to change any of the values under this new item.

To finish the process, click Apply then Save. Test to make sure that it's working by clicking icon Build Now.

For more information:

Please contact your customer TA if you need help. This Jenkins Android project setup page has some additional details and tips, though not all of them apply to our projects.

This document and its content are copyright © Marty Stepp, 2013. All rights reserved.
Any redistribution, reproduction, transmission, or storage of part or all of the contents in any form
is prohibited without the author's expressed written permission.