Creating a Jenkins project for building packages with poudriere

Now it is time to create a Jenkins job that will do continuous building and testing of our packages.

Installing and configuring required Jenkins plugins

First we would need two Jenkins plugins installed which are the Jenkins PostBuildScript Plugin and the Jenkins Publish Over SSH Plugin

Jenkins PostBuildScript Plugin will be used to execute a shell script in the Post-build Actions section of our project for syncing the already built packages and the Jenkins Publish Over SSH Plugin will be used for connecting to our build server using SSH keys, performing the build process and afterwards syncing back to Jenkins the already built packages.

In order to install the plugins navigate to Manage Jenkins -> Manage Plugins -> Available and install the Jenkins PostBuildScript Plugin and Jenkins Publish Over SSH Plugin.

Before creating the Jenkins job we need to do some configuration of the Jenkins Publish Over SSH Plugin first.

To do that navigate to Manage Jenkins -> Configure System and scroll down to the SSH section as shown in the screenshot below.

In the SSH section we need to specify the location to the private SSH key used by the Jenkins instance, which in the above screenshot is /usr/local/jenkins/.ssh/id_rsa. The private SSH key for Jenkins you can create by logging into the Jenkins instance and executing the below command:

$ sudo -u jenkins ssh-keygen

Now copy the public SSH key for user jenkins (which should be in /usr/local/jenkins/.ssh/id_rsa) from the Jenkins CI instance to the poudriere build server so that we can connect to our build server and run package builds. To do that from the Jenkins instance execute the command below:

$ sudo -u jenkins ssh-copy-id jenkins-builder@poudriere.example.org

Remember that we are using jenkins-builder user on the poudriere build server and jenkins user on the Jenkins instance.

If you do not have ssh-copy-id(1) package installed then just copy-paste the public ssh key of jenkins user to /home/jenkins-builder/.ssh/authorized_keys on the build server. Test that you can succesfully connect to the build server from Jenkins as well when ready.

Now that we have configured SSH key connection between the build server and Jenkins instance, lets configure Jenkins itself so that it can use that connection.

To do that lets add our poudriere build server to the SSH Servers section in Jenkins as shown on the screenshot below. When ready click on the Test Configuration button to test the connection between Jenkins and the poudriere builder server.

If the test was successful as shown on the screenshot above click on the Save button and we are ready with the configuration of the Jenkins Publish Over SSH Plugin.

Creating the Jenkins poudriere job

Now, lets create our Jenkins project for running poudriere package builds. Navigate to New Job and give the new project a meaningful name. In this example for the project name I will use the freebsd-9.0-amd64-builder name as shown on the screenshot below.

Once the new project is created we need to configure it - project description, SCM, number of builds to keep, etc.

We do not use an SCM system, so we leave the Source Code Management section of the project empty.

The next section we configure is the Build Triggers one. This is where we configure when a project is being scheduled for a build. I choose to leave this section of the project empty and trigger a build manually, but you may wish to enable periodic builds if needed. Feel free to play with this section of the project.

Now lets configure the build itself. This is where we define how we trigger the poudriere builds. To do that we configure the Build section of our project as shown in the screenshot below.

In order to add the build step click on the Add build step button from the Build section of the project and select the Send files or execute commands over SSH option.

Then select the poudriere-builder server and in the Exec command field we specify the commands used for building the packages as shown in the screenshot below.

The Build section of the project is a simple shell script that takes care of building our packages with poudriere.

Below is listed the shell script used for building the packages with poudriere:

# poudriere jail
POUDRIERE_JAIL="90amd64"

# packages to be built
PACKAGES_TO_BUILD="security/apg
ports-mgmt/pkg
sysutils/zfsnap
ports-mgmt/poudriere
shells/zsh"

# use a temp file to store the packages for building
tmpfile=`mktemp /tmp/poudriere.XXXXXX`
echo "${PACKAGES_TO_BUILD}" > ${tmpfile}

# update the poudriere default ports tree
sudo poudriere ports -u 

# build the packages
sudo poudriere bulk -f ${tmpfile} -j ${POUDRIERE_JAIL} -k

# show jail information after build has finished
sudo poudriere jails -i -j ${POUDRIERE_JAIL}

# remove the temp file
rm -f ${tmpfile}

There are only two things you need to change in the above script - the poudriere jail you are building packages on as defined by POUDRIERE_JAIL variable and the packages you want to build as defined by the PACKAGES_TO_BUILD variable.

In the above example script we are going to build the packages below on the 90amd64 poudriere jail:

• security/apg
• ports-mgmt/pkg
• sysutils/zfsnap
• ports-mgmt/poudriere
• shells/zsh

Now lets configure the Advanced settings of our project, to do that click on the Advanced* button and check the Fail the build if an error occurs as shown in the screenshot below:

One last thing we need to configure in the Build section of our project is to check the Exec in pty option and set the Exec timeout as shown in the screenshot below. We set the Exec timeout to 0 as we don't know how long a build is going to last.

The last thing we need to configure about our project is to sync the already built packages back to Jenkins so that we can make them available for installation. To do that we configure the Post-build Actions section of our project.

In the Post-build Actions section of your project click on the Post-build action button and select [PostBuildScript] - Execute a set of scripts then from the Build steps select Execute shell as shown in the screenshot below:

Our shell script in the Post-build Actions section only serves for syncing the packages from the build server to the Jenkins instance and is configured as shown on the screenshot below.

The sync script used in the Post-build Actions section is listed below:

BUILD_SERVER="poudriere.example.org"
POUDRIERE_JAIL="90amd64"
POUDRIERE_DATA="/poudriere_data"
POUDRIERE_PORTS_TREE="default"

[ ! -d ${WORKSPACE}/archive/packages ] && mkdir -p ${WORKSPACE}/archive/packages
[ ! -d ${WORKSPACE}/archive/logs ] && mkdir -p ${WORKSPACE}/archive/logs

echo ">>> Syncing packages from ${BUILD_SERVER} ..."
/usr/local/bin/rsync -av --progress jenkins-builder@${BUILD_SERVER}:${POUDRIERE_DATA}/packages/${POUDRIERE_JAIL}-${POUDRIERE_PORTS_TREE} ${WORKSPACE}/archive/packages

echo ">>> Syncing logs from ${BUILD_SERVER} ..."
/usr/local/bin/rsync -av --progress jenkins-builder@${BUILD_SERVER}:${POUDRIERE_DATA}/logs/${POUDRIERE_JAIL}-${POUDRIERE_PORTS_TREE}-*.log ${WORKSPACE}/archive/logs

You will need to adjust the below variables to match your setup:

• BUILD_SERVER - your poudriere build server
• POUDRIERE_JAIL - your poudriere jail used for building packages
• POUDRIERE_DATA as defined in /usr/local/etc/poudriere.conf
• POUDRIERE_PORTS_TREE the Ports Tree used by poudriere, which is set to default by default

Once ready click on the Save button and do a test build of the project.

And our poudriere Jenkins job is fully configured! Now we can do a test build of our project.