Visualize Jenkins build data through InfluxDB in Grafana

When coming to visualization of build data I want to rely on time series databases. The big differences between „common“ databases and time series oriented databases like InfluxDB is the specialization of last for time based data like log messages or even better: measurement data in relation to time. This can be used for temperature, air pressure, cpu usage … or build durations of our continous integration server called Jenkins. The first problem: how to get the measurement data from Jenkins to our influx instance? It’s not supported out of the box, but Jenkins is highly extendable by using plugins, so I developed my own simple-influxdb-plugin for Jenkins, that is mainly inspired from an already existing but a bit outdated plugin from jrajala.

This is how the context looks like:
jenkins_influxdb_grafana-overview

Versions

component version download
Ubuntu 14.04 http://www.ubuntu.com/download/desktop
Java Hotspot 1.8.0_72 http://www.oracle.com/technetwork/java/javase/downloads/jdk8-downloads-2133151.html
Jenkins 1.648 http://mirrors.jenkins-ci.org/war/latest/jenkins.war
InfluxDB 0.10.1 https://influxdata.com/downloads/
Grafana 2.6.0 http://grafana.org/download/

InfluxDB

Create the required database via InfluxDB’s frontend at http://localhost:3000
influxdb_createdb
influxdb_createdb2
and create also the users for the jenkins and grafana access:
influxdb_createusers
influxdb_createusers2
That’s all for InfluxDB as it acts only as time-series-optimized persistence.

Jenkins

Next step is to put the simple InfluxDB db plugin to Jenkins plugin directory and restart Jenkins, so in its bootstrap process it can be found and loaded. After successful restart of Jenkins, the plugin’s global config appears on Jenkins configuration page and the installed InfluxDB can be configured as publishing target:
influxdb_global_config

Next step is to prepare three dummy projects that are configured with the InfluxDB as publishing target.
jenkins_projects
As we want to visualize different build durations over time, a simple bash script with a random sleep has to be added as buildstep:
influxdb_job_config

Now it’s time to trigger the jobs and the magic happens: The jobs are executed and after a random duration they come to post build step where the simple-influxdb-plugin takes over the task of pushing the measured data job-wise to the InfluxDB. Now they can be obtained from the InfluxDB gui:
influxdb_measurements
influxdb_measurement_dummy1

Grafana

This is already quite nice, but it would even be more nicer to let it show up as graph, e.g. with Kibana or Grafana. Note: The guys from InfluxData also provide a graph engine called Chronograf but I think there are more people out there who are using more common technologies and so I choose Grafana.
The Grafana installation on my Ubuntu is straight forward and very good shown in their documenation: http://docs.grafana.org/installation/ so I rely on the defaults. After a bit playing around with several queries I got to following queries, one for each dummyproject:

SELECT (jobduration/1000) FROM "build_dummyproject1" WHERE "async" = 'true' AND $timeFilter

And after some modifications and experiments with the features of Grafana you’re able to visualize graphs like the following:
grafana_queries

That’s it 🙂