X-platform build of a QT Application using a Jenkins pipeline

16 min read

It's been many years since the last time I made a cross platform desktop-application where I wanted to provide binaries to end-users. I wrote whid in just two weeks, and had it running on Debian Linux, macOS and Windows 10 on my own machines. That was the fun and easy part. Now I have to set up a system to build the application (and eventually to test it) automatically for all the platforms that I want to support. It's simply too much work to build the application for all the target platforms manually. If I go down that road, I will eventually stop making updates simply to avoid all the time wasted on releasing anything. This is especially true for a "one man shop", where whid was something I wrote in a hurry because I needed it.

Jenkins

I have used Jenkins in the past. I never really liked it. Last year I made a SDK for a company in 5 programming languages. For each language I created a Jenkins project, using Ubuntu to build and test the target. For each of these Jenkins-projects, I had to provide detailed instructions to the company so that they could build it with their own Jenkins instance.

Jenkins Pipelines

Jenkins has several project types that can build my QT application. What I want to exploit is the "pipeline" with a "Jekinsfile". This allows me to check in the build instructions to Jenkins as part of my own source code. To build for Windows and macOS I will use Jenkins slaves running on those operating systems. For the various Linux distributions, I will create Docker containers on the fly, that can make native installation packages for the various Linux distributions. The Dockerfile files for these containers are checked in as part of my source code. Doing it this way makes it easy for other developers to test my code - but most importantly - I don't have to care too much about my Jenkins machine. Since all the interesting configuration is on github, I can create a new Jenkins virtual machine in no time if I need to. In other words, one less server to worry about.

Jenkins Plugins

I am not 100% sure what Jenkins plugins you actually need to do this, but on my current Jenkins server I have these (among others):

Docker plugin
docker-build-step
GitHub Branch Source Plugin
Pipeline
Pipeline: GitHub Groovy Libraries

Take 1: Set up the build with just one target.

To make things doable, it's often good to start with small steps and then iterate until you have th result you desire. therefore, I started with a simple build where I made a .deb package in Ubuntu xenial.

I made a new pipeline project, selected to use a Jenkinsfile, the path, relative to the root of my source code tree to the Jenkinsfile, and gave it the git repository to pull from and the branch. That was all the configuration in Jenkins itself.

Jenkins UI

Jenkins files in my source code repository

In the first iteration for this strategy, I built whid for Ubuntu xenial.

In my git repository, I have the following files:

ci/jenkins/Jenkinsfile

#!groovy

env.WHID_VERSION="2.0.1"

pipeline {
    agent { label 'master' }

    stages {
        stage('Build') {
            parallel {
                stage('Docker-build inside: Ubuntu-xenial') {
                    agent {
                        dockerfile {
                            filename 'Dockefile.ubuntu-xenial'
                            dir 'ci/jenkins'
                            label 'master'
                        }
                    }

                    steps {
                        echo "Building on ubuntu-xenial-AMD64 in ${WORKSPACE}"
                        checkout scm
                        withEnv(["DIST_DIR=${WORKSPACE}/dist", "BUILD_DIR=${WORKSPACE}/build", "SRC_DIR=${WORKSPACE}"]) {
                            sh 'pwd; ls -la;'
                            sh './scripts/package-deb.sh'
                        }
                    }

                    post {
                        success {
                            echo "Build of debian package suceeded!"
                            archive "**/whid*.deb"
                        }
                    }
                }
            }
        }
    }
}

Then, I have ci/jenkins/Dockefile.ubuntu-xenial

# Based on: https://github.com/evarga/docker-images/blob/master/jenkins-slave/Dockerfile
FROM ubuntu:xenial

MAINTAINER Jarle Aase <jgaa@jgaa.com>

# In case you need proxy
#RUN echo 'Acquire::http::Proxy "http://127.0.0.1:8080";' >> /etc/apt/apt.conf

RUN apt-get -q update &&\
    DEBIAN_FRONTEND="noninteractive" apt-get -q upgrade -y -o Dpkg::Options::="--force-confnew" --no-install-recommends &&\
    DEBIAN_FRONTEND="noninteractive" apt-get -q install -y -o Dpkg::Options::="--force-confnew" --no-install-recommends openssh-server &&\
    DEBIAN_FRONTEND="noninteractive" apt-get -q install -y g++ git make cmake libboost-all-dev libssl-dev zlib1g-dev \
    qtdeclarative5-dev  qt5-default ruby ruby-dev rubygems build-essential &&\
    gem install --no-ri --no-rdoc fpm &&\
    apt-get -q autoremove &&\
    apt-get -q clean -y && rm -rf /var/lib/apt/lists/* && rm -f /var/cache/apt/*.bin &&\
    sed -i 's|session    required     pam_loginuid.so|session    optional     pam_loginuid.so|g' /etc/pam.d/sshd &&\
    mkdir -p /var/run/sshd

# Install JDK 8 (latest edition)
RUN apt-get -q update &&\
    DEBIAN_FRONTEND="noninteractive" apt-get -q install -y -o Dpkg::Options::="--force-confnew" --no-install-recommends software-properties-common &&\
    add-apt-repository -y ppa:openjdk-r/ppa &&\
    apt-get -q update &&\
    DEBIAN_FRONTEND="noninteractive" apt-get -q install -y -o Dpkg::Options::="--force-confnew" --no-install-recommends openjdk-8-jre-headless &&\
    apt-get -q clean -y && rm -rf /var/lib/apt/lists/* && rm -f /var/cache/apt/*.bin

# Set user jenkins to the image
RUN useradd -m -d /home/jenkins -s /bin/sh jenkins &&\
    echo "jenkins:jenkins" | chpasswd

# Standard SSH port
EXPOSE 22

# Default command
CMD ["/usr/sbin/sshd", "-D"]

In addition, I have a script to build the project and create a .deb package in scripts/package-deb.sh

#!/bin/bash

# Compile and prepare a .deb package for distribution
# Uses the QT libraries from the lilnux ditribution
#
# Example:
#   WHID_VERSION="2.0.1" DIST_DIR=`pwd`/dist BUILD_DIR=`pwd`/build SRC_DIR=`pwd`/whid  ./whid/scripts/package-deb.sh

if [ -z "$WHID_VERSION" ]; then
    WHID_VERSION="2.0.0"
    echo "Warning: Missing WHID_VERSION variable!"
fi

if [ -z ${DIST_DIR:-} ];
then
    DIST_DIR=`pwd`/dist/linux
fi

if [ -z ${BUILD_DIR:-} ];
then
    BUILD_DIR=`pwd`/build
fi

if [ -z ${SRC_DIR:-} ];
then
# Just assume we are run from the scipts directory
    SRC_DIR=`pwd`/..
fi

echo "Building whid for linux into ${DIST_DIR} from ${SRC_DIR}"

rm -rf $DIST_DIR $BUILD_DIR

mkdir -p $DIST_DIR &&\
pushd $DIST_DIR &&\
mkdir -p $BUILD_DIR &&\
pushd $BUILD_DIR &&\
qmake $SRC_DIR/whid.pro &&\
make -j8 && make install &&\
popd &&\
fpm --input-type dir \
    --output-type deb \
    --force \
    --name whid \
    --version ${WHID_VERSION} \
    --vendor "The Last Viking LTD" \
    --description "Time Tracking for Freelancers and Independenet Contractors" \
    --depends qt5-default --depends libsqqlite3 \
    --chdir ${DIST_DIR}/root/ \
    --package whid-VERSION_ARCH.deb &&\
echo "Debian package is available in $PWD" &&\
popd

And in the whid.pro file, I have a section that let 'make install' copy the binary to the expected location:

linux {
    DIST_DIR = $$(DIST_DIR)
    target.path = $${DIST_DIR}/root/usr/bin
    INSTALLS += target
}

Take 2: Set up the build with several, similar Linux builds

This time we will build for Debian stable and testing, and Ubuntu LTS. I like to build with Debian testing to catch problems with my code in new versions of g++ or the libraries.

All the builds will make the same target; whid-2.0.1_amd64.deb. And that is a problem. We need to get different artifacts for each build, so we can distribute the correct file to the right people. I spent quite some time trying to figure out how to do that. Jenkins does not have any options in the pipeline commands to rename or prefix the deliverables. I don't know the programming language groovy that the pipeline use, so I will not write code to handle it. Eventually I avoided the problem by delegating the responsibility of prefixing the .deb file to the script that builds it. It's simple, reliable, but not really what I want. It will do for now.

Another problem is that apt-get has changed behavior, so our Dockerfile for Debian Testing must handle that. The -y parameter must now be given before the apt-get command to execute.

The modified files to build on 3 Linux distributions

ci/jenkins/Jenkinsfile

#!/usr/bin/env groovy

pipeline {
    agent { label 'master' }

    environment {
        WHID_VERSION = "2.0.1"
    }

    stages {
        stage('Build') {
           parallel {
                stage('Docker-build inside: ubuntu-xenial') {
                    agent {
                        dockerfile {
                            filename 'Dockefile.ubuntu-xenial'
                            dir 'ci/jenkins'
                            label 'master'
                        }
                    }

                    environment {
                        DIST_DIR = "${WORKSPACE}/dist"
                        BUILD_DIR = "${WORKSPACE}/build"
                        SRC_DIR = "${WORKSPACE}"
                        DIST_NAME = 'ubuntu-xenial-'
                    }

                    steps {
                        echo "Building on ubuntu-xenial-AMD64 in ${WORKSPACE}"
                        checkout scm
                        sh 'pwd; ls -la;'
                        sh './scripts/package-deb.sh'
                    }

                    post {
                        success {
                            echo "Build of debian package suceeded!"
                            archive "dist/*.deb"
                        }
                    }
                }
                stage('Docker-build inside: debian-stretch') {
                    agent {
                        dockerfile {
                            filename 'Dockefile.debian-stretch'
                            dir 'ci/jenkins'
                            label 'master'
                        }
                    }

                    environment {
                        DIST_DIR = "${WORKSPACE}/dist"
                        BUILD_DIR = "${WORKSPACE}/build"
                        SRC_DIR = "${WORKSPACE}"
                        DIST_NAME = 'debian-stretch-'
                    }

                    steps {
                        echo "Building on debian-stretch-AMD64 in ${WORKSPACE}"
                        checkout scm
                        sh 'pwd; ls -la;'
                        sh './scripts/package-deb.sh'
                    }

                    post {
                        success {
                            echo "Build of debian package suceeded!"
                            archive "dist/*.deb"
                        }
                    }
                }
                stage('Docker-build inside: debian-testing') {
                    agent {
                        dockerfile {
                            filename 'Dockefile.debian-testing'
                            dir 'ci/jenkins'
                            label 'master'
                        }
                    }

                    environment {
                        DIST_DIR = "${WORKSPACE}/dist"
                        BUILD_DIR = "${WORKSPACE}/build"
                        SRC_DIR = "${WORKSPACE}"
                        DIST_NAME = 'debian-testing-'
                    }

                    steps {
                        echo "Building on debian-testing-AMD64 in ${WORKSPACE}"
                        checkout scm
                        sh 'pwd; ls -la;'
                        sh './scripts/package-deb.sh'
                    }

                    post {
                        success {
                            echo "Build of debian package suceeded!"
                            archive "dist/*.deb"
                        }
                    }
                }
            }
        }
    }
}

Here is a bit repeated code here, so we might want to change from declarative style to programming with a loop in the future.

ci/jenkins/Dockefile.ubuntu-xenial

# Based on: https://github.com/evarga/docker-images/blob/master/jenkins-slave/Dockerfile
FROM ubuntu:xenial

MAINTAINER Jarle Aase <jgaa@jgaa.com>

# In case you need proxy
#RUN echo 'Acquire::http::Proxy "http://127.0.0.1:8080";' >> /etc/apt/apt.conf

RUN apt-get -q update &&\
    DEBIAN_FRONTEND="noninteractive" apt-get -q upgrade -y -o Dpkg::Options::="--force-confnew" --no-install-recommends &&\
    DEBIAN_FRONTEND="noninteractive" apt-get -q install -y -o Dpkg::Options::="--force-confnew" --no-install-recommends openssh-server &&\
    DEBIAN_FRONTEND="noninteractive" apt-get -q install -y g++ git make \
    qtdeclarative5-dev  qt5-default ruby ruby-dev rubygems build-essential &&\
    gem install --no-ri --no-rdoc fpm &&\
    apt-get -q autoremove &&\
    apt-get -q clean -y && rm -rf /var/lib/apt/lists/* && rm -f /var/cache/apt/*.bin &&\
    sed -i 's|session    required     pam_loginuid.so|session    optional     pam_loginuid.so|g' /etc/pam.d/sshd &&\
    mkdir -p /var/run/sshd

# Install JDK 8 (latest edition)
RUN apt-get -q update &&\
    DEBIAN_FRONTEND="noninteractive" apt-get -q install -y -o Dpkg::Options::="--force-confnew" --no-install-recommends software-properties-common &&\
    add-apt-repository -y ppa:openjdk-r/ppa &&\
    apt-get -q update &&\
    DEBIAN_FRONTEND="noninteractive" apt-get -q install -y -o Dpkg::Options::="--force-confnew" --no-install-recommends openjdk-8-jre-headless &&\
    apt-get -q clean -y && rm -rf /var/lib/apt/lists/* && rm -f /var/cache/apt/*.bin

# Set user jenkins to the image
RUN useradd -m -d /home/jenkins -s /bin/sh jenkins &&\
    echo "jenkins:jenkins" | chpasswd

# Standard SSH port
EXPOSE 22

# Default command
CMD ["/usr/sbin/sshd", "-D"]

ci/jenkins/Dockefile.debian-stretch

# Based on: https://github.com/evarga/docker-images/blob/master/jenkins-slave/Dockerfile
FROM debian:stretch

MAINTAINER Jarle Aase <jgaa@jgaa.com>

# In case you need proxy
#RUN echo 'Acquire::http::Proxy "http://127.0.0.1:8080";' >> /etc/apt/apt.conf

RUN apt-get -q update &&\
    DEBIAN_FRONTEND="noninteractive" apt-get -q upgrade -y -o Dpkg::Options::="--force-confnew" --no-install-recommends &&\
    DEBIAN_FRONTEND="noninteractive" apt-get -q install -y -o Dpkg::Options::="--force-confnew" --no-install-recommends openssh-server &&\
    DEBIAN_FRONTEND="noninteractive" apt-get -q install -y g++ git make \
    qtdeclarative5-dev  qt5-default ruby ruby-dev rubygems build-essential \
    openjdk-8-jdk &&\
    gem install --no-ri --no-rdoc fpm &&\
    apt-get -q autoremove &&\
    apt-get -q clean -y && rm -rf /var/lib/apt/lists/* && rm -f /var/cache/apt/*.bin &&\
    sed -i 's|session    required     pam_loginuid.so|session    optional     pam_loginuid.so|g' /etc/pam.d/sshd &&\
    mkdir -p /var/run/sshd


# Set user jenkins to the image
RUN useradd -m -d /home/jenkins -s /bin/sh jenkins &&\
    echo "jenkins:jenkins" | chpasswd

# Standard SSH port
EXPOSE 22

# Default command
CMD ["/usr/sbin/sshd", "-D"]

ci/jenkins/Dockefile.debian-testing

# Based on: https://github.com/evarga/docker-images/blob/master/jenkins-slave/Dockerfile
FROM debian:stretch

MAINTAINER Jarle Aase <jgaa@jgaa.com>

# In case you need proxy
#RUN echo 'Acquire::http::Proxy "http://127.0.0.1:8080";' >> /etc/apt/apt.conf

RUN apt-get -q update &&\
    DEBIAN_FRONTEND="noninteractive" apt-get -q upgrade -y -o Dpkg::Options::="--force-confnew" --no-install-recommends &&\
    DEBIAN_FRONTEND="noninteractive" apt-get -q install -y -o Dpkg::Options::="--force-confnew" --no-install-recommends openssh-server &&\
    DEBIAN_FRONTEND="noninteractive" apt-get -q install -y g++ git make \
    qtdeclarative5-dev  qt5-default ruby ruby-dev rubygems build-essential \
    openjdk-8-jdk &&\
    gem install --no-ri --no-rdoc fpm &&\
    apt-get -q autoremove &&\
    apt-get -q clean -y && rm -rf /var/lib/apt/lists/* && rm -f /var/cache/apt/*.bin &&\
    sed -i 's|session    required     pam_loginuid.so|session    optional     pam_loginuid.so|g' /etc/pam.d/sshd &&\
    mkdir -p /var/run/sshd


# Set user jenkins to the image
RUN useradd -m -d /home/jenkins -s /bin/sh jenkins &&\
    echo "jenkins:jenkins" | chpasswd

# Standard SSH port
EXPOSE 22

# Default command
CMD ["/usr/sbin/sshd", "-D"]

scripts/package-deb.sh

#!/bin/bash

# Compile and prepare a .deb package for distribution
# Uses the QT libraries from the lilnux ditribution
#
# Example:
#   WHID_VERSION="2.0.1" DIST_DIR=`pwd`/dist BUILD_DIR=`pwd`/build SRC_DIR=`pwd`/whid  ./whid/scripts/package-deb.sh

if [ -z "$WHID_VERSION" ]; then
    WHID_VERSION="2.0.0"
    echo "Warning: Missing WHID_VERSION variable!"
fi

if [ -z ${DIST_DIR:-} ]; then
    DIST_DIR=`pwd`/dist/linux
fi

if [ -z ${BUILD_DIR:-} ]; then
    BUILD_DIR=`pwd`/build
fi

if [ -z ${SRC_DIR:-} ]; then
# Just assume we are run from the scipts directory
    SRC_DIR=`pwd`/..
fi

echo "Building whid for linux into ${DIST_DIR} from ${SRC_DIR}"

rm -rf $DIST_DIR $BUILD_DIR

mkdir -p $DIST_DIR &&\
pushd $DIST_DIR &&\
mkdir -p $BUILD_DIR &&\
pushd $BUILD_DIR &&\
qmake $SRC_DIR/whid.pro &&\
make && make install &&\
popd &&\
fpm --input-type dir \
    --output-type deb \
    --force \
    --name whid \
    --version ${WHID_VERSION} \
    --vendor "The Last Viking LTD" \
    --description "Time Tracking for Freelancers and Independenet Contractors" \
    --depends qt5-default --depends libsqqlite3 \
    --chdir ${DIST_DIR}/root/ \
    --package ${DIST_NAME}whid-VERSION_ARCH.deb &&\
echo "Debian package is available in $PWD" &&\
popd

After the build, Jenkins now shows 3 .deb files.

Jenkins UI with 3 .deb packages

Take 3: Adding macOS

Building QT applications on macOS is simple when we have it working for Linux.

I used a similar script, and after figuring out how to build and sign a dmg package, the build process is simple.

For this build I used a Jenkins slave running on a Mac Mini. Basically I created a slave in the Jenkins user interface, selected Webstart, and then started the java-command to connect the slave in a shell on that machine. It the keeps a connection to the Jenkins server as long as that script is running.

Take 4: Adding Windows.

This turned out to be a challenge.

Building a Windows .exe file is trivial with QT, and they have a utility, windeployqt that bundles all the required QT libraries (and sqlite) into a directory structure. It works beautifully, except, it's useless for ordinary windows users. They expect an installer that will copy the application to an appropriate location, a start menu entry to magically appear, and the ability to uninstall the application from "Add or Remove Programs".

There are people who have solved this problem before. My goto solution for a scriptable installer is the WiX tooset, released as open source by Microsoft. This however require that you carefully describes each individual file in your distribution in an xml file. Since windeployqt copies a lot of files, I felt no strong desire to add them each, manually, to a xml file, and the see my build broken each time QT is updated. In stead, I wrote a front-end to WiX as a separate project on github, mkmsi. Now, all I needed was to tell mkmsi where the files were, and it would make me a nice installer package that people can download and install on their windows PC's and laptops. I tested it with Windows Vista SP2, Windows 7 and Windows 10. All 64 bit versions.

At the moment I don't sign the Windows package, as it makes no sense, what so ever, from a security perspective. (All the bad guys already have stolen code signing certificates, so signing a package proves nothing. It's just a way to throw money out at the Windows). That means that Windows Installer at the client machines will try to prevent users from figuring out how to install the application by being silly and confusing and hiding the install button from clear sight. I don't really understand why Microsoft hate their customers so much...

Anyway. At this point we have a pipeline that will build installable packages for 3 flavors of Linux, macOS and Windows! That means that I can improve the application and provide new binaries in a convenient and manageable way.

I could have cross-compiled for Windows from Linux, but it was simpler to just start a VM with Windows on my server (especially since we also want to run WiX), and run a Jenkins slave there. That VM has Java, Visual Studio 2017, QT 5.10 and the WiX toolkit installed.

Our final files

Our final files looks like:

ci/jenkins/Jenkinsfile

#!/usr/bin/env groovy

pipeline {
    agent { label 'master' }

    environment {
        WHID_VERSION = "2.0.2"
    }

    stages {
        stage('Build') {
           parallel {
                stage('Docker-build inside: ubuntu-xenial') {
                    agent {
                        dockerfile {
                            filename 'Dockefile.ubuntu-xenial'
                            dir 'ci/jenkins'
                            label 'master'
                        }
                    }

                    environment {
                        DIST_DIR = "${WORKSPACE}/dist"
                        BUILD_DIR = "${WORKSPACE}/build"
                        SRC_DIR = "${WORKSPACE}"
                        DIST_NAME = 'ubuntu-xenial-'
                    }

                    steps {
                        echo "Building on ubuntu-xenial-AMD64 in ${WORKSPACE}"
                        checkout scm
                        sh 'pwd; ls -la;'
                        sh './scripts/package-deb.sh'
                    }

                    post {
                        success {
                            echo "Build of debian package suceeded!"
                            archive "dist/*.deb"
                        }
                    }
                }
                stage('Docker-build inside: debian-stretch') {
                    agent {
                        dockerfile {
                            filename 'Dockefile.debian-stretch'
                            dir 'ci/jenkins'
                            label 'master'
                        }
                    }

                    environment {
                        DIST_DIR = "${WORKSPACE}/dist"
                        BUILD_DIR = "${WORKSPACE}/build"
                        SRC_DIR = "${WORKSPACE}"
                        DIST_NAME = 'debian-stretch-'
                    }

                    steps {
                        echo "Building on debian-stretch-AMD64 in ${WORKSPACE}"
                        checkout scm
                        sh 'pwd; ls -la;'
                        sh './scripts/package-deb.sh'
                    }

                    post {
                        success {
                            echo "Build of debian package suceeded!"
                            archive "dist/*.deb"
                        }
                    }
                }
                stage('Docker-build inside: debian-testing') {
                    agent {
                        dockerfile {
                            filename 'Dockefile.debian-testing'
                            dir 'ci/jenkins'
                            label 'master'
                        }
                    }

                    environment {
                        DIST_DIR = "${WORKSPACE}/dist"
                        BUILD_DIR = "${WORKSPACE}/build"
                        SRC_DIR = "${WORKSPACE}"
                        DIST_NAME = 'debian-testing-'
                    }

                    steps {
                        echo "Building on debian-testing-AMD64 in ${WORKSPACE}"
                        checkout scm
                        sh 'pwd; ls -la;'
                        sh './scripts/package-deb.sh'
                    }

                    post {
                        success {
                            echo "Build of debian package suceeded!"
                            archive "dist/*.deb"
                        }
                    }
                }
                stage('Build on macOS') {
                    agent {label 'macos'}

                    environment {
                        DIST_DIR = "${WORKSPACE}/dist"
                        BUILD_DIR = "${WORKSPACE}/build"
                        SRC_DIR = "${WORKSPACE}"
                        QTDIR="/opt/Qt/5.10.0/clang_64"
                    }

                    steps {
                        echo "Building on macOS in ${WORKSPACE}"
                        checkout scm
                        sh 'pwd; ls -la;'
                        sh './scripts/package-macos.sh'
                    }

                    post {
                        success {
                            echo "Build of macOS package succeeded!"
                            archive "dist/*.dmg"
                        }
                    }
                }
                stage('Build on Windows') {
                    agent {label 'windows'}

                    environment {
                        QTDIR="C:\\Qt\\5.10.0\\msvc2017_64"
                    }

                    // The ${WORKSPACE} has the wrong slashes for Windows,
                    // so we will use %cd% to get a path to the workspace
                    // in a format Windows can work with. From that we will
                    // make the other paths.
                    //
                    steps {
                        echo "Building on Windows in ${WORKSPACE}"
                        checkout scm

                        bat script: '''
                        set SRC_DIR=%cd%
                        set DIST_DIR=%SRC_DIR%\\dist
                        set BUILD_DIR=%SRC_DIR%\\build
                        set MSI_TARGET_DIR=C:\\jenkins\\build\\whid-x64
                        call "C:\\Program Files (x86)\\Microsoft Visual Studio\\2017\\Community\\VC\\Auxiliary\\Build\\vcvars64.bat"
                        cd
                        cd %SRC_DIR%
                        dir
                        call scripts\\package-windows.bat
                        if %errorlevel% neq 0 exit /b %errorlevel%
                        copy "%SRC_DIR%\\res\\icons\\whid.ico" "%DIST_DIR%\\whid"
                        cd %MSI_TARGET_DIR%
                        del whid.msi
                        C:\\devel\\mkmsi\\mkmsi.py --auto-create qt --source-dir "%DIST_DIR%\\whid" --wix-root "C:\\Program Files (x86)\\WiX Toolset v3.11" --license C:\\devel\\mkmsi\\licenses\\GPL3.rtf --merge-module "C:\\Program Files (x86)\\Common Files\\Merge Modules\\Microsoft_VC140_CRT_x64.msm" --add-desktop-shortcut --project-version %WHID_VERSION% --description "Time Tracking for Freelancers and Independent Contractors" --manufacturer "The Last Viking LTD" whid
                        if %errorlevel% neq 0 exit /b %errorlevel%
                        copy whid.msi %DIST_DIR%\\whid-%WHID_VERSION%-x64.msi
                        if %errorlevel% neq 0 exit /b %errorlevel%
                        copy whid.json %DIST_DIR%\\whid-%WHID_VERSION%-x64.json
                        if %errorlevel% neq 0 exit /b %errorlevel%
                        echo "Everything is OK"
                        '''
                    }

                    post {
                        success {
                            echo "Build of Windows package succeeded!"
                            archive "dist/*.msi"
                            archive "dist/*.json"
                        }
                    }
                }
            }
        }
    }
}

ci/jenkins/Dockefile.ubuntu-xenial

# Based on: https://github.com/evarga/docker-images/blob/master/jenkins-slave/Dockerfile
FROM ubuntu:xenial

MAINTAINER Jarle Aase <jgaa@jgaa.com>

# In case you need proxy
#RUN echo 'Acquire::http::Proxy "http://127.0.0.1:8080";' >> /etc/apt/apt.conf

RUN apt-get -q update &&\
    DEBIAN_FRONTEND="noninteractive" apt-get -q upgrade -y -o Dpkg::Options::="--force-confnew" --no-install-recommends &&\
    DEBIAN_FRONTEND="noninteractive" apt-get -q install -y -o Dpkg::Options::="--force-confnew" --no-install-recommends openssh-server &&\
    DEBIAN_FRONTEND="noninteractive" apt-get -q install -y g++ git make \
    qtdeclarative5-dev  qt5-default ruby ruby-dev rubygems build-essential &&\
    gem install --no-ri --no-rdoc fpm &&\
    apt-get -q autoremove &&\
    apt-get -q clean -y && rm -rf /var/lib/apt/lists/* && rm -f /var/cache/apt/*.bin &&\
    sed -i 's|session    required     pam_loginuid.so|session    optional     pam_loginuid.so|g' /etc/pam.d/sshd &&\
    mkdir -p /var/run/sshd

# Install JDK 8 (latest edition)
RUN apt-get -q update &&\
    DEBIAN_FRONTEND="noninteractive" apt-get -q install -y -o Dpkg::Options::="--force-confnew" --no-install-recommends software-properties-common &&\
    add-apt-repository -y ppa:openjdk-r/ppa &&\
    apt-get -q update &&\
    DEBIAN_FRONTEND="noninteractive" apt-get -q install -y -o Dpkg::Options::="--force-confnew" --no-install-recommends openjdk-8-jre-headless &&\
    apt-get -q clean -y && rm -rf /var/lib/apt/lists/* && rm -f /var/cache/apt/*.bin

# Set user jenkins to the image
RUN useradd -m -d /home/jenkins -s /bin/sh jenkins &&\
    echo "jenkins:jenkins" | chpasswd

# Standard SSH port
EXPOSE 22

# Default command
CMD ["/usr/sbin/sshd", "-D"]

ci/jenkins/Dockefile.debian-stretch

# Based on: https://github.com/evarga/docker-images/blob/master/jenkins-slave/Dockerfile
FROM debian:stretch

MAINTAINER Jarle Aase <jgaa@jgaa.com>

# In case you need proxy
#RUN echo 'Acquire::http::Proxy "http://127.0.0.1:8080";' >> /etc/apt/apt.conf

RUN apt-get -q update &&\
    DEBIAN_FRONTEND="noninteractive" apt-get -q upgrade -y -o Dpkg::Options::="--force-confnew" --no-install-recommends &&\
    DEBIAN_FRONTEND="noninteractive" apt-get -q install -y -o Dpkg::Options::="--force-confnew" --no-install-recommends openssh-server &&\
    DEBIAN_FRONTEND="noninteractive" apt-get -q install -y g++ git make \
    qtdeclarative5-dev  qt5-default ruby ruby-dev rubygems build-essential \
    openjdk-8-jdk &&\
    gem install --no-ri --no-rdoc fpm &&\
    apt-get -q autoremove &&\
    apt-get -q clean -y && rm -rf /var/lib/apt/lists/* && rm -f /var/cache/apt/*.bin &&\
    sed -i 's|session    required     pam_loginuid.so|session    optional     pam_loginuid.so|g' /etc/pam.d/sshd &&\
    mkdir -p /var/run/sshd


# Set user jenkins to the image
RUN useradd -m -d /home/jenkins -s /bin/sh jenkins &&\
    echo "jenkins:jenkins" | chpasswd

# Standard SSH port
EXPOSE 22

# Default command
CMD ["/usr/sbin/sshd", "-D"]

ci/jenkins/Dockefile.debian-testing

FROM debian:testing

MAINTAINER Jarle Aase <jgaa@jgaa.com>

# In case you need proxy
#RUN echo 'Acquire::http::Proxy "http://127.0.0.1:8080";' >> /etc/apt/apt.conf

RUN apt-get -q update &&\
    apt-get -y -q --no-install-recommends upgrade &&\
    apt-get -y -q --no-install-recommends install openssh-server g++ git make \
    qtdeclarative5-dev  qt5-default ruby ruby-dev rubygems build-essential \
    openjdk-8-jdk &&\
    gem install --no-ri --no-rdoc fpm &&\
    apt-get -y -q autoremove &&\
    apt-get -y -q clean

# Set user jenkins to the image
RUN useradd -m -d /home/jenkins -s /bin/sh jenkins &&\
    echo "jenkins:jenkins" | chpasswd

# Standard SSH port
EXPOSE 22

# Default command
CMD ["/usr/sbin/sshd", "-D"]

scripts/package-deb.sh

 1#!/bin/bash
 2
 3# Compile and prepare a .deb package for distribution
 4# Uses the QT libraries from the lilnux ditribution
 5#
 6# Example:
 7#   WHID_VERSION="2.0.1" DIST_DIR=`pwd`/dist BUILD_DIR=`pwd`/build SRC_DIR=`pwd`/whid  ./whid/scripts/package-deb.sh
 8
 9if [ -z "$WHID_VERSION" ]; then
10    WHID_VERSION="2.0.0"
11    echo "Warning: Missing WHID_VERSION variable!"
12fi
13
14if [ -z ${DIST_DIR:-} ]; then
15    DIST_DIR=`pwd`/dist/linux
16fi
17
18if [ -z ${BUILD_DIR:-} ]; then
19    BUILD_DIR=`pwd`/build
20fi
21
22if [ -z ${SRC_DIR:-} ]; then
23# Just assume we are run from the scipts directory
24    SRC_DIR=`pwd`/..
25fi
26
27echo "Building whid for linux into ${DIST_DIR} from ${SRC_DIR}"
28
29rm -rf $DIST_DIR $BUILD_DIR
30
31mkdir -p $DIST_DIR &&\
32pushd $DIST_DIR &&\
33mkdir -p $BUILD_DIR &&\
34pushd $BUILD_DIR &&\
35qmake $SRC_DIR/whid.pro &&\
36make && make install &&\
37popd &&\
38fpm --input-type dir \
39    --output-type deb \
40    --force \
41    --name whid \
42    --version ${WHID_VERSION} \
43    --vendor "The Last Viking LTD" \
44    --description "Time Tracking for Freelancers and Independenet Contractors" \
45    --depends qt5-default --depends libsqlite3-0 \
46    --chdir ${DIST_DIR}/root/ \
47    --package ${DIST_NAME}whid-VERSION_ARCH.deb &&\
48echo "Debian package is available in $PWD" &&\
49popd

scripts/package-macos.sh

 1#!/bin/bash
 2
 3# Compile and prepare a signed .dmg package for distribution
 4# Assimes the environment-variable QTDIR to point to the
 5# Qt installation
 6#
 7# Example:
 8#     Jarles-Mac-mini:scripts jgaa$ QTDIR=/Users/jgaa/Qt/5.10.0/clang_64 ./package-macos.sh
 9
10if [ -z "$WHID_VERSION" ]; then
11    WHID_VERSION="2.0.0"
12    echo "Warning: Missing WHID_VERSION variable!"
13fi
14
15if [ -z ${DIST_DIR:-} ];
16then
17    DIST_DIR=`pwd`/dist/macos
18fi
19
20if [ -z ${SIGN_CERT:-} ];
21then
22    SIGN_CERT="Developer ID Application"
23fi
24
25if [ -z ${BUILD_DIR:-} ]; then
26    BUILD_DIR=`pwd`/build
27fi
28
29if [ -z ${SRC_DIR:-} ];
30then
31# Just assume we are run from the scipts directory
32    SRC_DIR=`pwd`/..
33fi
34
35echo "Building whid for macos into ${DIST_DIR} from ${SRC_DIR}"
36
37rm -rf $DIST_DIR $BUILD_DIR
38
39mkdir -p $DIST_DIR && cd $DIST_DIR
40mkdir -p $BUILD_DIR
41
42pushd $BUILD_DIR
43
44$QTDIR/bin/qmake \
45    -spec macx-clang \
46    "CONFIG += release x86_64" \
47    $SRC_DIR/whid.pro
48
49make -j4
50
51popd
52
53pushd $DIST_DIR
54
55mv $BUILD_DIR/whid.app $BUILD_DIR/whid-${WHID_VERSION}.app
56
57echo "Making dmg package with $QTDIR/bin/macdeployqt"
58$QTDIR/bin/macdeployqt $BUILD_DIR/whid-${WHID_VERSION}.app -dmg -appstore-compliant -codesign="$SIGN_CERT"
59
60mv  $BUILD_DIR/whid-${WHID_VERSION}.dmg .
61
62popd

scripts/package-windows.bat

rem On my machine, I execute the build script from a script with these commands:
rem
rem SET QTDIR=C:\Qt\5.10.0\msvc2017_64
rem call "C:\Program Files (x86)\Microsoft Visual Studio 14.0\VC\bin\amd64\vcvars64.bat"
rem PATH=%PATH%;C:\Program Files (x86)\Windows Kits\10\bin\10.0.16299.0\x64
rem cd "C:\Users\Jarle Aase\src\whid\scripts"
rem call .\package-windows

echo on

IF NOT DEFINED DIST_DIR (set DIST_DIR=%cd%\dist\windows)
IF NOT DEFINED BUILD_DIR (set BUILD_DIR=%DIST_DIR%\build)
IF NOT DEFINED SRC_DIR (set SRC_DIR=%cd%\..)
IF NOT DEFINED OUT_DIR (set OUT_DIR=%DIST_DIR%\whid)

rmdir /S /Q "%DIST_DIR%"
mkdir "%DIST_DIR%"
mkdir "%BUILD_DIR%"
mkdir "%OUT_DIR%"

pushd "%BUILD_DIR%"

%QTDIR%\bin\qmake.exe ^
  -spec win32-msvc ^
  "CONFIG += release" ^
  "%SRC_DIR%\whid.pro"

nmake

popd

echo "Copying: %BUILD_DIR%\release\whid.exe" "%OUT_DIR%"
copy "%BUILD_DIR%\release\whid.exe" "%OUT_DIR%"
copy "%SRC_DIR%\res\icons\whid.ico" "%OUT_DIR%"

%QTDIR%\bin\windeployqt "%OUT_DIR%\whid.exe"

echo "The prepared package is in: "%OUT_DIR%"

After the build, Jenkins now shows 5 deliverables :)

(The json file is a backup of the configuration-file for mkmsi, in case I need to re-deploy the Windows slave. It has some uiid's required to handle upgrades to future versions of whid).

Screenshot, 5 deliverables

These files are also available at github in the whid repository.

The Last Viking's Nest

X-platform build of a QT Application using a Jenkins pipeline

Jenkins

Jenkins Pipelines

Jenkins Plugins

Take 1: Set up the build with just one target.

Jenkins files in my source code repository

Take 2: Set up the build with several, similar Linux builds

The modified files to build on 3 Linux distributions

Take 3: Adding macOS

Take 4: Adding Windows.

Our final files

Tags

Comments