The move of KDE’s continuous integration infrastructure from Jenkins to Gitlab gave us pre-integration compilation and license checks, and now we can also require passing unit tests prior to integrating merge requests.

Configuration

In its default configuration, KDE’s Gitlab CI will only consider compilation or license validation failures a failure of the CI job, which follows what the previous Jenkins-based system did. Unit test failures would be shown in the results of a job, but would not be considered fatal and stop integration.

In projects with reliable and maintained tests this is obviously not ideal, and therefore this can now be changed. The require-passing-tests-on setting controls on which platform(s) tests need to pass for the CI to pass. Making this a per-platform setting allows phasing this in gradually, without the least well supported platform holding everything back.

The global default for this is an empty list for now (ie. no behavior change), but it can be configured per repository in the .kde-ci.yml file:

Options:
  require-passing-tests-on: [ 'Linux', 'FreeBSD', 'Windows' ]

Platforms

Currently the CI serves four platforms:

  • Linux
  • FreeBSD
  • Android
  • Windows

Since we cross-compile for Android we cannot run unit tests there at this point, and it thus is pointless to add that to the require-passing-tests-on setting.

Windows support is brand new and is only just now being rolled out to the repositories supporting Windows builds, with unit tests often still needing more work to pass there.

An important point is that we cannot distinguish between Qt5 and Qt6 based builds here, the platform setting always covers all Qt versions. This limitation is temporary though, once we branch for 6 the need to support both major versions in the same code base goes away.

Contribute

There’s an easy way to help here, check the CI state of your favorite repository and if the unit tests pass reliably there mark them as required to keep it that way!