Keyboard shortcuts

Press ← or → to navigate between chapters

Press S or / to search in the book

Press ? to show this help

Press Esc to hide this help

Travis-CI

Travis-CI

What is Continuous Integration (CI)?

  • How to get feedback to your project?

  • How often do you compile your code and run your tests?

  • How often do you integrate your code with the other team members?

  • The basis of all learning and all improvement is getting feedback as soon as possible.

  • Hence introducing Continuous Integration is one of the best ways to improve the quality of your project.

What is the status of CI in companies?

  • Report from March 2018

  • Only 45% of developers in organizations with five employees or less are using continuous integration.

  • Organizations with over 1,000 employees, 68% of developers report using continuous integration.

Why use Travis-CI?

  • Cloud based Continuous Integration system
  • Well integrated with GitHub
  • Free for Open Source projects
  • Open Source
  • Private repos

Empty Git repository for demo

To get started we are going to use a Git repository that is almost empty. It only has a README file in. We need to visit GitHub with our browse to create the repository at first and then we can decide if we'd like to contnue use the GitHub UI or if primarily we'd like to use our editor and git client on our computer.

  • Use the GitHub UI
  • Use a Git client on your computer

Create repository directly on GitHub

In either case first on GitHub create a project called travis-demo and make sure you check the box next to "Initialize this repository with a README" and then clone the repos.

  • If you'd like to use your editor and git client then clone the project:
git clone git@github.com:szabgab/travis-demo.git
  • I assume you know how to edit files, commit and push them out.

YAML

Minimal Travis-CI

Minimal configuration

Setting up Travis-CI is very easy you only need to follow the following steps:

  • Connect your GitHub account with Travis-CI
  • If the accounts were already connected and this is a new repo you might need to tell Travis-CI to sync from GitHub
  • Enable Travis for the specific repository
  • Add the .travis.yml to the repository (see the minimal file below)
  • Push out your changes
language: minimal

This will trigger the build on Travis-CI. As it has nothing to do, it will pass.

For any interesting project you will set the language field properly, but for now we do this without any code and this we tell Travis to set up a minimal virtual environment for us.

Minimal Travis-CI echo

The first thing we can do is specify some code to run in the environment. For this we need to add the script field to the YAML file with a command. A simple one would be just to echo some text.

If we make this change in the GitHub editor then immediately after you commit the change it will trigger a new build on Travis-CI. If you edit the file locally on your computer, then you need to commit the change and push it out to GitHub. In this case the push operation will trigger the build on Travis-CI.

language: minimal
script: echo Hello World

Minimal Travis-CI exit 1 failure

In the previous example our script was a simple echo that was successful. Now let's see what would happen if the script exited with an exit code different from 0. For this we replace our script with a simple exit . After this triggers the build you'll probably get an e-mail reporting that the build failed. You can also look at the UI in Travis-CI to see the failure.

language: minimal
script: exit 1

Minimal Travis-CI installations

Even though the machine this runs on has a minimal installation it already has quite a few tools that you can use. Instead of having a single value for the script key we convert it to a list of commands. Travis executes all the other commands, but if any of these fails (has an exit code different from 0) then the whole process is reported as failure.

If they are all successful the it is reported as success.

You can see the detailed results in the console output.

language: minimal
script:
  - echo Hello World
  - python -V
  - python3 -V
  - ruby -v
  - perl -v
  - docker -v
  - git --version
  - lsb_release -a
  - uname -a

Release:	16.04
Codename:	xenial

Minimal Travis-CI installations - run shell script

language: minimal
script: ./run.sh

before_script: chmod +x run.sh

{% embed include file="src/examples/minimal-shell-script/run.sh)

Minimal Travis-CI installations setting the dist

  • dist
  • Linux
  • Ubuntu
  • trusty
  • xenial
  • bionic
  • focal
dist: trusty
language: minimal
script:
  - echo Hello World
  - python -V
  - python3 -V
  - ruby -v
  - perl -v
  - docker -v
  - git --version
  - lsb_release -a
  - uname -a
  • dist: trusty
Release:	14.04
Codename:	trusty
  • dist: bionic
Release:	18.04
Codename:	bionic
  • dist: focal
Release:	20.04
Codename:	focal

Build environments

Travis-CI on OSX

Travis allows the use of Mac OSX for your tests. In order to do that you need to include os: osx in the configuration file. (os defaults to linux).

Setting dist: osx is a mistake. It will mean no build is triggered. Unfortunately I did not even see an error message from Travis telling me that my configuration is incorrect.

  • Setting os: osx

  • This image has different default tools. For example it does not have docker, nor lsb_release.

os: osx
language: minimal
script: ./run.sh

before_script: chmod +x run.sh

{% embed include file="src/examples/minimal-osx/run.sh)

Build environments

Travis-CI on MS Windows

  • OSX

  • Experimentaly support

  • Run in a shell so we can run shell commands

os: windows
language: minimal
script:
  - echo Hello World
  - python -V
  - ruby -v
  - perl -v
  - git --version
  - uname -a

Build environments

The UI of Travis-CI

  • Travis-CI
  • Current
  • Branches
  • Build History
  • Pull Requests

Travis-CI scheduled cron jobs

  • Travis-CI
  • More options / Settings
  • Cron Jobs
  • Monthly/Weekly/Daily
TRAVIS_EVENT_TYPE=cron

instead of the usual push.

Trigger a custom build

  • Can be good for experimentation with Travis without he need to make commits to the repository

  • More Options / Trigger Build

  • Set a commit message

  • copy (or type in) the content of a YAML file, it will be merged into the regular .travis.yml

  • (e.g. add more versions of the language)

Setting

TRAVIS_EVENT_TYPE=api

instead of the usual push.

Travis-CI and languages

Build status

  • passing (success)
  • failing (the script exited with non 0 exit code)
  • error (Some other step failed = exited with non 0 exit code)

Add badge

# travis-demo

[![Build Status](https://travis-ci.org/szabgab/travis-demo.png)](https://travis-ci.org/szabgab/travis-demo)

Job Lifecycle

  • Defaults for each language
language: minimal

before_install:
    - echo "Before install"

install:
    - echo "Install"

before_script:
    - echo "Before the script phase"

script:
    - echo "The main task"

after_script:
    - echo "After the script phase."
    - echo $TRAVIS_TEST_RESULT

after_success:
    - echo "After success"

after_failure:
    - echo "After failure"

$ echo "Before install"
$ echo "Install"
$ echo "Before the script phase"
$ echo "The main task"
$ echo "After success"
$ echo "After the script phase."
  • Add a - ls qqrq to the steps to see how we get the "After failure" message.

OS Matrix

language: minimal
script: uname -a
os:
   - osx
   - windows
   - linux

true as no-operation to skip a step

  • true

Sometimes, for some languages, there is some default behavior for the index and the script step but you'd like to skip that operation. You can put true as the command to be executed. A a shell command the only thing it does is exit with success.

So you could write:

install: true

Languages

Install travis-cli command line tool

sudo apt-get install ruby
gem install --user-install travis

Add the following to ~/.bash_profile

export PATH=~/.gem/ruby/2.7.0/bin/:$PATH

notifications email secure - sending to multiple addresses

travis encrypt "joe@example.com" --add notifications.email.recipients

This will want to rewrite and reformat yout .travis.yml. If you are ok with it then let it do it. If you don't want it to change your config file, then instead of that you might copy the new section it offers to add to your file and then you can add it manually. It will look something like this:

notifications:
  email:
    recipients:
      secure: c6AsRDQshSjrw+JIlaH7XbusT2zqVMAdhbFTWFogKc4LY9ytK2K2i2FVYp015p/14SRSK8HBYOpXJ3uy+vGBS2Eyovq0WSnTki7MLpx1GXhPOUyuiiLhtgHiTRzTK/3BdTlIOc9uKnw7Urw=
  • Add another email address:
travis encrypt "jane@example.com" --append --add notifications.email.recipients

The result will look like this:

notifications:
  email:
    recipients:
      - secure: c6AsRDQshSjrw+JIlaH7XbusT2zqVMAdhbFTWFogKc4LY9ytK2K2i2FVYp015p/14SRSK8HBYOpXJ3uy+vGBS2Eyovq0WSnTki7MLpx1GXhPOUyuiiLhtgHiTRzTK/3BdTlIOc9uKnw7Urw=
      - secure: AsAAlsjfkshkSADQshSjrw+JIlaH7XbusT2zqVMAdhbFTWFogKc4LY9ytK2K2i2FVYp015p/14SRSK8HBYOpXJ3uy+vGBS2Eyovq0WSnTki7MLpx1GXhPOUyuiiLhtgHiTRzTK/3BdTlSFF=

It seems this will work without the "recipients" key as well but then only with one email address, even if you supply two.

environment variables in reposiotry settings

Deployment

Python

Travis-CI and Python

  • Travis-CI and Python

  • language: python

  • pip install -r requirements.txt is executed automatically

  • empty requirements.txt file will do it.

language: python
script: python -V

Travis-CI and Python with Pytest

  • Simply adding script: pytest will not work.
  • pytest will fail with exit code 5 if it cannot find any test to run.
language: python
script: pytest

def test_anything():
    pass

Python version matrix

language: python
script: pytest -s

python:
  - "3.7"
  - "3.8"

import sys

def test_anything():
    print(sys.version)
    pass

The environment variables set by Travis - Python

language: python
script: pytest -s

python:
  - "3.8"

import os

for env in sorted(os.environ.keys()):
    print(f"{env:25} = {os.environ[env]}")



def test_anything():
    pass

Set environment variables for Python

language: python
script: pytest -s

python:
  - "3.7"

env:
    DATABASE=postgresql


import os

print(f"DATABASE = {os.environ['DATABASE']}")


def test_anything():
    pass

Python version and environment matrix

  • matrix
language: python
script: pytest -s -m $DATABASE

python:
  - "3.7"
  - "3.8"

env:
    - DATABASE=postgresql
    - DATABASE=mysql

matrix:
  exclude:
  - python: "3.7"
    env: DATABASE=mysql


import os
import pytest

print(f"DATABASE = {os.environ['DATABASE']}")

@pytest.mark.postgresql
def test_postgresql():
    print("Testing postgresql")
    pass

@pytest.mark.mysql
def test_mysql():
    print("Testing mysql")
    pass

[pytest]
markers =
  postgresql
  mysql

Perl

Travis-CI and Perl 5

  • language: perl
  • runs cpanm --quiet --installdeps --notest . so we need a Makefile.PL
language: perl

use strict;
use warnings;
use ExtUtils::MakeMaker;

WriteMakefile(
	NAME         => 'Project',
	AUTHOR       => q{Gabor Szabo <szabgab@cpan.org>},
	VERSION      => '0.01',
	ABSTRACT     => 'Demo Perl Makefile.PL',
	( $ExtUtils::MakeMaker::VERSION >= 6.3002
		? ( 'LICENSE' => 'perl' )
		: () ),
	PL_FILES  => {},
	PREREQ_PM => {
	},
	dist  => { COMPRESS => 'gzip -9f', SUFFIX => 'gz', },
);

Perl version matrix

language: perl
perl:
   - "5.30"
   - "5.28"

use strict;
use warnings;
use ExtUtils::MakeMaker;

WriteMakefile(
	NAME         => 'Project',
	AUTHOR       => q{Gabor Szabo <szabgab@cpan.org>},
	VERSION      => '0.01',
	ABSTRACT     => 'Demo Perl Makefile.PL',
	( $ExtUtils::MakeMaker::VERSION >= 6.3002
		? ( 'LICENSE' => 'perl' )
		: () ),
	PL_FILES  => {},
	PREREQ_PM => {
	},
	dist  => { COMPRESS => 'gzip -9f', SUFFIX => 'gz', },
);
  • With xenial (the default) perl 5.14 and newer are supported.
  • You can use old versions of perl (starting from 5.8) if you set the dist: trusty
  • On newer versions of linux also the travis-perl helpers

The environment variables set by Travis - Perl

language: perl
perl:
   - "5.30"
   - "5.28"

use strict;
use warnings;
use ExtUtils::MakeMaker;

for my $key (sort keys %ENV) {
    print "$key=$ENV{$key}\n";
}

WriteMakefile(
	NAME         => 'Project',
	AUTHOR       => q{Gabor Szabo <szabgab@cpan.org>},
	VERSION      => '0.01',
	ABSTRACT     => 'Demo Perl Makefile.PL',
	( $ExtUtils::MakeMaker::VERSION >= 6.3002
		? ( 'LICENSE' => 'perl' )
		: () ),
	PL_FILES  => {},
	PREREQ_PM => {
	},
	dist  => { COMPRESS => 'gzip -9f', SUFFIX => 'gz', },
);

Set environment variables for Perl

  • %ENV
  • $ENV
  • env
language: perl
perl:
   - "5.30"
   - "5.28"

env:
    - DEMO=Alpha


use strict;
use warnings;
use ExtUtils::MakeMaker;

print "$ENV{DEMO}\n";

WriteMakefile(
	NAME         => 'Project',
	AUTHOR       => q{Gabor Szabo <szabgab@cpan.org>},
	VERSION      => '0.01',
	ABSTRACT     => 'Demo Perl Makefile.PL',
	( $ExtUtils::MakeMaker::VERSION >= 6.3002
		? ( 'LICENSE' => 'perl' )
		: () ),
	PL_FILES  => {},
	PREREQ_PM => {
	},
	dist  => { COMPRESS => 'gzip -9f', SUFFIX => 'gz', },
);

Perl version and environment matrix

  • matrix
language: perl
perl:
   - "5.30"
   - "5.28"

env:
    - DEMO=Alpha
    - DEMO=Beta

matrix:
  exclude:
  - perl: "5.28"
    env: DEMO=Beta

use strict;
use warnings;
use ExtUtils::MakeMaker;

for my $key (sort keys %ENV) {
    print "$key=$ENV{$key}\n";
}

WriteMakefile(
	NAME         => 'Project',
	AUTHOR       => q{Gabor Szabo <szabgab@cpan.org>},
	VERSION      => '0.01',
	ABSTRACT     => 'Demo Perl Makefile.PL',
	( $ExtUtils::MakeMaker::VERSION >= 6.3002
		? ( 'LICENSE' => 'perl' )
		: () ),
	PL_FILES  => {},
	PREREQ_PM => {
	},
	dist  => { COMPRESS => 'gzip -9f', SUFFIX => 'gz', },
);