Ensuring record tests always runs

Some tools prevent launchable record tests from running when tests fail. Follow these instructions to fix this.

This page relates to Recording test results with the Launchable CLI.

The launchable record tests command must be executed after you run tests.

However, some tools exit the build process as soon as the test process finishes, preventing this from happening.

The way to fix this depends on your CI tool:

Jenkins

For declarative Pipeline jobs, use the post { always { ... } } option:

pipeline {
  ...
  sh 'bundle exec rails test -v $(cat launchable-subset.txt)'
  ...
  post {
    always {
      sh 'launchable record tests <BUILD NAME> [OPTIONS]'
    }
  }
}

For scripted Pipeline jobs, the catchError step should be used as described here: https://www.jenkins.io/doc/pipeline/steps/workflow-basic-steps/#catcherror-catch-error-and-set-build-result-to-failure.

Note that this is unnecessary for Maven builds that use the -Dmaven.test.failure.ignore option.

CircleCI

CircleCI has when: always option:

- jobs:
  - test:
    ...
    - run:
        name: Run tests
        command: bundle exec rails test -v $(cat launchable-subset.txt)
    - run:
        name: Record test results
        command: launchable record tests <BUILD NAME> [OPTIONS]
        when: always

GitHub Actions

GitHub Action has if: ${{ always() }} option:

jobs:
  test:
    steps:
      ...
      - name: Run tests
        run: bundle exec rails test -v $(cat launchable-subset.txt)
      - name: Record test result
        run: launchable record tests <BUILD NAME> [OPTIONS]
        if: always()

Bash

If you run tests on your local or other CI, you can use trap:

function record() {
  launchable record tests <BUILD NAME> [OPTIONS]
}
# set a trap to send test results to Launchable for this build either tests succeed/fail
trap record EXIT SIGHUP


bundle exec rails test

Last modified ago