GitHub Actions

First, follow the instructions on Getting Started to sign up and create a workspace for your test suite.

Then create an API key for your workspace in the Settings area (click the cog ⚙️ icon in the sidebar). This authentication token lets the CLI talk to your Launchable workspace.

Launchable record build and test results action

The Launchable record build and test results action enables you to integrate Launchable into your CI in simple way with less change. This action installs the CLI and runs launchable record build and launchable record test to send data to Launchable so that the test results will be analyzed in Launchable to improve your developer productivity.

Example usage

1name: Test
2
3on:
4  push:
5    branches: [main]
6  pull_request:
7    branches: [main]
8
9env:
10  LAUNCHABLE_TOKEN: ${{ secrets.LAUNCHABLE_TOKEN }}
11  LAUNCHABLE_DEBUG: 1
12  LAUNCHABLE_REPORT_ERROR: 1
13
14jobs:
15  tests:
16    runs-on: ubuntu-latest
17    steps:
18      - uses: actions/checkout@v2
19      - name: Test
20        run: <YOUR TEST COMMAND HERE>
21      - name: Record
22        uses: launchableinc/record-build-and-test-results-action@v1.0.0
23        with:
24          build_name: $GITHUB_RUN_ID
25          test_runner: <YOUR TEST RUNNER HERE>
26          report_path: <PATH TO TEST REPORT XML FILES>
27        if: always()

Instructions for test runners/build tools

Android Debug Bridge (adb)

Currently, the CLI doesn't have a record tests command for ADB. Use the command for #gradle instead.

Ant

No special steps.

Bazel

Behave

First, in order to generate reports that Launchable can consume, add the --junit option to your existing behave command:

# run the tests however you normally do
behave --junit

CTest

First, run your tests with ctest -T test --no-compress-output. These options ensure test results are written to the Testing directory.

cucumber

First, run cucumber with the -f junit option, like this:

bundle exec cucumber -f junit -o reports

(If you use JSON, use the Launchable CLI instead.)

Cypress

GoogleTest

First, configure GoogleTest to produce JUnit compatible report files. See their documentation for how to do this. You'll end up with a command something like this:

# run the tests however you normally do
./my-test --gtest_output=xml:./report/my-test.xml

Go Test

First, in order to generate reports that Launchable can consume, use go-junit-report to generate a JUnit XML file after you run tests:

# install JUnit report formatter
go get -u github.com/jstemmer/go-junit-report

# run the tests however you normally do, then produce a JUnit XML file
go test -v ./... | go-junit-report -set-exit-code > report.xml

Gradle

**/build/**/TEST-*.xml.

Jest

First, in order to generate reports that Launchable can consume, use jest-junit to generate a JUnit XML file after you run tests.

# install jest-junit reporter
npm install jest-junit --save-dev
# or
yarn add --dev jest-junit

You'll need to configure jest-junit to include file paths in reports.

You can do this using environment variables:

export JEST_JUNIT_CLASSNAME="{classname}"
export JEST_JUNIT_TITLE="{title}"
export JEST_JUNIT_SUITE_NAME="{filepath}"

export JEST_JUNIT_SUITE_NAME="{filepath}"

Then, run jest using jest-junit:

# run tests with jest-junit
jest --ci --reporters=default --reporters=jest-junit

Maven

'./**/target/surefire-reports'

Note: The invocation above relies on the CLI to expand GLOBs like **.

minitest

First, use minitest-ci to output test results to a file. If you already store your test results on your CI server, it may already be installed.

NUnit Console Runner

pytest

First, run tests with the --junit-xml option:

pytest --junit-xml=test-results/results.xml

Robot

output.xml

RSpec

First, use rspec_junit_formatter to output test results to a file in RSpec. If you already have a CI server storing your test results it may already be installed:

bundle exec rspec --format RspecJunitFormatter --out report/rspec.xml