pytest (Integration)

Details about how Launchable integrates with pytest.

This is a reference page. See Getting Started, Sending data to Launchable, and Predictive Test Selection for more comprehensive usage guidelines.

Launchable interfaces with pytest via

the Launchable pytest plugin
the Launchable CLI

Launchable pytest plugin

We offer a new way to integrate Launchable, a native pytest plugin.

Installing the plugin

The Launchable pytest plugin is a Python3 package that you can install from PyPI.

{% hint style="warning" %} The plugin requires Python 3.7+, Pytest 4.2.0+, and Java 8+. {% endhint %}

If you use Pipenv, you can install the plugin into your repository:

pipenv install --dev pytest-launchable

Or, you can install the CLI in your CI pipeline by adding this to the part of your CI script where you install dependencies:

pip3 install pytest-launchable

You don't need to install Lanchable CLI separately because the plugin automatically installs the CLI and uses it internally.

Setting your API key

First, create an API key for your workspace at app.launchableinc.com. This authentication token allows the pytest plugin to talk to Launchable.

Then, make this API key available as the LAUNCHABLE_TOKEN environment variable in your CI process. How you do this depends on your CI system:

CI system	Docs
Azure DevOps Pipelines	Set secret variables
Bitbucket Pipelines	Variables and secrets
CircleCI	Using Environment Variables
GitHub Actions	How to configure a secret
GitLab CI	GitLab CI/CD environment variables
GoCD	Setting variables on an environment
Jenkins	Injecting secrets into builds (Create a global "secret text" to use in your job)
Travis CI	Environment Variables

Generate a config file

launchable-config is a command-line tool to generate and validate configuration files. The Launchable pytest plugin uses this config.

First, generate a new config file:

# via pipenv
pipenv run launchable-config --create

# via pip
launchable-config --create

This generates a template .launchable.d/config.yml file in the current directory that looks like this:

# Launchable test session configuration file
# See https://docs.launchableinc.com/resources/cli-reference for detailed usage of these options
#  
schema-version: 1.0
build-name: commit_hash
record-build:
  # Put your git repository location here
  source: .
  max_days: 30
record-session:
subset:
  # mode is subset, subset-and-rest, or record-only
  mode: subset
  # you must specify one of target/confidence/time
  # examples:
  #   target: 30%  # Create a variable time-based subset of the given percentage. (0%-100%)
  #   confidence: 30%  # Create a confidence-based subset of the given percentage. (0%-100%)
  #   time: 30m  # Create a fixed time-based subset. Select the best set of tests that run within the given time bound. (e.g. 10m for 10 minutes, 2h30m for 2.5 hours, 1w3d for 7+3=10 days. )
  target: 30%
record-tests:
  # The test results are placed here in JUnit XML format
  result_dir: launchable-test-result

You can then edit the config file per the directions below.

Recording test results (pytest plugin)

Update your config file

In .launchable.d/config.yml:

Check that the source option in the record-build section points to your Git repository (the default is ., the current directory).
Check that the mode option in the subset section is set to record-only

Verify your config file

Verify the contents of the .launchable.d/config.yml file:

# via pipenv
pipenv run launchable-config --verify

# via pip
launchable-config --verify

If any problems are reported, edit the file accordingly.

Use the plugin with pytest

Then, just add an --launchable option to the pytest command. It is very easy:

pytest --launchable <your-pytest-project>

If the configuration file is not in the current directory, use the --launchable-conf-path option:

pytest --launchable --launchable-conf-path <path-to-launchable-configuration-file> <your-pytest-project>

This will:

Create a build in your Launchable workspace
Run your tests
Submit your test reports to Launchable
Leave XML reports in the launchable-test-result by default

Requesting and running a subset of tests