Zero Input Subsetting works better with some Test Session Layouts than others, so contact your Customer Success Manager before you start using this feature. We're here to help!
We've created a complementary approach called Zero Input Subsetting. With this approach, the CLI does not have to gather and submit the full list of tests. Instead, the server generates the full list of tests from the last two weeks of recorded sessions. To ensure new tests are run, the CLI outputs exclusion rules instead of inclusion rules.
Zero Input Subsetting works better with some Test Session Layouts than others, so contact your Customer Success Manager before you start using this feature. We're here to help!
You can adopt this approach by adding two options to launchable subset
:
--get-tests-from-previous-sessions
, and
--output-exclusion-rules
The subset request then returns a list of tests to exclude (i.e., don't run these tests):
The following CLI profiles/integrations support Zero Input Subsetting:
Let us know if you want to see support for another test runner!
Also, see Using groups to split subsets which expands this behavior.
Find your dotnet test
command in your CI script. These commands will go before that command.
First, request an exclusion pattern:
launchable subset --get-tests-from-previous-sessions --output-exclusion-rules --build <BUILD NAME> <OPTIMIZATION TARGET OPTION> dotnet > launchable_exclusion_filter.txt
See the CLI reference for details about <BUILD NAME>
and <OPTIMIZATION TARGET OPTION>
.
Then pass that exclusion pattern into dotnet test
using --filter
:
dotnet test --filter $(cat launchable_exclusion_filter.txt)
OR
set /p LAUNCHABLE_EXCLUSION_FILTER=<launchable_exclusion_filter.txtdotnet.exe test --filter %LAUNCHABLE_EXCLUSION_FILTER%
First, you'll request an exclusion list from your full test suite. Then, you'll pass this list to Gradle.
First, you need to add a snippet to your Gradle config to enable test exclusion via the Gradle command line:
test {if (project.hasProperty('excludeTests')) {exclude project.property('excludeTests').split(',')}}
Then, find the gradle
command used to run tests in your CI script.
Before that command, run launchable subset
to request an exclusion list. The subset and exclusion lists are generated from the union of tests recorded in the last two weeks.
launchable subset --build <BUILD NAME> <OPTIMIZATION TARGET OPTION> --get-tests-from-previous-sessions --output-exclusion-rules gradle > launchable-exclusion-list.txt
See the CLI reference for details about <BUILD NAME>
and <OPTIMIZATION TARGET OPTION>
.
This creates a file called launchable-exclusion-list.txt
. This file contains a list of test classes formatted for passing into Gradle like this:
-PexcludeTests=com/example/FooTest.class,com/example/BarTest.class
Then pass this file into your existing command, like shown below.
gradle test $(cat launchable-exclusion-list.txt)# equivalent to gradle test -PexcludeTests=com/example/FooTest.class,com/example/BarTest.class
Note: If the exclusion list is very large, it may be unable to specify it directly from the command. In that case, you can change the Gradle config to read from launchable-exclusion-list.txt
.
Change the Gradle config as follows:
test {if (project.hasProperty('excludeTestsTxt')) {exclude new File(project.property('excludeTestsTxt')).text.replaceFirst('-PexcludeTests=', '').trim().split(',')}}
Then, specify the exclusion tests file from the command.
gradle test -PexcludeTestsTxt=$PWD/launchable-exclusion-list.txt
In summary, here's the flow before:
# your normal command to run tests looks something like thisgradle test <OPTIONS>
And the flow after:
# request an exclusion list from all testslaunchable subset --build <BUILD NAME> <OPTIMIZATION TARGET OPTION> --get-tests-from-previous-sessions --output-exclusion-rules gradle > launchable-exclusion-list.txt# run tests, excluding deprioritized tests, leaving only the recommended subsetgradle test <OPTIONS> $(cat launchable-exclusion-list.txt)
First, you'll request an exclusion list from your full test suite. Then, you'll pass this list to Gradle.
First, find the gradle
command used to run tests in your CI script.
Before that command, run launchable subset
to request an exclusion list. The subset and exclusion lists are generated from the union of tests recorded in the last two weeks.
launchable subset --build <BUILD NAME> <OPTIMIZATION TARGET OPTION> --get-tests-from-previous-sessions --output-exclusion-rules gradle --bare > launchable-exclusion-list.txt
See the CLI reference for details about <BUILD NAME>
and <OPTIMIZATION TARGET OPTION>
.
Don't forget the --bare
option after gradle
!
This creates a file called launchable-exclusion-list.txt
. This file contains a list of test classes formatted for passing into Gradle, like this:
com.example.FooTestcom.example.BarTest...
First, you need to add a dependency declaration to build.gradle
so that the right tests get excluded when TestNG runs:
dependencies {...testRuntime 'com.launchableinc:launchable-testng:1.2.1'}
Then export the exclusion list file path as an environment variable before you run mvn test
, like shown below.
export LAUNCHABLE_REST_FILE_PATH=$PWD/launchable-exclusion-list.txtgradle test <OPTIONS>
In summary, here's the flow before:
# your normal command to run tests looks something like thisgradle test <OPTIONS>
And the flow after:
# request an exclusion list from all testslaunchable subset --build <BUILD NAME> <OPTIMIZATION TARGET OPTION> gradle --bare <PATH TO SOURCE> > launchable-exclusion-list.txt# run tests, excluding deprioritized tests, leaving only the recommended subsetexport LAUNCHABLE_REST_FILE_PATH=$PWD/launchable-exclusion-list.txtgradle test <OPTIONS>
First, you'll request an exclusion list from your full test suite. Then, you'll pass this list to Maven.
Find the mvn test
command used to run tests in your CI script.
Before that command, run launchable subset
to request an exclusion list. The subset and exclusion lists are generated from the union of tests recorded in the last two weeks.
launchable subset --build <BUILD NAME> <OPTIMIZATION TARGET OPTION> --get-tests-from-previous-sessions --output-exclusion-rules maven > launchable-exclusion-list.txt
See the CLI reference for details about <BUILD NAME>
and <OPTIMIZATION TARGET OPTION>
.
This creates a file called launchable-exclusion-list.txt
that you can pass into Maven.
To exclude deprioritized tests and only run the recommended subset, use the -Dsurefire.excludesFile
option. For example:
mvn test <OPTIONS> -Dsurefire.excludesFile=$PWD/launchable-exclusion-list.txt
If your build already depends on surefire.includesFile
, or <includes>/<includesFile>
, those and our exclusion list will collide and not work as expected. Contact us to resolve this problem.
In summary, here's the flow before:
# your normal command to run tests looks something like thismvn test <OPTIONS>
And the flow after:
# get an exclusion list from the serverlaunchable subset --build <BUILD NAME> <OPTIMIZATION TARGET OPTION> --get-tests-from-previous-sessions --output-exclusion-rules maven > launchable-exclusion-list.txt# run tests, excluding deprioritized tests, leaving only the recommended subsetmvn test <OPTIONS> -Dsurefire.excludesFile=$PWD/launchable-exclusion-list.txt
First, you'll request an exclusion list from your full test suite. Then, you'll pass this list to Maven.
Find the mvn test
command used to run tests in your CI script.
Before that command, run launchable subset
to request an exclusion list. The subset and exclusion lists are generated from the union of tests recorded in the last two weeks.
launchable subset --build <BUILD NAME> <OPTIMIZATION TARGET OPTION> --get-tests-from-previous-sessions --output-exclusion-rules maven > launchable-exclusion-list.txt
See the CLI reference for details about <BUILD NAME>
and <OPTIMIZATION TARGET OPTION>
.
This creates a file called launchable-exclusion-list.txt
that you can pass into Maven.
First, modify your pom.xml
so that it includes Launchable TestNG integration as a test scope dependency:
<dependency><groupId>com.launchableinc</groupId><artifactId>launchable-testng</artifactId><version>1.2.1</version><scope>test</scope></dependency>
Then export the exclusion list file path as an environment variable before you run mvn test
, like shown below.
export LAUNCHABLE_REST_FILE_PATH=$PWD/launchable-exclusion-list.txtmvn test <OPTIONS>
In summary, here's the flow before:
# your normal command to run tests looks something like thismvn test <OPTIONS>
And the flow after:
# get an exclusion list from the serverlaunchable subset --build <BUILD NAME> <OPTIMIZATION TARGET OPTION> --get-tests-from-previous-sessions --output-exclusion-rules maven > launchable-exclusion-list.txt# run tests, excluding deprioritized tests, leaving only the recommended subsetexport LAUNCHABLE_REST_FILE_PATH=$PWD/launchable-exclusion-list.txtmvn test <OPTIONS>