About GoCardless
GoCardless is a global leader in direct bank payments. They help more than 85,000 businesses, from start-ups to household names, collect both recurring and one-off payments, without the chasing, stress or expensive fees. Each year GoCardless processes more than US$35 billion of payments across 30+ countries. Their acquisition of Nordigen means they’re also helping businesses make faster and more informed decisions through easy access to bank account data. They are headquartered in the UK, with additional offices in Australia, France, Latvia and the United States. For more information, please visit www.gocardless.com and follow them on Twitter @GoCardless.
Meet the team
Launchable was brought in by a few different people at GoCardless. Sam, an engineer of the Developer Enablement team and the Continuous Integration (CI) lead at GoCardless, was the initial individual to reach out to our team. After the initial intro, Sam brought in Bastian, a product manager, and Burak, an engineer. Both are also part of the Developer Enablement team at GoCardless, which aims to improve the everyday lives of developers at GoCardless via tools and other resources.
Sam and Bastian came to Launchable to help improve developer productivity by reducing testing times, and Burak identified a specific test suite for a payments processing application. As a side goal, they were looking to also reduce their cloud spend for this test suite as well.
The Problem
Months before coming to Launchable, the Developer Enablement team was assigned an SLO (service level objective) to reduce testing time. The unit test suite for this spec payment processing application was the target of this SLO, and they were looking to achieve at least a 40% reduction.
They had already been using a lot of cloud resources at this suite, with it running at a 40x parallelization, and could not afford to spend any more on that.
The Solution: Run a subset of the unit test suite using Launchable to improve developer productivity and reduce resource costs
By using Launchable’s Predictive Test Selection (PTS), GoCardless is able to run a subset of tests to both help increase developer productivity by reducing testing times, and also save money by reducing cloud resources
Let's dive into the technical layout a little bit more. This unit test suite, which contains around 60,000 test cases, runs against every push function across every branch tied to this payments processing application. This results in around 200 runs per day of the unit test suite. With Launchable, GoCardless was able to implement subsetting and achieve a 50% reduction in machine hours (post parallelization) on each run. The team's first full month of subsetting resulted in a total of 8.5k total machine hours saved. Along with those savings, the team has noticed an uptick in overall CI runs.
This points to higher developer speed and reduction of waiting times due to the faster turnarounds on this test suite.
“CI times were a growing bottleneck for developer productivity for us at GoCardless; as our key monorepo expanded and more tests were added, CI times increased and consequently productivity slowed. Launchable offered a quick and easy-to-integrate solution that surpassed our expectations.”
- Sam Robson, CI lead at GoCardless
In addition to leveraging PTS, the team was able to leverage Launchable’s split subset feature, which evenly splits subsets across all 40 parallel runs of the test suite automatically. They simply make one call to request a subset on each of the 40 resources and Launchable splits the test durations evenly across all of them.
A snapshot of their model can be seen above. At 90% confidence, meaning
Launchable is 90% confident a failure can be found in the selected subset, GoCardless only needs to run ~48 minutes of tests. This is compare to an average total test time of ~300 mins(before parallelization) without using Launchable
“Thanks to Launchable, we’ve managed to increase the speed of our pipeline for our engineers, reducing waiting time and costs and increasing productivity and satisfaction all around. The product is thorough and easy to integrate, which allowed us to get it on production in just one month.”
Bastian Zamorano, Product Manager for Developer Enablement
Where to next?
With the success the team achieved on the unit test suite, they are already expanding and looking to subset across multiple test suites at GoCardless.