Using parallel CI/CD jobs to increase productivity is not a new concept – it’s a standard way to speed up your CI cycles. But there’s a limit to how much you can distribute with standard tools. It’s not just splitting jobs across machines.
To use a supermarket analogy, your build is a shopping cart. The eco-friendly bags you keep in your trunk are your ‘jobs.’ All the food in your bags is your ‘processes.’ One way to make your checkout faster would be to scan each one of your bags across every available counter. But even then, there’s still a way to complete the scan faster.
A more granular way to complete the task faster would be to take out each item from the bags and parallelize those across an infinite number of counters. If that’s not enough, imagine everyone else in your household had a scanner, and you’d be prompted if someone else had already stocked up on your favorite protein bars.
How to accelerate GitLab with Incredibuild
In our example, your CI (whether that’s GitLab, Jenkins, or another platform) is your first level of parallelization – your shopping bags. Incredibuild is the ability to take each item out of its bag to a separate scanner. It kicks in when your first-level CI parallelization completes, breaking the jobs into micro-processes. Taking it even further, our Build Cache technology is your household scanner, letting you shorten your times even further by reusing previously output build artifacts.
Here’s a quick guide how to get this working in your supermarket. NOTE: This procedure assumes that you have already installed Incredibuild and Gitlab.
- Install an Incredibuild Agent on your Gitlab Runner machine. This Agent will function as an Incredibuild Initiator. Other Agents that are connected to the same Incredibuild Coordinator will act as helpers. We recommend verifying that the initiator agent and all helper agents are connected to the same coordinator in the Coordinator Monitor.
- Create a file called gitlab-build.bat in the root directory of your GitLab project with the following content: ECHO Executing BuildConsole command to build %PROJECT_NAME%
BuildConsole.exe “%PROJECT_NAME%.sln” /Cfg=”%CONFIGURATION%|%PLATFORM%” /rebuild /showtime /showagent
This batch file is triggered during pipeline executions. It instructs Incredibuild to build your VS solution using the configuration variables listed in the pipeline. - In your GitLab environment, edit the .gitlab-ci.yml file. This is a standard GitLab ci declaration file.
In the script section, add a call to run the gitlab-build.bat file as seen here:
All of your GitLab builds will now be distributed among your helper agents.
Verifying Build Distribution
You can verify that connected helper agents are participating in the build process during a build in the command prompt. You will see which machine is running each task in parentheses as seen in this example:
You can also see a visual breakdown of each executed task in the Build Monitor of the agent on your Gitlab Runner machine (the initiator agent).
Powering your CI pipelines with Incredibuild
No matter what CI/CD tools you’re using, Incredibuild can help you accelerate your development. By breaking down your massive tasks to a granular level, you can shorten your build times, iterate faster, and ensure that your dev teams’ CI pipelines work smoothly and let you deploy as quickly as you need.
You can learn more about Incredibuild’s CI/CD acceleration here.
