Survey Report - 2022

2022Big DevBuild Times

Read the overview

Introduction and
Key Findings

Introduction & Methodology

We launched this survey with a single question in mind.
What is the true state of the software build process today, and what impact does it have on the broader business?
To uncover the answer, we spoke to respondents across a wide array of industries, roles, and company sizes, and with varied amounts of hands-on vs managerial experience. We asked them whether they suffer from long
build times, and whether this problem is getting worse year over year. We delved into the frustrations they face, the technologies they use, and how much time and
productivity they could regain if they had the right processes and tools in place.
The results reveal the nature of the build experience for today’s enterprises and paint a clear picture of how organizations can translate faster build times into true ROI.

Methodology

To collect these insights we commissioned a survey of 300 senior managers or above from North America, APAC, and the EU in companies with 1,000 employees or more that implement C++ software projects. The survey was completed by Global Surveyz, an independent survey company, and took place during Q4 2021. The titles of our respondents ranged from CTOs to software engineers and architects, as well as firmware engineers, embedded systems, build/release developers or those in configuration management.
The respondents were recruited through a global B2B research panel and invited via email to complete the survey. The average amount of time spent on the survey was 5 minutes.
The answers to the majority of the non-numerical questions were randomized to prevent order bias in the answers.

Key Findings

  • Today’s builds take a long time,
    and it’s getting longer

    Running a build takes an average of 20 minutes, with almost 25% of respondents spending more than half an hour on each build. For CI/Release builds, this percentage jumps to 32%. Respondents spent an average of 57 minutes waiting for builds to finish. This number is rising, with 96% of respondents saying build time increased an average of 15.9% in 2021.

  • Long builds have a direct impact on productivity,
    frustration, and costs

    Time spent on a build is time not spent on testing, iterating or being productive. 98% of respondents say they are wasting time waiting for builds to finish. 91% consider this a problem, and slow build times come in third in the top frustrations experienced in day-to-day development. Almost half of respondents are using the cloud for development and deployment, where slow build times equate directly to more compute utilization and higher costs.

  • The most commonly used tools are making
    build times longer

    76% of respondents use static code analysis, with an additional 11% planning to start in 2022. 88% of respondents use dynamic code analysis. These tools are overwhelmingly in use, and are required to increase code quality and shorten development cycles. However, they do not come without a cost to build time. This may cause frustration over long build times to become even more prominent.

  • Current solutions to the growing
    problem aren’t sustainable

    Organizations which successfully decreased their build times focused on better hardware (44%) and decreasing codebase size (33%). However, adding hardware is not a sustainable fix to growing build times, and is an extremely costly and limited solution. Decreasing codebase size might help at first, but once code is in good shape this also has limited impact. Organizations need to prioritize managing their hardware better over band-aid solutions.

  • The impact of the right tools and processes
    could be game-changing

    Putting the right tools and processes in place would add 65 minutes of productivity on average every day, with this number increasing to 85 minutes per day for R&D, and 80 minutes per day for DevOps. 65 minutes per day is 36 days a year, which on a $60/hour developer salary is a $20,000 saving overall. Productivity savings could be used to hire more staff, implement best practices, debug faster, or boost customer experience.

State of Development
Build Times and Challenges

Time Spent on Builds and CI/Release Builds

Running a build or a CI/Release build takes its toll. Companies are spending 20 minutes on average to run each build (figure 1) and 23 minutes to run a CI/Release build (figure 2).
69% of companies are spending 10 or more minutes to run a build, and 24% of companies are spending more than 30 minutes (figure 1).

82% of companies are spending 10 or more minutes to run a CI/Release build, and 32% of companies are spending more than 30 minutes (figure 2).
Instead of being able to build quickly, test and get results, builds are taking a long time, slowing down the whole development cycle and having a direct impact on productivity.

Weighted average: 20.3 (Min)

Figure 1: Time to Run a Build

Weighted average: 23.5 (Min)

Figure 2: Time to Run a CI/Release Build

Time Spent Per Day on
Waiting for Builds to Finish

98% of respondents admitted they waste time waiting for builds to finish (figure 3). The average time spent waiting for builds to finish is just under an hour at 57 minutes (figure 4). It varies by industry, with defense/military holding the slowest build times (72 minutes) and banking the fastest build times (38 minutes).
However, even 38 minutes is a lot of wasted time in terms of productivity. In many cases developers need to wait for builds to finish before they can move onto the next task, so this wait is literally time down the drain.

Figure 3: Are you Wasting Time Daily Waiting for Builds to Finish

Figure 4: Time Waiting per Day for Builds to Finish by Industry
(Average in Minutes)

Increase in Build Times 2020-2021
(and What Decreases Build Time)

96% of survey respondents said build time increased by an average of 15.9% in 2021 compared to 2020 (figure 5).
This can probably be attributed to increase in codebase size.
Only 4% reported that build time decreased. The top reasons they give for the decrease are better hardware (44%) and a decrease in codebase size (33%). Better hardware might seem like a quick fix, but it is a very costly solution which organizations will pay for whether it’s utilized or not. It’s also not sustainable, as a team can’t keep adding more hardware within a single machine to meet growing needs.
Decreasing the codebase size might be helpful at first, but will provide diminishing returns over time, and have marginal impact once an organization has made initial fixes.

Weighted average: 15.9%

Figure 5: Increase in Build Times 2020-2021

Figure 6: Top Driver that Decreased Build Time

Development Technologies in Use

The most used technologies and methodologies for accelerating or improving the development cycle are dev in the cloud (49%), static code analysis (46%) and dynamic code analysis (28%). Just 8% of survey respondents have no plans to use dev in the cloud, likely because of security/compliance issues.
About one quarter (23%) of respondents plan to start using build acceleration in 2022.

Figure 7: Development Technologies in Use

Are Slow Build Times A Problem?
Analysis by Industry

Slow build times are a problem for 91% of companies. Only 33% said this isn’t a big problem and 58% said it’s a significant problem or a problem to some extent.
When comparing by industry (figure 9), defense/military suffers the most from slow build times (73%), followed by IT services (67%) and automotive (63%).

Figure 8: Are Slow Build Times A Problem?

Figure 9: Reported Slow Build as a Problem by Industry

Top Developer Day-to-Day Frustrations

97% of survey respondents report that they experience frustrations in their day-to-day work.
The most frustrating issues for respondents are analyzing crashes from production (35%), learning new technologies (35%), and long build times (31%). While crashes and new technologies are somewhat necessary evils, long build times can be solved with the right technology.

Static code analysis is one method of reducing crashes, but this adds time to builds, which is likely to increase frustration overall.
This requires a solution since adoption of both static and dynamic code analysis are growing in the market (figure 7) as methods to improve testing and quality.

Figure 10: Top Developer Day-to-Day Frustrations

Top Time-Wasters According to Dev Teams

The top time-wasters are creating internal documentation that no one will read (82%), unnecessary changes in requirements (78%), and slow builds (69%). Also on the list are slow tests, highlighted by 67% of respondents. Slow builds and slow testing feed into one another. If a company can speed up builds, they can speed up tests.

Figure 11: Top Time-Wasters According to Dev Teams

Top Requested Areas to Reduce
Development Time

97% of respondents admit they have areas they wish they could improve in their day-to-day work which would save on development time.
The top requested areas are moving to new build systems (35%), optimizing build scripts (31%), and being able to accelerate their build times (28%). All of these top requests are aligned with improving build times, showing the significance of this need.

Figure 12: Top Areas to Reduce Development Time

C++ Codebase Size and 2022
Expected Growth

91% of survey respondents have hundreds of thousands of lines of code or more. 76% have millions of lines of code or more (figure 13).
Codebases are expected to grow by 18.6% in 2022. This will grow most at large companies with 5K+ employees (21.6%), while in companies of up to 5K employees, it is expected to grow 16% (figure 14). Bigger companies face bigger problems.
As codebases grow, build times will get even longer.

Figure 13: C++ Codebase in Lines of Code, 2021

Figure 14: Codebase Expected Growth by Company Size in 2022

Impact on Productivity of Having
the Right Tools and Processes

95% of survey respondents indicated that with the right tools and processes, they can save (average weight) 65 minutes per day. Based on 261 working days per year (in 2021) and 8 working hours per day, that translates into approximately 36 days per year. Translating this into cost savings, 36 days is almost $20,000 per year if we take $60/hour as a typical dev cost. Productivity in this area could realize serious ROI even for those who say it will only save 30 minutes or an hour per day. Saving two hours per day would mean a company could add another full-time developer to its team.
Faster builds also allow the adoption of best practices such as feature flagging, working in parallel on a few code branches and more, and allow for faster bug resolution, faster time to market and better customer success opportunities – all of which impact an organization’s bottom line.

Weighted average: 65.6 Minutes/day (36 days/year)

Figure 15: Impact on Productivity of Having the Right Tools and Processes

Impact of Having the Right Tools
and Processes on Productivity by Role

When looking at the impact on productivity of having the right tools and processes by role, we see that while the average time saved for all respondents is 65 minutes per day. The number grows by 30% when it comes to R&D, and the right tools and processes will save them 85 minutes per day.
In DevOps and R&D there are much higher issues with recruitment, so when translating the time savings to cost savings (figure 16), it’s clear to see how impactful the right tools and processes could be on these teams.

Figure 16: Impact of Having the Right Tools and Processes on Productivity by Role (in Minutes / Day)

Cloud Usage throughout Development Lifecycle

The cloud is already used throughout the entire development lifecycle. The top stages for using the cloud overall are in development (43%) and deployment (42%). When looking at the industry breakdown, banking has the highest rate of cloud usage. When we connect cloud usage to slow build times, we can see a direct correlation to costs. Time-consuming builds cost much more on the cloud in terms of compute utilization.
In some industries, it’s impossible to use money-saving options such as spot instances, and therefore costs can spiral quickly. Organizations need to look for ways to keep cloud costs low while optimizing for faster builds.

Figure 17: Cloud Usage throughout Development Lifecycle by Industry

Demographics and Development
Environments

Industry, Top Target Platforms

Figure 18: Industry

Figure 19: Top Target Platforms and Operating Systems

Country of Residence

Figure 20: Country of Residence

Figure 20: Country of Residence

Department, Job Role, Seniority and
Company Size

Figure 21: Hands-on vs. Managers

Figure 22: Role

Figure 23: Seniority

Figure 24: Company Size

About Incredibuild

Incredibuild turbocharges development from compilations to testing and release-automation by turning every host into a supercomputer with hundreds—even thousands—of cores. World-leading brands rely on Incredibuild to deliver better products to market - radically faster.

Get a free license

For more information please visit us:

© 2022 Incredibuild Software Ltd. All rights reserved.