Joseph Sibony
reading time:
Introduction
Developer productivity is a measure of a team’s ability to quickly and efficiently write high-quality software that performs well and is easy to maintain. By maximizing developer productivity, applications can be created and launched faster, shortening time to value and boosting return on investment.
With ROI and business performance as driving factors, it is of no surprise that developer productivity has been the focus of research for decades. Another recent factor is the Covid-19 pandemic, which saw many people in the software industry, including developers, transition to working remotely from home. Companies have had to deal with new challenges to keep development teams connected and collaborating healthily and efficiently. Their ability to do this properly will no doubt help to define the workplace model for years to come.
Developer productivity is related to developer velocity, which is a metric first published in 2020 by McKinsey & Company with a contribution from Microsoft. Developer velocity has to do with giving developers the proper tools in the right environment to bolster innovation, and it is closely correlated with business performance. You may want to download our whitepaper that explains ways to enhance developer velocity in your organization. In this context, productivity is a subset of velocity, and therefore, the topic here is more focused on a narrowed aspect of the issue. 
One last note before we begin. The subject of measuring and improving developer productivity is generally relevant to almost all development environments. However, since we love C++ (well, not only C++ but mostly), some of the references in this post come from the C++ world. This should not bother you, even if you have never dealt with C++ and are programming with any other language, it should be very easy to convert in one’s head any C++-specific reference to your daily used language and ecosystem.
Myths and Misconceptions about Developer Productivity
Over time, several myths and misconceptions about developer productivity have developed and taken hold in some organizations. A common misbelief is that there is a simple and effective way to measure developer productivity using one very good metric. In reality, no one gauge can be used to assess productivity because it depends on too many factors that can vary between industry and organization. In practice, a weighted combination of several metrics may be used, but even so, which metrics and specific weights will differ depending on the domain.
Another misconception is that developer productivity is primarily about the individual. In reality, it is deeper and of wider scope than any one person. Each team member’s performance is certainly of value on its own but in addition, their contribution to the team for group-based activities is critical. The success of the project depends on more than the independent success of several individuals. It also depends on how well they work together. Indeed, the whole is greater than the sum of the parts.
Some developers feel that measuring productivity is something that only managers benefit from. However, metrics can help at the level of the individual or team, either as a personal or collaborative effort. While it is true that managers are often interested in productivity metrics, they are not alone in wanting to estimate progress and improve the chances of success.
Can and Should Developer Productivity Be Measured?
Common sense dictates that you can’t compare two things or judge progress between revisions if you don’t have an effective way to measure them. It follows that if you can’t measure where you were and how far you’ve progressed, then you can’t improve. As such, it is not only reasonable to want to measure developer productivity, but important as well for any organization that wants to move forward, efficiently and effectively.
Now that we’ve established that we indeed should measure productivity, the question is how. One of the obstacles to maximizing developer productivity is that there isn’t one standard for measuring it. Companies have long had internal metrics for measuring developer productivity, some of which are tied to code quality, whereas others are linked to events such as major product releases. There is no all-in-one metric and the reason for this is that many challenges exist in trying to measure productivity in this context.
Challenges With Measuring Developer Productivity
On the face of it, measuring development productivity might seem like a straightforward task, although it is anything but. To illustrate this, we consider some of the traditional metrics. For example, counting the number of lines of source code (LOC) written in a single day might seem logical, but even novice programmers recognize that more is not always better.
First, there is a difference between physical lines of code and logical lines of code, where more than one action can happen on a single line. This is not a simple issue of counting one versus multiple lines because there are times when spacing out instructions makes code more readable, yet more compact code can be more efficient. Additional comment lines are intended to make code more maintainable, but too many will have the opposite effect. There is often a trade-off between parsimony and clarity that cannot be easily captured with such a simplistic metric.
There are other serious problems, as well. A developer aware that they are being judged by LOC can easily write more verbose code, trading eloquence for bulk. Different programming languages will require greater or fewer lines to accomplish the same task, which may not matter for language-uniform projects, but in large projects, mixing languages is not uncommon. For example, front-end development work is typically written in a different language than a routine that interacts with a database at a low level. Yet, these exist in the same project.
Also, measuring productivity using LOC effectively requires a weighting based on the stage of development. When new functionality is added, perhaps at inception, there is a lot of coding. When the effort shifts to debugging, very few lines of code might be added or even modified. Effort spent testing and debugging is at least as demanding as the initial development, yet would score poorly using this one-dimensional measure.
And in some cases, reducing LOC is an achievement, by integrating a new common library or unifying code parts into one, but this would be considered by LOC measure as a bad thing, thus not encouraging the developers to invest in it.
Similar arguments can be made for the number of hours worked, the rate of defects, the number of bugs that are discovered, the number of tickets that are resolved, static code analysis scores, the criticality of identified vulnerabilities, the number of story points in an Agile environment, the number of features, the number of releases, and so forth. Also, during a code review, subjective praise or criticism for aspects such as coding style and following coding conventions are all points that contribute to measuring developer productivity.
What makes sense in one industry, development environment, company’s culture, or even a specific project, may not make sense in another. Worse, choosing the wrong metrics can be harmful to a team’s morale. An example of this is the strict adherence to scheduled and regular releases. Having a large number of releases, each with notable improvements, is impressive. However, inflexible deadlines not only put undue pressure on team members but undoubtedly, will pave the way for bugs that later surface in production. Having these reported by end-users can cause damage to reputation, which is very much more costly than taking a proactive approach, even if it takes a little longer.
The bottom line is that using a simple measure misses context, and this can lead to bigger issues. The implementation of rarely used features will not drive significant business impact. Unless innovations have a reasonable path to success, driving business value, the only thing that looks good is the score for the metric. Similarly, releasing many versions of the software can mean sacrificing quality and introducing unnecessary risk. All of these things are challenges when it comes to measuring developer productivity.
Git-Based Metrics
Code metrics, just like other areas of software development, continue to evolve. There is no one perfect code metric but it doesn’t mean that better measures don’t exist and that we can’t strive for them. For example, several companies have started using metrics that are tied to their source control system. This analysis of the metadata represents a new perspective and meaningfully contributes to the question of whether developer productivity can be adequately measured.
Code Churn
One of the Git-based metrics currently in use is Code Churn. This idea, also known as rework, is related to how often a section of code is edited. The scope of the code varies in that it could be a short function, a large class, or perhaps an entire file. The problem with using a metric like this is that code churn isn’t necessarily a good thing or a bad thing. It’s about context. A high code churn may very well be proof of a frantic effort to get things working properly at the last minute. However, it might be a significant mid-release refactoring task intended to improve maintainability. This means that the volume of new changes matters and moreover, recency is a factor.
Consider a section of code that traditionally has a high rework rate. Perhaps this is because new functionality is regularly added, as it is closely tied to a dynamic environment. Perhaps the developers could just not seem to get the code right, and new bugs or performance issues are constantly being discovered. Regardless, the code in question has recently entered a period of very few edits and is now stable in that regard. To account for such changes, looking at a time-limited window for code churn is a healthy constraint.
One approach would be to look at the rate of churn within a month of the first code commit. Oft-changing code may indicate complexity that was not fully accounted for, or perhaps a design flaw that would ultimately impact the project at a higher level. Another approach, perhaps for a more mature product, is to look at the rework rate leading up to a planned release.
Typical levels of code churn will differ between organizations, teams, and even programming languages. We all know that when code is constantly changing, the likelihood of introducing bugs increases. Yet, code updates are a fact of life, so how much is too much? This is something that needs to be looked at and tested at the organizational level to determine what rate of code churn is reasonable.
Review Coverage
Another Git-based metric being used today is review coverage. Having team members review and critique their peers’ code is a common practice that has benefits for teams both large and small. It is not surprising that better code reviews lead to better software quality, so it makes sense to consider what percentage of the codebase has been subject to peer review.
The Future of Code Metrics
Traditional code metrics have served project managers and developers well but at the same time, it doesn’t mean that the domain is without room for improvement. In particular, with advancements in technology, the possibility of newer metrics opens up. For example, the Git-based metrics now being used were not possible before the widespread adoption of source control, and clearly, certain metrics are more useful depending on the programming language. Moving forward, as programming languages, environments, and technologies evolve, code metrics are sure to follow.
Different Approaches to Properly Measuring Developer Productivity
Granted, there are challenges when it comes to measuring developer productivity. At the same time, we know that it’s not only useful but essential in the path to a superior product. This combination leads to different approaches, each with advantages and disadvantages.
Input and Output Measurement
One such approach is input and output measurement, where people consider software development productivity in those terms. The aforementioned number of hours worked is an example of an input measure. There are several problems with this, not the least of which is that it leaves no way for people to showcase their value, other than by just being present. Unfortunately, just because an office is full on a Friday afternoon after a morning release, it doesn’t mean that the time is productive, even collectively.
It is important to remember that inputs are not just positive. Putting in a lot of hours leads to programmer burnout that detracts from productivity. Mistakes are made, and this can, in turn, fuel interpersonal problems including disputes between team members. Rather than promoting a healthy culture of innovation, it leads to a toxic environment with poor morale and non-existent collaboration. Choosing an arbitrary output measure, such as the number of software releases, is again a similar problem. Essentially, the over-promotion of one metric allows no room for everything else of value.
SPACE framework
The SPACE framework for developer productivity measures performance based on 5 pillars: (a) Satisfaction, (b) Performance, (c) Activity, (d) Communication and Collaboration, and (e) Efficiency. The authors of this framework well understand that no single metric can be used to measure developer productivity and in turn, have created a multidimensional measure that incorporates all of these concepts.
Satisfaction
SPACE focuses not only on building and improving the developer experience but promoting a healthy team-oriented culture. The rationale is that developer satisfaction and productivity are highly correlated, and a positive work environment clears the way for maintaining high productivity. On the other hand, excessive stress in the workplace leads to burnout and is highly detrimental to productivity.
Performance: Outcomes versus Output
Measuring performance in the SPACE framework is related to the outcome, and not strictly the output. Consider the aforementioned metric, counting the number of releases. A more pertinent question than whether a release is made on time is whether the goals of the release were achieved. Clearly, the outcome is more relevant to the project’s long-term health.
Activity
The SPACE framework includes the notion of an Activity, which represents a basic unit of work such as committing a block of code. Activities can be described and measured by a set of code metrics and as we have already discussed, tell only part of the story and are highly contextual. Nonetheless, code metrics are essential to measuring progress and need to be considered.
Communication and Collaboration
It is trite knowledge that people need to work together when a project is larger than a one-person job. Effective collaboration demands meaningful and effective communication, in addition to a diverse set of skills that are sufficient to handle all aspects of the project. An updated skills audit and the appropriate network metrics to show which team members are connected are invaluable documents for measuring and promoting collaboration.
Efficiency
Efficiency is the driving factor behind many tools, tips, and frameworks. Completing a project on time and budget is critical in many cases, yet regardless of industry, the price of doing so cannot be an unmaintainable project that can’t be scaled. An efficient process brings the project from inception to delivery, through a well-completed yet optimized series of steps in between. Efficiency is about cutting down on latency throughout the process, such as by reducing the number of teams involved and accounting for time lost due to such overhead.
The ultimate goal for SPACE is to evolve a codebase that is of higher quality with less technical debt and more maintainable code, leading to a better return on investment. This multifaceted approach adds a layer of administration but adds important aspects of accountability.
OKR
The OKR framework is for goal management that works off of the concept of Objectives and Key Results. Essentially, each OKR consists of an objective, which is a clear direction, and the key results, which is the measure indicating progress toward satisfying the objective. OKRs help developers to focus on the task at hand by breaking problems into manageable objectives and adding transparency by clearly pointing out what represents success. This is a very straightforward (and smart!) approach. You measure your actual goal but there is a clear price for that; it is the need to define new objectives and measurements for each different part of the project, sometimes even by iteration. If new objectives are not set or the measurements are not updated along the project lifetime, we get back to disoriented measurements.
Ways to Improve Developer Productivity
To this point, we understand that what works for one team or one project, or for one company, may not work elsewhere, even for a similar project. Choosing the proper measures that fit your team and project, even if not “the perfect measure”, should help in measuring developer productivity and getting a quantitive understanding, that can be then further analyzed. With a choice of measures available, we now consider how to go about with the improvement.
Productivity Tools
Myriad productivity tools exist to assist with improving developer productivity (and naturally, one that is high on our list is of course Incredibuild). Whether you consider productivity tools for C++ developers or a more general set of DevOps tools, all such solutions can make an important contribution when it comes to boosting productivity.
Even the simple efficiency-enhancing features included in modern IDEs can help to promote collaboration and build products with a high-quality codebase. For the best developer experience, it is important to optimize your IDE. For example, Microsoft provides tips for optimizing Visual Studio.
And for those who prefer a simple tool such as Vim, knowing the tricks of Vim, or any tool that you are using is of course very important to the developer’s productivity.
Which brings us to another measurement, which is much subjective and requires manual assessment, but is still important: are you using the right tools? Having a list of the tooling domains, checking that we use the appropriate tool per each domain, and use them properly, can be set as a measurement tool by itself (for more on C++ tooling, you may want to watch the 2022 Best C++ Ecosystems Virtual Round Table, discussing the C++ ecosystem and stack of tools).
Focus on Efficiency
Improving productivity is not just a plan but a mindset, which comes with a collection of techniques. For example, actively working to reduce the technical debt of a project will increase productivity in the long run. Similarly, one should strive to improve development time by using approaches to reduce your C++ compile time.
Efficiency can also be improved through the use of automation. It is clear that if our testing is mostly manual, we may be doing something wrong in terms of efficiency. This goes quite the same for DevOps processes.
Final Thoughts
Improving developer productivity is an important step in the goal of releasing high-quality applications with minimal time to market. There is no one metric or approach that is perfect for measuring developer productivity, although there are popular choices available that can be fitted to your environment. As the field of software development advances, new metrics and measuring code quality and developer productivity arise.
Adopting and developing code metrics that are efficient and effective in your environment will help to measure progress, not just for the individual coder but for the team’s collaborative efforts. This is important because productivity is not just about the individual. Once productivity can be accurately measured, a focus can be made on improving it by using proper tools and techniques. At the same time it is crucial to regularly check that your metrics drive you to your goals, and do not create a false sense of progress while deviating your path, something that may happen if following bad metrics (such as the naive LOC metric).
Improving code quality is one of the problems that can be addressed proactively by using the right tools and techniques, and measured sucessfully. Proper planning and collaboration will lead to fewer errors, and pre-production steps such as employing static code analysis tools will help to curb problems like software vulnerabilities. With the proper metrics, tools, and techniques in place, it is indeed possible to improve productivity and follow it.
Ultimately, a sensible push to improve and measure developer productivity will lead not only to a better product, higher quality, and improved time-to-market but also to a better work environment, more orderly managed and with much less surprises and need of putting out fires.
Table of Contents
Shorten your builds
Incredibuild empowers your teams to be productive and focus on innovating.