Experienced programmers of any language are familiar with code dependencies. Whether your code is relying on an internal dependency, or any external library or framework, it usually does not operate in isolation. Code reuse, the practice of using existing code, is an important tool for efficient development, but when code is reused, you’ve created a dependency. Unfortunately, dependencies have a dark side. They are difficult to manage.
What Is Dependency Management?
Dependency Management is a multifaceted process that includes building and maintaining a list of external entities being called from your codebase. Unfortunately, this is not as easy as it sounds. The first problem is that you have to make sure that you are bringing in the right dependencies. Then, you need to worry about dealing with dependency conflicts.
Maintaining a dependency list is something that needs to be efficient and reliable, and as such, should be as automated as possible. At the same time, tasks like resolving dependency conflicts require manual intervention. To assist with the process, developers make use of dependency management tools.
Dependency Management Tools
A dependency management tool is responsible not only for building the list of dependencies but keeping apprised of the latest updates. For example, when a new version of a library is released, it can have effects on the codebase that the tool will need to identify and deal with.
A dependency management tool will keep an eye on things like dynamically linked libraries, regularly working to identify and resolve any problems that should arise. Problems can occur, for example, by having different dependencies relying on different versions of the same artifact. If dependency “A” relies on a specific version of dependency “C”, and at the same time, dependency “B” relies on another version of dependency “C”, then we have a dependency conflict that needs to be solved.
A common problem in dependency management is keeping them up to date. Ideally, dependencies would be kept up to date to benefit from the latest changes. Even in a stable product that will not necessarily take advantage of new features, it is still important to implement the most recent bug fixes and other improvements. However, this is often not the case. In fact, it is commonplace that updates are ignored and the process can drag out for some time.
Done manually, the process is often a complex and time-consuming activity, which is why dependency management tools strive to automate the process. That said, it is important to remember that resolving dependency conflicts would still usually require manual intervention.
Dependency Management tools are common in any software development environment, regardless of the programming language. For example, you’ll find Maven and Gradle are popular with Java programmers, pip is common in Python circles, npm and Yarn are used for Node.js, and Composer works well for PHP projects. Even Rust, which is quite a young language, has had Cargo from its early stages as a useful tool for package management. Myriad other package management tools exist for these and other languages, but a C++ Dependency Management tool is indeed a rare breed.
Why Is Dependency Management in C++ Difficult?
If there is one thing that experienced C++ developers know about using dependencies, it’s that C++ dependency management is difficult to do. In fact, unlike other languages, C++ does not have a standard or dominant package manager. The consequence is that C++ dependency management is either trivial, such as by employing copy-and-paste on files with no automated management whatsoever, or instead, it relies on tools like APT that are operating-system-specific.
Complex with a lack of specification or standard
The C/C++ ecosystem is large, mature, and rather complex. Considering the plethora of extant third-party libraries, including those supported by the open-source community, and that they have evolved with time across platforms, across operating systems, and even with versions of the language, it has resulted in a fragmented ecosystem where dependency management can be far from the time-saving measure that it should be. There are different build systems, for example, each complete with idiosyncrasies and options to choose from. Dealing with the different build systems is difficult, in and of itself, yet this is only one part of the puzzle.
Portability
Another reason that C++ dependency management is difficult is that source code is portable, yet binary files are not. This is unlike Java, for example, which uses the JVM to execute compiled code across platforms. With C++, libraries are available for different platforms and when managing dependencies, if portability is to be maintained then the same libraries for different platforms all have to be maintained. Keeping different artifacts, for different platforms, might be a solution, but the number of combinations for ABI compatibility can be huge, if one needs to store a precompiled binary for each and every architecture and operating system out there (and in some cases this is not enough, having different ABIs for the same OS, based on compilation flags).
Dependency management tools use different techniques to deal with the issue of portability. For example, the Conan C/C++ Package Manager, which we cover below, uses a decentralized system to maintain a repository system for multi-binary packages. Cargo, which is the Rust package manager, instead retrieves dependencies from crates.io, which is a central package registry maintained by the community.
How Do We Manage C++ Dependencies?
Development shops use a variety of approaches for C++ dependency management ranging from manual downloading, transferring, or copying files, to writing or adopting more complex tools to assist with the task. Some programmers choose to use tools that are specific to the operating system, such as APT, whereas others will take a more manual approach.
For those following a more manual approach, it is all but guaranteed to be less efficient and cause errors, leading to a need to manually maintain a dependency graph to avoid versions conflict. As we strive for efficiency in both development and processing time, it is recommended to look at some of the popular C++ dependency management systems that exist.
Existing Tools
Several C++ dependency management tools are being actively developed, each with its features, advantages, and disadvantages. Popular features include integration with CMake and IDEs like Visual Studio, cross-platform support, and ease of use. The two most commonly used and mature tools are Conan and vcpkg, although there are other options such as Buckaroo, Hunter, and Build2, described below.
The Conan C/C++ Package Manager is an open-source project that uses a flexible, decentralized client-server model to store packages that can be retrieved by clients. It supports multiple platforms and toolsets, allows for the storage of both source-code and binary files, and has a rich set of options that allows you to fine-tune the process for different dev environments. The documentation is well-written and it has an active community that includes hundreds of companies and thousands of developers. Currently, the number of packages in the ConanCenter repo is 2,611.
The vcpkg package manager is an open-source, cross-platform dependency manager from Microsoft. It is used for retrieving and managing libraries from open-source and private repositories, is easy to use, and features Visual Studio and Visual Studio Code integration. There is good documentation available for beginners, including a helpful section on getting started. Vcpkg way for retrieving a specific version of a library is based on retrieving the relevant branch from the vcpkg repo, an approach that is viewed by some as cumbersome and too technical (without any declarative simple syntax for achieving that). Currently, the number of packages in the vcpkg repo is 1,822.
Buckaroo is an open-source solution that labels itself a Package Manager for C++ and Friends. The documentation includes tips for getting started, integration with popular IDEs, and some helpful tips and tricks. Unlike other dependency managers that fully support the CMake build system, Buckaroo defaults to the Buck Build system and has limited CMake support. Buck is a build system developed by Facebook that encourages the use of small modules, but it does not support the packaging of binary dependencies. Dependencies can be pulled directly from GitHub, BitBucket, and GitLab.
Hunter is a dependency manager that supports multiple platforms and can be used for C/C++, as well as other languages. It is designed to manage packages using the CMake build system, although it supports non-CMake packages through the use of custom templates. The documentation includes a Quick Start guide, as well as instructions to integrate it with popular IDEs. The major advantage of Hunter is that it only needs CMake, so it is relatively transparent to the developers. The primary complaint by users is that everything is built from source, and thus more time-consuming to use.
Build2 is a build system by itself. It offers both the building part and the package management part. The documentation includes instructions on installation and getting started, as well as usage examples. Its advantage is that by being a complete tool-chain, including the building part and the package management part, it only requires a C++ compiler and that’s it. On the other hand, many C++ developers would not consider moving to a new build system in order to gain the package management capabilities, and build2 is firstly a build system that doesn’t integrate or work with make, ninja, or CMake. So you have to move to build2 as your build system. The repository maintained for build2 is under https://cppget.org with currently only 95 hosted packages, which is not a high number.
Best Practices
Efficient dependency management that is working properly should be virtually transparent to developers. Of course, errors inevitably appear and something needs to be adjusted, but this can be minimized by following a set of best practices.
There are a few important takeaways, and the first is that builds should be stable and consistent across platforms and operating systems. If a build runs on one machine, then it should build and run the same way on other machines in different environments. This means that dependencies need to be properly maintained not only according to version but also according to the platform. Though not related directly to dependency management, developers must remember that if building for multiple environments, the outcome shall be fully tested under each environment, even if your code is exactly the same.
Second, dependencies should be reused as often as practical, pushing for a parsimonious dependency graph. Components within a project rarely are unique in terms of what they do or what functionality they rely on to operate. As such, make sure of what is there and don’t scale or overcomplicate things unnecessarily. A very simple example of this is there is usually no need to depend on two different logging libraries in a single project. That said, it is understood that the reason dependency management is needed in the first place is that the environment is complex.
Finally, whenever possible, don’t do anything manually.
Conclusion
Dependency management is a collection of tools and techniques used to identify and resolve issues related to dependencies in your codebase. Dependencies come in many forms including libraries and frameworks and are found in projects ranging from small to large, from proof of concept to maturity, across industries, across coder skills levels, and across programming languages.
C++ dependency management is more difficult than it is in other languages because of a lack of standards, issues with portability, maturity of the global codebase, and inherent complexity of the language. There are both good and bad ways to tackle the problem but fortunately, several tools are available to assist you with the task. The most popular of these tools are Conan and vcpkg, but there are several more to try if you feel so inclined. Regardless of the package or toolset that you choose, unquestionably it is a reprieve from the manual copy-and-paste from days past (and not so past for others).
Relevant Links
A nice comparison table – vcpkg vs. Conan, a bit old but still mostly relevant.
Posts discussing C++ dependency management, you may take inspiration from, but don’t take it as set in stone:
Best practice of C/C++ dependency management on build servers?
Dependency Management Best Practices?
And last, Maven NAR plugin for C++ as another option for dependency management in C++.
