A recent IDC report on DevOps trends predicts that “the DevOps market expects to grow from $2.9 billion to $8 billion by 2022” in part due to the continued rise in DevOps tools offerings.
Although adopting a DevOps culture is more than the tools used, the best DevOps tools certainly play a critical role in shaping other decisions and attitudes and often seamlessly integrate. Here, I’ll draw on my own DevOps experience to highlight a few of the best DevOps tools in the arena. Unfortunately, it is not feasible to touch on every tool currently available – and new ones emerge often – so many excellent tools are omitted by necessity in order to focus on the best DevOps tools. Many different tools exist for each facet of DevOps work – such as project management, version control, infrastructure as code, continuous deployment, feature flagging, and more.
1. Git – Version Control
What: Git is a version control system designed to manage work for distributed projects and teams. Although widely adopted implementation for Git is via hosted services such as GitHub and GitLab, many organizations choose to self-host Git on-premise in accordance with policies or governance.
Why Git: Adoption of Git as a technology has exploded since its release in 2005 to become the de facto standard of software version control.
- GitHub is perhaps the best known Git as a service provider on the market, with widespread adoption and a vibrant user community. Acquired by Microsoft in 2018, Github supports actions, security alerting, discussions, and subscriptions among other features. GitHub provides a built-in container registry and Kubernetes integration. These capabilities move GitHub out of purely a code version management space and places it at the intersection of development, communication, and security.
- GitLab offers many of the same features as GitHub, and more, including data analysis of your project to suggest possible impact to the systems resulting from a code change. GitLab also includes a built-in container repository and Kubernetes integration for DevOps support, quite similar to GitHub. For more info regarding what is GitLab.
Both GitHub and GitLab support a type of code execution platform that can be integrated into the software development lifecycle (SDLC) – an important DevOps capability.
In addition to these solutions, a GitOps approach to software management is rapidly becoming synonymous with Git itself.
Where: https://git-scm.com
Other Options: Bitbucket, Sourceforge
2. Docker – Containers
What: Docker offers abstraction of the environment in which application would run, through reusable packages called containers; containers are isolated from one another and bundle their own software, libraries, and configuration files.
Why: Few technologies have had as significant an impact on software development as Docker. In 2013 when Docker was released, containerization was not a new concept nor was Docker the first platform to implement containers, however, Docker Engine commoditized containers and ignited the imagination of engineers around the world. More recently, Docker has become a critical component of the adoption of the microservices architecture by supporting the highly reusable, service-oriented container components.
Where: https://www.docker.com/
Other Options: Linux LXC, Microsoft Hyper-V, rkt.
Related: Docker vs VM. To briefly summarize, Docker containers are only isolated to a limited extent, they all still run on the same host machine and there is much less isolation compared to virtualization solutions.
3. Terraform – Infrastructure As Code
What: Terraform is an infrastructure as a code tool built by Hashicorp. Resources are described using declarative syntax to support natural and highly reproducible configuration.
Terraform runs in two phases – a planning step where an execution plan is generated, and an execution step where the plan is applied.
Why: Terraform has commoditized the pre-existing infrastructure as code space to become the most widely adopted solution supporting public cloud infrastructure, private cloud infrastructure, network appliances, software as a service, and platform as a service.
Where: https://www.terraform.io/
Other Options: Pulumi, Ansible, Salt
4. Kubernetes – Container Orchestration
What: Kubernetes is a container orchestration platform supporting automated deployment, scaling, and management of applications. The Kubernetes ecosystem also supports many different management strategies and utilities such as etcd for key / value pair management and Rancher or Portainer for operational management. Many organizations also find value in implementing a service mesh such as Istio or Linkerd to manage interactions between containers and services.
Why: Since its initial release in 2014, Kubernetes (k8s) has become the preeminent platform for container management and orchestration. In addition to the managed implementations, both on-premise and Cloud, k8s has been adopted by the major Cloud providers as a managed service offering – such as in the case of AWS Elastic Kubernetes Service (EKS), GCP Kubernetes Engine (GKE), and Azure Kubernetes Service (AKS). Kubernetes also underpins some hosted container platforms such as AWS Elastic Container Service (ECS).
The Kubernetes platform also supports many extensions and many additional services work with Kubernetes. The list includes tools and services such as Helm, knative, and Keda. These are not discussed in detail here, but are surely worth a look in the future.
Where: https://kubernetes.io/
Other Options: Openshift, Rancher, Nomad
Related:
Docker vs Kubernetes – Should We Really Compare?
Using Kubernetes for CI Build Jobs and Generic Processing Tasks
5. Jenkins – Continuous Integration
What: Jenkins is still the most popular CI pipeline tool, widely adopted as an open source platform for automating tasks during the software development lifecycle (SDLC). Supporting a wide variety of plugins to extend its native capabilities, Jenkins directs the traffic at the crossroads between development, automation, release management, and more. For more information about what is Jenkins click here (including Incredibuild integration with Jenkins).
Why: Jenkins is the Swiss Army Knife of the SDLC with either support or a plugin sure to be available for any DevOps needs.
Where: https://www.jenkins.io/
Other Options: In a previous posts we already discussed Jenkins vs bamboo, Jenkins vs TeamCity and Jenkins vs Azure DevOps.
6. ArgoCD – Continuous Deployment
What: ArgoCD is a declarative continuous delivery (CD) tool for Kubernetes supporting GitOps. ArgoCD manages Kubernetes resources declaratively – supporting Helm charts, kustomize applications, bare YAML configuration, and more – in an application pattern of directly managing resources, or in an inherited model of applications abstracted into parent applications.
Why: Deploying infrastructure and installing container management frameworks is all in the service of deploying the containers themselves; for this purpose, ArgoCD is one of the frontrunners. Seamlessly integrating upstream with tools like Jenkins and CircleCI, or directly with Git repositories, ArgoCD waits for notification that a change is needed then performs a diff of current and target states before taking action to rationalize the diff.
Where: https://argoproj.github.io/argo-cd
Other Options: GitHub, Harness, Spinnaker
7. ELK Stack – Logging and Data Analysis
What: “ELK” refers to the combination of Elasticsearch, Logstash, and Kibana. Logstash gathers log data from remote endpoints such as systems or containers. Elasticsearch provides search and analytics capabilities. Kibana visualizes analytics results as charts and graphs.
Why: The ELK stack has been a mainstay of logging and analytics for many years due to its ease of use and high degree of flexibility. The stack serves to not only collect and aggregate data, but also to provide insights into its meaning and relationship to a myriad other data points. These insights can back-feed into DevOps decisions concerning scaling, stability, security, and more.
Where: https://www.elastic.co/what-is/elk-stack
Other Options: Splunk, Sumo Logic, Grafana-Loki
8. Cloud – Virtual Infrastructure and Managed Services
What: Cloud generally refers to virtualization of hardware abstracted by groups of services for allocating and managing resources.
Why: Data center virtualization was not new in 2002 when AWS was founded, but the growth of public Cloud adoption has without a doubt been one of the most significant trends of technology in the past 20 years. Cloud adoption remains commonplace, especially in large enterprise organizations during ongoing technology refresh and retirement of legacy systems, and Cloud has become a critical component of the technology industry.
In part, Cloud adoption remains strong due to the constant combined innovation of providers – such as Amazon, Google, and Microsoft – and in part due to the continued novelty of uses and configurations implemented by Cloud customers. This virtuous cycle suggests that Cloud still has much more value to give well into the future.
Where: https://aws.amazon.com/ | https://cloud.google.com/ | https://azure.microsoft.com/en-us/
Other Options: VMWare, Openshift
9. LaunchDarkly – Feature Flag Management
What: LaunchDarkly is a software tool for dynamically controlling software application availability through managing feature flags.
Why: Feature flagging is a key enabler for DevOps velocity, and its effective management can greatly improve a team’s effectiveness by decoupling development, and associated deployment, from release and impact. This supports frequent, confident release to production without necessitating a “test in production” approach.
LaunchDarkly consolidates management of flags to present a simplified view for easy visibility as well as offering support for targeted exposure of flags to end or internal users.
Where: https://launchdarkly.com/
Other Options: Split, ConfigCat
10. Prometheus – Monitoring & Alerting
What: Prometheus records metrics as real-time events stored in a time series database that can be accessed and aggregated using their own query language – PromQL. The software integrates upstream with other tools for visualization – such as Grafana – and alerting platforms – such as PagerDuty.
Why: The the pull-driven approach and the discovery of services – allowing to support the dynamic nature of microservices / containers / Kubernetes, metric-driven monitoring, combined with the time series approach to data storage, make Prometheus an ideal fit for highly dynamic operations such as Kubernetes containers or public Cloud compute resources. As an open-source project supported by the CNCF, Prometheus enjoys a vibrant user community that won’t break the bank.
Where: https://prometheus.io
Other Options: Datadog, New Relic, Zabbix
11. Dependabot – Code Dependency Updates
What: Dependabot is an open source tool for daily scanning of code dependencies leading to pull requests to rationalize outdated requirements.
Why: Numerous security breaches every year can be traced back to software that has not applied available patches for vulnerabilities. Although Dependabot won’t solve every patching issue that may arise, automating the work out of the SDLC not only saves developers time and energy but helps to ensure that changes are applied in a timely fashion while still subject to code practices via Git pull requests.
Where: https://dependabot.com
Other Options: Snyk, SonarQube
12. Slack – Communications
What: Slack is a business communications platform including features such as ad hoc group chats, video calls, and broadcast announcements (@channel is not for the shy). Slack also supports cross-organization communication such as with public forums like the Cloud Native Computing Foundation (CNCF) – cloud-native.slack.com as well as a C++ quite active forum – cpplang.slack.com – or for organizations to include external parties in discussions such as working with a vendor on troubleshooting an issue. Slack API also allows integration with automation scripts as well as with external platforms and other providers, thus allowing for example to automate slack messages based on external events.
Why: Effective communication is the lifeblood of any organization; more so one pursuing a DevOps culture. Over the past 18 months, this has been thrown into sharp reflection by world events. In addition to the resounding popularity of Slack, the software features are a compelling case for its adoption in organizations large to small.
Where: https://slack.com
Other Options: Microsoft Teams, Google Chat, Discord
13. Jira – Project Management
What: Jira is a project management and issue tracking platform from Atlassian.
Why: Teams follow many different strategies for project management – from Agile to Kanban to Lean – and Jira consistently remains the most widely adopted tool for making sense of work effort regardless of strategy. DevOps teams are most effective when communicating, both in a sync mode (e.g. with Slack) or in an async mode, through dashboards and project plans such as are offered by Jira.
Where: https://www.atlassian.com/software/jira
Other Options: Zepel, Basecamp, Trello
What About Incredibuild?
Have you examined Incredibuild as part of your DevOps journey? Incredibuild turbocharges development from compilation to testing and other compute-intensive tasks.
Software delivery velocity can be influenced by a wide range of factors – such as the time required to launch compute resources in the Cloud or time required to complete testing of software before release – but for compiled languages – such as C / C++ – the time required to compile large projects can be a significant factor in delivery velocity. By distributing compilation processes across other machines in your network of public cloud, Incredibuild helps to reduce compilation time and improve both velocity and the time required to deliver fixes and iterative changes. Try our free license.
The Best DevOps Tools – Final Note
DevOps should be a symbiosis between culture and tools and while many organizations have found those listed to be the best fit for them, the space is rich with alternatives and options which are constantly being updated and expanded with new options emerging constantly. Not every DevOps journey is the same – in fact, almost all are unique – but exploring and deploying the best DevOps tools is an excellent first step.
