Introduction
The software development and deployment cycle usually involves numerous and tedious steps other than just “coding”. These steps often includes: download dependencies, build with different options, run various static analysis tools to check if the project meets the minimum predefined set of quality metrics, execute unit tests and check code coverage, execute functional and other types of tests, generate documentation, packaging, etc.
The ability to run all the development steps easily and frequently in a consistent and automatic way is important to minimize human errors, improve the software quality and speeds up the whole development cycle.
The Problem
Some of the development steps can be automated by Continuous Integration (CI) or Continuous Delivery (CD) systems but others needs to be performed frequently on the development machine before committing the code, and this poses some challenges:
- Asking every developer contributing to a particular software project to manually execute a long list of Quality-Assurance (QA) steps before committing the code is impractical and prone to errors. Indeed, each development step may require different options and settings depending of the development machine environment.
- Different sets of language-dependent tools exist to automate various parts of the development process but they don’t play well in case of multiple-language projects, and they often have peculiarities and extra dependencies to be managed manually.
- As there is not a common standard, even projects written in the same programming language may require different development steps.
- The time spent by developers to successfully interact for the first time with a software project can be seen as an entry barrier. This is aspect is particularly important in Open-Source-Software (OSS) projects because it may significantly affect the adoption rate.
- All the important development steps should be written in a Single-Source-Of-Truth (SSOT), avoiding duplication between documentation, local automation systems and CI/CD systems.
- Having the development steps configured in an external CI/CD tool may break the build of a new software version that requires some build changes (e.g. different dependencies).
Proposed Solution
Amongst various potential solutions, one that stands out particularly is the usage of GNU Make as a top-level entry point with a set of common targets.
First released in 1977, GNU Make is a well known and extremely stable software, available virtually for any existing Operating System. As it is not limited to any particular language, Make can be used to execute arbitrary commands. In our case these commands are the development steps organized or grouped in common targets.
All is required to use Make with a software project is to add a file named Makefile in the root directory of the project. This file will act as a Single-Source-Of-Truth (SSOT) for all the development steps divided in several targets.
The two fundamental targets are:
help: display the help menu with the list of available targets.all: alias for the “help” target.
With these two targets everyone can easily and immediately
access the list of all available options just typing the command
“make” when in the root directory of
the project.
Other recommended common target names are:
qa: Run all the Quality-Assurance (QA) steps and generate reports.test: Run the unit tests (already included in the “qa” target).format: Format the source code.docs: Generate source code documentation.deps: Get the software dependencies.build: Compile/build the software.dbuild: Build everything inside a Docker container to guarantee the consistency of the building environment.clean: Remove any build artifact.install: Install the software in the current environment.uninstall: Uninstall the software from the current environment.rpm: Build an RPM package.deb: Build a DEB package.bz2: Build a tar bz2 (tbz2) compressed archive.docker: Build a docker container to run this service.
The commands inside each target can vary deeply between different projects, but they tend to be very similar for the same programming language, so it is possible to write a generic Makefile for each language to be reused with few minor tweaks. This is even easier if the projects shares a common Software Structure.
As a general rule, if specific language or project build tools already exist for a particular project, then the Makefile should act as a wrapper for these tools and not as a replacement. The main intent is to provide a common entry point regardless of the specificity and complexity of the internal commands.
Once such Makefile is in place, executing all the QA
tasks either locally or using a CI/CD system is easy as invoking
the “make qa” command. This means
that configuring a CI/CD system will be very easy and it will
require minimal maintenance.
To guarantee the building environment we can always execute the make commands inside a Virtual Machine (VM) or a Docker container. On another article I will explain how to automate the build inside a Docker container using make.