Using DevOps? Combine CI/CD With Multi-Repository

Implementing continuous integration/continuous development (CI/CD) in DevOps is an effective way to improve your company’s performance. That’s a given, but you can go even further by applying microservices to the same DevOps architecture to reap real rewards especially when compared with monolithic architectures. Sounds like something you could use for your team?

So let’s explore how CI/CD and microservices relate to one another, the products that use microservices, and how they apply to both mono and multi-repository solutions. And that’s not all, we’ll also examine the mono repo versus multi repo debate, and which one we think is best for DevOps teams like yours.

DevOps

Before we go any further however it would be worth our time to go back to the basics on what exactly DevOps is and why this concept is so important. The term ‘DevOps’ is a portmanteau of the words ‘development’ and ‘operations,’ intended to highlight a collaborative approach utilized by developers while working towards application development as well as general IT operations. Certainly, DevOps is a term with considerable practical application, but it perhaps can be best understood from a conceptual perspective.

The entire concept of DevOps is predicated on improving communication between the various teams that make up a company’s IT operations. This is achieved via the practical aspect of DevOps, by driving the iterative development of software as well as automation, programmable infrastructure deployment and maintenance, to encourage and create cultural changes within a company. This, in turns, creates a corporate environment characterized by increased trust and cohesion, generating tangible benefits for the software delivery chain, services, job roles, IT tools and best practices, all thanks to DevOps.

Continuous Integration/Continuous Development

Continuous integration (CI) is a process flow designed to assist developers in making small changes and checking code to version control repositories on a routine basis. As most applications and systems today need to code a mechanism to integrate and validate any changes that are made as required. This occurs by running automated tests to ensure integration challenges can easily be avoided when working with merge changes into the release branch.

Continuous development (CD) works as an extension to CI as all the code changes are deployed to a testing or production environment after the build stage is complete. This allows the operator to deliver information on a daily, weekly, fortnightly basis and provide CI/CD services on a regular basis.

This highlights the benefits of CI/CD as companies that adopt CI/CD best practices have a considerable degree of adaptability. In particular, if you work in non-production areas, including research & development, this CI/CD flow is useful as it can be used to push code changes.

If you use a multi-repository system for your microservices you need to select one that integrates well with your CI/CD processes. Make sure it can implement the following practical steps;

• Build all services involved as separate components.

• Test entire platforms and deploy them as one single entity.

• Provide clear visioning and maintenance for old versions of all platforms.

• Smooth integration with developer workflow and overall CI/CD pipelines.

• Easily scaled deployment in developmental workflows.

Use a tool that work with Git Lab, a platform you can use to develop, secure, and operate software in a single application. You should then use git-flow, specifically, a rebase flow, as this approach will provide you with clean git history.

Microservices

Microservices were developed so that companies could solve complex IT challenges and represent a suite of small services (hence the name) where each performs a specific role like shipping, product searches, deliveries, etc. Communication between these services takes place via API gateways and is becoming more popular as companies increasingly adopt Agile methodology, along with DevOps and continuous testing.

KaaIoT IAM enables companies to create highly granular access control policies for their digital and cyber-physical assets based on high-level parameters, such as resource type, as well as low-level attributes, such as device status or IP address.

Microservices are highly modular, developed as an alternative to monolithic architecture where each modification can require an entirely new version of software. This is really beneficial as it enables developers to build an application as a suite of services, each running in its own processes and independently deployable.

These services are often written in different programming languages and can also utilize different data storage techniques, which results in a system characterized by scalability and flexibility. There’s no set microservice standard, as you may expect from such an adaptable concept, but most systems usually include multiple components, simple routing, failure resistance, and fail-safes, all marked by a decentralized approach to storage.

Who Uses Microservices

As you can imagine, microservices are utilized by the DevOps teams of numerous companies in a myriad number of industries like the following:

E-commerce

Amazon achieved much of the success that it did after it used microservices to shorten its development pipeline for customers. Previously, its internal systems had been highly centralized, but Amazon used microservices to identify bottlenecks and replace them with service-oriented architecture.

Media

Netflix was a pioneer of microservices back in 2009, now the streaming company leverages 500+ microservices with API Gateways that handle over 2 billion API edge requests on a daily basis.

Transport

When Uber entered the market it faced scalability and continuous integration issues as the company’s monolithic and centralized system wasn’t fit for purpose. In response, Uber’s DevOps team introduced an API gateway combined with independent services.

Fashion

German fashion giant Zalando once relied on a bloated monolithic structure with the resulting dependency and coordination problems, leading to slow development cycles. The company adopted microservices to provide contingencies for the company’s code by making each internal team act as an SaaS to its 69 other counterparts.

Mono-Repository & Multi-Repository

When you start using microservices you need to decide how you’re going to organize your codebase and there are two major strategies used to plan a code repository system; Mono-repository and multi-repository.

The mono-repository strategy employs one single repository to host all the code for the multiple libraries or services that comprise a DevOps team’s ongoing processes and usually means that all of a company’s codebase is stored in one location as part of the DevOps process.

The benefits of the mono-repository system include the fact that it provides visibility of all the code used by a company to all of its developers, making troubleshooting simple. This also makes it easy to perform code modifications in a company’s library under a single pull request. Also, it provides DevOps security as teams are able to share a development culture efficiently and securely, reducing the chance of miscommunication and mistakes.

However, the amount of code stored in a mono-repository can be huge and the system experiences slower development cycles as there’s a tendency for programmers to have to follow the slowest member of each team. Also, if you use a conventional CI system that only works per project, you may find that using a mono-repository system can experience problems with constant builds.

Benefits of Multi-Repository

Using a multi-repository system has a number of benefits that should make it the first choice for CI/CD best practices. For starters, when a repository is tagged its whole codebase is assigned the new tag which creates independent library versioning.

Your code enjoys its own development cycle as your multi-repository only contains the code for some services, ensuring a fast release cycle as well as continuous delivery for your team. Control access across your organization is easier to define, and you will find that your DevOps teams will be able to work more autonomously as they will be able to design your library’s architecture and implement code in isolation.

If running a multi-repository system you should try to meet a few standards that will ensure your system will run to an optimal standard, and you should start off by ensuring that your library is resynced frequently for optimal performance. Watch out for team fragmentation by ensuring high-level communication between the different autonomous units working in isolation from one another. You need to ensure that these two points are covered for an efficient multi-repository for CI/CD systems.

CI/CD & Multi-Repository

If you choose to use a multi-repository system for your microservices you need to select one that integrates well with your CI/CD processes. Such a system needs to perform a number of functions, including being able to build all services involved as separate components, test each of these services, test whole platforms as well as being able to deploy them as one single entity, and provide clear versioning and maintenance for old versions of said platforms.

At Kaa, we recommend using a tool that works with Git Lab, a platform you can use to develop, secure, and operate software in a single application. You should then use git-flow, specifically, a rebase flow, as this approach will provide you with clean git history. You’ll find that this is very useful when multiple team members work on the same service.

Takeaway

Using a version control system for microservices that brings together the best aspects of CI/CD and multi-repository solutions will offer considerable benefit to your team. Read our next article about CI processing for automated versioning and leverage this for your company to optimize DevOps costs for your enterprise.