JenkinsX - new Kubernetes dream? Part 1

Hi there!

Today we’re going to share about Jenkins X.

I think every developer knows what Jenkins is. Or at least has heard of it once. But what is ‘X’?

Basically, Jenkins X is not just a CI/CD tool to run your builds and deployments, it is an attempt to automate the whole development process end to end for containerised applications based on Docker and Kubernetes. It is obviously open source, as all best applications.

This sounds very promising at a high level. So, let’s see what’s behind the glam cover of Jenkins X!

TL;DR:

Here is our takeaway  from the Jenkins X project:

Though project has been made public just recently, it appears to be mature enough to make use of it in Production systems.

Although migration of existing CI/CD pipelines into Jenkins X might be found difficult due to existing custom processes, new projects may benefit from using Jenkins X a lot, as it saves significant amount of effort by setting up basic GitOps flow out of the box.

Customisation of the Jenkins X system shouldn’t be a problem as it utilizes already familiar tools like Jenkins, Make, Docker, Helm

Strengths of Jenkins X:

• Entry level for DevOps/GitOps implementation lowers significantly due to E2E automation of GitOps processes
• Strong concept: GitOps processes as a foundation of the project
• Good toolset out of the box, which is already configured and works (k8s, Jenkins, Docker registry, Chartmuseum, Monokular, Nexus)
• Quickstarts make a creation of new apps an easy ride
• Ability to customise the pipelines and their templates

Not that good points of Jenkins X:

• Jenkins X is another framework to learn. It requires understanding of its concepts, though still requires understanding of Kubernetes, Helm, Docker concepts
• It is only the early beginning of the project, still a lot of things to implement and improve
• Documentation is not always comprehensive and only provides basic information. That entails lots of questions and makes it sometimes difficult to understand all the caveats.
• Only a few quickstarts are available and it requires to raise an issue on Jenkins X Github to create new ones.

Our opinion on Jenkins X is that it goes into right direction, addressing the pain points and streamlining implementation of GitOps principles. So it’s effectively DevOps for everyone, providing basic, standardised implementation, though there is still a room for customization.

Most probably, we’ll see more radical implementations of similar tools in future, making PaaS implementations even easier, but less customizable. However, this is the common technological trend: deliver things simpler, faster, cheaper.

Ok, if you want to get to know Jenkins X closer, let’s jump into details!

The problem

The main problem, which Jenkins X (‘JX’ hereafter) is going to address is formulated by its creators as ‘make it simple for developers to work to DevOps principles and best practices’. Some of that are:

• Frequent deployments
• Low Mean Time to Recover
• CI/CD
• Configuration as Code
• Automated Release Management

In plain English, that means JX takes a challenge to automate a creation of development environments and facilitate development process. So, once JX is installed and set up, it does:

• create a Git repo for a new application
• create a pipeline configuration in Jenkins for a new application and connect it with a Git repo
• automate the DevOps processes (like builds, artefacts and containers creation and publishing, deployments) based on Git operations (branching, commits, PR creating, PR merging)

Jenkins X building blocks:

In order to achieve the goal and solve the problem set, JX makes use of 4 main building blocks:

• Jenkins X – a standalone platform (not just an extension of Jenkins), that coordinates all the components below. Provides an abstraction layer which facilitates the communication and management of those system components. The abstractions of Jenkins X utilise the terms of DevOps (e.g. application, environment, promotion) explicitly, thus converging terminologies of different technologies. Comes with a CLI to manage the platform.
• Git – stores all the code and configurations, including environments setup. Serves as a source of truth for everything. Jenkins X manages remote Git repositories and follows the GitOps principles (explained further in the article)
• Kubernetes – manages containers at scale, providing work environments for all developing and administrative applications. The resources inside k8s are managed with help of Helm and organized in Helm Charts for convenience. Jenkins X designed to install all the modules on k8s cluster. It also can create a k8s cluster itself with the following cloud providers supported:
  • AWS
  • GKE
  • Azure
  • Minikube
  • OpenShift (planned)
  • EKS (planned)
• Jenkins – open source CI/CD solution. Jenkins X make use of Jenkins to create and run CI/CD pipelines.

Jenkins X abstractions:

As different components of the system utilise different vocabulary, Jenkins X tries to span across all domains with its own abstraction layer. And the most important entities of that one include the following items:

• Environment – is a container where Applications are deployed to. By default, ‘staging’ and ‘production’ environments are created during JX installation to k8s
• Application – is a representation of an application being developed.

There is much more entities than that in JX, but the ones above are essential for understanding the concept of JX.

Jenkins X components relationship

Now, that we know the main building blocks, let’s try to put them together. Next picture shows how parts of JX system are connected in static:

Figure 1. Components of the system

As you can see, associations between different entities of the components are pretty straight forward: JX abstractions are mapped 1:1 to Git, k8s and Helm objects.

To get more understanding of how everything works, It’s worth to consider each of the components in more details:

Kubernetes and Helm

Kubernetes is the foundation of a development infrastructure. Its ecosystem covers most of the software development and operational needs in modern IT world. Distributed cluster manages Docker containers and provides features like auto-scaling, self-healing, secrets and configuration management, automatic rollout/rollback and others.

For more about Kubernetes, please check out following articles:

https://blog.octo.com/en/how-does-it-work-kubernetes-episode-1-kubernetes-general-architecture/

https://blog.octo.com/en/the-twelve-factors-kubernetes/

Kubernetes hosts all services deployed by JX, including administrative ones (Jenkins, Chartmuseum, Monocular etc). Deployment of the services (or applications) is coordinated via Helm. Helm’s Charts allow sharing of application templates and makes versioning easy. Helm also takes care of upgrade and rollback cases, which makes it quite useful.

Each Helm Chart is a folder, that contains the following files:

• Chart.yaml: contains Chart’s metadata like name, version, description etc
• requirements.yaml: describes Chart’s dependencies on other charts. Each dependency consists of chart’s name, alias, version and repository of that dependency.
• templates/ directory: contains template files written in Go template language. Templates describe what containers to use, their replication factors, services descriptions etc
• values.yaml: contains default values for templates

As part of JX installation, following services are deployed to facilitate Helm Charts managements:

• Chartmuseum: Helm Charts repository, helps to manage charts via rest api
• Monocular: web-based UI to Helm Charts repository

Jenkins

Jenkins instance is deployed to Kubernetes cluster as part of the installation of JX. It runs in “master/agent” distributed mode. That means there is always a ‘master’ process running, serving requests to Jenkins console and distributing the load to multiple ‘agents’. This allows to process multiple pipeline runs in parallel and evenly distribute the load, create/delete agent nodes based on demand.

Master and each of the agents are deployed as separate Pods in Kubernetes, this can be easily monitored via kubectl CLI or a dashboard.

Jenkins pipelines are typically described in a configuration file called Jenkinsfile. This file contains set of stages to execute for each pipeline. It’s normally checked into VCS along with an application code.

Git

In order to fulfil GitOps requirements, Jx uses Git to store two types of data:

1. Application repo: this is the code of particular applications itself, something that developers work on. Each application is associated with one Git repository. Apart from the code, such repos also include:
   1. Jenkinsfile: describes Jenkins pipeline configuration for the app
   2. Dockerfile: contains Docker  image building instructions for an application
   3. charts/ directory: contains a set of charts for an application:
      1. Chart config for app version. Used when app version is created (first time promoted to any environment). Also includes Makefile to describe how to build a version
      2. Chart config for preview. Used only when Git PR is created for feature branch. Facilitates a process of review and merge approval for a PR. Also includes Makefile to describe how to build a preview
2. Environments repo: these contain a set of configuration files, which describe environments metadata. Each environment is associated with one Git repo. Also includes:
   1. Jenkinsfile: contains Jenkins pipeline configuration for the environment. In conjunction with Makefile describes steps on how to (re)build the environment
   2. Makefile: contains environment building commands. In current implementation, it completely relies on Helm capabilities, in particular ‘helm upgrade’ command
   3. env/ directory: contains configuration of environment contents. That means, what applications of which versions should be deployed there.

So, JX is connecting all these pieces together with its smartly weaved fabric of automation scripts and configurations. And the pattern of that fabric is known as GitOps**.**

GitOps

What is GitOps? This is a set of principles for managing software and infrastructure based on Git:

• Git is considered to be a Source of Truth for everything, from code to environments setup.
• Any operational changes, including environments config updates, are done via Pull Requests
• A declarative configuration of infrastructure is implied
• Any divergence from Git version is detected by diff tools and convergence mechanism is triggered
• In case if rollback is needed or disaster happens, the state of the system can easily be pulled from Git

As you can see, there is a direct association between JX Environments, Git Repos and k8s Namespaces. Actually, JX environment is nothing more than just an abstraction, which keeps configuration in specific Git repo and deploys to the corresponding k8s namespace.

Applications (in our case it’s myGreatApp) are another abstraction in JX. For each of the applications, corresponding Git repo is created. Different versions of apps can be deployed (promoted) to different environments in k8s, but only through changing of the configuration of particular environment in Git.

Jenkins X CLI

Now, we’re coming to the main element of JX, the tool which does all the magic, glueing all building blocks together and providing an entry point to a system management and orchestration. It’s a JX CLI interface and it’s written in Go. The JX CLI is utilized by end users to manage resources (apps, environments, urls etc) as well as in Jenkins pipelines created by JX. Some of the most important commands are provided below:

• jx install - installs JX on k8s cluster
• jx create - creates JX resources and associated underlying services (e.g. k8s namespaces, pods)
• jx import - imports a project (code) into JX. It then creates all required objects (e.g. Git repo, Helm Charts, JX objects)
• jx preview - creates a temporary preview environment for an application version
• jx promote - promotes an application version to a specified environment

Jenkins X Flow

As a next step, let’s consider the lifecycle of the application development, from the initiation and all the way to the Production deployment. At a high level, it’s presented on the diagram below:

The picture above shows, that JX covers not only operational processes based on Git, but also provides the ways of initiating an application, as well as promoting to Production, thus covering e2e cycle.

Let’s have a look at how the JX processes correlate to Git actions. For simplicity, let’s use only master and one feature branch. The table details the steps of the flow.

Figure 2. Git workflow events

Jenkins X installation

Alright, so once we’ve covered got some overview of JX, let’s see what happens when we install it! But before that, the main prerequisite for JX installation is to have Kubernetes cluster (or Minikube - simplified single-node Kubernetes cluster) installed, as well as command-line Kubernetes CLI - Kubectl and lastly - VM environment if we choose to go with Minikube (either VirtualBox, VMware Fusion, or HyperKit). There is a lot of things happening with our Kubernetes cluster in background of the installation. Below is a diagram showing the process of what JX does:

Figure 3. JenkinsX moved functionality

Once JX is installed, it initialises Pods with the following applications in jx namespace of k8s:

1. Chartmuseum (Open-source Helm Chart repository)
2. Docker registry (Docker image storage)
3. Jenkins (Automation server)
4. MongoDB (NoSQL Database)
5. Nexus repository (Artefact storage)

This jx namespace of k8s is associated with the Dev environment in JX. Those host all utility tools which are used for development, but are not being developed themselves.

Apart from Pod installation, JX creates staging and production environments that hold versions of applications being developed. These environments configurations are pushed to remote Git repos and webhook is created to link to Jenkins installation in k8s.

It is worth to mention that JX exposes IPs of some Pods via Ingress. Which means you can access all the utility tools like Docker Registry, Monocular or Nexus from an external to k8s network.

Jenkins X: create new application

JX supports the creation of a Quickstart application based on Node JS, Spring boot, GoLang or Rust (as of now) . This is basically a creation of application structure from template. When quickstart is created, JX initializes a local repository, unpacks code template, adds the Jenkinsfile template, Helm charts, Dockerfile and Makefile from so called ‘draft’, and then pushes the new repo to the provided remote Git.

Figure 4. Jenkins X quickstart installation and first pipeline run

As a part of quickstart installation, JX initializes the first pipeline as following:

1. Create an app image, push it to Docker registry
2. Create local Jenkins repository and push it to remote Git
3. Create Application chart and publish it to Chartmuseum (could be checked on Monocular)

Considering the type of application, if any artefacts are created (such as Spring boot app), they will be pushed to Nexus when a new version of app is created/promoted.

Jenkins X community

Even though JX positions itself as a sub project of the Jenkins Foundation, it has a very vibrant and responsive community. During our initial product exploration, we opened issues on GitHub page regarding JX functionality. Surprisingly, major contributors resolved our issues within a couple of hours with immediate actions. We are very pleased with such a fast feedback.

Figure 5. Example of GitHub issue discussion

Conclusion

Jenkins X spans quite a number of technologies and though it is intended to make application development lifecycle easier, it still requires an understanding of many technical concepts.

Apart from the complexity of the toolset, Jenkins X appears to be a promising project which can level up a DevOps experience on Kubernetes and reduce initial implementation costs.

Its Quickstarts and configurable drafts can fulfil the customisation needs of picky DevOps engineers allowing (with some additional effort) to go beyond basic GitOps flow and standard application templates.

The well articulated roadmap shows the width of intentions and instills confidence in the ability of Jenkins X to cover a variety of technical cases.

It would be nice to see more detailed documentation on main principles and architecture of the system, as well as technical details on some CLI commands. There is also very limited information available on making custom extensions like Quickstarts and drafts.

Table of definitions

Authors downunder:

Ilya Trofimov and Nick Shulhin