CI/CD Pipeline Using Jenkins Unleashed : Solutions While Setting Up CI/CD Processes

✍ Scribed by Pranoday Pramod Dingare

• Publisher: Apress
• Year: 2022
• Tongue: English
• Leaves: 432
• Category: Library

✦ Synopsis

Understand continuous integration (CI), continuous delivery, and continuous deployment (CD) with Jenkins. These processes allow users as well as administrators to catch problems as soon as they get injected into software systems.
This book starts with an introduction to Jenkins and covers its architecture and role in CI/CD. The basics are covered, including installing and configuring Jenkins. Tool configuration and plugins are discussed as well as available security measures such as credentials. You will learn what is meant by Job in Jenkins, its types, sections, and much more. You will look at Java API: projects, jobs, configuration. The concluding chapters take you through creating pipelines, their role in managing web apps, and distributed pipelines. The book also covers unit testing using TestNG as well as end-to-end testing using Selenium Python as a part of building a life cycle and setting up Jenkins on different physical and Docker environments as well as Jenkins integration with cloud environments such as AWS. And you will learn how to create reusable libraries for use in Jenkins Pipeline and control Jenkins servers using Jenkins CLI and REST APIs. The new Jenkins Blue Ocean also is covered.

The book helps you understand CI/CD implementation using Jenkins from scratch in your projects and prepare for end-to-end DevOps practices.

What You Will Learn

Apply Jenkins to create end-to-end pipelines
Integrate Jenkins with AWS, Docker, Git, and many more tools
Use Selenium automation for end-to-end testing
Create distributed pipelines

Who Is This Book For

Developers and test automation professionals who are involved in creating CI/CD pipelines as well as prospective DevOps aspirants who want to make their way ahead as professionals

✦ Table of Contents

Table of Contents
About the Author
About the Technical Reviewer
Acknowledgments
Introduction
Chapter 1: Understanding CI/CD
The Development Workflow
Running Unit Tests Locally
Pushing and Merging Code to the Central Repository
Compiling Code after the Merge
Running Tests on the Merged Code
Deploying an Artifact
Continuous Delivery/Continuous Deployment
CI/CD Workflow Case Study
Pulling the Application’s Latest Code
Developing and Running Unit Test Cases
Code Development
Rerunning the Unit Test Case
Pushing and Merging the Code to the Central Repository
Compiling the Code after the Merge
Running Tests on the Merged Code
Deploying an Artifact
Run e-e Tests on a Deployed Application
Summary
Chapter 2: Introducing Jenkins
What Is Jenkins?
A History of Jenkins
Implementing CI/CD Using Jenkins
The Jenkins Architecture
Summary
Chapter 3: Installing Jenkins
Installing Jenkins on Windows
Hardware/Software Requirements
Different Ways to Install Jenkins on Windows
Understanding the Configuration Files and Directory Structure of Jenkins
Understanding Important Settings in Jenkins.xml
Summary
Chapter 4: Configuring Jenkins
Configuring Global Settings and Paths
Signing into Jenkins
Starting the Jenkins Server
Starting the Jenkins Service on Linux
Opening the Browser and Signing In
Understanding the Configure System Options
Resetting the Username and Password in Jenkins
Adding a New User
Summary
Chapter 5: Managing Plugins in Jenkins
What Are Plugins
Commonly Used Jenkins Plugins
Installing the Plugins in Jenkins
Understanding the Plugin Manager
The Updates Tab
The Available Tab
The Installed Tab
The Advanced Tab
Troubleshooting Installation Problems
Problem 1
Problem 2
Summary
Chapter 6: Understanding the Global Tool Configuration Page
Global Tool Configuration Settings
Understanding the Global Tool Configuration Settings
Maven Configuration
Java Configuration
Summary
Chapter 7: Managing Security with Jenkins
Configuring Global Security in Jenkins
Configuring LDAP with Jenkins
Why We Need to Configure LDAP with Jenkins
How to Configure LDAP with Jenkins
Summary
Chapter 8: Managing Credentials
Understanding Credentials in Jenkins
Creating Credential Entries in Jenkins
Understanding Scope and Domains
Creating Credential Entries in Jenkins
Creating a Credentials Entry in a Global Domain (Default Domain) and a Global Scope
Updating a Credentials Entry
Creating a Credentials Entry in a Particular Domain
Configuring a Credentials Provider
Summary
Chapter 9: Managing Users
Creating Users in Jenkins
Assigning Roles to Users in Jenkins
Installing the Role-Based Authorization Strategy Plugin
Enabling Role-Based Strategy in Jenkins
Creating User Roles in Jenkins
Assigning Roles to Users in Jenkins
Checking the Assignment of a Role to a User
Creating Project-Based Roles in Jenkins
Assigning Project-Based Roles to Users
Verifying the Assignment of the Project-Based Role to the User
Understanding Matrix-Based Security in Jenkins
Understanding the Project-Based Matrix Authorization Strategy in Jenkins
Summary
Chapter 10: Understanding Jobs in Jenkins
What Is a Job in Jenkins?
What Is a Build in Jenkins?
What Is a Free-Style Job in Jenkins?
How to Create a Job in Jenkins
How to Configure a Job in Jenkins
This Project Is Parameterized
Disabling a Project
Executing Concurrent Builds
Quiet Period
Block Build When Upstream Project Is Building
Block Build When Downstream Project Is Building
Use Custom Workspace
Display Name
Keep the Build Logs of Dependencies
Source Code Management
Branches to Build
Build Triggers
Trigger Build Remotely (e.g. from Scripts)
Build After Other Projects Are Built
Build Periodically
Poll SCM
Build Step
Post-Build Actions
How to Run a Job in Jenkins and Check Its Output
How to Edit a Job in Jenkins
How to View a Job’s Workspace
How to Clear a Job’s Workspace
How to Delete a Job
Summary
Chapter 11: Preparing a Java API Project Using Maven
Understanding the Maven Build Tool
Typical Development Flow of a Java API Project
Downloading Third-Party Libraries
Adding Downloaded Libraries to the Project Build Path
Coding and Writing Unit Test Cases
Compiling the Application and Unit Test Cases Code
Running Unit Test Cases
Bundling/Packaging the Application
Releasing it on the Artifact Repository
How the Build Tool Helps Developers
How to Use the Maven Build Tool to Build a Java API Project
Step 1: Creating a Workspace in Eclipse
Step 2: Creating a Maven Project
Understanding the Maven Project Directory Structure
src/main/java
src/test/java
Understanding Java API Project Code Files
Understanding the pom.xml File in the Java API Project
Build Lifecycle Phases and Their Order
How to Use Maven from the CLI
Setting Up Maven
Using Maven CLI Commands
Understanding Maven’s settings.xml File
Summary
Chapter 12: Integrating Maven with the Nexus Repository and Creating Free-Style Jobs to Release the Java API on the Nexus Repository
Understanding Git
Installing Git
Understanding GitHub/GitLab
Understanding End-End Use of Git for the API Project
Step 1: Creating a Local Repository
Step 2: Creating a Central Repository on GitLab
Step 3: Committing Code to the Local Repository
Step 4: Pushing the Code from the Local Repository to the Central Repository on GitLab
Step 5: Creating a Master Branch in the Central Repository on GitLab
Understanding the Nexus Repository
What is an Artifact?
What is the Nexus Repository?
Installing the Nexus Repository
How to Start the Nexus Repository System
Installing Nexus as a Service
Accessing the Nexus Repository Manager
Creating a Hosted Repository to Release the Artifacts
Integrating Maven and the Nexus Repository
Releasing the CalculatorAPI.jar File in the Nexus Repository
Creating a Jenkins Free-style Job to Release the CalculatorAPI.jar in the Nexus Repository
Step 1: Setting Up Maven in Jenkins
Step 2: Adding Git Repository Credentials to Jenkins
Step 3: Creating a Free-Style Job from the Jenkins Dashboard
Step 4: Add a Subtraction Function and Unit Test Cases to the API Project
Step 5: Commit Changes in the Local Repository and Push them to the Central GitLab Repository
Step 6: Merge the SubtractionFunction Branch with the Master Branch on the Central GitLab Repository
Running a Jenkins Free-Style Job to Perform a Release of the CalculatorAPI.jar in the Nexus Repository
Summary
Chapter 13: Creating an Auto-Trigger Free-Style Job to Manage Java API Releases
How to Add a New Code Contributor to a Private GitLab Repository
How to Invite a Team Member to the Code Repository
Understanding SSH Authentication
Why You Need SSH Authentication
How SSH Authentication Works with GitLab
Applying SSH Authentication to the Java API Code Repository
Step 1: Generating the Public and Private Keys
Step 2: Adding the Public Key to the  JenkinsBookCalculatorAPI GitLab Repository
Accessing the JenkinsBookCalculatorAPI Repository Using SSH URL and Adding new Arithmetic Functions to the CalculatorAPI
Step 1: Cloning the JenkinsBookCalculatorAPI Repository
Step 2: Adding a Multiplication Function to the Calculator Class
Step 3: Adding Unit Test Cases for the Multiplication Function
Step 4: Changing the Version Number to 3.0 in pom.xml
Step 5: Unit Testing the Recent Multiplication Function and Regression Testing for the New Functionalities
Step 6: Committing New Functionality Changes in a Branch Named Multiplication Function in the Local Repository
Step 7: Pushing the MultiplicationFunction Branch to the Remote Repository
Step 8: Creating a Merge Request for this New Branch
Creating an Auto-Trigger Jenkins Job with Email Notification
Step 1: Creating a Free-Style Job from Jenkins Dashboard: Click the New Item link on Jenkins Dashboard
Setting Up Jenkins to Send Email Notifications
Triggering the New Jenkins Job
Understanding the Execution of Jobs with SCM Set
Failing the Build to See the Email Notification
Summary
Chapter 14: Understanding the Jenkins Pipeline
What Is a Jenkins Pipeline?
Why Use a Jenkins Pipeline?
Understanding Different Pipeline Concepts
Pipeline Syntax Overview
Fundamentals of Declarative Pipelines
Fundamentals of Scripted Pipelines
Defining a Simple Pipeline in Jenkins UI
Pipeline Speed/Durability Override
Resolving java.lang.IllegalArgumentException: Unsupported Class File Major Version Error
Understanding String Interpolation in the in Jenkins Pipeline
String Interpolation Example
Creating a Pipeline Job to Release the Java API
Running a Pipeline Job and Release the Calculator API
Summary
Chapter 15: Creating Jenkins Jobs to Manage a Web Application Project
Understanding the Calculator Web Application Source Code
Building the Calculator Web Application
Deploying the Calculator Web Application
Accessing the Calculator Web Application
Understanding E-E Testing the Calculator Web Application Using the Selenium WebDriver
What Is UI Automation Testing
How UI Automation Tools Work
What Is a Selenium WebDriver
Understanding the Selenium Tests Written in Python
EnvVars.csv
CalculatorPage.csv
AutomationFramework Package
CalculatorWebApp_Pages Package
CalculatorWebApp_TestCases Package
Software Setup to Run these Test Cases
Running Selenium Python Tests
Step 1: Open the Command Prompt and go to the TestCalculatorWebApplicationUsing SeleniumPython Project Directory
Step 2: Create a Virtual Python Environment and Activate It
Step 3: Install the Required Python Packages
Pushing the WebApplication and Automation Project to the GitHub Repository
Step 1: Sign up on Github.com
Step 2: Creating a New Repository
Step 3: Creating a New SSH Key Pair for the GitHub Repository
Step 4: Include the SSH Private Key File Path in the $(user.name).ssh\config File
Step 5: Adding an SSH Public Key to the GitHub Repository
Step 6: Committing the Calculator Web Application to the Master Branch in the Local Repository
Step 7: Pushing the Master Branch to the Remote Repository
Pushing the Selenium Python Automation Project to GitHub
Creating Parameterized Auto-Trigger Free-Style Jenkins Jobs
Step 1: Installing the Parameterized Trigger Plugin
Step 2: Creating a BuildAndDeployCalculator WebApplication Job
Step 3: Creating the TestCalculatorWeb Application Job
Configuring an Extended Email Notification
Step 1: Installing the Email Extension Plugin
Step 2: Configuring Extended Email Extension settings
Step 3: Adding an Email Notification Step in the Post-Build Section of the TestCalculatorWeb Application Job
Running a Parameterized Free-Style Job Manually
Auto-Triggering a Parameterized Free-Style Job
Creating a Parameterized Pipeline Job
Creating a Build and Deploying the Calculator Web Application Parameterized Pipeline Job
Creating a TestCalculatorWebApplication Pipeline Job
Running the Parameterized Pipeline Job Manually
Automatically Triggering the Parameterized Pipeline Job
Summary
Chapter 16: Understanding Pipeline as Code
What Is API Authentication
How to Apply API Authentication to GitHub Repositories
How to Use an API Token to Access a Private GitHub Repository
Creating an API Access Token in GitLab
How to Use the API Token to Access a Private GitLab Repository
How to Access a GitHub/GitLab Repository Using an API Token in Jenkins
Step 1: Setting Up Maven in Jenkins
Step 2: Creating a Free-Style Job from the Jenkins Dashboard
Step 3: Change the Version in pom.xml of the Cloned API Project
Step 4: Commit Changes in the Local Repository and Push them to the Central GitLab Repository
Running Free-Style Jobs Accessing the GitLab Repository with the API Token
Understanding Pipeline as Code (Jenkinsfile)
What Is a Jenkinsfile
Advantages of Pipeline as Code
Writing a Pipeline in a Jenkinsfile
Scripted vs Declarative Syntax
Creating a Jenkins Job to Run a Jenkinsfile
Saving the Pipeline Code in a Jenkinsfile and Pushing it to the GitLab Repository
Triggering a Jenkins Job Using a GitLab Webhook
Creating a Pipeline Job to Trigger Using GitHub Webhooks
Triggering Pipeline Jobs Using GitHub Webhook
Summary
Chapter 17: Jenkins Distributed Builds
Jenkins Distributed Architecture
Ways to Connect the Master and Slaves
Understanding the Configuration to Connect the Master to the Agent Using SSH
Step 1: Install the SSH Build Agent's Plugin
Step 2: Install Java on the Slave Node
Step 3: Create an SSH Public-Private Key Pair
Step 4: Add a Credentials Entry with a Private Key to the Master Machine
Step 5: Add a Node Entry to the Jenkins Master
Step 6: Add a Public Key to the authorized_keys File on the Slave Machine
Step 7: Change the Permissions of the  authorized_keys File on the Slave Machine
Step 8: Restart the sshd Service
Step 9: Check the Connection to the Agent Machine from the Master
Step 10: Launch the New Node from the Master Machine
Creating a Free-Style Job to Run on the Node1 Agent
Running the New Free-Style Job on the Node1 Agent
Understanding the Configuration to Connect the Agent to the Master Using JNLP
Step 1: Configure the Jenkins Master to Receive JNLP Agent Connection Requests
Step 2: Set the Jenkins URL
Step 3: Create a New Node Entry from the  Agent (Slave) Machine
Creating a Job to Run on the JNLP Node
Summary
Chapter 18: Integrating Jenkins with AWS
Understanding an EC2 Instance on AWS
Creating an EC2 Instance on AWS
Step 1: Sign Up on AWS
Step 2: Sign in to AWS
Step 3: Create an EC2 Instance
Step 4: Start an EC2 Instance
Configuring the EC2 Instance to Deploy the Calculator Web Application
Step 1: Install the IIS Web Server
Step 2: Configure the IIS Web Server
Step 3: Configure the SSH Connection Between the Local Computer and the EC2 Instance
Copying the Calculator.html File to the EC2 Instance
Accessing Calculator.html Copied to the EC2 Instance from a Browser on a Local Computer
Creating a Jenkins Job to Deploy the Calculator Web Application on an EC2 instance
Changing the Calculator Application URL in Selenium Framework
Running the BuildAndDeployCalculator WebApplication OnEC2Instance Job
Summary
Chapter 19: Miscellaneous Topics Part 1
Understanding the Jenkins CLI
How to Interact with Jenkins Using its CLI
How to Create a Job Using the Jenkins CLI
Authenticating Users Using Basic Authentication (Username-Password/API Token)
Authenticating Users Through SSH While Using CLI Commands
Configuring an SSH Server in Jenkins
Configuring an Inbound Rule in a Firewall
How to Build Jobs with the Jenkins CLI Using SSH Authentication
Using the OpenSSH Client to Run Jenkins CLI Commands
Using the jenkins-cli.jar Client to Run Jenkins CLI Commands with SSH
How to Export All Jobs
How to Import All Jobs
Summary
Chapter 20: Miscellaneous Topics Part 2
Understanding Jenkins Remote Access API
Using Jenkins Remote Access API
Getting the Configuration of Existing Jenkins Jobs Using Jenkins Remote API
Creating New Jenkins Jobs Using Jenkins Remote API
Triggering Parameterized Jenkins Job Using Jenkins Remote API
Triggering Normal (Non-Parameterized) Jenkins Job Using Jenkins Remote API
Working with the Jenkins Server Using Python-Jenkins
Using Python-Jenkins Package Libraries
How to Use Shared Libraries in the Jenkins Pipeline
Step 1: Creating a Shared Library in the .Groovy File
Step 2: Push the Created Shared Library File to the GitLab Repository
Step 3: Configure the Shared Library in Jenkins
Step 4: Create a Pipeline Job to Use the Shared Library
Step 5: Running the Pipeline Job
Summary
Index