How to Integrate JENKINS With Cloud Formation and Step Funtions | Hands on Cloud
From my personal experience, many companies rely on Jenkins and Terraform while automating their cloud infrastructure. There’s no problem if you’d like to deploy something from Jenkins to the cloud, but what to do, if you want to call Jenkins job from to deploy a Terraform module as a Service Catalog product, or include Jenkins pipeline to a workflow orchestrated by StepFunctions? This article will describe a unified integration pattern, which will allow you to integrate Jenkins jobs with CloudFormation custom resources and Step Functions workflows.
There are many ways of deploying Jenkins to the AWS cloud.
For the automated installation process of a single Jenkins server to AWS cloud use our How to Install Jenkins on the Latest Ubuntu in 5 Minutes article.
You may also be interested in Deploying Jenkins on Amazon EKS with Amazon EFS if you’re looking for a way to use Docker containers as Jenkins workers.
Whatever deployment method you use; as a result, you’ll get a Jenkins server whose workers can assume IAM roles in one or many AWS accounts to deploy required applications or services.
Now, we can integrate your Jenkins server with CloudFormation and Step Functions services.
If you’re reading this article, you’re probably trying to solve one of the following problems:
How to call Jenkins APIs from other AWS services
How to provide access to your central Jenkins APIs from multiple AWS accounts
The solution is to use AWS Lambda, SQS, and SNS:
How to integrate Jenkins with CloudFormation and Step Functions — Integration pattern — Simple diagram
In this integration pattern:
AWS Lambda’s role is to make API calls to Jenkins using python-jenkins library.
Amazon SNS allows you to receive events from Step Function and CloudFormation services seamlessly.
Amazon SQS is responsible for sending, storing, and receiving messages for Jenkins at any volume without losing those messages if Jenkins (for any reason) becomes unavailable.