Jenkins, Powershell, AWS and Cloudflare Automated Deployment. (Part3 Cloudflare)

If you’re interested in only the code, export of the Jenkins template and list of needed plugins it can be found on my Github. I have made some changes since part 2 section was written. I would suggest updating if you’re using an old version of the script.

This is Part 3 of my series on deploying an EC2 instance and Integrating Cloudflare using Powershell and Jenkins. You can find my previous posts here Part 1 and Part 2.

In Part 2 section I walked you through the creating the Jenkins build and populating the Parameters, and at the end you should have been able to launch a successful EC2 instance. This section I will cover Injecting the Environmental variables into a second build step and utilizing the cloudflare API to create a non-proxied DNS entry. Non-proxied DNS entries act as standard DNS entries and are not affected by the cloud-bleed issue from a few weeks ago.

This post assumes you have a configured cloudflare account for the domain you want to auto deploy.  You will also need the Email and API key for your cloudflare account.

In your Jenkins instance go to the build that was created in Part 2 for launching EC2 instance. Scroll to the domain parameter and make sure you enter domains that match what is available in cloudflare, and that you want available for deployment.

Next  we need to add a new build step. Select “Inject environment Variables” and name the Properties file path “Build.prop” Make sure the build step is after the Powershell EC2 launch:

 

 


Connect to the console of your Jenkins Server, and go to the workspace. Normally that is C:\Program Files\Jenkins\workspace\<build name>. You can
Go back to one of your previous test builds and find the workspace path under the console output to verify.

Create a Blank file called build.prop


I go into more detail about the environment inject variable here

Returning back to the Jenkins web control panel create another Powershell build job and add the build2_Cloudflare.ps1 contents to jenkins

At the top of the script configure your API information:

Where the $email and $api_key variables come directly from cloudflare.

You can also decide on the default behavior on IP collisions within cloudflare. If a subdomain already exists in cloudflare this decides if the script should overwrite the IP. If the ip isn’t created the console will output the IP that was assigned to the new instance so it is reachable. If the variable $overwriteip is set to $true the script will attempt to update the domain, and if that fails try to create the domain. If it is set to $false then only the attempt to create the domain will be tried. If a failure registers the Elastic IP is output in the console so the user can still contact the new instance.

No other configuration is needed for the cloudflare segment. The flow simply grabs the domain and the AWS instance name and uses that to generate the FQDN. The Elastic IP injected into the file from the first buildstep like so:

$PublicIP = $ellastic_ip_allocation | select -expandproperty PublicIP
echo "Passing Env variable $PublicIP"
"ElasticIP = $PublicIP" | Out-file build.prop -Encoding ASCII -force

The second buildstep picks it up injected variable like any other variable:

cd $env:WORKSPACE
$EIP = $env:ElasticIP
$domain_partial = $ENV:Domain
$domain_Instance_Name = $ENV:Instance_Name
$domain_FQDN = $domain_Instance_Name + “.” + $domain_partial

For the cloudflare script and code  I go into a lot more detail on the functions on this post here.

I hope this blog series provided inspiration on how to build out a DevOPs friendly deployment. Something that allows the Development team the freedom to deploy servers, but allows for a consistent and secure environment that every Operations team needs to thrive.

Thanks for reading,
I_Script_Stuff

Jenkins, Powershell, AWS and Cloudflare Automated Deployment.

If your interested in only the code, and export of the Jenkins template and list of needed plugins it can be found on my Github

Introduction:

This multipart series is about using Jenkins to spin up an EC2 instance, add an elastic IP, and deploying the DNS to cloudflare using powershell. This is meant to be a template configuration for Windows DevOps teams to start building out environments. From here you can build your own AMI, or tie in your own github/svn code deploy or use ansible/chef/puppet/powershell/million other options to complete an automated deployment.

Some advantages to this method:

The user doing the build will be able to pick an instance, pick a security group, and deploy into multiple regions while Jenkins 100% controls the options available. This insures consistency across builds and allows you to limit who requires credentials to critical infrastructure. We can also apply tags and enforce mandatory tagging. This method also utilizes the concept of “only right answers.”  Though the user is given options, those options are limited within jenkins, and those limitations make sense for the project. This project is a template that will let you have confidence that tomorrow morning you will not wake up to a dozen d2.8xlarge with 5tbs of ESD storage to act as a cluster of DNS servers.

I have written posts about many of the tools used for this series ( AWS Report using Powershell ,Managing Cloudflare with Activedirectory, Jenkins EnvInject Plugin, Migrating Powershell Scheduled tasks to Jenkins). So if you want some other posts that cover the getting started concepts I’d recommend those.  I will note that  Matthew Hodgkins’ wrote a really great 2 part blog entry on getting started with Jenkins and Powershell, and the AWS plugin getting started  docs are a great resources if you are trying to do this project from scratch.

Plugins and Configuration:

For this project to work there are a few plugins you will absolutely need:

Jenkins (latest version)

Powershell 4.0

Aws Powershell plugin : Used to integrate the powershell script and AWS.

Environment Injector Plugin : Used to keep some variables between Jenkins build steps

User build vars plugin : Used to get data about who triggered the Jenkins build.

Role-based Authorization Strategy (Optional) : This can be used to limit which builds a user has access to. I’ll cover this in a later post.

 

Additional configuration notes:

I have Jenkins running as a service account. I did this due to wanting to run powershell plugins, and a few times I have run into odd behavior without a full user space provided. I havn’t tested to see if this works without it.

Workflow:

User has connected to Jenkins and found the build the want.

When initializing the build they encounter configuration options:

They Click build and Jenkins launches the build. The user waits a few minutes and on the AWS console, a new EC2 instance has spawned with Tags:

The Name has been Tagged “WebDeploy”, and two new tags have been created. The BuildTag that shows the Jenkins job that launched the instance. In this case since we have 256 Unicode characters to work with in aws and 200+ in Jenkins I was able to name the Build after the environment it was launched in (Prod or production), and a Billing code (Billcode0001), the last -2 is which Jenkins build the deploy is from.  The other tag is BuiltBy which is the user account in Jenkins that triggered the build.

After the AWS instance is fully online Powershell does API calls back to Cloudflare. A new entry is created, or an old entry is updated. Webdeploy is given the FQDN of webdeploy.electric-horizons.com:

Under the Hood:


#Import AWS powershell module
import-module awspowershell

#Enforce working in our current Jenkins workspace.
cd $env:WORKSPACE

#
#ENV:AWS_Profile is from the build parameters earlier it provides the AWS profile credentials
#

$aws_profile = $ENV:AWS_Profile
Set-AWSCredentials -ProfileName $aws_profile

#Load all the other environment variables AWS needs to to create an instance

$region = $ENV:Region
$instance_name = $ENV:Instance_Name
$builder = $ENV:BUILD_USER
$buildtag = $ENV:BUILD_TAG
$image_type = $ENV:image_type
$Instance_Type = $ENV:Instance_Type
$domain = $ENV:Domain
$public_key = $ENV:Public_Key
$SecurityGroup = $ENV:Security_Group

#Search for the Security group Name tag Value. More on this in the next post.

try {
$SecurityGroup_Id = Get-EC2SecurityGroup -Region “$region” | where { $_.Groupname -eq “$SecurityGroup” } | select -expandproperty GroupId
echo “Security group Identification response:”
$SecurityGroup_Id

} catch {
$_
exit 1
}

#Make sure that the Instance name is not blank.

if($instance_name.length -le 1) {
echo “ERROR: Instance must be named and the length must be greater than 1.”
echo “ERROR: Instance name: $instance_name”
echo “ERROR: Instance name length” $instance_name.length
exit 1
}

#Select AWS AMI. This is limited to the ones owned by Amazon. And gets the most up to date image.

try {
$image_id = Get-EC2Image -Owner amazon, self -Region $region | where { $_.Description -eq $image_type } | select -first 1 -expandproperty ImageId
echo “EC2 Image ID Response:”
$image_id
} catch {
$_
exit 1
}

#Generate the instance, with all environmental variables provided from Jenkins build.

try {
$instance_info = New-EC2Instance -ImageId $image_id -MinCount 1 -MaxCount 1 -KeyName $public_key -SecurityGroupId $SecurityGroup_Id -InstanceType $instance_type -Region $region
echo “Image generation response”
$instance_info
} catch {
$_
exit 1
}

#Let the user know things are working as intended and to please wait while we wait for the instance to reach the running state.

echo “Please wait for image to fully generate”
while($(Get-Ec2instance -instanceid $instance_info.instances.instanceid -region $region).Instances.State.Name.value -ne “running”) {
sleep 1
}

#Apply tags to the instance

echo “Naming Instance”
$tag = New-Object Amazon.EC2.Model.Tag
$tag.Key = “Name”
$tag.Value = “$instance_name”

New-EC2Tag -Resource $instance_info.instances.instanceid -Tag $tag -Region $region
echo “Tagging build information”
$tag.Key = “BuiltBy”
$tag.Value = “$builder”
New-EC2Tag -Resource $instance_info.instances.instanceid -Tag $tag -Region $region

$tag.Key = “BuildTag”
$tag.Value = “$BUILDTAG”

New-EC2Tag -Resource $instance_info.instances.instanceid -Tag $tag -Region $region

#Attach an elastic IP to the instance

try {
$ellastic_ip_allocation = New-EC2Address -Region $region
echo “Elastip IP registered:”
$ellastic_ip_allocation
} catch {
echo “ERROR: Registering Ec2Address”
$_
exit 1
#return $false
}

#Assign the elastic IP to the instance

try {
$response = Register-Ec2Address -instanceid $instance_info.instances.instanceid -AllocationID $ellastic_ip_allocation.allocationid -Region $region
echo “Register EC2Address Response:”
$response
} catch {
echo “ERROR: Associating EC2Address:”
$_
exit 1
}

#Send the elastic IP value to the EnvInj plugin:

$PublicIP = $ellastic_ip_allocation | select -expandproperty PublicIP
echo “Passing Env variable $PublicIP”
“ElasticIP = $PublicIP” | Out-file build.prop -Encoding ASCII -force

exit 0

In the next post I’ll cover prepping the AWS Environment, prepping the Jenkins Build and Key parts to look at to add customization for your environment.

If you want the whole project you can get it from my Github.

Part Two can be found here

Thanks for reading,

I_script_stuff