Jenkins, Powershell, AWS and Cloudflare Automated Deployment.

If your interested in only the code, and export of the Jenkins template and list of needed plugins it can be found on my Github

Introduction:

This multipart series is about using Jenkins to spin up an EC2 instance, add an elastic IP, and deploying the DNS to cloudflare using powershell. This is meant to be a template configuration for Windows DevOps teams to start building out environments. From here you can build your own AMI, or tie in your own github/svn code deploy or use ansible/chef/puppet/powershell/million other options to complete an automated deployment.

Some advantages to this method:

The user doing the build will be able to pick an instance, pick a security group, and deploy into multiple regions while Jenkins 100% controls the options available. This insures consistency across builds and allows you to limit who requires credentials to critical infrastructure. We can also apply tags and enforce mandatory tagging. This method also utilizes the concept of “only right answers.”  Though the user is given options, those options are limited within jenkins, and those limitations make sense for the project. This project is a template that will let you have confidence that tomorrow morning you will not wake up to a dozen d2.8xlarge with 5tbs of ESD storage to act as a cluster of DNS servers.

I have written posts about many of the tools used for this series ( AWS Report using Powershell ,Managing Cloudflare with Activedirectory, Jenkins EnvInject Plugin, Migrating Powershell Scheduled tasks to Jenkins). So if you want some other posts that cover the getting started concepts I’d recommend those.  I will note that  Matthew Hodgkins’ wrote a really great 2 part blog entry on getting started with Jenkins and Powershell, and the AWS plugin getting started  docs are a great resources if you are trying to do this project from scratch.

Plugins and Configuration:

For this project to work there are a few plugins you will absolutely need:

Jenkins (latest version)

Powershell 4.0

Aws Powershell plugin : Used to integrate the powershell script and AWS.

Environment Injector Plugin : Used to keep some variables between Jenkins build steps

User build vars plugin : Used to get data about who triggered the Jenkins build.

Role-based Authorization Strategy (Optional) : This can be used to limit which builds a user has access to. I’ll cover this in a later post.

 

Additional configuration notes:

I have Jenkins running as a service account. I did this due to wanting to run powershell plugins, and a few times I have run into odd behavior without a full user space provided. I havn’t tested to see if this works without it.

Workflow:

User has connected to Jenkins and found the build the want.

When initializing the build they encounter configuration options:

They Click build and Jenkins launches the build. The user waits a few minutes and on the AWS console, a new EC2 instance has spawned with Tags:

The Name has been Tagged “WebDeploy”, and two new tags have been created. The BuildTag that shows the Jenkins job that launched the instance. In this case since we have 256 Unicode characters to work with in aws and 200+ in Jenkins I was able to name the Build after the environment it was launched in (Prod or production), and a Billing code (Billcode0001), the last -2 is which Jenkins build the deploy is from.  The other tag is BuiltBy which is the user account in Jenkins that triggered the build.

After the AWS instance is fully online Powershell does API calls back to Cloudflare. A new entry is created, or an old entry is updated. Webdeploy is given the FQDN of webdeploy.electric-horizons.com:

Under the Hood:


#Import AWS powershell module
import-module awspowershell

#Enforce working in our current Jenkins workspace.
cd $env:WORKSPACE

#
#ENV:AWS_Profile is from the build parameters earlier it provides the AWS profile credentials
#

$aws_profile = $ENV:AWS_Profile
Set-AWSCredentials -ProfileName $aws_profile

#Load all the other environment variables AWS needs to to create an instance

$region = $ENV:Region
$instance_name = $ENV:Instance_Name
$builder = $ENV:BUILD_USER
$buildtag = $ENV:BUILD_TAG
$image_type = $ENV:image_type
$Instance_Type = $ENV:Instance_Type
$domain = $ENV:Domain
$public_key = $ENV:Public_Key
$SecurityGroup = $ENV:Security_Group

#Search for the Security group Name tag Value. More on this in the next post.

try {
$SecurityGroup_Id = Get-EC2SecurityGroup -Region “$region” | where { $_.Groupname -eq “$SecurityGroup” } | select -expandproperty GroupId
echo “Security group Identification response:”
$SecurityGroup_Id

} catch {
$_
exit 1
}

#Make sure that the Instance name is not blank.

if($instance_name.length -le 1) {
echo “ERROR: Instance must be named and the length must be greater than 1.”
echo “ERROR: Instance name: $instance_name”
echo “ERROR: Instance name length” $instance_name.length
exit 1
}

#Select AWS AMI. This is limited to the ones owned by Amazon. And gets the most up to date image.

try {
$image_id = Get-EC2Image -Owner amazon, self -Region $region | where { $_.Description -eq $image_type } | select -first 1 -expandproperty ImageId
echo “EC2 Image ID Response:”
$image_id
} catch {
$_
exit 1
}

#Generate the instance, with all environmental variables provided from Jenkins build.

try {
$instance_info = New-EC2Instance -ImageId $image_id -MinCount 1 -MaxCount 1 -KeyName $public_key -SecurityGroupId $SecurityGroup_Id -InstanceType $instance_type -Region $region
echo “Image generation response”
$instance_info
} catch {
$_
exit 1
}

#Let the user know things are working as intended and to please wait while we wait for the instance to reach the running state.

echo “Please wait for image to fully generate”
while($(Get-Ec2instance -instanceid $instance_info.instances.instanceid -region $region).Instances.State.Name.value -ne “running”) {
sleep 1
}

#Apply tags to the instance

echo “Naming Instance”
$tag = New-Object Amazon.EC2.Model.Tag
$tag.Key = “Name”
$tag.Value = “$instance_name”

New-EC2Tag -Resource $instance_info.instances.instanceid -Tag $tag -Region $region
echo “Tagging build information”
$tag.Key = “BuiltBy”
$tag.Value = “$builder”
New-EC2Tag -Resource $instance_info.instances.instanceid -Tag $tag -Region $region

$tag.Key = “BuildTag”
$tag.Value = “$BUILDTAG”

New-EC2Tag -Resource $instance_info.instances.instanceid -Tag $tag -Region $region

#Attach an elastic IP to the instance

try {
$ellastic_ip_allocation = New-EC2Address -Region $region
echo “Elastip IP registered:”
$ellastic_ip_allocation
} catch {
echo “ERROR: Registering Ec2Address”
$_
exit 1
#return $false
}

#Assign the elastic IP to the instance

try {
$response = Register-Ec2Address -instanceid $instance_info.instances.instanceid -AllocationID $ellastic_ip_allocation.allocationid -Region $region
echo “Register EC2Address Response:”
$response
} catch {
echo “ERROR: Associating EC2Address:”
$_
exit 1
}

#Send the elastic IP value to the EnvInj plugin:

$PublicIP = $ellastic_ip_allocation | select -expandproperty PublicIP
echo “Passing Env variable $PublicIP”
“ElasticIP = $PublicIP” | Out-file build.prop -Encoding ASCII -force

exit 0

In the next post I’ll cover prepping the AWS Environment, prepping the Jenkins Build and Key parts to look at to add customization for your environment.

If you want the whole project you can get it from my Github.

Part Two can be found here

Thanks for reading,

I_script_stuff

Jenkins EnvInject Plugin and Powershell

I have been working on larger projects in Jenkins. I hope to have a post put together in the next few weeks that shows better uses for multiple build steps. For now I thought I’d talk about a plugin I came across and some of the quirks you’ll run into when trying to do Powershell, Jenkins and the EnvInject plugin.

The plug-in allows you to create a file and define environment variables. These variables can persist between build steps and builds. It also allows you to override the default ENV variables Jenkins gets from the server. This can be used to enforce an environment over different Jenkin servers. For example if you need to modify the windows Path variable ($env:Path in jenkins) you can add a config file that makes the modification temporarily for the build rather than making a permanent change to the Windows OS. You can read more on the Environment Inject Plugin wiki.

For this post I’ll just use a few simple scripts to create a new Environment variable and hand it to a second build step. You could use a CSV file to do this as well, but I like the uniform cleanliness of using the Environment variables. It simplifies script re-use for me, your mileage of course may vary with your work flow.

 

First we will install the plugin. You’ll need to go to Manage Jenkins → Manage Plugins → Available

Filter for “Envinject” and install the plugin:

 

The first build we won’t be enabling the plugin. This is to show the default behavior of Jenkins to contrast the difference between the plug in behavior. Click New Item. I named the build “Env-Inject Example 1” and used a Free style project. Once created the build properties are as show:

The non-working code can be found here on pastebin.

The code simple outputs some text to so we can keep track of what is going on in from the console output.

Unsurprisingly we see a blank spot before Finished: SUCCESS where the $Keep_me variable didn’t make it to the second build step.

The Environment Inject Plugin requires a file to read from. We can define one anywhere and provide the path. For simplicity I’ll add the file to the workspace for the project. The workspace path can be seen on our console output above. Look for the line “Building in Workspace” for my configuration it is found on “C:\Program Files\Jenkins\workspace\Env-Inject Example1”.  Lets create a blank file on the Jenkins Server called build.prop

Once the file is in place go back to jenkins and we will add  “Inject environment Variables” to the build.

We will then move the step to the middle of the Jenkins build like so:

With Injected file at the top of the build we will need to change the code to write to the file. The code (showed above) can be found on pastebin here.

For the first build step the  the changes are at the end step:


echo "build 1"
$keep_me = "fubar"
echo "do more stuff"
$save_variable = "Keep_me=" + $keep_me
$save_variable |Out-file build.prop -Encoding ASCII

We output the variable with the environment tag we want. In this case I called it “Keep_me” just like the variable name. I then output to the build.prop file. Notice how I didn’t define a path. Since we put the original file in the workspace Jenkins checks there first for assets. If you wanted to output this to a different location, simply define the path to use. I also declared explicitly to encode the output as ASCII. Powershell defaults to UTF8 and Jenkins doesn’t like that encoding. So writing the code like:


echo "KEEP_ME=Fubar" >> build.prop

would fail since the encoding would be utf and Jenkins would miss interpret the result.

The second build step we declare the environmental variable and assign it a variable:

echo "build 2"
$keep_me =$env:Keep_me
echo "Did it make it"
echo "$Keep_me"

This is the same way you would declare any other environment variable available to Jenkins. For example the Path environmental variable found in Windows is $env:Path.
When we run the build we see that the console output now shows the expected output just above Finished: Success is now “fubar” like we declared.
If you go back to your build.prop file you will see that it has now been set with the environmental variable.
That is a persistent change. You can overwrite the variable so each build is different, but the file stays with the variables from the last build. You can also prep multiple overriding Environment variables from within the file.
Besides the persistent variables in the file a few other things you should know.
If the file is missing you will see errors like:
To help with troubleshooting and to keep track of what environmental variables were used with each build step a new option has been added to the Jenkins post build report:
The environmental variables used for the build are reported. As you can see the Keep_Me variable we used is shown:
How are you keeping track of variables across builds in Jenkins? Any other preferred methods? Let me know in the comments!
Thanks for reading.
I_Script_Stuff