Skip to content

Commit

Permalink
Merge pull request #1 from wearefine/v2
Browse files Browse the repository at this point in the history
Release v2.0.0
  • Loading branch information
cpitkin authored Jun 8, 2018
2 parents e29c904 + 4d78774 commit d917f86
Show file tree
Hide file tree
Showing 9 changed files with 275 additions and 78 deletions.
23 changes: 20 additions & 3 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
@@ -1,5 +1,22 @@
# puppet-jenkins-shared-libraries Changelog
# Changelog

## 1.0.0
All notable changes to this project will be documented in this file.

- EVERYTHING :tada:
The format is based on [Keep a Changelog](http://keepachangelog.com/en/1.0.0/)
and this project adheres to [Semantic Versioning](http://semver.org/spec/v2.0.0.html).

## [v2.0.0] - 6-1-2018

### Added

- Docker build support based on presence of a docker-compose.yml file
- Docker build information

### Updated

- CHANGELOG format
- Linted README markdown

## [v1.0.0]

- EVERYTHING :tada:
13 changes: 13 additions & 0 deletions Jenkinsfile.clean_example
Original file line number Diff line number Diff line change
@@ -0,0 +1,13 @@
node {
try {
stage('Volume Clean') {
sh 'docker volume prune -f'
}
currentBuild.result = 'SUCCESS'
} catch(Exception e) {
currentBuild.result = 'FAILURE'
slackSend channel: '#devops', failOnError: true, color: 'danger', message: 'Jenkins Docker cleanup run *FAILED*!'
throw e
}
slackSend channel: '#devops', failOnError: true, color: 'good', message: 'Jenkins Docker cleanup run *SUCCESSFUL*!'
}
3 changes: 3 additions & 0 deletions Jenkinsfile.example
Original file line number Diff line number Diff line change
Expand Up @@ -18,4 +18,7 @@ puppet {
R10K_DEPLOY_BRANCH = ['production', 'support']
SLACK_CHANNEL = '#puppet'
DEBUG = 'false'
DOCKER_REGISTRY_CREDS_ID = 'creds_id_to_registry'
DOCKER_REGISTRY_URL = 'https://hub.docker.io'
AWS_DEFAULT_REGION = 'us-east-1'
}
70 changes: 44 additions & 26 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -7,26 +7,31 @@ Testing your Puppet on every change should be a smooth process. With Jenkins pip
If you're new to Jenkins pipelines you should go read the [documentation](https://jenkins.io/doc/book/pipeline/) before proceeding to get a sense for what to expect using this code. The rest of the setup process will assume you have basic knowledge of Jenkins or CI/CD jobs in general.

OS
- rvm installed in the jenkins user
- git
- build-essential
- docker if needed for acceptance tests

- rvm installed in the jenkins user
- git
- build-essential
- docker if needed for acceptance tests

Jenkins
- Version: > 2.7.3 - tested on (2.19.4 LTS)


- Version: > 2.7.3 - tested on (2.89.4 LTS)

Plugins
- slack
- pipeline (workflow-aggregator)
- git
- timestamper
- credentials
- junit

- slack
- pipeline (workflow-aggregator)
- git
- timestamper
- credentials
- junit

Scripts Approval
- When the job runs the first time you will need to work through allowing certain functions to execute in the groovy sandbox. This is normal as not all high use groovy functions are in the default safelist but more are added all the time.

When the job runs the first time you will need to work through allowing certain functions to execute in the groovy sandbox. This is normal as not all high use groovy functions are in the default safelist but more are added all the time.

### Manage with Puppet

The following modules work great to manage a Jenkins instance.

- maestrodev/rvm
Expand Down Expand Up @@ -57,25 +62,31 @@ puppet {
R10K_DEPLOY_BRANCH = ['production', 'support']
SLACK_CHANNEL = '#puppet'
DEBUG = 'false'
DOCKER_REGISTRY_CREDS_ID = 'creds_id_to_registry'
DOCKER_REGISTRY_URL = 'https://hub.docker.io'
AWS_DEFAULT_REGION = 'us-east-1'
}
```

### Required Parameters

- RUBY_VERSION: Ruby version to use. [String]
- RUBY_GEMSET: Name of the gemset to create. [String]
- **RUBY_VERSION:** Ruby version to use. [String]
- **RUBY_GEMSET:** Name of the gemset to create. [String]

### Optional Parameters

- RUN_ACCEPTANCE: Run acceptance tests? [String] (true|false) Default: false
- DEPLOY_WITH_R10K: Deploy branch with r10k. [String] (true|false) Default: false **NOTE:** This requires r10k to be configured web hook support. (https://forge.puppet.com/puppet/r10k#webhook-support)
- TEST_RESULTS_DIR: Directory to look for junit output of test results. [String] Default: testresults
- ACCEPTANCE_TESTS: Required if RUN_ACCEPTANCE is true. Map of values to use for running in the parallel step. [Map] See [below](#Acceptance Test Configuration) for more details
- R10K_DEPLOY_URL: Required if DEPLOY_WITH_R10K is true. The URL of the Puppet server. [String]
- R10K_DEPLOY_BASIC_AUTH_CRED_ID: If your Puppet server is behind basic auth then set a credential in Jenkins. This is the credentialsId set in the Jenkins credentials plugin. [String]
- R10K_DEPLOY_BRANCH: Env branch(s) to deploy with r10k [List]
- SLACK_CHANNEL: Specify the Slack channel to use for notifications. [String] Default: #puppet
- DEBUG: Turn off Slack notifications and turn on more console output. [String] Default: false
- **RUN_ACCEPTANCE:** Run acceptance tests? [String] (true|false) Default: false
- **DEPLOY_WITH_R10K:** Deploy branch with r10k. [String] (true|false) Default: false **NOTE:** This requires r10k to be configured [web hook support](https://forge.puppet.com/puppet/r10k#webhook-support)
- **TEST_RESULTS_DIR:** Directory to look for junit output of test results. [String] Default: testresults
- **ACCEPTANCE_TESTS:** Required if RUN_ACCEPTANCE is true. Map of values to use for running in the parallel step. [Map] See [below](#Acceptance Test Configuration) for more details
- **R10K_DEPLOY_URL:** Required if DEPLOY_WITH_R10K is true. The URL of the Puppet server. [String]
- **R10K_DEPLOY_BASIC_AUTH_CRED_ID:** If your Puppet server is behind basic auth then set a credential in Jenkins. This is the credentialsId set in the Jenkins credentials plugin. [String]
- **R10K_DEPLOY_BRANCH:** Env branch(s) to deploy with r10k [List]
- **SLACK_CHANNEL:** Specify the Slack channel to use for notifications. [String] Default: #puppet
- **DEBUG:** Turn off Slack notifications and turn on more console output. [String] Default: false
- **DOCKER_REGISTRY_URL:** The private Docker registry URL. Required to build with Docker. [String]
- **DOCKER_REGISTRY_CREDS_ID:** The private Docker registry credentials ID in Jenkins. Required to build with Docker. [String]
- **AWS_DEFAULT_REGION:** The AWS region of you Elastic Container Registry

## Acceptance Test Configuration

Expand All @@ -92,12 +103,19 @@ ACCEPTANCE_TESTS = [
failFast: false
]
```

I went ahead and left the first item in the map the same as above but changed the second to include explanations of the values. The main part to point out here is the `puppetRvm()` wrapper function. This function is used in the shared library to run all the commands within the scope of the specified ruby version and gemset name as specified by the `RUBY_VERSION` and `RUBY_GEMSET` parameters. The wrapper function is also safe to use for anything not needing ruby as well. It is however highly advised you use the puppetRvm wrapper function to run the actual acceptance tests since that does require the ruby version and gemset that was installed earlier in the build process.

## Test Results

All test results are assumed to be in JUnit format and placed in a single directory.

## [Changelog](CHANGELOG.md)
## Docker Builds

## [MIT License](LICENSE)
If a docker-compose.yml is present in the repo and `DOCKER_REGISTRY_URL` and `DOCKER_REGISTRY_CREDS_ID` are set builds will run with Docker. All containers are cleaned up after the build completes. Gems are stored in a Docker volume per branch (eg. puppet_master-gems). This allows for faster builds since gems are cached between runs. This can, however, lead to filling up the disk quickly it is recommended that you run a periodic clean job to remove old volumes. See Jenkinsfile.clean_example.

**Note:** The current setup only works with AWS ECR but can easily be adapted to work with other private registries.

## [Changelog](CHANGELOG.md)

## [MIT License](LICENSE)
108 changes: 59 additions & 49 deletions vars/puppet.groovy
Original file line number Diff line number Diff line change
Expand Up @@ -88,68 +88,39 @@ def call(body) {
echo "BRANCH_NAME: ${env.BRANCH_NAME}"
}

try {
stage('Install Dependancies'){
milestone label: 'Install Dependancies'
retry(2) {
puppetRvm('bundle install')
}
currentBuild.result = 'SUCCESS'
}
} catch(Exception e) {
currentBuild.result = 'FAILURE'
if (config.DEBUG == 'false') {
puppetSlack(config.SLACK_CHANNEL)
}
throw e
}

try {
stage('Unit Test'){
milestone label: 'Unit Test'
dir(config.TEST_RESULTS_DIR) {
deleteDir()
}
puppetRvm("rake test")
junit allowEmptyResults: true, keepLongStdio: true, testResults: "${config.TEST_RESULTS_DIR}/*.xml"
currentBuild.result = 'SUCCESS'
}
} catch(Exception e) {
junit allowEmptyResults: true, keepLongStdio: true, testResults: "${config.TEST_RESULTS_DIR}/*.xml"
currentBuild.result = 'FAILURE'
def dockerBuild = fileExists 'docker-compose.yml'
if (dockerBuild) {
puppetDocker(config)
if (config.DEBUG == 'false') {
puppetSlack(config.SLACK_CHANNEL)
}
throw e
}
} else {

if(config.RUN_ACCEPTANCE == 'true') {
try {
stage('Acceptance Test'){
milestone label: 'Acceptance Test'
parallel config.ACCEPTANCE_TESTS
junit allowEmptyResults: true, keepLongStdio: true, testResults: "${config.TEST_RESULTS_DIR}/*.xml"
stage('Install Dependancies'){
milestone label: 'Install Dependancies'
retry(2) {
puppetRvm('bundle install')
}
currentBuild.result = 'SUCCESS'
}
} catch(Exception e) {
junit allowEmptyResults: true, keepLongStdio: true, testResults: "${config.TEST_RESULTS_DIR}/*.xml"
currentBuild.result = 'FAILURE'
if (config.DEBUG == 'false') {
puppetSlack(config.SLACK_CHANNEL)
}
throw e
}
}

if (config.DEPLOY_WITH_R10K == 'true') {
try {
stage('Deploy'){
milestone label: 'Deploy'
def deploy_branch = config.R10K_DEPLOY_BRANCH.any {it == env.BRANCH_NAME}
if (deploy_branch) {
sh returnStdout: true, script: "curl --request POST -k --url ${config.R10K_DEPLOY_URL}/payload --header \'content-type: application/json\' ${config.BASIC_AUTH_HEADER} --data \'{\"push\":{\"changes\":[{\"new\":{\"name\":\"${env.BRANCH_NAME}\"}}]}}\'"
currentBuild.result = 'SUCCESS'
}
stage('Unit Test'){
milestone label: 'Unit Test'
dir(config.TEST_RESULTS_DIR) {
deleteDir()
}
puppetRvm("rake test")
junit allowEmptyResults: true, keepLongStdio: true, testResults: "${config.TEST_RESULTS_DIR}/*.xml"
currentBuild.result = 'SUCCESS'
}
} catch(Exception e) {
junit allowEmptyResults: true, keepLongStdio: true, testResults: "${config.TEST_RESULTS_DIR}/*.xml"
Expand All @@ -159,10 +130,49 @@ def call(body) {
}
throw e
}

if(config.RUN_ACCEPTANCE == 'true') {
try {
stage('Acceptance Test'){
milestone label: 'Acceptance Test'
parallel config.ACCEPTANCE_TESTS
junit allowEmptyResults: true, keepLongStdio: true, testResults: "${config.TEST_RESULTS_DIR}/*.xml"
currentBuild.result = 'SUCCESS'
}
} catch(Exception e) {
junit allowEmptyResults: true, keepLongStdio: true, testResults: "${config.TEST_RESULTS_DIR}/*.xml"
currentBuild.result = 'FAILURE'
if (config.DEBUG == 'false') {
puppetSlack(config.SLACK_CHANNEL)
}
throw e
}
}

if (config.DEPLOY_WITH_R10K == 'true') {
try {
stage('Deploy'){
milestone label: 'Deploy'
def deploy_branch = config.R10K_DEPLOY_BRANCH.any {it == env.BRANCH_NAME}
if (deploy_branch) {
sh returnStdout: true, script: "curl --request POST -k --url ${config.R10K_DEPLOY_URL}/payload --header \'content-type: application/json\' ${config.BASIC_AUTH_HEADER} --data \'{\"push\":{\"changes\":[{\"new\":{\"name\":\"${env.BRANCH_NAME}\"}}]}}\'"
currentBuild.result = 'SUCCESS'
}
}
} catch(Exception e) {
junit allowEmptyResults: true, keepLongStdio: true, testResults: "${config.TEST_RESULTS_DIR}/*.xml"
currentBuild.result = 'FAILURE'
if (config.DEBUG == 'false') {
puppetSlack(config.SLACK_CHANNEL)
}
throw e
}
}
if (config.DEBUG == 'false') {
puppetSlack(config.SLACK_CHANNEL)
}
}
cleanWs notFailBuild: true
} // timestamps
if (config.DEBUG == 'false') {
puppetSlack(config.SLACK_CHANNEL)
}
} //node
}
39 changes: 39 additions & 0 deletions vars/puppetDeployDocker.groovy
Original file line number Diff line number Diff line change
@@ -0,0 +1,39 @@
#!/usr/bin/env groovy

def call(Map config) {
try {
stage('Deploy') {
milestone label: 'Deploy'

if (config.DEPLOY_WITH_R10K == 'true') {
try {
stage('Deploy'){
milestone label: 'Deploy'
def deploy_branch = config.R10K_DEPLOY_BRANCH.any {it == env.BRANCH_NAME}

if (deploy_branch) {
sh returnStdout: true, script: "curl --request POST -k --url ${config.R10K_DEPLOY_URL}/payload --header \'content-type: application/json\' ${config.BASIC_AUTH_HEADER} --data \'{\"push\":{\"changes\":[{\"new\":{\"name\":\"${env.BRANCH_NAME}\"}}]}}\'"

currentBuild.result = 'SUCCESS'
}
}
} catch(Exception e) {
junit allowEmptyResults: true, keepLongStdio: true, testResults: "${config.TEST_RESULTS_DIR}/*.xml"
currentBuild.result = 'FAILURE'
if (config.DEBUG == 'false') {
puppetSlack(config.SLACK_CHANNEL)
}
throw e
}
}

currentBuild.result = 'SUCCESS'
}
} catch(Exception e) {
currentBuild.result = 'FAILURE'
if (config.DEBUG == 'false') {
puppetSlack(config.SLACK_CHANNEL)
}
throw e
}
}
35 changes: 35 additions & 0 deletions vars/puppetDocker.groovy
Original file line number Diff line number Diff line change
@@ -0,0 +1,35 @@
#!/usr/bin/env groovy

def call(Map config) {
if (!config.DOCKER_REGISTRY_CREDS_ID) {
error 'DOCKER_REGISTRY_CREDS_ID is required to use Docker builds'
}
if (!config.DOCKER_REGISTRY_URL){
error 'DOCKER_REGISTRY_URL is required to use Docker builds'
}
if (!config.AWS_DEFAULT_REGION){
env.AWS_DEFAULT_REGION = 'us-west-2'
} else {
env.AWS_DEFAULT_REGION = config.AWS_DEFAULT_REGION
}

env.BRANCH_NAME = env.BRANCH_NAME.split('-')[0].toLowerCase()
echo "BRANCH_NAME: ${env.BRANCH_NAME}"

config.DOCKER_REGISTRY = config.DOCKER_REGISTRY_URL.split('https://')[1]
env.REPO_NAME = env.JOB_NAME.split('/')[0]
env.GEM_VOLUME = "${env.REPO_NAME}_${env.BRANCH_NAME}_gems"

docker.withRegistry(config.DOCKER_REGISTRY_URL, "ecr:${env.AWS_DEFAULT_REGION}:${config.DOCKER_REGISTRY_CREDS_ID}") {

config.container = "docker run -t --rm --name ${env.BUILD_TAG} -w /app -v ${env.WORKSPACE}:/app -v ${env.GEM_VOLUME}:/gems -e RAILS_ENV=${env.RAILS_ENV} ${config.DOCKER_REGISTRY}:${env.RUBY_VERSION}"

puppetInstallDepsDocker(config)

puppetTestsDocker(config)

puppetDeployDocker(config)

sh "${config.container} chown -R 1003:1004 /app"
} // withRegistry
} // top level function
19 changes: 19 additions & 0 deletions vars/puppetInstallDepsDocker.groovy
Original file line number Diff line number Diff line change
@@ -0,0 +1,19 @@
#!/usr/bin/env groovy

def call(Map config) {
try {
stage('Install Dependancies') {
milestone label: 'Install Dependancies'
retry(2) {
sh "${config.container} bundle install --quiet --clean --jobs=4"
}
currentBuild.result = 'SUCCESS'
}
} catch(Exception e) {
currentBuild.result = 'FAILURE'
if (config.DEBUG == 'false') {
puppetSlack(config.SLACK_CHANNEL)
}
throw e
}
}
Loading

0 comments on commit d917f86

Please sign in to comment.