Automate provisioning of CCO management server

[adborden]

User Story

As an operator, I want the management server to be provisioned automatically through terraform so that I can deploy and manage CCO with all the benefits of infrastructure-as-code and release engineering principles, including Identifiability, Reproducibility, Consistency, and Agility.

Details / tasks

CCO documents the initial setup of the management server which is necessary for CCO. These instructions should be captured as terraform templates so they are incorporated into our existing CI/CD pipeline.

Acceptance Criteria

• Management server is deployed automatically through our automated CI/CD process.
• A pre-built AMI is used, so that deployment of new versions of the Management server can be deployed in an immutable fashion.
• State is persisted via EBS or similar so that the instance can be destroyed and re-created without losing state.

[pwalsh]

This is similar to the existing issue I have on the CCO tracker: datopian/ckan-cloud-operator#69

I will investigate the additional step of AMI creation.

Related, automating is partly covered by datopian/ckan-cloud-operator#68

[adborden]

I think the distinction between this and datopian/ckan-cloud-operator#69 is that this issue is specific to GSA, while the CCO issue is a broader and more generic solution.

For example, datopian/ckan-cloud-operator#69 could include work to create a reusable Terraform module, then this issue would create the Terraform templates and GSA-configuration to consume that module. I wanted to make sure the GSA-specific stuff was captured explicitly and separate from the CCO work because I think we haven't been great about that in the past.

[pwalsh]

@adborden

To make things easier to follow, we added a new, properly documented EPIC for CCO here:

https://app.zenhub.com/workspaces/datagov-multi-tenant-ckan-5cf12b601437ff1e943f0aa2/issues/gsa/datagov-ckan-multi/209

Then following, the tasks that are in this particular issue are fully covered in other backlog items:

• https://app.zenhub.com/workspaces/datagov-multi-tenant-ckan-5cf12b601437ff1e943f0aa2/issues/datopian/ckan-cloud-operator/78
• Expose the state that CCO manages within a cluster datopian/ckan-cloud-operator#81
• Create (versioned) image for CCO and its direct dependencies datopian/ckan-cloud-operator#82

I suggest - we close this issue then.

cc @estebanruseler @kimwdavidson

[adborden]

For datopian/ckan-cloud-operator#78, I'm not seeing anything specific to GSA. My suggestion was to scope it down, or add GSA specific tasks. However, I think this issue captures that work best for GSA.

[akariv]

We now have an AMI containing terraform, cco and Jenkins.
It has all the required code to provision an EKS cluster & other required services using Terraform & CCO.
Everything is built and continuously tested in CI.

[akariv]

Would be good to do a peer session with @adborden to provision a new cluster (to replace the current, unstable one).

[kimwdavidson]

Adam completed documentation today. @estebanruseler to schedule a session for GSA to deploy themselves.

[adborden]

AC for deploying into the GSA environment isn't met, but we're going to call this done and move on.