At the DevOps Conference 2016 in Berlin I will be giving a talk about Zero Downtime Deployment with Ansible. There are many good reasons for choosing Ansible for automating deployment and provisioning, but when moving from shell scripts and ad hoc operations it can be hard to do the transition. When the uptime command displays several years, everything has been evolving without proper documentation or audit, and your environment is a blizzard of snowflakes. What do you do? Somehow you need to understand what your environment looks like before automating changes.
Several years ago when my development team took over operations from an external vendor we simply didn't know how the servers were set up. Although the vendor had some documentation, it was far from complete, and we discovered many differences between servers and environments regarding patch level and configuration. We wanted to automate the setup, but had to make sure we didn't break anything. After all, our systems were operational at the moment. We had to find some way to replicate production at the given time without making the same mistake of applying changes manually. Checking each server manually or by hand written shell scripts would be to much work and error prone.
Serverspec is a framework for testing your server configuration using RSpec. Serverspec, like Ansible, does not require any agents on your servers and is executed from your local machine over SSH. This is ideal for test-driven development of your infrastructure automation code, but can also be used to verify your current setup. We decided to create tests for one of the production servers and run them against the others. That helped us weed out many differences between them. When we had written hundreds of tests we were pretty sure we were close to the intentions of the current setup.
What we ended up with was documentation of our environment as test code. Not only did we know the state of all our servers and environments. We also had an excellent starting point for automating provisioning. Using Ansible we were able to automate creation of identical servers using the Serverspec code as documentation and automated test suite. It didn't take much time to implement all the steps once we got used to how Ansible works. Luckily Ansible is easy to learn as it uses a language that approaches plain English and has excellent documentation. Once we had reproduced everything based on the tests we vere able to evolve and improve our infrastructure, applying patches and configuration changes to all environments super fast. Since Ansible is also documentation as code and is easy to test drive we didn't see the need for the Serverspec tests anymore. Maintining both is just wasteful work, so we threw them away.
You don't have to, and often can't, start from scratch while going from manual ad hoc operations to automated provisioning. To gain control over your existing environment Serverspec is a great tool. Use it to gain control, and after you're done automating all your provisioning throw away the tests.
I hope to see you in Berlin in June. If not, Auf Wiedersehen!