When I started web development, I has great anxiety when deploying. I was worried that I would crash the website or introduce a horrendous bug that would cost my employer lots of money. Today I deploy very relaxed. It is not because I have changed my personality, I'm still a cautious programmer worried about introducing horrendous bugs. But now I use and understand tools that allow me to confidently deploy. Should any issue occur, I know I will (most likely) notice it quickly and fix it.
9 days ago a reddit thread was created by a user being very worried about getting an angry phonecall when deploying, and judging by the number of comments it was pretty clear that lots of developers have post-deploy anxiety.
The following is a list of the practices that I used to turn me from an anxious deployer to a confident one. I've tried to keep it both generel and with a simple setup to make it as useful to as many people as possible.
1. Develop in an environment like production.
"It works on my machine" is no guarantee that it works in production, small differences can have big consequences. I learned this the very hard way after spending 5 hours debugging a very mysterious issue and eventually realising it came from using python 2.7.4 on the development machine and 2.7.3 on production.
Bugs arising from environment differences are rare, but they are also hard to debug and obviously impossible to test for while development on your local machine.
2. Deploy Script
Deploying a newer version of the codebase is composed of multiple steps and forgetting to do one can bring the site down. If we have and use a deploy script, we ensure that a step won't be forgotten and that everyone deploys in the same way.
A deploy script should contain the following steps.
- Take a backup of the project directory including any libraries.
- Update the codebase to the latest version.
- Install and/or update libraries if neccesary.
- Update the database schema and do any database migrations.
- Collect static files and potentially push them to a CDN.
- Clean the project directory for temporary files.
- Reload the uWSGI server.
Taking a backup of the entire directory might seem excessive, but it is very handy in emergency situations. Rolling back is then a simple 3 step process.
- Rollback any DB migration
- Swap the project directory with the backup directory.
- Reload the uWSGI server.
We know it worked with this directory and that the only changes where within the project directory and possibly the DB. So rolling back those two will mean everything has been rolled back. Knowing that you can easily roll back makes deployments far less stressful. If it goes bad, you have an undo button.
Tests are reproducable proof that the code works. Having them makes me gives me the confidence that I haven't accidentally broken something elsewhere in the codebase. The test suite should be run every time a commit has been made to the
master branch. Ideally from a continious integration server that matches the live production environment in every regard, except for not containing eny live security credentials or having access to the live database.