It's 9:30 p.m. on a Saturday. Rather than spending the evening enjoying a dinner party or a good movie, you're sitting in a windowless basement, hunched over a tiny keyboard and staring at a blurry monitor. The installation process for your company's latest software—the application upon which your job depends—has not gone according to plan. The latest code just won't run, and until you figure out what to do about it, you're stuck there.
You type frantic messages to the programmers, most of whom aren't responding. The one who does get back to you says only, "it works on my machine! ¯\_(ツ)_/¯" Your phone is blowing up with furious texts from executives demanding explanations. You're tired, frustrated, confused, and more than a little afraid of the conversation you're going to have with your boss in the morning. Welcome to Operations!
If you talk to anyone who has worked in Ops for a long time, we all have some stories like this. The details are different—sometimes we were in a datacenter, deafened by the mind-numbing white noise of tens of thousands of whirring computer fans—other times we were livid at unreliable hotel Wi-Fi while nominally on vacation. The essence of these horror stories is the same. Business-critical systems were going dark with only the fraying sanity of a few exhausted humans between unexpected downtime and total disaster.
The real tragedy is that these nerve-wracking outages still happen. In the past, limitations of technology and budgets could excuse some of the tension surrounding rollouts of new versions of software. Today, hassle-free software deployments are within the reach of even modest organizations. All it really requires is a small cultural shift. Of course, changing people can be the most difficult upgrade of all.
The Evolution of Software Delivery
Once upon a time, software was delivered to users on physical media; tapes, floppy disks, CD-ROM, DVD-ROM. In those bygone days, shipping literally meant shipping. The software needed to be completed so that delivery media could be manufactured, packaged, and shipped to the end users. Once those media went out the door, there was no getting them back and no easy way to supply updates to whoever installed them.
In such a world, it's easy to understand why software developers adopted processes similar to the construction industry: determine the requirements, design the solution, build the product, test it, and eventually declare the work completed. As software moved online, it was only natural that some of that mentality carried over. Developers still wanted to continue to deliver reliable, feature-rich software that met all of their users' needs. Selling anything less than a finished product might feel downright dishonest.
Unfortunately, the nature of software is that it's never really perfected. No matter how detailed your plan or extensive your testing, there are always flaws that aren't found in time and feature improvements that no one thought of in advance. In the days of physical media, updates and new features required another shipment. With the Internet, it became possible to distribute new software over the wire, almost instantaneously, as soon as any improvement to the source code is made.
Another expectation that arrived with the debut of online software was uninterrupted access. One of the best features of the Internet is being able to use our favorite services at any time of day or night. This put a whole new level of pressure on development and operations teams, who now had to squash bugs and add features without ever taking the systems offline if at all possible.
Automation is Key
There's an economy of scale when it comes to software updates: the smaller an update is, the less likely it is to break something. A few lines of new code mean only a few places where something could go wrong. Thousands of lines of new code mean thousands more possible mistakes. When organizations wait weeks or even months between deployments, each one becomes a massive and stressful undertaking.
Computers offer us a solution to this. With deployment automation, new code can be deployed without human intervention as soon as it's written. Testing automation allows software to be verified before it's deployed. Security automation can check it for vulnerabilities. Automated monitoring can send alerts when human attention is actually required.
DevOps approaches software operations like a software problem; any repetitive process can be automated. Product management approaches software development as a continuous service, rather than like a massive construction project. Delivering software can now be more like publishing a newspaper or running a restaurant, providing users with something new every single day.
These techniques are not science fiction. Anyone who creates software today can use them. Software deployments no longer have to be miserable; humans have better things to do on Saturday nights than fight with computers!
Craig Butler
Craig Butler is an instructor in the Certificate in Product Management program at the Georgetown School for Continuing Studies. He is a cybersecurity and cloud infrastructure professional specializing in DevOps and incident response management, with a background in business and network administration.