Stream Step

There is a lot of blog fodder out there about the "new" movement or field of DevOps. I wanted to make a flat-out statement: DevOps is NOT new. It's a new term. It basically refers -- not to put too fine a point on it -- to the old Mythical Man-Month toss over the "transom" or "fence" problem that has always existed between development and operations/IT.

What is new is that methodologies like agile have turned that transom into a gap (or that fence into a . . . border crossing?) and laws like Sarbannes-Oxley have turned it into a regulatory issue. Thankfully, some folks are starting to blog the same thing: this is not a new problem, it's an old one that has gotten bigger over time because it never got addressed since it was very first identified. Now that we have a working term for it, we can start to ennumerate the pains and problems and needs. Getting a term for a thing is a big deal, but 'the thing' has been around awhile.

I think everyone involved in enterprise development or operations has – at some point or another – fallen under the influence of the “big red button” idea. Put simply the big red button is the button you push when you want all the messy realities of your current process to disappear and be replaced by a flawless, repeatable and automatic system for achieving your desired goal. You want to deploy a few components to qa-a – click - boom – done! You want a new server with these specs – click – boom – done!

It’s not that we can’t get there – we can and many companies over the next few years will get there. No, it’s about how we get there. In my experience the best way to get to higher levels of process automation is to rapidly structure your existing process. And then ratchet up your process automation. If your current state process under control and have the staff and environments to test in – you can automate very quickly. But don’t take my word for it – let’s look at a couple of customer case studies that prove the point.

The Big Script Team

We worked with a customer that had previously determined that their deployment process was too manual and error-prone. A cross-functional team was established to build a fully automated process. They launched the effort and quickly had a big script that invoked all of the lower level scripts – and even had a simple web ui for selecting certain environments. This worked really well for about a month. Then the next release came along and there were some changes required. The next release had a couple of new application components and a new application server version – and suddenly the big script was on its knees and in serious need of refactoring. In fact the first run of the big script had completely trashed one of the critical testing environments. The damage to took hours to fix – and involved several team members that had better things to do. The big script had gone from “time saver” to “time waster” in a matter of days. So the team was at a decision point – invest more time in “enhancing” the big script with more error handling and testing? Or cut bait and get another strategy?

Evolution or Intelligent Design?

Most of us are mortals – which means that we should break big problems into smaller chunks – and then solve each smaller problem in turn. Then we iterate where appropriate to make sure that the entire problem is solved. This is our approach to achieving the big red button. Understanding, as we all do that we will “only” achieve an approximation of perfection as it relates to automation. We understand the constraints under which many enterprise development and operations teams operate and this building block approach is delivering success for our enterprise customers. And let’s not confuse this approach with going slow. This approach will allow you and your team to deliver value in days not months.

When you hear the call for the big red button – think carefully about the implications and downsides of the big script approach. There is a better way to get there.

If the CEO or GM of a business unit producing software using agile in a modern development, operations and QA structure, were to sit in on a release coordination meeting, they'd wonder what decade they'd been transported back to. They'd sit in a room full of folks -- expensive staff, folks like developers, MBA project and product management, QA and IT support folks, a marketing person or two and some extras -- and they'd watch them all gather over a set of spreadsheets and post-it notes and . . . basically decide the future of the company by planning the release steps of its products. They'd see these meetings happen weekly or more as product development and release details are communicated and coordinated using very old -- and poorly suited -- techniques and technologies indeed.

So, yes, they'd think they'd fallen backwards in time, as they wouldn't recognize the painful, time-consuming, staffing hours consuming sink-hole of a practice that defines most release coordination today. They'd see their multi-million and billion dollar business managed at the grass-root level by developers, MBAs and DBAs working in real-time on multi-layer spreadsheets. Using two-hundred dollar tools to attempt to manage billion dollar processes.

They'd see the DevOps gap in terms of salaries paid to expensive talent to sit in a room while a RC or a project manager or equivalent goes through the meeting by making notations in spreadsheets and post-it notes and a notepad for follow-up to get answers that weren't available during the meeting. He'd see IT explain why they couldn't provision everything development wanted and development explain why the release delayed because QA and IT had coordination problems. He'd take the 10 to 20 staff headcount and the process and see that the core of his business, producing product to be sold and installed and counted as revenue, was really more a sort of wonky professional coincidence of deliverables than scientific process.

The CEO would leave the meeting in a state of shock, probably even wondering if he needed to report the state of "DevOps" in his organization to the board, because frankly the DevOps gap is material information on the state of the business. That is, if the CEO ever found out how release management works in an organization that isn't DevOps savvy.

It was a four hour web event hosted by CM Crossroads that started with a panel of experts from Electric Cloud, Thoughtworks, Urbancode and Streamstep. It closed with a great presentation (if I may say so myself, but I wasn't alone in finding it illuminating) by Clyde Logue of Streamstep on "Bridging the DevOps Gap."

Core points included: this gap isn't new. It's just now large enough that we no longer use Mythical Man-Month terms like "the transom effect" because the frequency and velocity has changed to a bullet fired out a machinegun barrel, not a report tossed through window open over a door (do they still make those?). Methodologies like agile have made it a cultural and organizational issue: the DevOps Gap. Also core: while tooling won't address the cultural components, as became clear over the whole session, there is tooling for all of the required functions a DevOps-capable organization must encompass (planning, coordination and automation).

I've been bumping up against the frequent assertion that DevOps requires a specific culture to flourish, much less ultimately succeed. This ties in with many other ride-along issues that crop up when conversations turn to DevOps and anything related. Of course, some folks even argue that DevOps is only a cultural thing and has no tools for it at all (a view which I disagree with and strikes me as -- frankly -- parochial: only the DevOps priesthood can practice it and its not amenable to tools designed to, well, bridge the DevOps gap). DevOps is not new: its basically the chasm that has yawned open as new methodologies have taken what used to be called the "transom effect" and turned it over the last decade into the grand canyon.

At DevOps 2010, there were presentations on this topic. One that you should not miss is about Changing Culture to Enable DevOps.

One speaker commented that they thought that culture was about 70% of influence. I've noticed over the years and in the various start-ups I've done that company culture occurs pretty quickly: it seems to be in place by about employee number 4 and certainly firmly operational by 20 people. And its extraordinarily hard to alter once it's there. And it is both helpful and mischievous in its effects: something to pay attention to and manage directly.