One of the biggest value propositions of the cloud is agility. Enterprises are leveraging the public cloud to accelerate delivery of new services and features at speeds not seen before. But in order to achieve agility, enterprises need to do more than just learn how to leverage cloud services. They also need to assess their legacy baggage and transform the way they think about software and delivery.
Legacy baggage comes in many forms. Many of the vendor solutions that enterprises have embraced in the era of the data center do not have a viable equivalent in the public cloud. Many legacy processes were derived in the days of biannual releases, which get in the way of progress in the era of continuous delivery. To be agile, legacy organizational structures, built to control costs and dictate policy, must give way to new organizational structures that promote high levels of collaboration, transparency, and shared goals.
As I have written in the past — and many of my colleagues agree — agility is the new currency. Cloud is one part of the equation. Getting out of the data center business and embracing cloud APIs is a great step forward. However, embracing the cloud without embracing the need for transformational change is a losing proposition.
The Solution: Go Big on DevOps
Many people equate DevOps to IT Automation or even CI/CD. Automation and CI/CD are components of DevOps that you will typically incorporate into your DevOps journey but they are not DevOps in and of themselves. At CTP, our definition of DevOps is “a culture shift or a movement that encourages great communication and collaboration to foster building better quality software more quickly with more reliability.” DevOps is the progression of the software development lifecycle (SDLC) from Waterfall to Agile to Lean. When we perform DevOps maturity assessments with our clients, we assess their maturity across three spectrums: People, Process, and Technology.
DevOps & People
Within the people domain, we evaluate enterprises in four categories: innovations, skills, culture, and collaboration. Innovation is a much overlooked area when it comes to transformation. In the days of physical hardware, it was often unfeasible to invest the time and money it took to procure or create new environments to try out experiments or a hypothesis. Many enterprises outlawed experimenting with new technologies without going through a proper vetting process. This mindset often stifled innovation. I have interviewed clients who have told me they stopped trying out new ideas because it was so painful to go through the processes, or because it was discouraged due to perceived risks. Employees would experiment at home but could not bring these learnings to work. Stifling innovation often leads to employee turnover because the best and the brightest love to innovate.
In the pre-cloud days, embracing new technologies was a lot harder than it is today. For example, let’s say an engineer determines that for a particular workload, a non-relational database like Hadoop is a better solution than the existing Oracle relational database that he or she is forced to use. The engineer would have to go through a committee to get approval to bring in a new vendor. Hardware and software licenses would have to be procured. People would have to be trained or hired to manage and maintain this new technology, and so on. All of this takes weeks or months to discuss, plan and execute, which almost always exceeds the time that the engineer has to complete the task. After going through this process once or twice to no avail, the engineer will not even bother to bring up new solutions any more.
In the cloud, databases like Hadoop are available as a managed service with a simple API call. No hardware or licenses to procure, no army of people to manage and maintain, and very little time and money is needed to experiment. If allowed, the engineer could test his or her hypothesis the same day without involving any other people. If the experiment proved that Hadoop was indeed the best solution for the specific workload, then the engineer could work with others to make it an official solution for the enterprise. In order to take advantage of the innovation capabilities that the cloud offers, people need to embrace innovation as an enabler, as opposed to it being a risk item to be carefully controlled.
People skills is another area we evaluate. Not only do people need to be trained to use the cloud provider’s services, but they also must learn new methods and approaches required to take advantage of the cloud. Too often enterprises treat the cloud as just another datacenter instead of what it really is: a game changing agility platform. In addition to skills, we look to ensure that employees’ incentives are aligned with the new way of doing things in the cloud. If incentives don’t change, how can we expect people to transform to the new way of building services in the cloud?
Silos are a big blocker for any transformation. We look at how different silos collaborate and whether they work together in unison or work against each other. Traditionally, silos owned singular parts of the software development lifecycle (SDLC). One team owned development, another QA, a third owned Ops, a fourth security, etc. In the DevOps model, these are all shared responsibilities. Everyone owns security. Everyone owns quality. And not just IT people. The product owner and the business sponsor also share the ownership. When ownership is shared, everyone works towards a common goal.
DevOps & Processes
One common mistake I see in almost every engagement that involves legacy processes is a lack of focus and analysis of those legacy processes before proceeding with automation. As a result, enterprises are automating waste and not realizing the benefits of agility they were expecting. This is often the result of silos and mismatched incentives across those silos. Developers will work in their own silo and implement CI/CD. They see great improvements in both time and quality of their build process but often see little to no change to their time to market metrics. Why is that? Because there are 20 to 30 years of legacy change control processes created in the era of biannual releases that have not been addressed.
I have seen numerous instances where red tape can mire the process for weeks or months prior to being able to perform an automated build, followed by more weeks and months of red tape in order to promote the code to production after the automated build. Yet teams still focus solely on perfecting CI/CD.
If you have ever read Eliyahu Goldratt’s, The Goal, you will have learned that working on the wrong bottlenecks does not improve the overall flow of a system. Instead it moves the bottleneck to another part of the system. If enterprises only implement CI/CD without performing a value stream assessment of the complete system, they will only move bottlenecks from the build process to another part of the system, thus never achieving the desired agility. Engineers must think about the system as a whole instead of just focusing on automating a component of the system. System thinking can be a foreign topic to a silo-based organization.
Another area we look at is the SDLC practices. Enterprises planning to move to frequent deployments need to embrace lean principles and move away from traditional waterfall or immature scrum approaches. Governance is another important area. The old method of governing with an iron st need to give away to baking controls, policy, and governance into the code. The days of holding multiple weekly review boards for architecture, security, and governance must be put to bed. These processes and mindsets simply don’t work in the era of continuous deployment. In this new age, we must trust in our automation and institute proactive and continuous monitoring to check for ongoing security and compliance. Manual review by humans just doesn’t scale when multiple teams are able to perform push-button deployments. We must audit ourselves in real-time in the new world.
DevOps & Technology
It is here where we finally start focusing on IT automation and the famous CI/ CD processes. What many call DevOps is just one small piece of the DevOps puzzle. Running systems in the cloud requires new tooling and methods. Many of the legacy tools we have used in the past require state and physical infrastructure. We recommend born-in-the-cloud solutions in the areas of security, monitoring, logging, code repositories, etc. Providing visibility into system health and application state is crucial in providing high SLAs in the new world where deployments happen frequently. Much thought needs to go into to building a robust security and monitoring framework that feeds into a central logging solution and can be accessed through a single pane of glass.
The build process should perform security and coding standards scans. Testing should be automated and part of the build process. The build process should produce a score for security, programming standards, and quality. The build should fail if any one of those scores are not at an acceptable level. The goal of this approach is to not let issues progress downstream, because it is much more expensive and time consuming to x defects later in the life cycle.
After evaluating our client’s capabilities in the area of people, process, and technology we provide a maturity score.
The score is a snapshot of the client’s current state of maturity. Next, we provide a list of gaps in each area that shows the delta between the current state and the desired future state. Clients often want to start with a maturity of level 3 so that they can get to a consistent, secure, and reliable state for deployments, while achieving a higher level of agility.
The score provides us a place to begin the conversation, but where the rubber meets the road is in the details behind the score. We provide a list of gaps, in a roadmap format, and the recommendations for each gap. The number of gaps can be quite overwhelming, especially if a customer is early in their journey. But it is this roadmap that brings clients the most value.
What bottlenecks should they work on first and in what order? How can they work on people, process, and technology changes concurrently? Too often we see clients only implement the technology recommendations and make little to no progress on the people and process recommendations. The end result is a suboptimal experience in the cloud and a missed opportunity to achieve the ROI that could be achieved by treating the cloud journey as a transformational project, as opposed to just a technology project.
Cloud computing is providing enterprises with the capabilities to greatly improve their overall agility as an organization. To take advantage of this opportunity we recommend that enterprises embrace the DevOps mindset. We also reiterate that DevOps is more than just IT automation and CI/CD. Embracing the cloud calls for a complete transformation of an enterprise’s people, processes, and technologies. Looking at the cloud any other way will diminish your returns on your investment.