If you are like me and dabble in technology more than once, you will see three-letter acronyms come and go. Sometimes, the technology they are referring to is just a flood. Sometimes it will linger for a while before it is included in the platform we build.
The same is true for Value Stream Management (VSM), which has become more and more popular in DevOps circles. The first question I tend to ask is, why? Is this a new innovation that requires a name, or has someone discovered weaknesses in existing tools and methods?
In short, VSM refers to the need or ability to have visibility into the way the software is built. As the entire process of functional units from concept to deployment continues, managers can benefit from understanding how this happens, the speed of development, where the bottleneck is, the value of delivery, and so on.
The question of whether we need a VSM is particularly important in the field of software development, especially because people have spent a long time building applications. You might think that we will now know how to do this and how to manage the process.
So, has the DevOps world suddenly become an epiphany, suddenly discovering the secrets of life, the universe and how to develop software? Incomplete. VSM (also traditional) exists as a response to current needs, so let’s look at the reasons.
First, let’s face reality: software development has been on the right track for decades. As systems become larger and more complex, linear processes cannot keep up, or, more accurately, things become slower and slower. Maybe there are still some waterfall advocates there, but many times, the process itself is a bottleneck and hinders innovation.
In the 1990s, many people began to study different ways of doing things. Some people choose lean production methods and Japanese efficiency technologies. Others focus on results, use case-driven design and eXtreme programming are just getting things done. However, when I was training on agile development methods such as DSDM, this method was largely an exception.
Then, driven by the Web, open source, RESTful API, etc., a new reality emerged, where children are getting their jobs done and surpassing older and trickier methods. Websites and applications need to be developed quickly; they need to be put together and put there quickly. People started to say: Look, can we get that website before next week?
The need for speed is largely driven by fear, and we still see this today, because organizations (correctly but exaggeratedly) are told how they can change or risk bankruptcy. However, with the acceleration of software development, it has encountered new challenges and bottlenecks, the most important of which is the need to control changes (one of the founding principles of DevOps, in 2007).
Fast forward to today, facing a whole new set of challenges. The fact is that if any method is widely adopted, it will eventually show weakness. In this case, developing things “fairly” quickly will harm other interests, such as developing well (see Shift Left Quality and Safety) or delivering positive things.
The latter is where the VSM comes into play. It basically fills in the gaps: if you are not thinking about whether you are doing the right thing in the right way, then it may be time to start. We are now in an era where agile practices (previously the exception) have become the norm. But agility itself is not enough: “controlled agility” is required.
This brings us another challenge. The world has changed from a scenario where everyone builds things in the same way (waterfalls) to a development process that uses flexible changes according to what people want to do. This is great when you are just starting out and want to focus on building things, but when you want to (for example) change teams and move on without knowing everything is working again, it doesn’t work so well.
Frankly speaking, the development process has become fragmented, inefficient, cumbersome and costly. This is not good-the team doesn’t want to spend time on processes and tools that might build great new applications. This is the origin of VSM.
The term value stream comes from manufacturing. I think the easiest way to think about it is to consider what you offer in an activity that is trying to build value on top of each other as a series of activities. Therefore, first, start to consider the development process as a value stream; make it efficient and effective, and then focus on standardizing the value stream of the entire organization.
Someone asked me recently: Does VSM not just apply business process modeling and management techniques to software? My answer: This is all about applying business process modeling and management to software development. This can be traced back to Hammer and Champy’s good definition of business processes: this is a series of activities that can bring value to customers, and so should software delivery.
Value stream management exists because it must be executed immediately, although it does not exist in many places where DevOps practices are attempted. Therefore, this is actually reinserting the tried and true principles of management governance and visibility into a fast-moving, dynamic and agile environment.
Will VSM last? This is another good question. I heard that some organizations find that VSM is another overhead (hint: they may be doing it wrong). I also have the idea that if we, as advocates of development and operations, can agree that we don’t need each individual project to reinvent best practices, then we might be able to standardize our pipelines so that we have more time to move on. Yes, cool stuff.
I don’t want to see the return of onerous methodologies (such as waterfall). I really want to see innovators innovate with minimal pressure, developer development and operator operations. I watched with interest the DevOps Institute moving towards the direction of evaluation capabilities. I am very happy to see the product-based approach adopted in software development, and I am discussing with multiple vendors how to convert pipelines into Terraform as a form of code. . The same open standards.
All these threads are providing a more consistent approach for the future. All of these have a catalyst, and that is the microservice approach, which begs for frankness in the face of the complexity they create. cf. A recent conversation with Tracy Regan of DeployHub about application configuration management requirements.
I know I haven’t answered the question yet. I believe VSM will prevail, but it is a more comprehensive end-to-end tool and platform function, not an additional layer. Hosting the value stream is a good thing, but you don’t need a separate tool isolated from other tool chains for this.
Therefore, in the end, VSM is not a huge epiphany. This is just a symptom of where we are, because we want to provide software-based innovation on a large scale. The journey is far from over, but it has reached the place of integration based on microservices (a bit ironically, it can be traced back to the principle of modular design in 1974). Best practices are emerging and will provide the standards and platforms we need as we move into the future.