A few weeks ago at Thought Crunch (Thought Ensemble’s digital idea brainstorming event), one of my teammates (Brent Lemons) brought up a touchy subject. It was a topic about how to organize people.
We work with many organizations looking to build digital products (apps, websites, educational tools, safety tools, etc.) and through that work, we have encountered every kind of organizational structure. Brent’s question was, “Which organizational structure is best for creating digital products?”
I can’t remember exactly what I said to this, but it was probably something along the lines of, “It depends” (always a good answer from a consultant). And yet, even as I said it, I knew there was a better answer. I even knew what the better answer was, I just didn’t say it because, as I said before, it’s a touchy subject.
It’s touchy because, in our experience, if making digital products really is the most important thing you are doing, then there really is a best way to organize. And most businesses (and most of our clients) aren’t organizationally structured that way. And no one likes to hear that.
Most organizations are divided into two major groups: engineering (or IT) and product development (or the business). It is notoriously difficult to get these two groups to work together. For the last 20 years or so, much of the intellectual work in software engineering and management has focused on making this divided relationship work to its full potential. And you can make it work, especially if you are building software for internal use.
The traditional approach is to implement some kind of agile or lean methodology and use that methodology to bring the two groups together: user stories, reveals, challenge sessions, etc. Using these types of methods, you can indeed get the performance of digital product development teams to improve. As you can see from the chart below, with the best methods and tools, you can get an organization to about a 72% success rate for software projects. But you may also notice that no matter what new method the gurus can come up with, there appears to be a natural limit for improving this kind of work. No matter the method, 72% is about as good as you can get.
Source: The 2013 IT Project Success Rates Survey from Scott Ambler + Associates
But, but, but, in most of these cases, the teams building this software were organizationally split between IT and the business. What happens when the engineers and the product designers are on the same team, with the same boss, and the same incentives?
This is, in fact, how most startups are organized. Teams organized by product, with business people and engineers all on one team. And startup teams are so productive, so creative, and so much more likely to deliver.
It’s not hard to see why. When people are on different teams, they have different incentives, different goals, and different bosses. Even in the best split-organizations, with the best processes, these differences make it more difficult to build software.
Add to that challenge that the nature of software engineering has changed in many companies. In the last 5-10 years, software development has gone from being a primarily internal facing activity to being an externally facing, product development function. In other words, the technology is becoming a greater and greater component of the product.
I’ve never seen data on the difference between the productivity of split vs combined software product teams, (if anyone reading this has seen studies, I’d love to read them) but anecdotal evidence in our practice, as well as in the public literature, is compelling: organizations that combine their engineering and business staffs, do better.
