Why it didn’t work
- Matrix management solved the wrong problem
- It fixated on team autonomy
- Collaboration was an assumed competency
- Mythology became difficult to change
Matrix management solved the wrong problem
The “full stack” agile team worked well, but the matrix management of software engineers introduced more problems than it solved.
Teams at Spotify were rather long-lived. The benefit of not having to change manager when moving to another team was limited.
Engineering managers in this model had little responsibility beyond the career development of the people they managed. Even then, their ability to help with interpersonal people skills development was limited. An engineering manager would not manage enough of the other people on the team or be involved enough in the daily context to independently assess conflict within the team.
Without a single engineering manager responsible for the engineers on a team, the product manager lacked an equivalent peer—the mini-CTO to their mini-CEO role. There was no single person accountable for the engineering team’s delivery or who could negotiate prioritization of work at an equivalent level of responsibility.
When disagreements within the engineering team arose, the product manager needed to negotiate with all of the engineers on the team. If the engineers could not reach a consensus, the product manager needed to escalate to as many engineering managers as there were engineering specializations within the team. A team with Back-end, Web app, and mobile app engineers would have at least 3 engineering managers who might need to get involved. If those engineering managers could not reach a consensus, a single team’s issue would have to escalate to the department’s engineering director.
“Chapter leads are servant-leaders who help you grow as an individual. They don’t really work with any team. They have direct reports on all the teams. They don’t have really any accountability for the delivery. They aren’t taking that responsibility. It’s easy to see the product owner as the manager for the team.”Joakim Sundén, agile coach at Spotify
- A product—design—engineering team typically contains more engineers than designers or product managers. Having a single engineering manager for the engineers on the team creates an accountable escalation path for conflict within the team.
- Product managers should have an equivalent peer for engineering. Product managers should be accountable for the prioritization of work. Engineering managers should be accountable for the engineers’ execution, which includes being able to negotiate speed and quality trade-offs with the product manager.
Spotify fixated on team autonomy
When a company is small, teams have to do a wide range of work to deliver and have to shift initiatives frequently. As a company grows from startup to scale-up, duplicated functions across teams move to new teams dedicated to increasing organization efficiency by reducing duplication. With more teams, the need for a team to shift initiative decreases in frequency. Both of these changes allow for teams to think more deeply and long term about the problems they are scoped to solve. Faster iteration, however, is not guaranteed. Every responsibility a team cedes to increase its focus becomes a new cross-team dependency.
Spotify did not define a common process for cross-team collaboration. Allowing every team to have a unique way of working meant each team needed a unique way of engagement when collaborating. Overall organization productivity suffered.
The Spotify model was documented when Spotify was a much smaller company. It was supposed to be a multiple part series, according to Anders Ivarsson. Autonomy made the first cut, but the parts on alignment and accountability were never completed.
Learn from Spotify’s mistakes:
- Autonomy requires alignment. Company priorities must be defined by leadership. Autonomy does not mean teams get to do whatever they want.
- Processes for cross-team collaboration must be defined. Autonomy does not mean leaving teams to self-organize every problem.
- How success is measured must be defined by leadership so people can effectively negotiate cross-team dependency prioritization.
- Autonomy requires accountability. Product management is accountable for value. The team is accountable for delivering ‘done’ increments. Mature teams can justify their independence with their ability to articulate business value, risk, learning, and the next optimal move
Collaboration was an assumed competency
While Spotify gave teams control over their way of working, many people did not have a basic understanding of Agile practices. This resulted in teams iterating through process tweaks in blind hope of finding the combination that would help them improve their delivery. People lacked a common language to effectively discuss the process problems, the education to solve them, and the experience to evaluate performance. It was not really agile. It was just not-waterfall.
“Agile coaches” were internal consultants Spotify provided teams to teach and suggest process improvements. While well-intention, there were not enough coaches to help every team. A coach’s engagement with a team was rarely long enough to span a project’s completion to help a team evaluate performance. More so, they were not accountable for anything.
Learn from Spotify’s mistakes:
- Collaboration is a skill that requires knowledge and practice. Managers should not assume people have an existing comprehension of Agile practices.
- When a company becomes big enough, teams will need dedicated support to guide planning within the team and structure collaboration between teams. Program management can be accountable for the planning process. Dedicated program managers enable teams in a manner similar to how dedicated product managers and engineering managers do with their respective competencies.