Guest Blog: Is Spotify the holy grail of software development?

Guest blog from Kevin Goldsmith, Chief Technology Officer (CTO) at Avvo, Strategic Advisor at Godel, former Spotify, Adobe, Microsoft

Spotify has long been regarded as a shining example of an organisation that is wholly dedicated to the culture of software development. In my time working at the company, I’d often get asked if organisations should follow the same approach – that is to say, adopt the same organisational model, and I think on balance more often than not, I’ve said that they probably shouldn’t. By way of explanation, it probably helps to provide some background to the model.

Spotify’s engineering and product organisation (well over 800 people at the time I moved onto to Avvo) is split into several large groups called Tribes. Each Tribe is responsible for a set of related features or engineering functions. For example, the Infrastructure and Operations Tribe, whose name is pretty self-explanatory. When I joined the company it was as the Tribe lead in the Music Player tribe. We handled importing audio from our label and distribution partners, storing and streaming the music, search, collection and playlists, artist pages, music metadata and the music knowledge graph that supports things like the above, but also ads, discover, radio and the like.

Autonomy

While the whole company works on the same product, Spotify, each tribe was set up so that it can work as independently as possible. A fundamental aspect of the organisational model was to provide autonomy at every level. This helps remove decision-making bottlenecks and unnecessary dependencies, which improves velocity.

Each tribe is composed of squads. A squad is a team that is responsible for a single feature or component. For example, there is a squad that is responsible for search, a squad responsible for the AB test infrastructure, etc. As each tribe is set up to be as autonomous as possible, each squad is also set up to be autonomous. In the context of a feature development team, this means that each team is a full-stack team. A full-stack team is responsible for both backend implementation as well as the user interface implementation, on all platforms.

A typical feature squad would have web service engineers, iOS, Android, web and desktop engineers as well as testers, an agile coach, a product owner and UX designer. With this staffing, the squad has everything they need to implement anything related to their feature. They don’t have to wait on another team to implement the pieces they need. They also have autonomy and local decision-making ability, so there are few impediments on their speed of execution.

To this point with Tribes and Squads described only, Spotify may have seemed like a traditional, hierarchical engineering organisation, but this is where the similarity ended. Unlike a traditional organisation, a squad does not have a single engineering leader whom everyone on the team reports to. In fact there is not a single leader for the squad. The Product Owner and UX Designer work with the engineers and testers collectively to make decisions about their features.

Spotify, however, is not a ‘no manager’ culture – managers have roles as career mentors and organisational communication conduits. Rather than have management hierarchies follow organisational ones (creating a de facto command-and-control structure), we instead had first level managers responsible for technical functional areas across multiple squads.

We called these reporting and functional groupings “chapters.” Chapter leads reported to Tribe leads. In my tribe, there were three backend (services) development chapters, two front-end development chapters (including all the UI developers), a core library chapter, and a test chapter.  These seven Chapters spanned eight different squads. Almost every chapter lead had reports in two squads, and a few of them had reports in three squads. Almost all chapter leads work within a squad in some capacity as well, either as developer or technical lead, and not necessarily within a squad that has members of their chapter.

This chapters/squads matrix organisation is critical to Spotify’s organisational agility. It allows the squads and the tribe to be more fluid. New squads could be created to take advantage of an opportunity or handle an issue without worry about changing reporting structures. If a squad completes its goals and has no reason to exist anymore, we could dissolve it without punishing a manager. This is a very important difference to a traditional hierarchy, because it gave us a lot of flexibility and helped us avoid the old political issues around empire building and resource contention.

In addition to our Tribes, Squads and Chapters, we also had virtual organisations called Guilds. Guilds are cross-tribe organisations centered on different technical or interest areas and their membership was voluntary. The guilds serve as ways to promote cross-tribe collaboration and communication, especially around things like best practices.

Should you adopt the same model for success?

If you are considering using the Spotify organisational model within your company, there are a few things that will be critical to success. If you implement only parts of the model or try to impose it on a very different corporate culture; you will have a difficult time achieving the same level of success with it that we had.

The organisational model assumed that engineering is doing development with agile methodologies. Our goals for autonomy meant that we did not prescribe any particular development framework our squads must subscribe to. However, all of squads used agile development methodologies. While we did our best to minimise dependencies between squads and tribes, they still occurred as we are all working on the same product. Any individual squad choosing to build using a traditional waterfall or other non-agile process would not be able to keep up with the rapidly changing teams around them. If they tried to impose some sort of process on other teams so that they could follow a longer-term development plan, they would start slowing down the rest of the organisation.

A critical requirement in making the organisational model work well was that the entire company worked with and understood agile practices and processes. While the legal team isn’t doing scrum or kanban, they are used to working with engineering teams that use agile processes. Having the entire corporation understand and agree with agile means that no line of business area becomes an impediment to the speed of implementation. If your development teams are working quickly in parallel, but marketing or legal is not supportive of an agile approach, they will become a bottleneck that will slow down the overall speed of the company. Similarly, implementing this with just one team as a test in a larger engineering organisation will be prone to issues. A traditional engineering organisation is not usually set up for autonomy. Adding a single autonomous team within that web of dependencies is likely to hamper and frustrate the team and skew the results of the experiment.

While I’ve mentioned autonomy in several places already, I cannot understate its criticality. Each squad must be empowered to make their own decisions, not only on features, but also on development model, infrastructure, and implementation. Every decision that has to be approved outside the team means a delay that slows development. Each dictated implementation or infrastructure decision means that a technology that doesn’t fit to the way the team works or something new that must be learned before the team can build. This is a challenge to coordination, but in practice it isn’t as bad as it might seem. Best practices and technologies do spread from team to team through avenues like guilds. Teams adopt these practices and technologies on their own schedule or pioneer new ways of working if it makes it easier for them to deliver value to our customers and then spread their learnings to the other teams.

Trying to layer the tribe and squads model over a traditional reporting hierarchy would be very problematic. While we had many long-lived squads at Spotify, we were constantly creating and disbanding squads as new needs arose or missions were fulfilled. Squad membership ebbs and flows as required by the needs of a squad’s mission. Traditional hierarchical organisations are self-perpetuating and restructuring them is very disruptive both to the management chains as well as the individual team members. You would gain some of the benefits of the Spotify model by building full-stack teams in a traditional organisational hierarchy, but you would lose a lot of the overall speed benefits that we leverage with our matrix organisation.

In conclusion, if you are looking to improve the speed of your development and are inspired by Spotify’s organisational model, there are a few things that you need to understand. The model works because it is layered on top of corporate culture. Spotify’s culture values autonomy, agile processes, democratic teams, and servant leadership, amongst other things. You can certainly take some of the ideas from the way it works and apply them in your organisation, but without the cultural underpinnings you may not get the same returns.