Thinking about "Agile" projects
I recently attended an event that was looking at applying aspects of Agile Product Development to research and other areas. This got me thinking about all the things I've observed and struggled with.
What is Agile?
There are lots of definitions and a wide range of real-world applications of Agile. Broadly speaking it is an approach to managing a ‘project’ in a manner that is explicitly adaptive to what you learn at each step of that project. Learn something new at the very start of your project? Incorporate it and change or cancel the project.
This makes it seem like common sense but there are many situations where projects require so much upfront investment or planning that “full Agile” isn’t possible. Once the construction starts, you are unlikely to decide to call off the project. There are also many human-factor problems with Agile. Teams have power dynamics, invested interests and sunk costs. Even the most Agile labeled organization and team will struggle to make their projects truly adaptive. Any good project will have aspects that are Agile in nature. Anyone who insists that learnings as the project progresses cannot be incorporated into the planning is crazy. But actually making it work isn’t as easy as the label makes it seems.
Different project management techniques are developed to address reoccurring problems that people face. If all projects went well and people were happy with the outcomes, no one would bother with the overhead of a process to decide how to manage a process.
Within this category of Agile, there is a whole spectrum of approaches - some like SAFe end up looking a lot like a top-down, highly structured process to deliver a fixed outcome by working a process machine (SAFe has a process for everything). The other side of the spectrum is a homebrew that the squad or team working on a given outcome comes up with a way-of-working that aligns with their styles and objectives or outcomes.
Definitions
I use ‘Solution’ and ‘Stakeholder’ repeatedly in this post and wanted to provide a short definition:
Solution: any project or set of tasks that are being done to provide a solution to some kind of problem. My most common experience with this is building technology solutions like apps or automations. The definition can apply to any project to build something to solve a problem.
Stakeholder: I use this in the broadest sense to mean someone involved in the project outcomes but who isn’t a team member. Stakeholders are most commonly other business people involved in a project/providing input or who have a stake in the final result.
15 Digital Agile/Project Observations
These are my own observations from working in various explicitly agile and non-agile teams during the last 20 years:
It isn’t faster or cheaper by default - Agile approaches are often advocated for as a faster and cheaper way to build solutions. However, the opposite is often true. If your objective is a clear and specific set of functionality or validation of something, it is much cheaper and faster to run it as a best-practice, top-down project. An Agile project will often produce a functional concept quickly, but simply by the nature of an interactive, iterative approach, it will take much longer to produce a final, stable solution.
Multi-stakeholder Agile solutions aren’t always “the best” - Often Agile projects are seen as fusion-teams or multi-stakeholder projects where different people from across an organization are tasked to work together to achieve some objective. This rarely results in the best solution. As the process involves spending considerable time with alignment and involvement of a diverse group of stakeholders, projects take considerably longer and require excellent facilitation and leadership. Be especially careful in how the stakeholder involvement is structured and what, how much and in what way you want input and involvement of stakeholders. Unbounded involvement of diverse stakeholders sounds great and results in chaos.
User Research is a well established profession, use it - Agile teams can often ignore industry research and best practices with the naive belief that the team can figure out the best solutions through collaborative iteration. This is simply wrong. There are well established areas like User Research that any team needs to take seriously if they want to build a good solution. URs have a lot of good approaches to minimizing negative impacts and getting more accurate insights from users.
Having a walk-away point - in any large group of stakeholders there will often be groups that insist on terms of participation and the solution development team can end up feeling like they need to agree to the demands of all stakeholders. This is common in large corporates trying to build new apps or solutions. This will quickly become a mess. It’s critical to define stakeholder requirements upfront and limit the power of individual stakeholder to co-opt the project.
Separate ‘learning how to build a solution’ from ‘building a solution’ - for any new team looking to build a solution they may end up justifying everything as ‘this is important for us to learn how to build a solution’. That is valid and important, but needs to be clearly separated from ‘we now need to build a solution’. Otherwise, you can end up with stakeholders being increasingly frustrated by the process as no solution is being generated but the involved team is very happy with the learnings.
Issues with awareness, consent and power - a lot of multi-stakeholder projects with digital will put a focus on participants having full power to opt-out / control their data / etc… However, it is also important to consider the reality that many users may not have the knowledge or expertise to judge whether something is a good choice or not. So sufficient care should also be exercised to not offer choices which can later be problematic when users finally do understand what they have agreed to.
Team Design/User/Stakeholder Recruitment and Filtering is Critical - the success of any solution development is dependent on the selection of the people who will participate. This needs to be controlled and carefully managed like facilitation and leadership of the project overall. If Recruitment and Filtering isn’t handled well most energy will be spent on stakeholder and team conflicts.
Security and Data Handling is a Massive Issue (much, much more than you think it is) - every stakeholder involved in the process increases the risk of data leaks / security breaches by at least double. Especially if there are sub-stakeholders who are not directly managed by the team. Agreements, storage, processing, handling, etc should all be designed with the expectation that there will be leaks and breaches and thus sensitive data should simply not be retained at all. There is also a direct relationship between success and risk - the more successful the projects, the more visible and the more likely to be targeted by bad actors. Success also leads quickly to complacency within the teams. Make sure you are building secure-by-default and related design principles.
Stopping is just as important as continuing to try - many projects suffer from continued iteration into a blackhole. Even if a project or a budget is approved for a given period, the team needs to be ready to regularly stop projects that are not validating, reset and try something new. This is incredibly difficult and most corporates fail to manage it well. There is a lot invested in any given project and it can seem much better to “keep trying” “keep learning” rather than accept that it is better to stop and reset. Stakeholders may struggle to understand this as they may go into the process expecting a usable “outcome” yet for such projects a result that the approach didn’t work and needed to be stopped (and was stopped) is success.
Iterative projects can become self-serving ego echo-chambers - even with the best teams in place, given the unstructured nature of iterative projects, there is a risk that the project team can become an echo chamber of self-reenforcing good news. Example: “this feature wasn’t popular with users but look how much we are learning!”. Teams can define success of each iteration and often have influencing power over users and stakeholders. This can be cooked into a boiling pot of “we are the best and this is amazing” pretty quickly. Defining success criteria upfront and cultivating a culture of harsh external and internal feedback helps, but will never completely remove this risk.
Consider the impacts/costs of the project and also removing the project - too many projects are completed successfully and then stopped for some reason. Yet the costs and impacts of the stopping of the project are rarely incorporated into the planning. For example: A new feature could completely change the way a user interacts with the app. Then removing that feature will cause chaos for that user. A user could become dependent on the interactions provided by the support team or stakeholders and may not handle the conclusion to the project in the same way. Projects should be considered in terms of potential impact and the difficulty to close-out the interactions. Without this, there is risk of Hotel California styled never-ending projects which have ended but people can never leave.
Many solutions are valid and impractical - a lot of projects are successful in small-scale pilot stage and then fail to scale. It is generally much easier to make a pilot successful as there is more dependency on the interactions of the core team and more engagement with a user or stakeholder group. It should be considered early on whether the potential project is reliant on a lot of team hand-holding or is already designed for scale. This can be very difficult to think about as by nature of an iterative project the early team is very hands on and constantly iterating the approaches. The question needs to be constantly asked “yes, this sounds like a good solution, but can we replicate it beyond this team at a dramatically cheaper cost?”. If there are doubts, unpack them, and incorporate these kinds of validations into the early pilots. In many cases, it won’t be possible to figure this out until you attempt to scale a solution. Thus, it’s good to catch those cases early and figure out a way to test scale as early as possible. This point assumes that scale is an objective. Sometimes it isn’t - see point 14 below.
Seriously track the cost of value - I can pay a user to do pretty much anything. A lot of successful apps hit scale pretty much via paying users to use their apps. It’s a very risky way to reach scale. It is important to keep track of what is your cost of value/impact and track it carefully. It will likely turn out that there are many, highly impactful approaches which are simply too expensive to be realistic. You want to discover this in early iterations, so you can modify the approach or stop the project and try something else.
Keep in mind niche, high impact, viable solutions that don’t scale - while I’ve written a lot about scale, there are lots of solutions that are not scalable as they are highly impactful to only a small group of users. Those are very good niche solutions and shouldn’t be ignored. Often a very niche, unscalable solution can be so impactful that it justifies the cost of having a bespoke solution just for that small segment. Keep this in mind and watch for it.
Find a way to get an unbiased baseline before starting - teams building a solution often get excited by early feedback as users will modify their perceptions and behavior simply because they are involved in a pilot. It makes them feel special and heard. This can lead them to report (honestly) very positive results from the pilots and interactions while never actually changing their behavior. If you can get some kind of unbiased baseline before starting the project, you can assess user behavior against that as you iterate through the project.
More to come…
I’ll keep adding on and tweaking this post as I go. For now these are the thoughts that come to mind. What do you think? What have been your observations working with Agile projects?