The Fry that Shorted Agile Manifesto

The Agile Manifesto prioritizes “Working software over comprehensive documentation,” acknowledging the significance of ongoing development. However, like any principle, this approach has its drawbacks, and I am trying to avoid statements like “completely flawed.” Having observed instances where projects experienced setbacks or even total failure due to strict adherence to this Agile tenet, today, I invite you to explore with me why comprehensive documentation is essential for successful software development.

Areas and phases of product development that may be affected by the lack of comprehensive documentation

Product Scope

Comprehensive documentation is essential for gaining a clear vision of the full product scope. Without it, understanding project deadlines, team composition, dependencies, and budgeting becomes challenging.

Case Study: A product revamp, initiated with a very limited list of high-level feature updates without specification on complexity and a list of functionalities, ended in hiring an offshore agency that provided a timeframe and development cost that proved to be unrealistic. The project ended up with significant flaws, insufficient testing, and a massive impact on the business-agency relationship, affecting team morale.

Product Roadmap

A high-level product development roadmap is crucial for aligning all team members and stakeholders. Comprehensive documentation helps create and maintain this vision.

Case Study: The lack of a lean roadmap resulted in the business continuously adding extra features to the MVP, expanding its scope while insisting on retaining deadlines and product team composition. This made the MVP no longer lean and created unrealistic expectations from the team, resulting in a fatal product failure.

Business and User Goals

Lack of comprehensive documentation increases the risk of misinterpreting key factors, business goals, and user objectives, leading to errors in the design and development stages.

Case Study: Ambiguous documentation led to the misinterpretation of the user goal and the creation of a product that didn’t meet customer expectations. The business didn’t notice soon enough, and the product had to undergo significant rework, creating a risk of a missed market opportunity.

Product Functionality

Comprehensive documentation is essential for a complete understanding of how the product will work strategically and in execution. Insufficient details may create a significant detour from the original idea of the exact functionality, an idea that may not always be captured in an Epic or a Story. In some cases, it may be severe, where one erroneous strategy may result in going too far from the original functional idea.

Case Study: A simple example would be using a dropdown in UI where leveraging a radio box is more efficient. It may be as simple as that. And then, once used, it had to be repeated in similar cases, therefore complicating the user experience.

Product Concepts and Modifications

Documentation ensures that initial concepts and updates are well-tracked, preventing the product from deviating from its original vision. It also provides a way to understand why alterations had to be made and what the initial and following versions of such modifications were.

Case Study: The product, once changed from its original course with a validated reason, returned back to taking the original course because the team forgot that there was a reason for the change, since it was not specifically documented.

Knowledge Management and Loss

Comprehensive documentation reduces the risk of knowledge loss when team members leave, avoiding significant rework caused by the departure of key personnel.

Case Study: Unfortunately, it happens way too often—when a key team member who possesses unique knowledge leaves the company without properly documenting that knowledge, the team often struggles to understand their course and has to redevelop pieces of a product or even start the work from scratch, resulting in a delay in product releases. It’s even worse when it comes to high-level leaders who hold company or product strategy. Sometimes, groups of products get put on hold or stopped completely because the strategy owner is no longer there to defend the product’s necessity.

Organized and User-Friendly Documentation

Well-organized and templated documentation enhances user experience, facilitating faster navigation and access to relevant information for everyone who is involved in the product development and strategy.

Case Study: A team working with unstructured documentation faced challenges in finding critical information that was documented but not well-organized, leading to increased frustration and decreased efficiency.

Development Processes

Comprehensive documentation simplifies development and QA, facilitating tracking of all performed tests, including regression and integration testing.

Case Study: In a product where testing documentation was not sufficient, the testing team missed a significant set of scenario testing that resulted in launching a feature that, following certain user actions, rendered the user’s mobile device unusable. Due to the majority of features being inaccessible in a Sleep mode that was no longer accessible to be turned off, the only possible action to give the user access to their phone features back was a factory reset of their device. This caused hardship for the user, requiring them to reset all their applications, passwords, and reinstall all their smart home procedures.

Leveraging Documentation for New Releases

Well-documented solutions, methodologies, and architectures can be leveraged in new releases and products, promoting efficiency and consistency.

Case Study: Well-documented full user process flow of already developed and proposed processes gave a team great visibility into which set of user processes may benefit from prioritizing it above the others.

Legal, Security and other Compliance Standards

Comprehensive documentation is crucial for products related to legal requirements, audit, and security standards, creating a paper trail for accountability. Approvals documented fall into this category as well.

Case Study: In working on a feature subject to legal approval, only the keen internal awareness of a product manager who remembered that the legal copy for a specific process was pending approval helped the team avoid significant conflicts and potential legal liabilities. If the requirement and the approval were documented as incomplete, the team didn’t have to rely solely on someone’s memory.

What do we typically find in product documentation

Here is a very common set of documentation that is usually developed for digital products:

• Lean business plan
• Product roadmap
• Product budget
• User research
• High fidelity designs
• High fidelity prototypes
• Epics
• Stories
• Test cases
• Some API documentation
• UAT/SIT plan
• Release plan

It’s, of course, absolutely necessary and very good to have, although significantly insufficient. However, I have seen products that didn’t even have all the listed above; just epics, stories, and clumsy wireframes created by business analysts…

So what is Considered Comprehensive Documentation?

Despite the common concern, comprehensive documentation is not hard or time-consuming to develop. If you know what you need to create and how, it’s very easy to complete a set of the necessary documentation. Templates may help, but despite the very common template frenzy, I try to stay lean and simple about them.

In my opinion, comprehensive documentation should be:

1. Version controlled – for tracking purposes
2. Simple – well organized, containing only necessary content, and ideally supplied with visuals such as flow charts and schematic views.
3. Templated across products within the same company without going overboard with templates themselves
4. Supplied with approvals for accountability where required and agreements where needed
5. Supported with procedure outlines, for example, for change management requests, and especially where risk mitigation is needed.

Here’s a list that I consider comprehensive and necessary for each digital product development:

Ideation

1. Business Plans – lean and elaborated with:
   • Idea
   • Viability
   • Value proposition
   • Customer channels
   • Customer segments
   • Cost structure
   • Revenue streams
   • Ideation sessions documented
2. Ideation results visualized high-level overall full product structure

MVP, Roadmap, Budgeting

1. Initial MVP identified in the ideation visualization
2. MVP features
3. MVP complete functional list outline, later used as the design intake
4. Cost to profit analysis
5. Market/user research strategy and findings
6. Overall product roadmap
7. Projected MVP completion timeframe
8. Team composition
9. Budget
10. Final MVP approvals

Design, Proof of Concept, and Compliance

1. Initial MVP user process flows
2. Low-fidelity wireframes
3. Organized in accordance with the process flows
4. Complete business requirements and their approvals
5. High-fidelity designs
6. High-fidelity prototypes following process flows
7. Proof of concept
8. Proof of concept research
9. Proof of concept caused modifications
10. UX and Content architecture charts
11. Legal requirements and approvals
12. Audit reviews and approvals
13. Security requirements and approvals

Development

1. Detailed business requirements
2. Risks
3. Data dictionaries
4. Non-data-related content
5. Non-functional requirements
6. Integration strategy
7. Cross-functional interaction documentation
8. Data and integration architecture
9. API documentation
10. Team agreements
11. Epics and stories
12. Testing strategies including integration and regression testing
13. Implementation strategies
14. Solutions documentation
15. Change management mitigation
16. Knowledge management mitigation
17. Risk mitigation strategy

Release

1. Pre-release UAT, SIT strategy and test cases
2. Release strategy
3. Release plan and procedures
4. Release risk mitigation strategy
5. Hypercare strategy and plan

And last, but not least – we are not saying that all that documentation must be developed before the development starts. It may be developed, as the product development is in progress, but all MVP documentation should be completed by the time when the product moves to the development stage.

This list looks massive and may even seem unrealistic to complete within the Agile development methodology, but I assure you, it’s neither. Once created, many of these documents may be reused or repurposed for further product development or releases and don’t need to be revisited. If you still remain puzzled as to why so much documentation is required, in my opinion, for the completion of a successful product development, please let me know, and I will explain in further postings.

While the Agile Manifesto values working software over comprehensive documentation, this outlook reveals that comprehensive documentation is indispensable for successful software development. Through real-world case studies, we’ve seen the tangible impacts of lacking detailed documentation on project understanding, clarity, and accountability.

Striking a balance between working software and comprehensive documentation is essential for achieving long-term success in the dynamic field of software development.