Debunking the myths around agile development
George Orwell once said: “Myths which are believed in tend to become true.” Of course, a story can become a myth or indeed the myth often becomes a good story.
This particular story has its genesis in the mountains of Utah, in February 2001, when a gathering of some of the greatest modern minds emerged with a set of 12 revolutionary software development principles under a proclamation called the Agile Manifesto.
These evangelists declared that requirements should evolve through collaborative efforts using self-organized cross-functional teams. They maintained that constant change, adaptive planning, faster delivery, continuous improvement and evolutionary development were all in fact practices to be embraced.
To traditionalists this sounded like organized anarchy, where control is handed from the executive team to what appears as a group of socially inept software developers where general mayhem and chaos would prevail. The agile atheists were uncomfortable and the war stories germinated.
But it is time to debunk some of the myths that have flourished since the Agile Manifesto proclamation.
The agile methodology helps to maximize the productivity of the development teams by dividing projects into short iterations. One methodologist stated that: “Agile is not just a methodology, but a set of principles and philosophy.”
The main principles of an agile development methodology are:
- Develop small (using Sprints) while building and releasing the software to the client regularly.
- The requirements evolve constantly but the timescale is fixed.
- Requirements should be captured at a high level; using wire frames or visuals where possible. Ensuring active user involvement is imperative.
- A collaborative and cooperative approach between all stakeholders is essential.
- A focus on frequent delivery of the product.
- Testing is integrated throughout the project lifecycle by testing early and often.
VersionOne, in its 2016 “10th Annual State of Agile” report - based on a survey of 3,880 practitioners - clearly outlined that there is an increasing number of enterprises adopting agile. Some 94 percent of all organizations surveyed by VersionOne now practice agile. Approximately 24 percent of respondents to the latest research worked in organizations that have practiced agile for greater than five years, up from 19 percent in 2013.
The report states “while agile adoption is increasing, there are still obstacles to overcome. The key barriers to further adoption usually hinge around culture, including the ability to change, general resistance to change, and management support. Interestingly, the majority of respondents pointed toward company culture as the reason for failed agile projects as well. Once these barriers are overcome, the limiting factor most often cited has been availability of personnel with the necessary agile experience.”
The top reasons cited for using agile for the third year in a row are: Accelerating product delivery (cited by 62 percent) and enhancing their ability to manage changing priorities (cited by 56 percent), which is not a surprise as organizations respond to increasing market demands and customer expectations.
By all indications, agile is helping enterprises around the world succeed. For the past five years, the top three cited benefits of agile include: manage changing priorities (cited by 87 percent), team productivity (cited by 85 percent), and project visibility (cited by 84 percent).
Still, projects still have interdependent tasks and the percentage complete must still be tracked and reported to completion. A project is still a project, a deliverable is still a deliverable, and as such project management principles still apply.
So myth one debunked: Agile does not mean you don’t project manage. Agile means you project manage constantly, to the very heartbeat of the development teams. Keep your basic project management practices as your guiding principles. And stop thinking that the scale or complexity is new or unique.
Another myth of agile is that it is a “new” methodology. It isn’t. Agile has been around since 2001 and it was simply the next generation in software development techniques that evolved from some incredible movements during the early Internet Age of the 90’s.
Agile was heavily influenced by the Iterative and Incremental, Rapid Application Development and the eXtreme Programming (XP) movements. It was a “lift and shift” of certain practices from object orientation, emerging real-time modeling languages, code generators and automated testing methodologies. Agile has been around for 16 years and like a good wine it has matured well. This is good news for new adopters, as there is a multitude of successful case studies, literature and expertise to turn to for help.
The earliest and most vocal anti-agile outcries came from the systems architects, who traditionally built conceptual models of the structure and behavior of the system through formal descriptions called software architectures.
Experts warned in the early 2000’s that the architecture would become “brittle” with pervasive “spaghetti code” in the system because agile did not prescribe formal design. The fear was understandable because as new code is layered on top of older code, with bigger user bases it naturally becomes less malleable.
For example, how many IT systems have you worked with whose architecture consists mostly of embedded “query calls”? Worryingly, the Agile Manifesto did not include, perhaps intentionally, the definition of systems architecture.
The reality is that in agile, one needs more design; it’s just hidden inside the daily activities of the development teams. The design is inherent all the way through development, during every stand-up or sprint planning meeting.
Today, the veracity is that all systems need an architecture, it just may not necessarily have formal architectural models.
So who is responsible then for the overall architecture? The simple answer is that everyone on the team is responsible. However, if this is a large scale or complex distributed system you're going to need to invest some time architecting it.
Do some up-front architectural design and then identify or hire someone into the role of "architecture owner,” or in agile vernacular, a solution architect.
In regards to this myth, agile does mean that developers improve the design of working code during the refactoring process, so make sure to add refactoring into your planning.
Another myth of agile is that there won’t be any project documentation.
Before debunking this myth, let’s discuss why projects need documentation. Documents are a tool for collaboration across the organization by providing all the stakeholders with the necessary level of detail to communicate “up or down” on key project information.
For example, stopping or course correcting a project, (dis)continuing investment, describing the system, training or user documentation and information on how to support the product. Oftentimes documentation is produced just for the comfort factor; the “we always did it this way” factor. The paying customer usually doesn't see under the “hood” at the code base so being able to touch and feel documents gives them the comfort that they are paying for something tangible.
Of course, developers and technical personnel abhor doing documentation. So at the start of every project, agreement should be reached between all stakeholders on what documentation needs to be produced throughout the project. Documents take time and time is money, so documents carry a hidden cost. Therefore, documents should be sized, the cost agreed before being scheduled into a project.
In the past, requirement documentation was vast with a lot of repeated or unnecessary information within each requirement specification. In agile, requirements are compiled as a collection of user stories that can be actively updated and maintained using product backlog management software.
Increased collaboration throughout agile development projects provides all stakeholders with a better understanding of the end product as it is being created, thus reducing the need for some specification documentation. There is redundant documentation and useful documentation. The trick is to find the right level of detail.
Still, let’s say it as it is: Agile probably does mean less documentation. In agile you document only what has to be documented. The difference with other methodologies is the efficiency of the documentation.
There is a strange interpretation that agile means there is little to no planning, when nothing is further from the truth. In agile software development, work is confined to a regular, repeatable work cycle, known as a sprint. Usually sprint cycles are 2 -3 weeks in duration, where user stories are designed and built.
At the start of each sprint, the entire team partakes in the sprint planning session. A story is a short description of a discrete piece of functionality. A story must include the usage scenario, the action taken and the resulting behavior. Specific acceptance criteria, mockups or wireframes can be added to provide additional details and guidance.
The agile product backlog is described as user stories, which is a prioritized features list, containing short descriptions of all functionality desired in the product. The main input to the product backlog will be the requirements represented as epics.
An epic is essentially a large user story that can be broken down into a number of smaller stories. It may take several sprints to complete an epic. It is important to note that an epic can span more than one project, if multiple projects are included in the planning to which the epic belongs. Therefore, sprints enable prediction of feature completion based on the story and epic point estimations by the engineering team, the product priority and the team velocity.
The vast majority of agile frameworks involve frequent, regular and evolutionary planning. If there are release plans, a high-level agreement will be in place of what product is being delivered, and at what timescale and cost. However, the agreement and the plans are specifically set up to enable change, and there will be significant regular ongoing planning.
Therefore, in agile there is rigorous daily planning. The problem is not the planning; the problem is the competence of those carrying out the planning. It requires well-trained and specialized teams capable of self-management, communication and decision-making. Invest in a specialist – an agile planning coach for a period of time to mitigate this potential issue.
One of the biggest myths about agile is that it is only to be used in software-based projects.
It is true that when the Agile Manifesto was published it was in the context of software delivery, but agile can be applied successfully in any business environment.
Given the proven success of agile development, an inevitable question for companies in healthcare, pharmaceuticals and other regulated industries is: ‘Can we adopt agile approaches in our environment?’ While medical device or pharmaceutical regulations do not prescribe a fixed lifecycle model, activities are still presented in a sequential manner, which naturally drive organizations back into waterfall development.
Several medical device manufacturers have adopted agile practices while keeping development in compliance with regulations, but conflicts still arise and decisions have to be taken in favor of agility or formality.
Agile team members have to master the technology and know the regulatory guidelines (for example the FDA regulations, security or technical aspects). And there are agile guidelines or standards available.
In 2012, the Association for the Advancement of Medical Implementation (AAMI) released the TIR45, which guides medical device development companies on how to use agile under stringent quality regulatory processes. TIR45 is an excellent place for neophytes or hardened agile veterans alike to gain insight into using an agile framework in the medical device industry. It’s a great reference point and demonstrates clear alignment with the all-important IEC 62304 standard.
Undoubtedly, over the coming years we will see continued growth of agile practices in non-software based projects.
In conclusion, there is still a lot of conjecture and debate about agile practices and processes. The harsh truth is that today, many projects still fail.
VersionOne states that the top causes of failure are lack of experience with agile methods (44%), company culture/philosophy at odds with agile (cited by 42 percent) and lack of management support (cited by 38 percent).The top barriers to success are the ability to change culture (cited by 44 percent), not enough personnel with agile experience (cited by 35 percent) and organizational resistance to change (cited by 44 percent).
One can conclude that the main problems with software development today are organizational culture and the ubiquitous problem of poor collaboration. But as long as you recognize this, and you keep the above information in mind, your chances of getting better results with agile will significantly improve.
An agile guru recently stated that: “Agility within and of itself is a strategy.” It’s not a silver bullet; it’s simply a set of proven guiding principles to develop software faster. And don’t forget: “Fairy tales are myths, and myths are only myths because there's a grain of truth in them.”