Agile Development: A Brief History

To understand Agile Development, you may have to go back to the 1950’s and Toyota’s lean manufacturing. Without going into a lot of detail, the basic idea of lean, kaizen, etc. is the idea of continual, iterative improvement. That’s really what Agile software development is all about. More about this later. Let’s start from almost ancient history – at least where software development is concerned.

Admittedly, I am a child of the 80’s when we were taught the basics of SDLC or the Software Development Life Cycle. As a computer science student and a young whippersnapper just coming up, I was intrigued and excited about a well-defined process that I could follow to develop software and deliver to my customer’s requirements and expectations. How could this go wrong? It was perfect. On paper that is.

Following the SDLC process, you meet with the customer (business users/stakeholders) and gather requirements and enter into a back and forth analysis of said requirements until you come to agreement with them, and the requirements are documented and written into a document that you can both assent to. Nicely outlined, annotated with supporting diagrams (entity-relationship, state, data flow, CRUD, etc.). Again, what could possibly go wrong? We all have our acts together. We have just spent 2-3 months meeting, documenting requirements, sweating over a hot terminal or workstation in Word, Visio, maybe ERWin, making sure that everything is sufficiently analyzed, diagrammed, outlined, and ultimately approved.

Mr. Business User is going to sign on the bottom line saying that this document represents what he wants. It’s a complete specification.

Let’s pause here for a sec.

Does anyone believe that the document under consideration specifies in its entirety and without reservation, the system the user wants to solve his business problem? Remember, this is not our first rodeo. We’ve been down this path before. Is there a clause in the document for change requests? Of course there is. Why are we already anticipating changes to a document and requirements that we have just spent 2-3 months laboring over to make sure we have covered all the bases and gotten everything just right?

Because it’s simply impossible to get it exactly right the first time. Change is inevitable. Sometimes it is because the business changes. This happens constantly. Sometimes it is because, even with great care being taken, the analysts may not get the requirements and specifications just right the first time.

This Waterfall methodology of requirements gathering and analysis, design, coding, testing, and implementation has proven to be ineffective in my own experience and to many others as well.

By the time you get to implementation several months or even years after beginning, the system may not be what the business expected, or the business has probably changed and the system needs to change with it.

The solution to this Waterfall problem is a set of software development methodologies that emphasize quick iterative development, working software, and frequent feedback from the customer and incorporation of change in the process. These methodologies are collectively called Agile software methodologies.

Although the roots of Agile may go back to the 50’s with Toyota and Test Driven Development with Project Mercury, things really began to pickup in the early 90’s with James Martin’s RAD (Rapid Application Development). Martin was well known for Information Engineering in the 80’s and 90’s. The idea of RAD was to reduce pre-planning and to quickly get into development, so the business could begin to collaborate right away with the development team by seeing a working prototype in just a few days or a couple of weeks.

Then things picked up in the mid-90’s with the advent of RUP (Rational Unified Process) and Scrum, followed by XP (Extreme Programming) in the late 90’s.

And then in 2001, the Agile Manifesto was written. From the Agile Manifesto website:

Manifesto for Agile Software Development

We are uncovering better ways of developing
software by doing it and helping others do it.
Through this work we have come to value:

Individuals and interactions over processes and tools
Working software over comprehensive documentation
Customer collaboration over contract negotiation
Responding to change over following a plan

That is, while there is value in the items on
the right, we value the items on the left more.

• Kent Beck
• Mike Beedle
• Arie van Bennekum
• Alistair Cockburn
• Ward Cunningham
• Martin Fowler
• James Grenning
• Jim Highsmith
• Andrew Hunt
• Ron Jeffries
• Jon Kern
• Brian Marick
• Robert C. Martin
• Steve Mellor
• Ken Schwaber
• Jeff Sutherland
• Dave Thomas

© 2001, the above authors
this declaration may be freely copied in any form, 
but only in its entirety through this notice. 

Finally, even though Agile has its critics and is not a good fit for all software development projects, the methodologies have gained a strong hold in recent years. Agile has also been further legitimized with certifications from the PMI (Project Management Institute), the Agile Certified Practitioner (PMI-ACP)in 2011 and Scrum Alliance certifications like Certified ScrumMaster (CSM).