Adapting agile techniques to open source and personal projects

Agile techniques are a great way to build applications in a flexible manner – allowing requirements to change during the lifetime of a project and delivering frequent working releases. Most agile methods are geared towards corporate development environments – where all contributors are available for face-to-face or instant messaging-based communication and each can guarantee a consistent and high level of contribution.

I’ve recently been looking at how agile techniques can be adapted to the benefit of both open source collaborations and personal projects.

Some key benefits of agile are:

• Sustained engagement of the stakeholders – by allow the stakeholders both visibility into the project, frequent working releases and the ability to change things if requirements change.
  
• Frequently available working releases – by time-boxing activities and test-driven development methods.
  
• A high level of confidence in the delivered software – by test-driven development methods, a large set of automated unit tests and pair programming.
  
• An engaged, productive development team – by frequent short meetings, pair programming and repeated complete delivery cycles.

There are a number of obstacles that open source and personal projects put in the way of using pure corporate agile techniques:

• A highly distributed team – it is unlikely to be able to get people together in person, or even online at a scheduled time.
  
• Lumpy time commitments – these projects are typically background work for the contributors who cannot commit a specific amount of time during any given period.
  
• Contributor churn – existing contributors may disappear from the project at any given point, and new contributors may join.
  
• A variable level / patchwork coverage of skills – dependent on the team of contributors, the areas that require effort may have no skills coverage or a variable level of skills coverage.
  
• Loose or non-existent chain of command – due to the community nature of open source projects and potential churn of original project members there may be no formal day-to-day direction of the project.

Using collaborations and interactions

The distributed nature of the contributors to an open source project means that in most cases it is not feasible to have the frequent face-to-face meetings of standard agile implementations.

An adaptation of the methodology is needed and one option is to use online collaborations (shared documents / spreadsheets / etc.) and interactions to connect contributors.

The methodology is focused on scope-boxing the changes to the project (rather than time-boxing where a consistent level of effort can be sustained.) This means that each contributor should ensure that the project remains in a releasable state (does not degrade) with every contribution. Automated unit-tests (preferably integrated with the IDE build environment) should be created for each contribution to enable future contributors to ensure the same.

The methodology also aims to be as lightweight as possible – so that contributors can focus on progressing the project.

Collaborations

• The project backlog – Mimicking the product backlog of Scrum, the project backlog contains the set of stories that define the future modifications considered or targeted for the project. Stories can include: user stories – the raw features and interactions for end users, design stories – object model creation, code-internal refactorings and migration of code to use a given pattern, bug stories – the tracking list of bugs in the project.
  Any contributor can add any type of story to the project backlog. The priority of each story is maintained in the backlog and can be set either by those directing the effort or by a summation of contributor votes.
  
• Personal treadmills – Each contributor maintains a community-visible personal treadmill – which contains the current story they are working on along with the history of stories they have previously completed. When the user claims a story and puts it on their treadmill the fact that it has been claimed is registered within the project backlog.
  
• The delivery log – As each story is completed and submitted to the code repository it is removed from the project backlog and added to the appropriate section of the delivery log. This contains the set of features and changes that have occurred on the project. Sections within the delivery log include features (for user stories), designs (for design stories) and fixes (for bug stories.)

Interactions

• Adding a story – Any contributor can add a story to the project backlog. The story is tagged with the type of story it is (user, design, etc.) and contains a description of the desired state of the application once the story is completed. Stories should be as atomic as possible such that they lead to treadmill work with a short duration.
  
• Prioritizing stories – A community-wide method of prioritizing stories is to allow each contributor a number of movable votes (e.g. 5) which they can then apply to separate outstanding stories in the project backlog. The stories are then prioritized by the count of votes for each story. As stories get completed contributor votes get freed up to be applied to other outstanding stories in the backlog.
  
• Contribution – A contributor claims an outstanding story and adds it to their treadmill. The story is marked as claimed in the project backlog and tagged with the date for the claim. Each contribution should be delivered such that a releasable project is available at the end of the contribution. The use of test-driven techniques strongly supports the continual verification of existing functionality as contributors add stories.
  
• Release phase lockout – If needed a release phase lockout can be performed on the project backlog. In the run up to a release a line can be drawn under the set of user stories targeted for the release. Only bug stories are allowed to be added above the line. Contributors working on the release branch should focus on completing stories above the line.
  
• Exceptions – Any stories that have been claimed by a contributor who has subsequently become inactive will be available for claim by other contributors. An appropriate timescale is agreed by the community (e.g. a week) before a story can be claimed by another contributor. Any stories on the backlog that are persistently being left unclaimed can have their priority increased.

The same methodology easily scales down to personal projects.