What is Agile … Really?

A I am constantly amazed at the misconceptions that most people have regarding what agile is and how it is used.  Often times when I bring up the topic of agile, people immediately start talking about “Scrum” and how they like it or not; or whether it can be used on big projects or not.

Agile is hard to pin down as it’s somewhat of a moving target so I have written this article to provide a short overview to those people interested in understanding what Agile really is.

Most people view “Scrum” as agile, and while sometimes a few people may mention concepts from “Lean” or “eXtreme Program” like Kanban boards, pair programming or test driven development, generally people think Scrum is agile.

However, agile is much bigger then Scrum. For example take a look at the diagram below.

Agile Frameworks

Agile really is about “delivery value by satisfying the customers needs by frequently delivering working software and quickly responding to change”.

Most agile teams welcome changing requirements if it helps the customer’s competitive advantage. And, by frequently, we mean from a couple of days to a couple of weeks using a face-to-face mean, as its often the most efficient and effective method of conveying information.

The key is most times, “the customer doesn’t know what they want”, so course of development ideas will flow which will cause customers to change their minds.  Therefore, if you want to deliver customer satisfaction you must respond to change.

In summary, here are four main benefits of agile, (if done correctly!)

  1. Delivers the Right Solution  – agile helps the team, users and other stakeholders to align with the right people to deliver the solution that the business actually needs
  2. Accelerates Delivery – agile shortens the development cycle and provides incremental value by using a set of short time-boxed iterations to get to the right solution faster
  3. Increases Flexibility – agile reduces the upfront investment by creating a more efficient IT development process which helps maximizes return on investment (ROI)
  4. Reduces Risk and Increases Quality – agile provides users, managers and stakeholders with greater stakeholder visibility and more granular control

Scaling Agile – Why Scrum isn’t enough

Agile Best Practices – Scaling Best Practices for large organizations

While most teams can start using agile practices, such as Scrum and XP, they often run into hurtles when adoption is extended to other parts of the organization.

Most often these challenges (or agile scaling factors) severely limits it adoption.

Recently delivered a session at the Agile Brazil 2012 conference on why most larger teams often must scale Scrum to overcome challenges (such as geographical distribution, organizational change issues) in their environments to success use agile.

Here it the link to the actual presentation.

Presentation Summary

Agile is a highly collaborative approach to software development that focuses on producing high quality “working code” that is potentially shippable at regular intervals. While many aspects of agile are new and innovative, agile practices are derived from many other successful methodologies such as traditional SDLC, iterative development and lean.

Over the last 5 years adopting practices such as Scrum or XP for small collocated projects has become fairly straightforward but we have found that most teams run into problems as their projects move away from an ideal agile team (e.g., 7 +/- 2 people, co-located team, cross-functional skills, self motivated resources all sharing the same vision and drive to deliver potentially shippable code).

Scaling agile to support larger more complex projects and problems often requires strategies that address both the team’s and the organization’s challenges and perceptions with regards to agile practices.

This session focuses on some of the key agile practices that must be scaled to overcome these challenges. The session uses a number case studies illustrating approaches for adopting more disciplined agile practices. In addition we will also cover 2 or 3 specific agile practices that must be scaled.

What is an agile methodology anyways?

Agile methodologies is a collection of lightweight software development approaches (some would call them “best practices”) that evolved in the mid-1990s and early 2000s in reaction to heavy weight waterfall processes common in most organizations.

Common agile methodologies include Scrum, Extreme Programming, Adaptive Software Development, and Feature Driven Development.

Scrum, one of the most commonly adopted agile method, focuses on quickly delivering value to the customer by breaking up tasks into small units, called “Stories”, which developed in short time boxed iterations lasting from one to four weeks.

Teams are typically between 5-9 people with cross functional skills. Small teams help simplify collaboration and team communication. In some cases multiple small teams are formed when projects require a larger number of development resource to meet the customer goals.

Each iteration involves a team working through a full software development cycle, including planning, requirements analysis, design, coding, unit testing, and acceptance testing when a working product is demonstrated to stakeholders.

Using this approach, teams are able to reduce project risk by providing teams with frequent points for customer input along with focused planning sessions that helps the project to adapt to changes quickly. Stakeholders produce documentation as required.

These and other agile methods promote collaborative planning, teamwork and lightweight development and testing practices. All support one or more of the twelve principles, first articulated in the Agile Manifesto.

    • Customer satisfaction by rapid delivery of useful software
    • Sustainable development, able to maintain a constant pace
    • Close, daily co-operation between business people and developers
    • Face-to-face conversation is the best form of communication (co-location)
    • Projects are built around motivated individuals, who should be trusted
    • Continuous attention to technical excellence and good design
    • Simplicity- The art of maximizing the amount of work not done – is essential
    • Self-organizing teams
    • Regular adaptation to changing circumstances

Most promote development, teamwork, collaboration, and process adaptability throughout the life-cycle of the project.  Finally, all agile practices use “working software” as the primary measure of progress and value  face-to-face communication over detailed written documentation. And focus on working with the customer (sometimes called the “Product Owner”) to prioritize “needs” based on business value that can be achieved at the start of the iteration

Agile Manifesto, http://agilemanifesto.org/ Ambler, S.W. “Examining the Agile Manifesto”. Retrieved 12 December 2011.http://www.ambysoft.com/essays/agileManifesto.html
Beck, Kent; et al. (2001). “Principles behind the Agile Manifesto”. Agile Alliance. Archived from the original on 14 June 2010. Retrieved 6 June 2010.

Retrospectives What are they and how do they help teams adopt Agile practices?

Retrospectives – “What purpose do they server?”

Many small to medium size projects teams have successfully adopted agile practices such as Scrum but some agile teams do fail.  Often as not they fail because of factors outside their control but here are some common reasons why agile project teams fail that are within their control:

  • Not implementing the whole process
  • Scaling Factors not addressed
  • Team not co-located or team governed like a Waterfall project)
  • Team too big or team has strong functional role boundaries
  • Team / Management not committed to Agile principles
  • Roles not followed (Organization compromise on the Product Owner / Scrum Master responsibilities)

When we look at the Principles behind the Agile Manifesto we find that the first principle is “Our highest priority is to satisfy the customer through early and continuous delivery of valuable

So a failure in the context of an agile project could be stated as an “unsatisfied customer due to late or sporadic delivery of valueless software”.

But Agile provides teams with the opportunity inspects and adapt the process to make changes that help overcome these changes. In Agile a Retrospective is used to help teams improve their processes and practices. Similar to a postmortem performed atthe end of a traditional project, retrospectives allow teams to quickly identify things

  • They are doing well;
  • They should improve; and
  • Outside their control but are risks to the overall project

Here are some some practical tips and tricks for conducting an effective Retrospective along with some best practices for using Rational Team Concert to collect, manage and communicate the impact of the teams improvement activities.

Retrospective help teams improve and holding them at the end of each iteration has several advantages

  • Standing meeting focused on improving team performance (every 2 to 4 weeks) at the end of each iteration or sprint
  • Team prioritizes improvements and takes ownership over making changes
  • Team knows where they stand. Retrospectives occur right after Sprint Demo so they have the feedback from the Prod Owner
  • Quick-wins can easily be implemented rapidly where with traditional post-mortems happened at the end of the project with limited ability to make real changes.

Common issues that come up during team retrospectives include:

  • Story estimates are not acturate
  • Some members of the team are being pulled into other projects limiting their time
  • Team missed Sprint due to resource issues
    • Team had several tasks (e.g., DBA tasks) that only one person on the team can perform.
    • During the sprint that resource worked very long hours to complete his tasks and became frustrated that other team members could not do more to help with those tasks.
  • In another case say a team has was spending too much time dealing with failed builds. That team decided to implement continuous integration but the team does not want to risk disrupting their regular build process.

How are each Work items estimated and do these estimates relate to each other?

Written by Reedy Feggins, IBM

RTC 3.0 uses three different attributes to help teams determine which work items should be done during the sprint / iteration. These attributes are: Priority, Ranking and Complexity which are covered in detail in the article Customizing the Agile Planning tools in Rational Team Concert 2.0 provides a thorough discussion of the topic with screenshots from RTC 2.x but still remains valid in the RTC 3.x release.

Can you provide some guidance on the relative ranking when assigning this to stories?

The Level of effort estimate is captured using the complexity attribute for Stories where the Story Points are often used.  In some implementations Epics don’t have a complexity attribute while level of effort estimate for a Task is almost always captured in Hours, see figure below.

By default RTC complexity ranking for the Story work item uses Story Points.  In Agile, a Story Point is a unit less measure of time that teams use to estimate to estimate the work that the team will commit to in a sprint. Story points are NOT meant to be directly mapped to calendar time (days/hours) for estimation but instead are mean to help teams create better estimates, often using relative estimating techniques called Poker planning.

RTC Epic - Story - Task Estimates

What is the default to be used for the complexity ranking?  We have 1,2,3,5,8,13, 20, and 40,100.  Is it recommended to modify this?

Estimates using story points mean that if you estimate two stories; one story was estimated as a one and the other story as a value of five, then the second story is 5 times bigger than the first. If a third story is estimated as a ten then it is 10 times bigger than the value of one. Why? so that when you know your team’s velocity equals 10 and you select a 1,2,3 and 5 point story for a sprint, you know they actually add up to a sum velocity of 10.

Studies have found that most people can estimate smaller numbers better but as the number become larger people estimates become less and less accurate so the larger the estimate, the larger the error.

We should give the scrum master some guidance on this relative ranking.

While some teams use a doubling sequence (1,2,4,8…), most Agile teams start off with Story estimates based on the Fibonacci sequence (1,2,3,5,8,13…), which is the default RTC complexity attribute. This mathematical sequence is used for relative ranking. In this approach when estimating, teams are not allowed to pick a number that doesn’t exist in the sequence.[1]

§         http://blog.mountaingoatsoftware.com/tag/story-p

§         http://agile-commentary.blogspot.com/2009/02/create-proper-estimate-scale.html

§         http://loicbar.me/2010/10/24/why-should-you-use-story-points-instead-of-hours/

§         http://agilesoftwaredevelopment.com/blog/jackmilunsky/significance-story-points

[1] APA: Agile Commentary: Create a Proper Estimate Scale… (n.d.). Retrieved 02-20-2011 from http://agile-commentary.blogspot.com/2009/02/create-proper-estimate-scale.html

How are Epics, Stories and Tasks related to each other?

Written by Reedy Feggins, IBM

Work items are the fundamental mechanism in Rational Team Concert (RTC) used to coordinate, and track development tasks.[1] RTC provides several out-of-the-box work items types including Epic, Story, Task and Defect.  Each work item has its own workflow and associated set of attributes used to govern progress status, reporting and collaboration.  Work items are the hub for linkage between many Rational Team Concert artifacts (e.g. other work items, builds, work items, and change sets), as well as providing support for integration with other products.

While RTC allows teams to link work items in any manner they choose, to keep it simple in when the customer is using the SCRUM Process Template, several of my customer implementation, I have recommended the following work item hierarchy: Epic story->Story -> Task/defect, see figure below.

Epics, Story andTask - Typical RTC work item hiearchy

User stories are a good approach to capturing customer requirements without having to create formalized requirement documents or incurring the extra effort to maintaining them. The intention is to speed up the requirement definition process, respond faster to changing requirements and communicate requirements in the language of the customer using scenarios that can be easily validated.

By default the RTC Scrum process template, Epics Stories are defined as top-level-item while Tasks are not.  Each Story can be broken down into smaller tasks but it is not required. In addition we recommend that the RTC process template be updated to make the Defect a top-level-item.

RTC Epic Work Item - Screen Capture

Finally, RTC allows team to use the Work Item customization to create new work item types or modify existing ones in order to support the development process that your team follows. For additional information see Work item tracking on Jazz.net.

[1] APA: Work item tracking – Rational Team Concert – Projects – Jazz … (n.d.). Retrieved 02/20/2011 from http://jazz.net/projects/rational-team-concert/features/wi

What guidance would you give when for assigning priorities to stories in RTC?

What is the default priority generally assigned? It was suggested to me that the default should be medium. We may need to provide some guidance on what constitutes a high, medium versus a low priority ranking.

  • Yes we recommend using Medium as the default value for prioritizing work items. Priorities are usually assigned by the Product Owner or other key stakeholder
RTC - Separating a Larger Task into several smaller tasks

How can you assign a task in a story to more than one person? Sometime we need to assign a task to multiple people

  • In RTC, each work item (Story, Defect, Task,) can only be assigned to 1 person at a time so we often recommend splitting a larger task into multiple smaller tasks, each estimated between 4 – 16 hrs of work, with a clear success criterion and linking them as children to the original task.