Agile antipattern: Burndown charts that hide the truth
See that burndown chart over there to the left? It looks beautiful doesn’t it? It is an actual burndown chart with no made up data. It looks like this team is kicking butt and having a great sprint. Unfortunately, the chart lies! It turns out this team is actually in difficulty and in fact are unlikely to make their sprint commitment. Are you confused yet? I know when I do this exercise during agile courses the course attendees all look at me like I’m crazy. I assure you, I am not crazy. This team is in trouble, it just doesn’t show up in this chart!
The problem is hidden from view. Notice the vertical axis has no units associated with it. This is the problem! It turns out this team is burning down task hours. Of course they go down every day, and they even do it reasonably predictably. However, there is NO correlation between task hours and the amount of completed users stories!!! This is an important distinction which is often glossed over in agile training. I like to say if you measure completed hours you will get completed hours, but that is no measure of completed stories or delivered value. Instead you MUST measure completed user stories in order to be assured of delivering value. From experience I can tell you the hours left unfinished on the burndown chart in the example are almost all testing hours (consider your own experience). In other words the team pushed testing to the end of the iteration, which is more or less guaranteed to fail. Most teams in this situation forget to account for rework hours once testing finds defects and the problems get worse.
On the other hand, the chart to the left of this paragraph not only looks nice, it really is nice! Notice the vertical axis is now labeled with “Story Points.” In other words the team has completed a significant number of story points and they look like they will finish their commitment based on the current trend. In fact, this team might even finish early! The only difference between these two charts is the legend on the left side. I tricked you with the first one because I didn’t give units at all. If I told you it was hours it may have led you toward the right answer a bit faster. However, I wanted to point out how charts can be made to lie VERY easily.
In agile, you WILL get what you measure, so make sure you measure the right things. One of the 3 Scrum artifacts is the burndown chart (the other two are the prioritized product backlog and the sprint backlog). Make sure the burndown chart is a valid artifact by measuring completed story points – oh, and NO PARTIAL CREDIT. Just because the coding is done does not mean half the story points are completed!!! Otherwise you are no better off than measuring hours. Measure success in order to achieve success.
Until next time work on Making Agile a Reality® by measuring properly! In particular be very suspicious of any metric which tends to look good all the time, yet the team doesn’t get the anticipated results.
[…] This post was mentioned on Twitter by Bob Hartman, Agile Carnival. Agile Carnival said: Agile Bob – Agile antipattern: Burndown charts that hide the truth: See that burndown chart over there to the left?… http://bit.ly/6h8kIq […]
Social comments and analytics for this post…
This post was mentioned on Twitter by AgileForAll: I just created a new #agile blog post: : Agile antipattern: Burndown charts that hide the truth http://bit.ly/8mSLwU…
Excellent post! This is exactly why I tell teams that I coach to focus on stories and not tasks. They may be completing tasks like there’s no tomorrow, but still not finishing stories (and thus delivering value).
I also tend not to use sprint burndown charts at all. Depending on the team’s preference, I use either a kanban-style board for the stories, or just the simple “4-dot” status on the story cards – Development Started, Development Complete, QA Complete, Accepted by Business (i.e. Done, Done).
Again, the tasks are secondary and just used to identify what needs to be done to deliver the stories.
Dave Rooney
The Agile Consortium
I’d like to agree to the general idea of the post, but isn’t the issue mostly caused by looking at events that already happened?
Tracking the time that the team already spent on a project doesn’t say anything about the grade of completion, the article got that right. The second chart uses story points instead of hours as unit for the vertical axis. The effect is as the article describes, the burndown chart now shows the amount of completed work instead of amount of time spent.
However I’d argue that the major benefit of the second chart is that it also shows the amount of work that is left which the first chart does not.
While the difference seems to be minor at first glance in my experience it is a critical shift in the way people have to think. I as a ScrumMaster do not care the least bit about how much time the team spends on User Story #4711. That information might be interesting for billing the client and for the sprint retrospective. All I care for in terms of “how well are we doing” is how much work is left vs. how much time is left.
The past is irrelevant. Just to be clear, the second chart in this article shows that, only the article text is not very clear about this difference.
my 2 cents,
Dominique
—
http://www.st-webdevelopment.com
Dominique, I think the point you raise is valid but so is mine. It is part of why I wrote a later post “Agile antipattern: Another burndown chart that lies” Using a burnUP chart shows more information. A burnUP chart for hours is more useful than a burndown because it COULD show the amount of hours left to complete (probably an upwardly trending line). I still prefer to measure story points rather than hours (what have you finished creating vs. what have you worked on) and I prefer to do it with a burnup. Cumulative flow charts (borrowed from lean/Kanban) can also be useful but are harder to automatically create. All of the burndown charts in these blog posts have been created with a simple Excel spreadsheet.
Hi Bob,
thanks for responding.
I didn’t mean to say that your point is not valid, sorry if it didn’t come across that way. Your second chart is telling the truth perfectly well. My argument is more towards what I read from it vs. what I don’t read from the first. In other words, personally I find the future (points / hours left) more relevant than the past (points / hours already ‘done’).
Hope that clarifies a bit. Anyways, great post and yes, I will read that other article of yours :).
Have a happy new year!
Dominique
—
http://www.st-webdevelopment.com
Very interesting post!
I never thought about the danger of tracking “remaining hours” and you are right, we could get to the end of the iteration with just 5h remaining, but still we wouldn’t have delivered any business value.
Perhaps this hasn’t happened to my teams yet because we inherently try to focus on delivering the stories and not just tasks.
Very useful article. Thank you, Bob