Agile antipattern: Burndown charts that hide the truth
See that burndown chart over there to the left? It looks beautiful doesn’t it? It is an actual burndown chart with no made up data. It looks like this team is kicking butt and having a great sprint. Unfortunately, the chart lies! It turns out this team is actually in difficulty and in fact are unlikely to make their sprint commitment. Are you confused yet? I know when I do this exercise during agile courses the course attendees all look at me like I’m crazy. I assure you, I am not crazy. This team is in trouble, it just doesn’t show up in this chart!
The problem is hidden from view. Notice the vertical axis has no units associated with it. This is the problem! It turns out this team is burning down task hours. Of course they go down every day, and they even do it reasonably predictably. However, there is NO correlation between task hours and the amount of completed users stories!!! This is an important distinction which is often glossed over in agile training. I like to say if you measure completed hours you will get completed hours, but that is no measure of completed stories or delivered value. Instead you MUST measure completed user stories in order to be assured of delivering value. From experience I can tell you the hours left unfinished on the burndown chart in the example are almost all testing hours (consider your own experience). In other words the team pushed testing to the end of the iteration, which is more or less guaranteed to fail. Most teams in this situation forget to account for rework hours once testing finds defects and the problems get worse.
On the other hand, the chart to the left of this paragraph not only looks nice, it really is nice! Notice the vertical axis is now labeled with “Story Points.” In other words the team has completed a significant number of story points and they look like they will finish their commitment based on the current trend. In fact, this team might even finish early! The only difference between these two charts is the legend on the left side. I tricked you with the first one because I didn’t give units at all. If I told you it was hours it may have led you toward the right answer a bit faster. However, I wanted to point out how charts can be made to lie VERY easily.
In agile, you WILL get what you measure, so make sure you measure the right things. One of the 3 Scrum artifacts is the burndown chart (the other two are the prioritized product backlog and the sprint backlog). Make sure the burndown chart is a valid artifact by measuring completed story points – oh, and NO PARTIAL CREDIT. Just because the coding is done does not mean half the story points are completed!!! Otherwise you are no better off than measuring hours. Measure success in order to achieve success.
Until next time work on Making Agile a Reality® by measuring properly! In particular be very suspicious of any metric which tends to look good all the time, yet the team doesn’t get the anticipated results.