Agile antipattern: Sizing or estimating bug fixes
Is the bug to the left a large bug or a small bug? It looks HUGE to me! Well, in reality it is probably between .5 and .75 inches long. Not really a very big bug at all. Why do we care? Because trying to size the fixing of software “bugs” is at least as hard as figuring out how big this bug is!
When I teach an Agile or Scrum course someone will almost always ask a question like “How do you handle bug fixes in iterations or sprints?” When I ask “How do you want to handle them?” we get into a pretty interesting discussion. Most people say something similar to “We should prioritize them with the user stories, size them like we do user stories and then see what fits into each iteration.” I usually smile and ask any developers if they know ahead of time how long it will take to fix a defect. They ALWAYS say “Sometimes.” And THAT is the problem!
How can you actually determine the size of fixing something which is broken in an unknown way? I tell people in my classes I only know two sizes for defect fixes: 1) Trivial because I already know what’s broken and how to fix it, or 2) Infinite because I have no idea what’s broken or how to fix it! If those are the only two sizes available to us how can we possibly put them into iterations effectively?
I have found one effective solution to be the use of Kanban techniques for defect fixing. I don’t want to get into what Kanban is or isn’t and when it should or shouldn’t be used, so I’ll just lay out what I have seen be effective for a number of teams:
- Prioritize the defect list. This is NOT done in the context of user stories, but separately. The list is prioritized however the Product Owner says it should be prioritized.
- The team and Product Owner decide on how much effort (time) should be used each iteration to work on defects. Hopefully this is not a large amount, but it might be for teams which have large numbers of defects in a legacy system.
- The team determines when the defect fixing time occurs and how they do it. Most effective is to put a gate or two in place on the defects. For example, gate 1 may say the developer needs to know within 2 hours if the defect is going to take more than a day to fix. If so, then put it off until a discussion can take place with the Product Owner. Gate 2 may be after a day if the defect is not fixed perhaps another discussion needs to take place. However the gates are set up (if they are) the defects are worked in priority order.
- Limit the number of bug fixes being worked at one time to a very small number. If you don’t do this you will have each developer working on at least one defect and run the serious risk of none of them getting fixed before the iteration or sprint ends!
This 3 step approach allows the team to work on defects in priority order while allowing a set amount of time to be spent on the defects. The amount of time spent can be changed as needed to address the business needs of the organization at any point in time.
The downside of this is no one can tell a stakeholder something like “that bug will be fixed by date X” or “we’ll knock out X bugs this iteration.” Saying anything like that is a lie anyway, so this shouldn’t be a big issue. I say these statements are lies under the assumption the defects are non-trivial.
How else have you managed a defect backlog that has been effective? I’d love to have more proven techniques for people to experiment with!
Until next time my clients will be Making Agile a Reality® by using sizing only when appropriate!
[…] This post was mentioned on Twitter by Bob Marshall, Bob Hartman. Bob Hartman said: New #agile blog post: Agile antipattern: Sizing or estimating bug fixes http://bit.ly/d23Ij6 #li […]
We always try to fix bugs asap. We usually put them on top of our backlog. As soon as a developer has finished his previous work he starts working on the bug. That way we came to have a system without any known bugs. If one shows up, it get’s fixed in the current iteration. This way, fixing bugs simply reduces our velocity. That makes it very clear to everyone how important it is to build in quality from the beginning.
Matthias, putting new bugs on the backlog is a good practice, but I’m not sure about always putting them first. Some bugs may not be worth fixing.
Also, the original post was mostly from the perspective of a legacy system with an existing “bug database” containing hundreds or in some cases, thousands of bugs. It represents a large mountain which will not easily be changed. I should have made that clearer.
– Bob –
Matthias, what about the bugs that are determined to be hard to fix but not so important? Do those really go at the top of the backlog?
I guess what I’m saying is: Is being bug-free always a higher priority than your other tasks?
Mark, being bug-free is very important to us, but there are situations, where other things are more important. But eventually we try to deal even with the small but hard to fix bugs as they usually point to a area of code, which urgently needs some refactoring (otherwise a bug should not be hard to fix 😉 )
Bob, I once lived with such a legacy application having a bug list of close to 1200 bugs. As harsh as it may sound, if a bug (or feature request for that matter) is so unimportant, that it can live happily in such a long list for years you’d better delete it right away. We spent so much time going through that huge list again and again (and again) without really changing anything – eventually we just deleted the last 1000 items on it. Man, was that a great feeling!
What I try to say here is: If it’s not worth fixing, don’t even record it. Why bother? If it’s hard to fix (but worth fixing) don’t push it around for too long as it points you to some area in your code, which needs your attention (it “smells”).
Oh, I really disagree with that. Maybe it’s because of my experience (quite a long time ago) with a project manager who kept redefining bugs out of existence if he thought they weren’t going to get fixed in the next release. So we kept “finding” bugs and having them flagged as “not a bug” or just deleted. Thus the illusion of bug-free releases when in fact there were many known and unacknowledged.
Call me old school. There is no shame in saying “Here’s your release, it has these bugs, we (you) decided it was okay to tolerate those bugs for now.” Embrace imperfection.
It really depends, what purpose your bug list has: If you need it as a tool for avoiding re-work (“finding” bugs again and again) or to manage expectations (“our release has these bugs…”)) then it makes sense to track all bugs.
But if you start re-prioritizing that list again and again it is really harmful. Maybe split it: Have only the bugs you need to fix on your backlog and have a separate list with known bugs.
Yes! That’s a good solution. I agree that you don’t want to spend time re-prioritizing the list of dormant bugs.
We record all issues/bugs/problems — call them what you want. However, once recorded — never deleted, and for good reason (later). We use state values including Deferred/Approved (we think we should fix it, just not now); Deferred/No-Fix-Planned (not worth the time/cost/effort to fix); Rejected/Approved (with a reason). By keeping these, we have proof (to ourselves) that we have seen an issue and made a decision about how to treat it. It also keeps folks from recording the same thing over-and-over again. Any issue can be reinstated later if it turns out to be bothersome to a customer or the product owner changes his mind. Note that our general policy is to fix what we can during the project in which they are found, but as you all know, that isn’t always possible, and so we use the stated mentioned above. Once in a while, we go thru the Deferred/Approved list and change some to Deferred/No-Fix-Planned, because it has become apparent that there is no need to fix.
[…] a programmer Are You Smart or Dumb? What Color is your Backlog? Backlog Grooming: Who, When and How Agile antipattern: Sizing or estimating bug fixes Learning About Security Vulnerabilities by Hacking Google’s Jarlsberg Sustainable Test-Driven […]