Back in December I moved to a new team. After 18 months on DevOps and email marketing projects I’m back to managing commercial product development. After a week of getting up to speed on the product and project (it was already in flight when I joined) I started looking at the bug backlog in Jira.
One problem with bug backlog tools is that they allow you to create bug backlogs. I considered shutting them all down but knew we’d find value in being more considered in our approach.
For this project I was pairing with another project manager; Dom Smith (yes, pairing isn’t just for developers). The project was a major new release for one of our longstanding successful tools.
This is the second time I’ve pair-led a project and every time the overall outcome and workload is so much better. One day the rest of the industry will realised that having a single PM running multiple projects will always get a suboptimal outcome. In fact another critical project (fixed time, fixed quality, fixed schedule, fixed scope!) elsewhere in the building is also being pair-led right now and is proving to be a great success.
Dom was a technical author in a previous life and is therefore rather good at keeping track of everything a team does over the life of a project. Another team were interested in what we’d done so fortunately Dom managed to capture everything we did to get the oubliette under control. Today’s post is brought to you courtesy of his work.
At the start of our review we had ~1200 open issues, of which about 400 targeted for the “next version”. The backlog had reached a state where nobody wanted to look at it as it was “too big” to resolve.
Fortunately strategies for breaking down big problems is something I tend to specialise in.
If I’d said at the start we were going to spend 4 months doing this, I think we’d have struggled. Looking back, we spent about 50-60 hours of our time and about 100 hours from the development team to get things under control.
We started with just one hour’s “space” and simply said. “OK, what can we achieve in 10 minutes”. Fortunately bug backlogs have many “seams” to work through. We worked through the smallest easiest first and whenever things slowed down or value started to diminish, we’d move to a fresh seam and started digging again.
We started by setting up an hourly slot once a week to pair up and pull the things we were going to fix into the current version. In the process of this, we found many duplicates we could close. We also ensured that rough priorities (priorities would be clarified later) were set on everything we pulled through, and that they all had components (the functional or technical product area) set.
After a few weeks repeating step one, we started tackling the other end of the list: things that were out-of-date or really not going to get done.
Anything more than 3 years old was checked on a case-by-case basis with the starting assumption that it would be closed as ‘won’t fix’ unless we could argue a reason not to (primarily based on prevalence and customer impact). Pairing made these a sensible conversation rather than a hasty decision.
“Autobugs” – we have an automated bug reporting system so that when users hit an exception they can choose to report it directly into Jira with no additional effort.
Any autobug raised prior to the last major shipped release that was not seen in the latest live release. We were working on v8.0 so any bugs reported against v6.x with no reports in v7.x, were closed
Any autobug raised only by one our development team on an unreleased build over 6 months old were closed.
After cleaning up the “noise”, we added a weekly, 45-minute meeting with the whole development team to review all the new issues added in the last 7 days, set a clear and appropriate priority, ensured component information was present, and set an explicit release target. (Either “current” (v8.0), “Next Minor” or “Next Major”. This helped contain the quality of new bugs – we “stopped the bleeding“.
We adapted our weekly pairing slot, to tackle the issues assigned to people no longer on the team (the aim being to ensure that they ended up with no issues assigned to them). Many of these could be closed as No Longer Valid, but there were a few which should have been prioritized that they were being sat on.
Again during the weekly pairing, we started working on closing bugs targeted at “improbably releases”. We had placeholder releases called ‘Future – Possible’ and ‘Future – Unlikely’ which, by their very nature, aren’t likely to happen. Again, we ruthlessly reviewed each in turn. Many had already have been fixed, and just needed 5 minutes testing to verifying they were resolved.
We added a further hourly weekly session with our product manager prioritize any issues we’d found through this process that looked more important but needed a product management decision. Having set the impacted component on everything we weren’t closing we’d found clusters of problem-areas that had been previously hidden, that became candidate focus for our next minor release (due to ship imminently). We also checked any queries coming from the development team’s review of issues from the last 7 days, and the scope of work for the current release to determine ‘Must Fix’ vs ‘High’ vs ‘Would like to fix’.
In parallel to these activities, we worked with our Sales team to ensure CRM references were added to any feature requests in JIRA (both new requests and existing ones that were being asked for repeatedly). This helps our UX specialists and product managers with prioritizing these in the future and gives them direct customer links to follow up on actual needs and problems.
In the space of 4 months we closed over 50% of the backlog issues by not doing any development work – just investing in three weekly meetings. We fixed another 25% by setting clear priorities, targets and sensible grouping of issues.
We aren’t there yet though. Bugs are still coming in and there’s still two big ugly piles in “Next Minor” and “Next Major” that we have yet to tackle but we’ll continue to put a little effort in on a regular basis to keep improving our position. Better still, the last release we shipped has significantly less bugs and the one we’re about to ship has less again.
We’re digging our way out!