I’ve been very busy for the last few months in my new job but will be presenting at a couple of conferences in the Summer. In the meantime, here’s a short thought to share…
When writing, people think in paper shape.
So why is it all our whiteboards are mounted in landscape?
Hang them up them portrait-style and panel them along a wall.
Just try it – portrait mounted whiteboards make a much better tool.
In my time working for a large multinational I led much of the design and layout work for redeveloping the office space at one of our sites to support agile teams. As part of this I reviewed and improved our available wall and whiteboard space.
As an experiment in one of the boardroom style rooms I arrange to have 3 large whiteboards mounted portrait style as 3 connected panels. They proved significantly more effective than normal boards and became some of the most heavily-used whiteboards in the building. As a result I replicated the same setup across as many other rooms as would accommodate them.
I can’t tell you why others liked them but from my own experience I found working in multiple portrait-format significantly easier than landscape. It allowed focus areas and footnotes in a way that a pair of normal whiteboards in the same space didn’t. This helped me arrange thoughts more cohesively with a natural information flow that landscape boards didn’t offer. They also worked as natural extra-large swim lanes during workshops and allowed multiple team members to work side-by-side on related aspects of a workshop.
It’s hard to remember all the benefits on a Friday afternoon. They just “felt” nicer, different, better.
Most of the really big successes and game-changers I’ve seen in my career have been initiated through individuals seeing a problem and fixing it themselves rather than waiting for others.
If you need something done that isn’t already happening, nobody else is going do it for you.
Here’s 4 lessons learned from adopting TDD…
It takes time & effort to get TDD into place
You will have to sacrifice “new code” time in the short term. From my experience, time is probably the biggest barrier to successfully introducing TDD on most teams (closely followed by motivation & accountability). Seeing the pay-off on the other side is notoriously hard to when you’re not there – either starting out or sitting at the bottom of the change curve.
If you’re coaching teams, sometimes you might just need to say “Trust me, I know what I’m doing”
“It can’t be unit-tested”
When you or someone else makes this statement they probably mean one or more of the following:
“I don’t believe I have the time to make this testable”
“Nobody has provided the infrastructure/harness/test data for me”
“I don’t know how to unit test this” (and need some help to figure it out)
“We have some architectural impediment to resolve in order to achieve this”
From these statements, which do you think is least common?
Break this cycle by encouraging accountability for making code testable.
This doesn’t just mean the product code it includes taking ownership of your testing tools and abilities as well.
If you’re struggling…
There’s 2 options for you; “abdicate” or “own”
Option 1 – Abdicate responsibility.
You’ll set the tone for your whole team. The next time someone hits the same problem, the cycle will repeat.
At best, someone better than you may eventually address the issue. (And I hope they make you feel mediocre or inferior).
At worst you may find them saying that you didn’t provide the solution for them.
The “best” case is highly unlikely to happen during your current project – maybe even the one after. Just think how much testing debt can build up over the life of a single project. We should be avoiding that, right?
Someone has to break the cycle. If you’re facing pain, why aren’t you responsible for fixing it?
Option 2 – Take ownership of the problem yourself
Great! You’ve recognized that your accountability is part of the problem.
Nobody else will do this stuff for you!
Establish a Personal strategy for implementing TDD – how are YOU going to solve the problem?
The teams and individuals I’ve seen that are successful with TDD have all got there by taking a personal responsibility to do so.
Just Do It
Here’s some suggested steps to get going – you can either do the first few alone or as part of a team.
(If there’s an accountability gap in your group you’ll probably need to start alone)
Round 1
1. Get a unit test harness and environment in place – there’s plenty online but write one yourself for starters if you have to!
2. If you have a database, break that dependency and channel it through something replaceable.
3. Write your first round of tests and see what hurts – now resolve those pains.
Limit the time spent on this first cycle and determine whether you should ask for time in advance or for forgiveness later – it’ll depend on your local context.
Round 2
4. Educate & support your team on what you’ve provided so far.
5. Get other team members to write their own round of unit tests (treat it “as an experiment” if there’s concerns over the effort required). See what hurts and resolve those pains.
6. Encourage and develop shared long-term ownership of the test harness, test data, stubs and mocks (if used).
Remember you’re setting the tone. If someone needs something that isn’t available, pair up, help them provide it and share the results with the team. Don’t leave them alone but don’t do it for them either.
Round 3
7. Repeat round 2 during development until writing tests is fast, easy and natural. (Don’t give up too soon – this is hard work and takes time!)
8. Capture any “wins” and “losses” during this cycle – what benefits are you seeing and what new pains are coming through?
9. Review the wins & losses. Decide what to do about the losses and how to promote the wins up the management chain and across the team. For example either publicise as you go or use as fuel for your retrospectives.
Think about the positive impact that little bit of extra effort and ownership has on you and your team in leading by example – doing a great job that others will want to share, not just a good job.
Abdication or accountability is a personal choice but it affects everyone around you. What are you doing to help your team?
If you’re losing capacity maintaining brittle code or infrastructure, the “litter patrol” in a related neighborhood may be good practice but how do you visualize and manage the true source of your pain?
This is based on a mix of my own experiences and great nugget of insight picked up from Jim Highsmith at Agile 2010 who in turn credited Israel Gatt.
First the paraphrased nugget from Jim…
“Teams need both debt reduction and debt prevention strategies.”
Here’s the quote from Israel back in December ’09.
“If your company relentlessly pursues growth, the quality/technical debt liability it is likely to incur could easily outweigh the benefits of growth. Consider the upside potential of growth vis-a-vis the downside of the resultant technical debt. When appropriate, monetize technical debt using the technique described in Technical Debt on Your Balance Sheet.”
Here’s my expansion to Israel’s article…
How do we know we have debt?
We “know” it’s there, we feel it. But do the right people know about it? You may be allocating permanent team capacity to keep bailing without seeing the hole.
Train all your teams in spotting technical debt
Develop and communicate your debt strategy to your teams and stakeholders.
Determine a common unit of currency (points, hours, money, NUTs) that your stakeholders can understand and engage with and use this to describe the transaction decisions you’re making.
Communicate current debt – “Balance”
This is obvious but hard to achieve. For the debt that hurts; quantify the cost and determine what you could do if it were resolved. Get that opportunity cost shared and get your solution sponsored. In most large companies if you can demonstrate a cost reduction or productivity improvement you can get support.
Communicate new debt – “Recent Transactions”
Not the decade-old pile in the corner! Focus on what you’re forced to add because of deadlines and market pressure. Every time you make a debt-related compromise, get that conversation exposed. Determine the maximum acceptable age for any debt being added, a cost (in your selected currency) and a priority. Whilst there are cases where a debt may never need to be repaid, chances are you’ll need to pay the next transaction off.
Limit debt – Set yourself a “Credit Limit”
What’s the maximum debt your team/project/product is willing and/or able to tolerate? Set a credit limit and stick to it. If we blow our limit, we’re in trouble. My personal rule-of-thumb is if it’ll take more than one whole sprint for the entire team to clear all new debt then you’re worryingly-leveraged and heading away from releasable software. This approach means a simple default credit limit for a new project (using story points as currency) is the same as your velocity – simple to calculate and remember. You could convert this into the average hours (and therefore financial impact) a story of that size takes if that means more to your leaders.
Whatever you do, don’t over-mortgage your product. You don’t want your product portfolio to collapse under unrecoverable bad debt.
Set a maximum age threshold – “Payment Terms”
Use topping & tailing approaches to control and ratchet your debt age to keep the span down. If your debt exists for more than N sprints (say 3) promote it to the top of the priority stack. Fixing new debt when it arrives is hard work and items do slip through. For those that escape, setting a maximum age means you’ll cycle all your debt through and keep it fresh. Even tough problems get resolved rather than rotting in the debt heap.
Visualize and report your debt – provide a “Debt Statement”
Put all debt visibly on your wall as cards or stickies, give it a different color, heading, whatever – make it visible. Now every sprint let’s highlight the numbers, the growth, reduction, reiterate your limit. If you’re using an electronic tool, try putting all the debt items under a single parent and tracking cumulative flow and burn-up of debt rolled up to that item.
Prioritize your debt – “Positions”
This is getting a little more advanced and difficult to manage – I’d reserve this for only projects that already have high debt where the simple strategies alone are too invasive.
Partition your debt into 3 positions. New:Short-term, New:Long-term, Old:All.
Set a different credit limit and payment terms for each of these and as part of prioritization set a ratio of effort/pay-off against each that ensures that at a minimum debt is sustained at a constant level but focus on cycling through so that the contents remain fresh.
Taking Action
Here are a few other points to explore – mostly around debt reduction.
• What “small wins” exist for you? Are there some simple debts than can be cleared quickly?
• If you sacrificed a team member for 1 sprint to pay off some short-term debt, would you be able to increase the overall team performance in later sprints?
• If a particular single debt item is too big for a team to swallow in a single sprint, can you call in or implement a parallel “SWAT Team” to fix it?
• What capacity do you need to reserve to sustain your current debt level?
• If you were going to reduce your debt burden by 10-20% what effort/capacity would that require and how long would it take?
• Can you demonstrate a return on investment for a particular piece of debt-reduction?
• Worst-case scenario: Could some debt control run “under the radar”? (And if this were discovered, what problems would be caused?)
Wrapping up:
Develop both debt prevention and reduction strategies for your teams, don’t just focus on reduction.
Treat technical debt like real personal or business debt: use “credit limits”, “statements”, “balance”, “transactions” & “payment terms” to your advantage.
To achieve effective reduction, work through the Oubliette strategies and in worst cases, consider a “SWAT Team”.
Some weeks ago I joined a new team. I don’t like causing disruption but I do like to get the pace of improvement going on my teams. Here’s how we started out together…
During my first week I observed my new team’s activities. They were trying hard and doing pretty well but hitting some hurdles. Their most obvious challenge was the impact of covering support activities and cleanup tasks in addition to new story delivery.
Other than the loss of expected delivery capacity, this unplanned work was causing problems during stand-ups. The flow and continuity of useful conversation was being lost as the team stopped to hunt around for the right coloured super-stickies, a working marker pen and figuring out where to put the tasks. (but… they were aiming to get these tasks visible!)
So…
Borrowing some 5S tricks from Lean, I started work…
Seiri (Sort); eliminate all unnecessary tools, parts, and instructions.
Seiton (Straighten); a place for everything and everything in its place.
Seiso (Shine); clean the workspace, and keep it clean.
Shitsuke (Sustain); don’t revert to old ways, continue looking for better.
I skipped Seiketsu (Standardize).
Each team I’m working with is establishing their own natural, effective way of working within a common framework. I don’t believe seeking further standardization is something that will help them right now. (Although I’ll be setting up a community of practice for sharing successful and failed ideas shortly).
We added a small new change to our board, stand-up and/or tracking every day over a week – generally this involved scissors, tape, markers and old cereal boxes!
We reviewed & cemented the successes and added a couple of bigger changes during our retrospective.
Here’s the current result. – I’ll post a photo in future (I’m writing this at home on the weekend).
Using the peripheral space for useful stuff
Clearing the decks, having specific right-sized places for consumables (colored sticky pads pens, story cards) and swim lanes for support items & cleanup tasks solved a number of problems quickly…
The support/new work split is clearly visible and we’re able to balance it better.
For new work, we’ve reduced the number of planned stories on the board to a minimum and have a clear priority order.
The team are now in the habit of making all new tasks visible on the board without interrupting the flow of the stand up – it’s not such an effort to add new tasks any more.
There’s always enough of each coloured sticky pad and some pens right by the board. If the holders are looking empty, one of us restocks.
Progress and status are clearly visible at a glance so we know if we have a problem and whether to adjust.
The team felt like things were under more control. Allowing them to focus on doing their best work and to try other improvement & collaboration activities.
When the team started their next round of sprint planning we had another 5S lesson – Seiri…
The great thing with this team is that all stakeholders are on-site and easily accessible. Having faced a painful hurdle on their previous sprint with a story that turned out to be significantly harder than expected, they discussed and adjusted priorities with their sponsor.
They pulled the part-completed story off the board, moved it right down the backlog and picked up something cheaper.
The team completely reset their board at the end of the sprint – even with work started or planned but unfinished – just to remove all the noise.
Interesting (but obvious) side-note – Perceived value is influenced by perceived cost. If something looks too expensive, previously less-valuable items become worth discussing and implementing.
IF the team were going to continue unfinished work, they would re-evaluate the remaining and completed tasks, understand why they didn’t finish, learn and re-plan appropriately rather than assuming what’s on the board was correct.
Longer-term, I’m expecting this approach to also encourage more flow/pull-scheduling of stories. I’m aiming for us to only pull new stories onto the board when there’s a to-do slot free.
Finally I stepped back for a while to let the team self-organize and see if they could sustain the changes (a trick from my friend Carl). About 80% of the changes have stuck with no support needed. Progress tracking and clearing the board down have needed some revisits. There were specific causes for these falling over but they’ve been recognised, addressed and are back on-track again.
In Part 2 I go into more depth on each of the areas of the board, what they do and most importantly why use them.
Here’s a great, simple tool for tuning and visualising WIP (Work In Progress) on your team.
Add a “Next” column between “Todo” and “In Progress” on your Scrum / Kanban board.
Team members only pull items into “In progress” from the “Next” column – and only when they actually start work on it, not when they plan to work on it.
Keep the “Next” column itself deliberately only wide enough for 1 task and replenish this with team input on priority/necessity when the “Next” column is empty.
This simple clarification has a huge impact on knowing who’s really doing what, when something has actually started and what we intend to do next.
For one of my teams I’m working with at the moment, this small tweak has had a knock-on impact of dramatically reducing the number of items in the WIP column to at most 1 per person (many tasks are covered through collaboration), per type of work (new features, debt/cleanup & support).
The team now use the full width (3 tasks wide) of the WIP column to indicate coarsely how “done” an individual task is (rather than filling the width of the column with potentially active tasks) which in turn has given us all much greater visual clarity on whether we’re actually going to reach our next goal – even when sat 10-15 feet from the board!
Here’s an example of what we have…
It’s really rewarding to see a team that instinctively “get” the value and benefit of simple visual management.
Give this a try with your teams and see how it works out.
I’m pretty sure next sprint we’ll only be pulling 1 story onto the board at a time – I’ll post about this in future.
Thanks to Adrian Wible for sharing this tip when I was in Nevada a couple of years ago. It’s now one of the most powerful simple tools I have at my disposal. I use it frequently with teams to rapidly size their backlogs and recalibrate their sizing.
This can be used with teams at any experience level but works with least pain if teams aren’t used to story points beforehand.
Teams new to story-points (or even working together) often get hung up on the numbers themselves and instinctively want to start converting to hours, ideal days, elapsed days or similar. The value of story points for teams during estimation sessions is in the relative sizing, not the actual size.
This approach keeps numbers entirely out of the mix until the end and allows participants to focus solely on the relative sizing of stories.
Once we’re over the relative sizing hurdle, adding numbers becomes a straightforward activity.
I’ve done this same activity with electronic tools but I’ll describe the manual version here…
Preparation:
Get your deck of incomplete user stories on readable cards or stickies.
If you also have a couple you have completed, these are useful triangulation points but not mandatory
In a room with your team, get yourself a large flat clear surface. I recommend tabletops in the middle of a space as you can swarm your whole team around all sides – this isn’t so simple with a wall.
Using string or sticky tape mark out 8 “swim-lanes” on the table (Requires 7 lines)
Your table should look something like this.
Don’t put any labels on the swim lanes
Activity:
Divide the deck of stories amongst your team and ask members to spend a maximum of 5 minutes silently placing their stories in the swim lanes – with stories of increasing size from left to right. E.g. smallest stories in the left-most lane.
You might want to lead the team through the first 1 or 2 random stories to give them some anchors or triangulation points to work from.
Don’t say anything about story points or numbers, if anyone asks just reiterate we’re looking for size relative only to each other.
If needed; guidance on complete unknown stories at this point should be to assume they are “very large”. The next round of sequencing will allow others to move them if they feel confident in doing so.
Once all stories are placed, you’ll probably have some “clumping” of stories, as in the table here. (assume X’s are story cards).
Give the team 5-10 minutes to again silently move any stories to different lanes where they feel their size relative to others makes them more closely aligned to those in another lane (stories must be wholly in a single lane – no overlaps).
Encourage team members to quickly look at and consider the location of each story on the table.
A team may occasionally find a deadlock where a story is “stuck” moving back & forth between lanes, you may intervene, pull the card out for discussion or allow the time boxing to encourage the card into a single location. Generally if there’s uncertainty, I choose the larger option but the last step of the process will resolve these.
Your table should now look something like the example here.
Once all the cards are placed and reviewed, ask the team to rate their satisfaction/confidence on the relative sizing and placement of the cards (use a scale of 1-5). For any team members indicating a low satisfaction level (1 or 2), ask them to describe their concern and for the team to decide upon how to act (either move a card or validate its current position). Time box this to 2 minutes per team member if possible. Use this opportunity to re-place any deadlocked stories – if a team really can’t decide, be cautious and go for the larger option.
Once relative sizing is agreed with a reasonable level of confidence for the whole team it’s time for a couple more questions.
For all the stories in the left-most column, do you think you’ll be working on anything smaller?
If the answer is “yes”, call this column 3 or 5; if the answer is “no”, call this column 1 or 2 (a single number from these options – use your judgement and instinct).
Now assign remaining numbers across the top of the columns using a subset of the common modified Fibonacci sequence used for story point estimation: (0, 0.5, 1, 2, 3, 5, 8, 13, 20, 40, 100, !/?)
This will give you a table of stories much like one of the following…
(I usually prefer the far-right column remains unsized to be broken down further to make it clear that this is a coarse exercise and cannot be used for commitments).
That’s it! You’ve just estimated & sized (using story points) your backlog in well under an hour.
It’s coarse, quick and dirty but chances are that even when you learn more, most of these won’t change relative to those in other columns .
More Depth:
The use of “silent sorting” is a powerful practice for agile teams. It removes debate and ensures focused thinking however in this example its use coupled with the lack of numbers is to avoid a common problem with inexperienced estimation teams during planning poker exercises known as “anchoring”. This is where an individual on a team will speak up with their view on what size a story should be before the remainder of the team have selected their view on size therefore influencing and anchoring all others estimates for the same story to a given point.
If you reuse this approach for additional rounds of backlog items I recommend preserving some of the older stories in the swim lanes to provide triangulation points for newer stories. Without these anchors your team’s velocity and estimation may see large shifts and greater unpredictability based on the shape of the new backlog.
Teams will quickly get used to which numbers match which lanes. This practice is still valuable however you may need to encourage teams to remain focused on relative sizes and not the numbers.
I generally hold the following views (and have found these actually do transfer well across teams)
5 is typically a “medium” sized story for a sprint
Anything greater than 8-13 requires further decomposition
Seriously consider also decomposing stories of size 8-13
Anything greater than 20 is in reality an unknown needing further investigation and decomposition before it can be sized sensibly.
These views can in themselves anchor others’ estimates towards some specific numbers. As a facilitator an awareness of the range of numbers used and their meanings is useful but be cautious that this information does not influence the outcome of the team’s process.
Update: March 2015 – 4 years after posting this article it’s still the second-most viewed page on the site. I’ve now added a second article offering some more theory to help explain how swim lanes and story points hang together with traditional estimation concepts.
It brought home what the last decade of international teams and ubiquitous business email access has achieved for many of us software professionals.
Since the late 1990’s I’ve worked in globally distributed or virtual teams. There’s a huge amount of positive things to be said about the working experiences I’ve had with these teams over the years.
My cultural boundaries have stretched
I’ve had opportunities to work with great people all over the world
I’ve learned a mountain of cool stuff
I’ve met hundreds of new friends
I’ve visited amazing places
and…
…
I’ve left my family at home most of the time.
Sadly small children and transcontinental business trips don’t really mix. My family have been incredibly tolerant and I always look to bring something back for them. They have a pretty tough ride but they support what I do and I appreciate their patience.
However…
With teams in the UK, India and the US there’s almost always a full working day of support, conversations and questions that happen outside the normal working timezone. As the pressure to deliver and support these teams has increased I find myself checking my work mail when I should be attending my family.
I recently reached the point where I was clearing my emails down during public holidays so that I could filter through and achieve something when I got in the following working day.
I check my email during breakfast at 6 or 7am and reply to things that came in during the US evening or India morning.
I commute to work and check my mail again to find another series of mails from India and a few UK early starters.
I check my mail when I get home from work to respond to anything urgent that came in during my commute.
I check my mail before bed in case there’s anything new that will derail my plans and priorities for the following day or that I can respond to before the US working day is over and 24 hours are lost on a decision.
(I also occasionally make time to write this blog, enjoy my family, study and maintain the house)
If I don’t clear my morning and evening international backlog my day job doesn’t have time & space to get done but this is all at the expense of other parts of life.
So how do I get things back under control?
WIP Limit vs Buffer Overrun
Here’s where Scrum, Lean & Kanban meet personal time management…
Set yourself a WIP limit. When that’s full, decide what doesn’t happen or has to be traded out. If you don’t make a decision, something will fall on the floor and chances are you’ll have a pile of half-done stuff. (a buffer overrun).
Build a visible backlog and keep it groomed. When new work comes in, prioritize and size it. (Take a look at the Covey Matrix as a powerful means of prioritizing). If I don’t have clear visibility to my backlog of work (not just my email inbox) then once again my mental buffer overruns and things fall on the floor.
This is where my problems are – relying on my mental buffer and inbox to be my primary and secondary backlogs!
Determine how big your backlog should be and whether it should be tiered (e.g. week, month, quarter). Just like a mature agile team, don’t build a backlog that’s bigger than your planning (or thinking and coping) horizon. If it’s important it’ll come back when you have the capacity.
Next, just like your agile projects, get your backlog visible. When new work comes in, take your stakeholders to the backlog and have a prioritization and trading outconversation.
Some things will have time deadlines and some of these you can’t avoid so what else has to give? If you have more items with time deadlines than you can cover, take your stakeholders back to your backlog and force the prioritization discussion again.
I recommend pipelining work into “emergencies“, “small“, and “not-small“. This is enough to provide an interesting mental mix but aim to limit multi-tasking to a maximum of one item in each area. (Clarke Ching has some fantastic insights and demonstrations on why multitasking is evil).
It’s also worth rewarding yourself. You’ll find items that fall in the “distraction” quadrant of the Covey matrix are often where some rewards lay hiding. Identify a few interesting, fun things and make sure they get some airtime in with all the priorities to pay off some of your priority fatigue.
Simple right?
OK, this won’t break the email addiction but it will help manage the personal backlog and priorities more effectively.
When it’s personal rather than a project this seems so much harder. With all that time teaching teams to trade out and prioritize, it’s time I started to eat my own dog-food.
Epilogue: This article has been waiting in my backlog for a couple of weeks to be rounded off before publishing. Yesterday the manager I’m pairing with planted a large kanban board by my desk with a list of the top priority management goals and activities that we have on our planning horizon right now down the left hand side and the associated tasks and states all prepped up! It’s not our entire backlog but it’s well more than we can achieve in the next 2 weeks and covers all the known top priority things.
Now it’s time to start managing the load properly again.
Every product team I’ve ever worked in had a bug blitz at some point and often one every year or two.
There’s no arguing that a decent bug blitz is a powerful way of getting the numbers down and clearing all the bugs in a good, solid drive feels good but the necessity for them is caused by a buildup from somewhere.
If you find you’re needing a bug blitz on every release, take a look at your defect and debt prevention activities and make sure you’ve got some topping & tailing practices going on.
Usually bug blitzes are performed at the end of a project but if you’ve not done so before, try having a 4-8 week blitz at the beginning instead. You’ll run quicker afterward and (if you keep things under control), you won’t have to worry about having time to mop up at the end.
Once you’ve got the numbers down, set your maximum defect threshold (or ratchet) at this level for the remainder of the project and keep this new lower level sustained throughout development.
What’s good about a bug blitz?
A blitz is particularly useful if you can focus on areas of the product you’ll be working with soon. It’ll get your team working together (particularly if they’re newly formed) and familiar with these areas before all the major work starts.
Couple this up with developing some decent automated tests in those areas as part of the defect fixing and you’ll be developing a much safer scaffold for your new work and reduce regression risks during your next release.
You could take things further and perform some refactoring but I suggest keeping different types of activities separate at this point and just stick to straight bugs. If you have a specific functional area needing a real overhaul, it doesn’t fit the bug blitz mold. I’ll cover this aspect in more depth when I talk about “sponsorship” for debt reduction.
I use bug blitz approaches when training up new staff. I review the defect backlog for a particularly grubby functional area and have a pair of staff take custody of it as caretakers. We pipeline the defects so that they can start with some simple introductory ones (usually low severity, noise or cosmetic stuff) and once we get confident in these we expand out into adjacent areas – it usually takes a month or two for them to really get warmed up.
The bug blitz is also a great opportunity to start new release development with a clean slate. It means no mixing types of work during feature development and gives you an opportunity to scrub the grime out and get a few new scaffolding tests in place.
What about the down-side?
First; they cost time and money. If you dedicate a team (or most of a team) to a blitz you’re not delivering anything new. This is obvious but important. What’s the impact of a month’s delay to your next release? (assuming you can ship with those bugs in there). And what will that delay do to your stakeholder relationships?
Be mindful of the impact a bug blitz can have on your customers. A high level of churn on existing functionality can be really dangerous. You might need to make a point of jumping through a few hoops to retain backwards compatibility.
Don’t be too hasty to refactor if customers are expecting certain things. If you don’t have decent automated regression tests, you really need to ensure you write at least a couple that hit the same area before you fix any defects. (I usually expect my developers to deliver at least 2 or 3 new automated tests with each bug fix).
Worse still, I’ve seen a batch of fixes in a functional area radically change behavior “for the better” according to the developers that raised the original internal defects who then discovered that customers were actually depending on existing behavior or had developed their business processes around the issues.
Sometimes, even if you think it’s ugly, it might be that way for a good reason.
With enterprise software, you’re often looking at heavily customized implementations, some of which have taken years and millions of dollars to evolve. Smacking these with a mountain of churn on existing functionality can be painful and expensive for your customers. Whilst it makes your numbers look good, consider whether some things should really be touched.
In summary. Bug blitzes are a great way of starting with a cleaner work area and ramping up teams but beware of backwards compatibility, customer impact, time and cost pitfalls.
Following my previous articles on topping and debt prevention, we’ll now focus on the easier parts. How to clear old defects and debt. we’ll cover 4 simple strategies; 2 for defects and 2 for debt. (there are paired similarities across these).
Unlike prevention and stop the bleeding activities which often require significant effort and commitment to accomplish, defect and debt removal have some simple and relatively low effort options. There are some high-effort/impact alternatives that we’ll cover later.
Today we’ll look at…
Tailing & Ratcheting
When you have a build up of defects over time you tend to have a “tail” of really old defects and issues. These are usually low or medium severity/priority items (with the occasional blip) and are often in functionally gray areas requiring difficult decisions or significant rework.
They age because we don’t want to touch them and would rather forget about them (the oubliette again). A typical defect age distribution looks something like this:
right-skewed beta distribution
It’s one of the most common statistical patterns I see in software development (and I’ll return to it in future) but for the purposes of this article, I want to look at the “tail” of this curve- that last 5-10% of your defect population – all your oldest items.
Addressing the tail is pretty straightforward. You can either set yourself a target “maximum defect age by a given date” or simply focus on continuous improvement. Whilst the target approach gives you a clear goal, you risk setting yourself up for failure or under-commitment. Defects are considered notoriously hard to size (I’ll cover this myth in future) but chances are if they’re old they’re probably a bit tricky too so being predictable about dates is something you might prefer to avoid for starters.
Whether you aim for a specific target or improving every day/week/month you’ll need the same stepped approach.
Identify your oldest defect, pick it off the end of the queue and commit to closing it quickly and fairly.
Here’s the first thing to accept… Closing doesn’t mean fix it in every case. Because you’ve not touched it for a while, chances are someone is expecting a solution so you’ll be looking at a difficult conversation if you don’t fix it but that might be far easier than “fixing” something that shouldn’t be changed or will derail your team and product. Make open, fair, honest decisions in each case.
If it is something you think you should be fixing, get on with it.
Aim to close at least one of your oldest defects every week or every sprint.
If you have a lot of items that are considered old, consider increasing your capacity on these in the short term to get the ball rolling. (at the cost of other delivery activities)
If we look back to your distribution of defects over time, when you do close out your oldest defects, put a ratchet mechanism in place that sets a continuously reducing maximum age. In very bad weeks the ratchet may not improve but don’t let the numbers get worse again or the effort and good faith from your teams and management will have been wasted.
With a ratchet, remember each “notch” is a step change from the last point. For example if your last point was 1000 days(!), clearing everything older than that should leave some defects nearly 1,000 days old. Set the next ratchet point to 950 days (rather than 997), determine what falls in that next block (950-999) and fix those. Ensure each ratchet point is a larger time window than your execution period otherwise you’ll end up stationary. (I usually go for a ratchet up of 50 days improvement per week or sprint).
Here’s what I mean…
Ratcheting out the oldest defects each week
3 months of this tailing practice every week will dramatically reduce the average and maximum age of issues in your queue. It’ll make you all feel better and it usually helps to draw the heat off on escalations for a while. (Bear in mind you should be doing this in addition to your “topping” approaches).
Occasionally you’ll reopen some old wounds this way but paying these some attention now stops them coming back up when the timing isn’t under your control.
Keep this up for six months or a year and you’ll reach a stable point where nothing gets too old and your team becomes adept at having difficult conversations with your customers early and backs them up with some successes as well.
Eventually in order to keep the ratcheting improving you’ll have to commit more people. This probably means you’re approaching your natural level of control or entitlement where you have aging defects levelling off and others being addressed in a reasonably consistent manner. It’s up to you whether you aggressively pursue the numbers down further or sustain them at this level.
In case this sounds too obvious or simple to work, I’m working with a team right now that introduced this ratcheting approach less than a month ago. We’ve reduced our oldest defect age by over 25% and customer escalations are consistently lower already. Perhaps most important to the team; we’re no longer at the top of the charts when our leaders review the defect stats. Although harsh-sounding. Sometimes, having someone else drawing the heat for a while lets us get back to focusing on value and priorities.
Great, I’ve got my incoming defect strategy nailed,
Now how do I prevent defects and debt in new code?
In 5 words…
Continuous attention to technical excellence.
Here’s my top 7 (there are plenty more)
Acceptance Criteria – Be really disciplined on your acceptance criteria & acceptance tests, team up with Analysts, Testers, Product Owners if you have them and attack your stories from every angle. A good approach to this is a “story kick-off” where the whole team dismantles a story before starting.
Thinking Time – don’t just start coding right away, task things out, try the 10 minute test plan, discuss your approach with your peers and for more complex or large items, try the “just enough design” approach.
TDD – It’s hard to start but has an immense impact. I’ve just seen a team complete their first project using TDD. 3 weeks into their final round of post feature-complete testing, their defect run-rate hasn’t had the testing spike seen on prior projects. In fact they’re keeping on top of all new incoming defects and have time to start paying down the historic backlog.
Pair Programming – Do it in half-day trial chunks if you don’t have the stomach for going full-tilt. I’ve performed remote pair-programming with colleagues across the Atlantic using decent phone headsets and online collaboration tools for hours at a time. The net result of 2 days of remote pairing was finding and fixing about 10 extra defects in a thousand lines of code that neither of us would have found coding alone.
Peer reviews – there is still a huge space for these in agile teams. But here’s the thing. Be really tough. A peer review is not a hurdle. It’s a shared learning exercise. Functional correctness is actually the smallest component of a peer review. You should trust your developers that far. But there’s a whole series of other aspects to review. See the joy of peer reviews.
Small tasks – I once worked with an outsourced team who when taking work would disappear into a hole for 2 weeks and return with a single task in our configuration management system containing edits to 200+ files and multiple condensed edits to the files. My rule of thumb is one reviewable task per activity. If you’re going to add new functionality and refactor, that’s 2 independent tasks that can be identified and reviewed separately. This means you should be able to easily deliver 2 reviewable, closable tasks per day.
Fast Builds – make it insanely simple for a developer to perform an incremental build that validates new code against the latest main code line. (small tasks are a big help here). This includes the right subset of unit and functional tests. Aim for a target of a 30 second response time or less between hitting the button and seeing the first results.
In the next article in this series I’ll focus on “Tailing” – How do you start reducing the old defects.
Your table should now look something like the example here.
