Escaping the Oubliette (Part 1) – Topping

Reading time ~ 2 minutes

As promised, how to Escape The Oubliette

So here you are, part of a team at the bottom of the defect & debt pit. What now?

The simple fact is, there’s no one answer but there are a selection of tools and there is a key.

The key comes courtesy of a comment from Jim Highsmith during one of his sessions at Agile 2010 – my paraphrasing below…

“Teams facing technical debt need both debt reduction and debt prevention strategies.”

When teams face technical debt they pick away, removing it piecemeal. After all, you have to eat an elephant one bite at a time right?

Trouble is, that only works if the elephant isn’t growing as fast as you’re chewing.

You need a two-pronged attack – what I like to call “topping and tailing”. The “top” is all the incoming issues and build-up of new debt. This is where you need to stop the bleeding. The “tail” is the backlog of existing debt. It’s already there – gently rotting in the corner of your product.

Topping

If you’ve read my post on stopping the bleeding, this is the crux of “topping”. You may have a variety of strategies for this but here’s the fundamentals.

Incoming Customer Defects

Ringfence enough capacity to address defects as soon as they come in. Make sure your burn rate matches your incoming rate. If you’re strapped, it’s your call whether you only cover high severity issues but for my money, I’d hit them all. A multi-year build-up of low severity issues makes products ugly, no pleasure for the users and a series of minor problems rapidly becomes a major burden and a big debt buildup.

Addressing every incoming defect doesn’t mean fixing every one. Whilst I generally recommend you do, when trying to dig yourself out of an oubliette the bigger issue is how to keep your goal achievable.

Be critical. The big, nasty bugs are usually pretty obvious but what about the rest?

    • Is it low severity and not important?
      • Is it quick & cheap to fix? – Just fix it.
      • If it’s not – Can you put a safety rail around it? Document it as a limitation or even ignore it?

Negotiate with your customers and make an explicit decision to fix now or never. Once decisions are made, communicated and agreed, close the defects in your backlog.

Getting these conversations happening immediately makes customers far happier than pretending they’ll just go away and never making a decision. (but not quite as happy as fixing their issues).

Incoming Internal Defects

The story here is the same. I’ve called it out independently as many companies and teams do.

Those incoming internal defects speak volumes about your approach to quality.

Get the discipline in place to fix them when you find them or be brutal and admit which ones will never be fixed but don’t pussyfoot around!

  • If you provide a service to other internal teams, treat them as customer defects.
  • If they’re raised by your own team, are they on new code or exposing existing issues?
    • On new code, fix them! – No excuses .
    • On existing code, the problems are already there. You need to decide if what you’ve done has made it more visible or harder to work with than before and make a judgement call. Fix now or never, resolve or close.

In the next article in this series I’ll cover debt prevention

The Joy of Peer Reviews (Part 1 – Code)

Reading time ~ 2 minutes

Pair programming replacing peer reviews is a myth in the same way that “agile projects have no documentation”.

From my experience peer reviews continue to hold a vital place in agile development and software craftsmanship. Unfortunately they are often misunderstood or misapplied.

“Humanizing Peer Reviews” by Karl Wiegers is the best primer I’ve read so far on peer reviews so I’m not going to duplicate Karl’s efforts – I strongly recommend a thorough read. In fact, print a copy and give one to every member of your delivery team to read and discuss.

Like everything else in modern software development, peer reviews are a collaborative team learning experience. Reviewing code properly means both the reviewer and reviewee walk away having learned something and improved their craft.

With code reviews; beyond reviewing for functional correctness, (the simplest, most obvious and potentially quickest part of a review) there’s a selection of considerations I expect reviewers to look for. (There are plenty more).

  1. No code without tests
  2. Good variable naming
  3. Correct use of classes and interfaces
  4. Small methods
  5. Adherence to standards and conventions
  6. Style consistency across the team
  7. Readability
  8. Good test coverage from small tests
  9. Test code follows the same quality standards as production code
  10. Tests describe expected behaviours
  11. All tests pass
  12. Evidence of some test preparation to consider boundary cases, failure modes and exceptions
  13. Clear exception handing, failure modes and explicit boundaries
  14. Sensible error messages
  15. Code smells
  16. Performance risks or tuning opportunities
  17. Security or other “ility” issues.
  18. Opportunities to learn new tricks
  19. New good practices & patterns
  20. Functional correctness

Once you have cultural acceptance on the breadth of technical peer reviews develop your own checklist that everyone supports and remember the goal of a review is to share improvement opportunities, not for lazy coders to have someone else find their bugs for them or for staff to step on each other.

Now…

If you think peer review of code is problematic for your teams, I guarantee that document peer reviews will be a whole lot worse! – see part 2.

Stop Working With Blunt Tools

Reading time ~ 2 minutes

Clarke Ching introduced me to a story of 2 woodcutters – one worked furiously but finished late whilst another stopped frequently to sharpen his tools and finished early.  (He tells it better than I do) – I think it’s actually based on an Abraham Lincoln quote:

“If I had eight hours to chop down a tree, I’d spend six hours sharpening my ax.”

Let’s think more about developing software the Lincoln way

What tool sharpening should you do before you start chopping?

If you paid off or prevented some of the debt you were facing before new work started, would it enable your project to run faster, more smoothly, with reduced risk, a lower chance of defects or a lower maintenance cost?

A previous employer had a great strategy. Every new release of the product had 2 top priority named features on the priority list for “cleaning up” and “levelling up”.

Cleaning up:

  1. Get all unit and regression tests passing (and keep them there).
  2. Address all build failures and warnings (and keep them under control).
  3. Delete all functionality, code and tests that will have been deprecated for more than 3 releases. (and add alerts to functionality that will be removed in the next release)
  4. Fix all defects that put us below releasable quality before we’ve even started (and keep them there).

Levelling up:

  1. Raise the tools we use to those that are best supported, newest in the market or offer improvements to our working conditions.
  2. Raise the libraries we use to the latest supported versions and address any issues.
  3. Raise the platform versions we’ll support when the product ships and address any issues. (and remove support for obsolete or out of date platforms).

No major release started full-tilt on functional work until these were cleared.

Like all good practices, this isn’t new thinking, it correlates to 3 components of the lean 5S strategy; sort (seiri), straighten (seiton) and shine (seiso). Rather than just describing what was done, here’s some tangible benefits for the clean up & level up approach…

  1. There were no unpleasant surprises for our customers on a new release. We had a standard platform, support and deprecation policy and kept to it. Our customers liked it when we did predictable things.
  2. For the development teams levelling up was a common risk. Addressing this at the beginning of a project was a valuable de-risking activity. Where we hit critical problems, we could make clear, early upgrade decisions and where there were fewer issues we would develop full-time on updated versions throughout the project with regression on older supported platforms available from prior development cycles.
  3. Removing old parts of the product made life significantly easier for the teams. The reduced testing, regression and maintenance load allowed us to speed up development – much like scraping barnacles off the hull of a boat to help it run faster. Cleaning up also allowed us to take some sensible baseline code metrics before any new work started.
  4. Giving teams space and time to clean and level up before starting functional work had a positive impact on morale. We felt that we were trusted to “do the right thing” rather than “just ship it”. This empowered us to continue doing the right thing throughout the rest of our work.

Give your teams some time out to sharpen their tools and sort, straighten & shine the workshop before new work starts. It will make a difference to the performance of your team and the quality of the end result.

Learning to Say “No”

Reading time ~ 2 minutes

The great thing about working with software development teams is that on the whole they’re a nice bunch of people who just want to get on and do the right thing.

The tricky thing in large-scale enterprise product development is that there’s often aggressive targets pushed through sales & marketing onto product managers.

A common response is to take a shotgun to the enhancement request backlog and ask for everything from the development teams and negotiate down from that point.

Craig Larman & Bas Vodde summarized this brilliantly using game theory in the second of their scaling agility books as the “your fault” game.

Assuming like many large companies you’re faced with the challenge of an annual commitment-based project/backlog and you’ve whittled that down to the best you can do; chances are your product manager will come back at some point through the year having visited a customer or partner and, like Columbo, ask for “just one more thing”.

Agile development is meant to give us the flexibility to do this right?

The typical nice guy response is “well… it’s going to hurt … but we’ll try”…

Great! When your team gets kicked around the room at the end of the year for failing to meet their release commitments, I’m pretty sure I know whose “fault” it’ll be.

FrAgile

So… Time to dig yourselves out of that abusive relationship, get a backbone and learn to say “No”.

OK. Maybe that’s a bit harsh. How about…

“Sure we’d absolutely love to do that but here’s everything we’ve already committed to for you. What would you like us to drop first?”

That still sounds pretty collaborative – but now it’s on your terms.

You’ve also just highlighted that being Agile doesn’t mean not making commitments if that’s what your business needs, it means managing your customers better.

If the previous “we’ll try” conversation is the norm for your teams, chances are this is the first time your product manager has ever been put in this situation. Don’t expect it to be pretty or easy. You’ll need to stand your ground and limit the options available.

I’m a fan of visual management so here’s my take.

  1. Take all the features you’ve committed to already and put them on cards or stickies.
  2. Draw a box around them so that there’s no room for anything else.
  3. Bring your product manager to the table, hand them the new feature or item they want on a card of similar size to an equivalent feature.
  4. Ask them to trade out something so that the card fits.
  5. Once the trade-out decision is made. Confirm any timing impact on the other features.
  6. Optional… (but recommended If you’re in a low-trust environment) Get the decision in writing recorded in whatever your system of record is.

If you can’t get your product manager in the room, do the same thing with excel and coloured blocks representing each feature in a pipeline. Adding a new feature in anywhere in that pipeline extends the end date. If you’re on a fixed-date commitment that means something falls off the end.

Simple?

Give this a try or adapt something similar for your own needs.

Captain, You’re Wrong!

Reading time ~ < 1 minute

With limited time and bandwidth, leading a large global development team on a large complex project requires rapid decision-making on difficult problems.

Some years ago my team found the cost of delay on a wrong decision corrected later was generally far lower than the delay of holding out to be sure of the right one.

Many times I had to look at what little information we had and either make (or support the team in making) an arbitrary judgement call.

The reason this approach worked for us was that we were all willing to step up as soon as we knew were wrong (usually in light of new, previously unknown information) and call each other out. In fact I took it as a matter of personal pride that I could be wrong so often and yet my team could still outperform those around us.  In that team it was socially acceptable to be fallible and we encouraged less-experienced staff to challenge their more experienced counterparts.

The phrase “Captain, you’re wrong” is still music to my ears. It usually meant someone on the team was right!

Try this: When you’re stuck making a decision, choose the one that seems least wrong. You’re an experienced professional, sometimes your instinct will be all you have left to work with. Make it clear to your teams that if they find a decision isn’t working for them they can revisit it – as long as we get on with it now with no debating.

Red Cards

Reading time ~ 3 minutes

One of the best facilitation tools I own. How to get a group out of a rat-hole & back on track without personal confrontation and minimal effort.

Name: The Red Card

Concept: When a group is in discussion on a particular topic they can often disappear down “rat holes” or off onto tangents. Every member of an agile team is empowered to “red card” a conversation that they feel is going off track. The group as a whole typically rapidly decide whether the red card is warranted or not.

Usage: I ensure that plenty of of small (playing card sized) red cards are available in the team rooms. To introduce them to a team that haven’t used them before, I will usually take a large session such as release planning and introduce the concept of red cards as part of the facilitation tools and ground rules at the start of a session. What I tell the teams is:

“Whilst I’m facilitating, I tend to get drawn into the conversations and need hauling out, especially if I start ranting. Therefore the red cards are required primarily to shut me up – although feel free to use them on each other too!”

Once a member of the team first uses a red card, that’s it – the lid is off. Expect use of cards to take off rapidly. (see “breaking the seal – part 2“).

Background: Chances are this has been used before me elsewhere in the world, but this is a tool I introduced to my teams after returning from Agile 2009. During one of the evening sessions there was a panel discussion. Questions were submitted in advance and each panelist had 2 minutes to discuss. After the whole panel had their say, the audience were given an opportunity to vote. On every seat was a large red and green paddle. If we wanted the discussion to continue we voted green. If we wanted to stop and move on, we voted red.

When I got back to Cambridge I introduced it during some training I was running. I “borrowed” my eldest daughter’s red & green art straws. There were a few “hot spots” on the course where 1 or 2 attendees would lose track. We had a great team who immediately raised a red straw. They enjoyed calling each other out so much that we had red straw warfare at one point!

After using the same in a couple more sessions it became clear the green straws weren’t needed. The red ones were getting tatty so I raided the stationary cupboard for some red card instead, cut this into pieces about the right size to hold up visibly and planted a few in the team rooms. These are now the social norm for facilitators on many teams worldwide but probably not well-known outside the company I’m at right now.

Impact: Of all the tools I’ve used over the last 2 years this one seems to have had one of the greatest impacts on teams and the most viral spread within the organization I work with. Even the management team now red card each other and they don’t even have the cards in the room. Like all good verbal anchors, everyone now knows what “red card” means during discussions. Better still – even on difficult teams I’ve not yet seen anyone use red cards in a socially unacceptable way.

Try red cards out on your next big retrospective – you might want a stooge to break the seal first of all and chances are you’ll need to set yourself up as the first target but once the team have been through this once, facilitating meetings will become more of a team sport than a job for you.

Communicating in Patterns

Reading time ~ < 1 minute

I’m a huge fan of “patterns” or what I describe as “pattern thinking”. My main source of inspiration was not “design patterns” but rather Mary-Lynn Manns & Linda Rising’s book “Fearless Change“.

Whilst I’m no expert at developing my own patterns, the concept of planting a seed – a verbal cue or anchor – to encapsulate a powerful concept in a couple of words is incredibly valuable when coaching and leading teams and managers.

2 Great examples of this that aren’t mine are “Technical Debt” and “Code Smells”. Terms that people can latch onto quickly and use in conversation with generally little problem in being understood.

As a line manager a few years ago I required every member of my team to read Steve McConnell’s white paper on managing Technical Debt.  Whilst he wasn’t the inventor, his treatment of the subject was a detailed and resonant enough introduction for it to stick with the teams. The term technical debt is now heavily used (and usually properly understood) all the way up to exec level within the group.

I was recently privileged enough to have Naresh Jain visit my site to coach TDD & refactoring. In a week of training he effectively introduced our entire group to the very sticky term “Code Smells“.

I’ll be writing up and sharing some of my own verbal seeds shortly. I doubt they’re unique  but they’re how I choose to socialize specific thoughts, ideas & behaviors from my own experiences.

10 Minute Test Plans

Reading time ~ 2 minutes

Struggling to introduce TDD or Acceptance Testing? Here’s a really powerful simple first step that you can perform immediately – How to get your development teams “thinking” about tests and how to test with little or no negative impact and a few seconds setup time.

This assumes you’ve already broken out your story/feature/activity into developer-sized tasks.

  1. Push the keyboard away and get a sheet of A4 or Letter paper and a pen or pencil.
  2. Draw 2 lines to Divide the paper into 3 roughly similar sized areas.
  3. Pair up with the developer who’s going to work on a task.
  4. Start a timer –  you have 10 minutes.
  5. Spend 3 minutes listing out as many “pass” cases as you can think of between you – all those simple “happy path” things.
  6. Spend 3 minutes listing out as many “fail” cases as you can think of – bad inputs, exceptions etc.
  7. Spend 4 minutes thinking of all the boundary cases you can think of – e.g. date/time/time zone, rounding, location etc. Often these boundaries depend on your problem domain.
  8. That’s it – you’re done!

OK, this isn’t going to give you perfect code and it does take a bit of practice. Try not to agonize too much about whether these are unit, functional or acceptance tests for now. We’re still learning to fly here.

Much as standing up collaborating at a whiteboard uses a different part of your brain to coding; paring up, brainstorming and writing with a good old pencil & paper does a similar job. The discipline of not touching the code for a short period of time and thinking about how you’re going to test and what could go right or wrong will significantly improve the quality of what you produce – it usually makes the coding easier too.

Better still – you should now have your first unit tests to write! – pick the simplest ones you have on there and get them working! (I usually start with input validation. I know it’s not directly related to end user value but once I get a good grasp on inputs the rest fits in my head more easily.)

What’s on Your Radar?

Reading time ~ 2 minutes

This is a great tool that I first saw used by Thoughtworks for showing changes in technology trends over time…  http://www.thoughtworks.com/radar/ (I don’t know who invented it). The TW example is very busy – there’s a mountain of new & changing technology trends out there!

But. This is a fantastic simple tool for tracking changes in your own domain or environment. And it doesn’t have to just be technology, this could be customers, prospects, types of work, focus areas, anything.

Here’s a sample I’ve put together for agile coaching . The Arrows show a change in focus since the last review (typically quarterly).  This is deliberately not an exhaustive sample but gives an idea of the what you can achieve.  It’s a great clarifying tool for both your coaching teams and your stakeholders.

Detail on the numbered items in this example:

1: TDD – Introduce, train & coach TDD practices. Ensure teams have the tools available to do so and the space in their schedules to do a decent job. Performance will turn the corner after an initial productivity dip so this needs a lot of care & attention.

2: Code Smells – Train teams on identifying code smells and when to act. We need to back this up with being polite & positive – perhaps some collaborative walk-throughs.

3: Refactoring – With TDD & Code smells, teach the “right” level of refactoring. There’s the natural refactoring needed during development and then there’s the open heart surgery of bad legacy code. We could refactor entire products and move nowhere (@see Netscape). Need to make sure this pragmatically taught.

4: Embedded coaching – with the major increase in XP and technical practices over scrum, we greater technical embedded coaching capacity.

5: No new code without tests – Make it socially unacceptable to check-in without tests unless there’s a real reason. (“It’s not testable” is usually an excuse, not a reason)

6: Shared code ownership – It might have been your baby once but it’s time for others to see how ugly it is and help you pretty it up. Nobody “owns” code any more, no matter how much of their creative heart & soul is invested.

7: Zero defects – We still have a crazy defect backlog. Let’s stop the bleeding this year and get it under control. Longer term we’re looking to get down to a stable level of entitlement.

8: Feature Teams – we’re working as product delivery teams and communities of practice right now which is ok but we need to get to a point where we can deliver fully working features through the product suite as a cohesive team without handovers.

9: Scrum – teams are all now using scrum as their overall operating framework, observing the “rituals” etc. We still need to watch & adjust but the main effort is over, this is now normal operation for the teams.

10: Agile Metrics – We’ve taught the teams and managers how to understand the new data they’re seeing, use it to their advantage, to make reasonable forecasts and highlight problems early. Again this will stay just over the horizon, it’s not going away but not something we plan to revisit for a while.

Just Enough Design

Reading time ~ 3 minutes

Some years ago at a prior employer I had the luxury of working with a team delivering a large green-field Java & Oracle project. The requirements were complex and the interfaces, APIs and business logic all needed some pretty exotic thinking to make everything work.

Prior to that project we’d delivered plenty of relatively simple work and been through requirements, design, code, unit test, integration, system test, documentation etc many times. We were generally a “pretty good” team.

We hired a new member – a very experienced and hands-on architect. He brought a whole load of knowledge we were looking for to the team and more…

After being on board about 2 weeks he called a meeting with the entire team. Hauled us into a room and pointed out just how poor we were at proper design. Moreover he took control of the situation, developed a series of design spec templates, guidance and examples, got the team fully ramped up on UML, capturing design decisions, practices, patterns – the works.

Using our new design knowledge and tools, we moved onto the first critical phase of our green-field project in 2 groups.

Group 1 had to get a working proof of concept to the customer in a matter of weeks, group 2 needed to start designing the way more complex second round of features.

For group 1 (a small pilot team of 2), one of the team did about a week’s research, wrote up the basics and hit the ground running (no real design). Group 2 were not allowed to touch a line of code until the designs were complete!

From memory, start to finish; that first phase took about 3 months.

After the initial work was completed, both groups 1 & 2 progressed onto the next round of features based on the design efforts group 2 had completed.

After about 2 weeks we realized we were having to sacrifice one of the team (our feature lead!) almost full-time to maintain the designs. Coding was completed in a total of about 6 weeks. The fastest coding turnaround we’d ever had for something of this scale and the functionality was way harder than the first round of work.

After our crash-course in the pain of full-on software design,  our architect reconvened the team to lead a design practices brainstorming session.

“OK, now you know how to do proper software design; of the tools, practices and documents you used, which do you want to keep and which do you want to ditch?”

Our management seemed to have had the foresight to allow our architect this social experiment knowing full-well that the net result would be a major overall team improvement (the same manager also helped us develop successful business cases for major refactoring efforts – a pretty forward thinking guy).

So what did we keep and what was our philosophy?

Philosophy first:

The greatest value in design after the fact is not in what was implemented but why we chose to do it that way (and why not another way).

The second greatest value in design after the fact is for team members (especially new joiners or maintainers) to get a foothold into the codebase and be able to navigate around safely.

With these cornerstones in mind we kept a few things…

1: High level architecture – a verbal or pictorial summary of the general concept and approach. Often just a photo of some legible whiteboard sketches. – our first foothold

2: Top level flow – a sequence diagram defining the overall flow of responsibility between actors. – our main “ladder”into the codebase.

3: Design decisions and rejections. (in a wiki/threaded discussion) – why did we choose to do things and why did we chose not to do others. – since learning the “why & why not” approach we saved days of ramp-up and maintenance pain on projects.

4: Complex algorithm annotations – for the really gnarly bits. (Avoid this where possible) – draw pictures for these where you can.

5: Public interfaces – as peer and tech-author-reviewed Javadoc post-implementation. I like public interfaces – they’re a great long-term commitment to communicate in a given stable way. In this case they were also a commitment to our customer. Doing a decent job on these saved a world of support pain later.

6: Unit & functional tests – yes, these are design too!

That’s it! – we ditched a whole world of class diagram hell and parameter definitions. We could still sketch out basic class diagrams when needed but not the level of depth needed to generate code from a CASE tool. We ditched all the noise and blurb and we made it clear why the product was written and behaved the way it did.

So – give your teams an easy leg-up into your code and then explain why it does things rather than telling people what it should do.