Tag Archives: test

The Hidden Dangers of an Independent Quality Organization

Posted on by
Reply
Reading time ~ 3 minutes

“We need an independent quality organization, to keep the project managers, developers and business honest”

Today I thought I’d let “Bad Captain” take the helm for a few minutes.

In talking about software quality at Agile Cambridge a couple of weeks ago, the audience were invited to share their insights. Mine was to “avoid independent quality hierarchies” – to which I received a small round of applause (likely from those that recognize the dangers). I’ve had this article in the shed for many months and although my context has since changed, there are many in the software world that face this pain and need your support.

We trust our managers and teams with the finances of multi-million dollar projects, we trust them with staff pay and appraisals when those knowledge workers are generating our primary source of income and we trust them to preserve our intellectual property!

OK so in many cases we have reviews but these are within the same organization. Some companies sadly don’t trust their teams with the quality of what they’re delivering in the same way?

  • What happened to damage or erode the trust?
  • What’s being done to rebuild it?
  • Is it beyond redemption?
  • Who can lead the change?

Most managers I know and most development teams succeed by having a high level of professional pride and integrity and by doing the right things for their team and product. In the situations where a manager or team doesn’t behave in that way it’s incredibly rare that they don’t get called out by their peers.

  • It’s professional and long-term product suicide to short-change quality, we just don’t do it.

There is no good reason for a manager and team to not set their own high quality standards and meet them.

Beyond “keeping teams honest”, the other argument I heard for an independent quality group was:

“Because it’s a required function further up the management chain, that’s just the way it is”

Don’t get me started on empowerment, continuous improvement and root cause analysis. It seems incredible that in the 21st century, staff in many organizations perpetuate this attitude. The truth is that this is still incredibly common.

Unfortunately I don’t have all the answers today but here’s a couple of thinking tools to present…

Let’s run on the assumption that we really do need an independent quality group. As I see things, that team has 2 options.

Option 1: Set a high quality bar and challenge/encourage teams to meet it. Negotiate down or stand your ground when the project teams challenge the setting of the bar.

  • We’ve just built a new source of conflict and waste in our organization with a management chain that have a vested interest in their group’s own continued existence. Nice move!

Option 2: Set the bar at a level we think the team can meet.

  • We’ve just low-balled our quality for the team. Better still, the team working on delivery are not accountable for raising their quality game because that’s been taken away from them!

Now let’s turn things around.

Option 3: Let teams and managers set their own quality bar. Arrange a proper review where the team use their professional pride to challenge the level they set and determine what’s achievable, what they could strive to achieve next and what are the business and engineering benefits of doing so?

Risk: The team low-balls their criteria.

  • Mitigation: Lead by example, drive a culture where everyone takes personal responsibility. Reward and recognise teams for raising their game.

Funny, for some reason Option 3 doesn’t seem to need independent QA.

Where I work right now doesn’t have an independent QA/QC organization. Our quality is high, as is the level of trust exhibited throughout the company.

This is the simple stuff, don’t break it with vested organizational interests.

Caveat: I understand regulated environments have a necessity for compliance and  independent oversight however that compliance environment is either for safety or due to exceptionally high trust-related risk . If that’s not relevant to your organization, why then are you still doing it?

I’ll return the rudder to Captain Crom in future.

Where’s My Tools?

Posted on by
Reply
Reading time ~ 3 minutes
hand

The most effective tool in the box

Most of the really big successes and game-changers I’ve seen in my career have been initiated through individuals seeing a problem and fixing it themselves rather than waiting for others. 

If you need something done that isn’t already happening, nobody else is going do it for you.

Here’s 4 lessons learned from adopting TDD…

It takes time & effort to get TDD into place

You will have to sacrifice “new code” time in the short term. From my experience, time is probably the biggest barrier to successfully introducing TDD on most teams (closely followed by motivation & accountability). Seeing the pay-off on the other side is notoriously hard to when you’re not there – either starting out or sitting at the bottom of the change curve.

If you’re coaching teams, sometimes you might just need to say “Trust me, I know what I’m doing”

“It can’t be unit-tested”

When you or someone else makes this statement they probably mean one or more of the following:

  • “I don’t believe I have the time to make this testable”
  • “Nobody has provided the infrastructure/harness/test data for me”
  • “I don’t know how to unit test this” (and need some help to figure it out)
  • “We have some architectural impediment to resolve in order to achieve this”

From these statements, which do you think is least common?

Break this cycle by encouraging accountability for making code testable. 

This doesn’t just mean the product code it includes taking ownership of your testing tools and abilities as well.

If you’re struggling…

There’s 2 options for you; “abdicate” or “own”

Option 1 – Abdicate responsibility.

  •  You’ll set the tone for your whole team. The next time someone hits the same problem, the cycle will repeat.
  •  At best, someone better than you may eventually address the issue. (And I hope they make you feel mediocre or inferior).
  •  At worst you may find them saying that you didn’t provide the solution for them.

The “best” case is highly unlikely to happen during your current project – maybe even the one after.  Just think how much testing debt can build up over the life of a single project. We should be avoiding that, right?

Someone has to break the cycle.  If you’re facing pain, why aren’t you responsible for fixing it?

Option 2 – Take ownership of the problem yourself

Great! You’ve recognized that your accountability is part of the problem.

Nobody else will do this stuff for you!

  • Establish a Personal strategy for implementing TDD – how are YOU going to solve the problem?
  •  The teams and individuals I’ve seen that are successful with TDD have all got there by taking a personal responsibility to do so.

Just Do It

Here’s some suggested steps to get going – you can either do the first few alone or as part of a team.

(If there’s an accountability gap in your group you’ll probably need to start alone)

Round 1

1. Get a unit test harness and environment in place – there’s plenty online but write one yourself for starters if you have to!

2. If you have a database, break that dependency and channel it through something replaceable.

3. Write your first round of tests and see what hurts – now resolve those pains.

Limit the time spent on this first cycle and determine whether you should ask for time in advance or for forgiveness later – it’ll depend on your local context.

Round 2

4. Educate & support your team on what you’ve provided so far.

5. Get other team members to write their own round of unit tests (treat it “as  an experiment” if there’s concerns over the effort required). See what hurts and resolve those pains.

6. Encourage and develop shared long-term ownership of the test harness, test data, stubs and mocks (if used).

Remember you’re setting the tone. If someone needs something that isn’t available, pair up, help them provide it and share the results with the team. Don’t leave them alone but don’t do it for them either.

Round 3

7. Repeat round 2 during development until writing tests is fast, easy and natural. (Don’t give up too soon – this is hard work and takes time!)

8. Capture any “wins” and “losses” during this cycle – what benefits are you seeing and what new pains are coming through?

9. Review the wins & losses. Decide what to do about the losses and how to promote the wins up the management chain and across the team. For example either publicise as you go or use as fuel for your retrospectives.

Think about the positive impact that little bit of extra effort and ownership has on you and your team in leading by example – doing a great job that others will want to share, not just a good job.

Abdication or accountability is a personal choice but it affects everyone around you. What are you doing to help your team?

Patterns For Collective Code Ownership

Posted on by
Reply
Reading time ~ 5 minutes

Following the somewhat schizophrenic challenges of dealing with person issues on collective code ownership, here we focus on the practices and practical aspects. How do we technically achieve collective ownership within teams?

We’re using the “simple pattern” approach again. For each pattern we have a suitable “anchor name”, a brief description and nothing more. This should be plenty to get moving  but feel free to expand on these and provide feedback.

Here’s the basic set of patterns to consider:

Code Caretaker

Let’s make it a bit less personal, encourage the team to understand what the caretaker role for code entails. It’s not a single person’s code – in fact the company paid us to write it for them. Code does still need occasional care and feeding – Enter the “Caretaker”.  Be wary though that caretakers may become owners.

Apprenticeship

Have your expert spend time teaching & explaining. An apprentice learns by doing – the old-fashioned way under the tutelage of a master craftsman guiding them full-time. This is time and effort-intensive for the pair involved but (unless you have a personality or performance problem) is a sure-fire way to get the knowledge shared.

If you need to expand to multiple team members in parallel,  skip forward to feature lead & tour of duty.

Ping-Pong Pairing

Try pair programming with an expert and novice together. Develop tests, then code, swap places. One person writes tests, the other codes. Depending on the confidence of the learner, they may focus more on writing the tests and understanding the code rather than coding solutions. Unequal pairing is quite tricky so monitor this carefully, your expert will need support in how to share, teach & coach.

Bug Squishing

Best with large backlogs of low-severity defects to start with. Cluster them into functional areas. Set a time-box to “learn by fixing” – delivery is not measured on volume fixed but by level of understanding (demonstrated by a high first-time pass rate on peer reviews). If someone needs help they learn to ask (or try, then ask) rather than hand over half-cut. This approach works with individuals having peer review support and successfully scales to multiple individuals operating on different areas in parallel.

A Third Pair of Eyes (Secondary Peer Review)

With either an “initiate” (new learner) or an experienced developer working on the coding, have both a learner and an expert participate in peer reviews on changes coming through – to learn and provide a safety net respectively. The newer developer should be encouraged to provide input and ask questions but has another expert as “backstop” for anything they might miss.

Sightseeing / Guided Tour

Run a guided walk-through for the team – typically a short round-table session. Tours are either led by the current SME (subject matter expert) or by a new learner (initiate). In the case of a new learner, the SME may wish to review their understanding first before encouraging them to lead a walk-through themselves.

Often an expert may fail to identify and share pieces of important but implicit information (tacit knowledge) whilst someone new to the code may spot these and emphasize different items or ask questions that an expert would not. Consider having your initiate lead a session with the expert as a backstop.

Feature Leads

The feature lead approach is one of my favourites. I’ve successfully acted as a feature lead on many areas with teams of all levels of experience where I needed to ramp up a large group on a functional area together.

By introducing a larger team, your expert will not have the capacity to remain hands-on and support the team at the same time. They must lead the team in a hands-off approach by providing review and subject/domain expertise only. This also addresses the single point of failure to “new” single point of failure risk seen with “apprenticeship”.

This is also a potential approach when an expert or prior owner steps in too frequently to undo or overwrite rather than coach other members activities. By ensuring they don’t have the bandwidth to get too involved you may be able to encourage some backing-off but use this approach with care in these situations to avoid personal flare-ups.

Cold Turkey

For extreme cases where “you can’t possibly survive” without a single point of failure, try cold turkey. Force yourself to work without them.

I read an article some years ago, (unfortunately I can’t track it down now) where the author explained how when he joined a project as manager, the leaders told him he must have “Dave” on the project as nobody else knew what Dave did. He spent a week of sleepless nights trying to figure out how to get Dave off the project. After removing Dave from the project, the team were forced to learn the bottleneck area themselves and delivered successfully.

(Apologies to any “Dave”s I know – this isn’t about any of you)

Business Rule Extraction

Get your initiate to study the code and write a short document, wiki article or blog defining the business rules in the order or hierarchy they’re hit. This is usually reserved for absolute new starters to learn the basics of an area and show they’ve understood it without damaging anything. It is however also a good precursor to a test retrofit.

Test Retrofit

A step up from business rule extraction whilst delivering some real value to the team. Encourage your learner to write a series of small functional tests for a specific functional area by prodding it, working out how to talk to it, what it does and what we believe it should do. Save the passing tests and get the results checked, peer reviewed and approved.

Chances are you might find some bugs too – decide whether to fix them now or later depending on your safety margin with your initiate.

Debt Payment

After a successful test retrofit, you can start refactoring and unit testing. As refactoring is not without risk this is generally an approach for a more experienced developer but offers a sound way of prizing out functionality and learning key areas in small parts.

This is also a natural point to start fixing any bugs found during a test retrofit.

National Service (also known as Jury Duty)

Have a couple of staff on short rotation (2-4 weeks) covering support & maintenance even on areas they don’t know. This is generally reserved for experienced team members who can assimilate new areas quickly but a great “leveller” in cross-training your team in hot areas.

Risks are generally around decision-making, customer responsiveness, turnaround time and overall lowering of performance but these are all short-term. I’ve often used this approach on mature teams to cross-train into small knowledge gaps or newly acquired legacy functional areas.

My preferred approach is to stagger allocation so that you have one member on “primary” support whilst another provides “backup” each sprint. For the member that covered primary in sprint one, they’re on backup in sprint 2. (expand this to meet your required capacity).

3-6 months of a national service approach should be enough to cover the most critical functional gaps on your team based on customer/user demand

Tour of Duty

Have your staff work on longer term rotations (1-6 months) working on a functional area or feature as part of a feature team. This couples up with the “feature lead” approach.

Again, this is an approach that is very useful for mature agile teams that understand and support working as feature teams but can be introduced on less-experienced teams without too much pain.

The tour of duty can also work beyond an individual role. For example a developer taking a 3-6 month rotation through support or testing will provide greater empathy and understanding of tester and user needs than simply rotating through feature delivery. (My time spent providing customer support permanently changed my attitude toward cosmetic defects as a developer)

Adjacency

This takes a lot more planning and management than the other approaches described but is the most comprehensive.

From the areas each team member knows; identify where functional or technical adjacent areas lay and then use a subset of the approaches described here to build up skills in that adjacency.

Develop a knowledge growth plan for every team member sharing and leveraging their growth across areas with each other. Although this is an increase in coordination and planning, the value here is that your teams get to see the big picture on their growth and have clear direction.

All these patterns have a selection of merits and pitfalls. some won’t work in your situation and some may be more successful than others. There are undoubtedly more that could be applied.

Starting in your next sprint, try some of what’s provided here to developing shared ownership and knowledge transfer for your teams. Pick one or more patterns, figure out the impacts and give them a try.

(Coming soon “Building a Case for Collective Code Ownership” – when solving the technical side isn’t enough)

Escaping the Oubliette (Part 1a) – Debt Prevention

Posted on by
Reply
Reading time ~ 2 minutes

This is a partial re-post of Escaping the Oubliette (Part 1). I’ve split the article into smaller readable components.

Great, I’ve got my incoming defect strategy nailed,

Now how do I prevent defects and debt in new code?

In 5 words…

Continuous attention to technical excellence.

Here’s my top 7 (there are plenty more)

  1. Acceptance Criteria – Be really disciplined on your acceptance criteria & acceptance tests, team up with Analysts, Testers, Product Owners if you have them and attack your stories from every angle. A good approach to this is a “story kick-off” where the whole team dismantles a story before starting.
  2. Thinking Time – don’t just start coding right away, task things out, try the 10 minute test plan, discuss your approach with your peers and for more complex or large items, try the “just enough design” approach.
  3. TDD – It’s hard to start but has an immense impact.  I’ve just seen a team complete their first project using TDD. 3 weeks into their final round of post feature-complete testing, their defect run-rate hasn’t had the testing spike seen on prior projects. In fact they’re keeping on top of all new incoming defects and have time to start paying down the historic backlog.
  4. Pair Programming – Do it in half-day trial chunks if you don’t have the stomach for going full-tilt. I’ve performed remote pair-programming with colleagues across the Atlantic using decent phone headsets and online collaboration tools for hours at a time. The net result of 2 days of remote pairing was finding and fixing about 10 extra defects in a thousand lines of code that neither of us would have found coding alone.
  5. Peer reviews – there is still a huge space for these in agile teams. But here’s the thing. Be really tough. A peer review is not a hurdle. It’s a shared learning exercise. Functional correctness is actually the smallest component of a peer review. You should trust your developers that far. But there’s a whole series of other aspects to review. See the joy of peer reviews.
  6. Small tasks – I once worked with an outsourced team who when taking work would disappear into a hole for 2 weeks and return with a single task in our configuration management system containing edits to 200+ files and multiple condensed edits to the files. My rule of thumb is one reviewable task per activity. If you’re going to add new functionality and refactor, that’s 2 independent tasks that can be identified and reviewed separately. This means you should be able to easily deliver 2 reviewable, closable tasks per day.
  7. Fast Builds – make it insanely simple for a developer to perform an incremental build that validates new code against the latest main code line. (small tasks are a big help here).  This includes the right subset of unit and functional tests. Aim for a target of a 30 second response time or less between hitting the button and seeing the first results.

In the next article in this series I’ll focus on “Tailing” – How do you start reducing the old defects.

Stop The Bleeding

Posted on by
1
Reading time ~ 2 minutes

One of my favourite verbal patterns is “Stop The Bleeding”. I heard the term come back to me in a management meeting last week in the right context and with the right response – Success!

I’m not exactly sure where I heard this first but it was sometime in the last 4 years. Mike Cohn references it in the context of agile testing but until I searched for the term just now I’d never read that particular article so I’m sure it’s also used elsewhere…

Name: “Stop The Bleeding” or “Plug The Hole”.

Concept: Consider the “flow” of work and issues. Many software activities generate or receive some sort of flow. Whilst you can use a Kanban approach to manage the flow, you need to address the site of the problem first to prevent it getting any worse and look upstream to consider prevention activites.

Analogies: You have a patient on a stretcher with a series of broken bones and a torn major artery, where do you focus your attention first? – or- You’re in a boat with a leak in the hull. You either stop rowing and start bailing or start rowing and start sinking. Maybe it’s time to fix the leak?  Once you’ve fixed the leak, how about staying away from the rocks?

Usage: The most common times I’ve used this term are in test automation and defect management.

I’ve seen major investments in writing automated regression test suites for existing functionality or old manual tests. Skilled testers are usually in limited supply. Why are we using their valuable capacity on a static problem?

We need to take that capacity and apply it to the issues we continue to create every day – the ones where we’re still bleeding. Write automated tests for our new functionality and changes that we’re introducing right now. These become your regression tests as soon as they start passing!

In defect management a common challenge is customer escalations. These usually happen because we failed to act on a problem in a timely manner. Stopping the bleeding here means addressing new incoming defects promptly and properly. For example; try replicating issues immediately and then negotiating when your customers want a solution rather than just focusing on traditional severity / impact numbers.

Once your flow is under control and you’re working to when customers want a fix, escalations for delinquent issues will trail off fast. Now you can start addressing your other priorities.  Note: The “bleeding” in this second example is customer escalations, not incoming defects – Stopping incoming defects is a subject in itself!

10 Minute Test Plans

Posted on by
1
Reading time ~ 2 minutes

Struggling to introduce TDD or Acceptance Testing? Here’s a really powerful simple first step that you can perform immediately – How to get your development teams “thinking” about tests and how to test with little or no negative impact and a few seconds setup time.

This assumes you’ve already broken out your story/feature/activity into developer-sized tasks.

  1. Push the keyboard away and get a sheet of A4 or Letter paper and a pen or pencil.
  2. Draw 2 lines to Divide the paper into 3 roughly similar sized areas.
  3. Pair up with the developer who’s going to work on a task.
  4. Start a timer –  you have 10 minutes.
  5. Spend 3 minutes listing out as many “pass” cases as you can think of between you – all those simple “happy path” things.
  6. Spend 3 minutes listing out as many “fail” cases as you can think of – bad inputs, exceptions etc.
  7. Spend 4 minutes thinking of all the boundary cases you can think of – e.g. date/time/time zone, rounding, location etc. Often these boundaries depend on your problem domain.
  8. That’s it – you’re done!

OK, this isn’t going to give you perfect code and it does take a bit of practice. Try not to agonize too much about whether these are unit, functional or acceptance tests for now. We’re still learning to fly here.

Much as standing up collaborating at a whiteboard uses a different part of your brain to coding; paring up, brainstorming and writing with a good old pencil & paper does a similar job. The discipline of not touching the code for a short period of time and thinking about how you’re going to test and what could go right or wrong will significantly improve the quality of what you produce – it usually makes the coding easier too.

Better still – you should now have your first unit tests to write! – pick the simplest ones you have on there and get them working! (I usually start with input validation. I know it’s not directly related to end user value but once I get a good grasp on inputs the rest fits in my head more easily.)