I’ve tried using 5 whys techniques on many projects and with teams and individuals in a variety of situations.
It’s generally used for root cause analysis but often (I think) misused for other situations. Something about it has always bothered me.
When you see it described in theory it makes so much sense. But most examples are already solved (and I suspect refactored for post-rationalization). When you use 5 whys techniques in practice it never quite hits the mark.
In most cases where I’d have previously considered a 5 whys line of questioning for understanding cause I’ve been inclined to use an Ishikawa (fishbone) diagram instead. This allows us to dig into multiple lines of questioning and link across related areas.
But this still only works well in understanding problems, not goals and reasoning.
Thanks to a great session from Paul Field at the Cambridge Agile Coaches Camp back in 2015 – I’ve finally been given a 5 whys alternative that actually works properly outside the context of examining root causes.
Paul’s own article goes into the full detail on the lines of questioning and the techniques involved so I strongly encourage you to give this a read.
A large part of the improvement here is in replacing the question “why”with directed alternatives.
“Why” in psychotherapy (and some coaching circles) is seen as a dangerous question. From a purely human perspective it can be confrontational and seen as judgmental. A good therapist or coach will instead ask very specific questions based on context that encourage consideration and introspection in a safe and well-judged manner.
Much like asking a child “Why did you punch little Davey in the face?”, asking “why” – particularly in challenging, negative, or political situations is unlikely to yield a well-reasoned answer. You’ll get a knee-jerk and/or self-justifying response.
Put another way. Repeatedly asking “Why” is like using a sledgehammer to open a jar of candies. You’ll open the jar but I wouldn’t recommend putting the results in anyone’s mouth – and you’ll still have to go fishing through the damaged results to find anything of real value afterward.
Next time you’re considering 5 whys, try asking “and what will X get from that?” instead.
I’ve been working in software product development companies for nearly 20 years.
Until 4 years ago I’d always been involved in “enterprise” software.
You know – the monolithic systems with great reporting capabilities that sell well to managers and (at least historically) poor UX for the real users. Those same products that promise the moon during slick demos, require 6-12 month sales cycles where complex pricing structures are worked through and year-long implementations are agreed.
I’ve helped set up demos, I’ve watched amazing presales engineers work all night to rewrite chunks of an application to demonstrate to a prospect that it’ll meet their unique requests.
And then during implementation you discover some of it just doesn’t work.
I implemented one company’s products internally. Dogfooding from the day V1.0 was released. In 6 months I raised over 50 showstopper bugs (almost all were found by their first real customer not long after us). On a visit to HQ in San Francisco, after a fair bit of wine at a great Italian restaurant in Burlingame (the House of Garlic if I remember correctly) I challenged the then VP of Development for that application on why they shipped blatantly unfinished software.
His answer was simple, logical and for a young, inexperienced graduate analyst programmer my first window into the commercial realities of product development. He said;
“If we released when the software was actually finished, we’d be beaten to market by our competitors.”
“It’s acceptable business practice because it takes 6 months to sell and we can’t start selling until the product is released and in our price books.”
“Even after the sale it takes months to implement so by the time users are ready to go live we’ve fixed all the major issues because you guys are dogfooding it for us”.
It made a whole lot of sense but it wasn’t something they ever told us on the project!
The thing is, everyone else is on the same bandwagon and the escalation games start rolling.
Products are brought to market earlier and earlier in their maturity with the knowledge that “nobody trusts a v1.0 version of a product anyway.” It continues today. Many large banks won’t touch an x.0 version of a product until the first major maintenance release is shipped.
My experience was the same. In most companies I worked for; when a new release of the DB platform shipped, we’d plan on adopting it sometime after the first 6 months out in the wild when we knew it was stable.
The game has moved on. It’s no longer just products that take a year to sell and implement so our exposure to early releases is increasing and in general so is the entire industry tolerance (there are obvious exceptions in safety-critical systems).
In my time at Oracle in the late ’90s I saw them try to change this world. They had a vision known as”Gray Iron”. It seemed brilliant – for business, development teams and customers. The idea was that based on a playbook of “best practice” business processes a customer could buy a server entirely preconfigured and working out of the box to support their entire financial, ERP and CRM set of processes. They could simplify the pain of users, ensure quality was high and radically reduce implementation times. Obviously this also offers a great potential competitive advantage.
Sadly it didn’t take off. Competitors were still playing the “ship early” escalation game so we had to play too – the poor system fed itself.
The surge of change brought about by Eric Ries’ book The Lean Startup has fanned the flames of this mentality. Ship early, get customer feedback and adjust.
The sad thing is this used to be only in the enterprise market but the world has turned. Consumer electronic devices now face the same market timing escalation issues as this article from December on Forbes highlights.
It’s now common for us to expect our phones to need a reboot or crash and even for consumer software to have significant glitches.
It’s great for the pace of innovation but spare a thought for the end users and customer support teams.
It’s a commercial reality we can’t avoid but let’s make sure we’re not sacrificing the end user experience. If you must use your customers as lab-rats, let them opt in or out of your experiments. Let them decide if they want “bleeding edge”, “new” or “stable” and honor their wishes.
If your product is even remotely valid you probably have enough early adopters willing to validate the bleeding edge without sacrificing laggards on the altar of possible futures.
At the beginning of this year the team I’ve been supporting hit a wall. After a deeply troubled project last year they’d started researching a “new” feature in December with a plan to get rolling in January.
After the holiday break, a couple of weeks into January we had a near-mutiny and hit the reset button.
Things are now back on track and working in a radically different way but here’s the story.
Early August – Halfway through a deeply troubled “rewrite” project. One of the paired project managers left and I was asked to support the remaining PM.
Mid August – Working with the remaining PM we visualised the entire scope of the UI on the team’s whiteboard and immediately excluded all dialogs and rarely used parts of the UI from the project. Essentially halving the scope of work
Late August – The second PM moved back to a development role elsewhere and I joined the team full-time. I started digging deeper into the scope and data. After reconciling the stories the team had on the wall with those in the electronic backlog we discovered as much additional scope as had been removed 2 weeks earlier – literally “hiding in the walls”.
Early September – Having reviewed the state of the project I talked to our engineering manager and product owner and said “this project is f***ed, do you want to kill it?” The response was “No, we believe this is the most important and valuable thing we can be doing right now.”
Late September – After the first attempts at recovery still hadn’t achieved enough I re-asked the same question – the reply was the same
Mid-October – And again. All the while the team are still working away trying to deliver.
Late October – Our Product Manager (not the Product Owner) finally decided the project needed a reset. The forecasts had finally tipped him over the edge and the total cost wasn’t acceptable. (This coincided with our push to get an Alpha release into the hands of users for concrete feedback.)
Mid-November – We shipped an alpha release to validate our new UI technology and started building plans and options for the finished product release.
Mid-November – We sought input from the team on options on how to exit the project gracefully. (there’s an entire article on this experience waiting to be written)
End November – We killed the project and I started on building a plan to move on – leaving the team with a new manager and have them successfully working on a new release by the end of January
December – A general lull. Recovery time. We had Down-tools week, holiday period, some bug fixing and a couple of the team started investigating the next feature. My replacement joined the team and we started on a gentle 2-3 month handover.
January – When the team returned from holidays we decided to start on the “new” feature (actually finishing a feature started 2 years earlier and never finished). One of my team had the task of reviewing current behaviour and delivering a demo to the team. This was eye-opening. The scope that had been implied from PO conversations didn’t address the fact that the feature was barely usable.
Mid-January – Over the 3 weeks this new feature had started (as a “3 month” project) scope had ballooned again to include rewriting a fundamental aspect of how data would be stored.
Mid-January – Team mutiny (During a retrospective). The feedback was honest and painful but most importantly it was actionable and had cemented the worries I already had – things were broken again and we needed to hit the reset button a second time. Although he was already on board, this was not the time to hand over to my replacement.
End January – Reset time. Feedback in hand, we had to act fast. I redesigned the entire way work flowed into and through the team, gave some direct and difficult feedback to our new product manager, engaged him in the reset and briefed the team on changes.
Mid February – With the reset well under way and team morale already on the increase I could safely hand ownership over without feeling guilty about the state of the project.
But what went wrong?
There were a catalogue of problems raised by the team. Here’ the points the team raised…
We talk about “quick wins” but never do them
Poor control of scope for the last 2 major projects
A team of 10 with 4 managers is consistently underperforming another team of 3 with no management – why is that?
The project backlog is only technical tasks, not user stories
We still don’t know what the focus of the next product version is
We’re deeply worried about the quality of the next feature (based on current scope)
We’re not bought into the new feature work – we’re repeatedly told users don’t need it and it’s just a checkbox
We want to be more reactive to support requests
We’re not delivering quickly to users or responding to their feedback
We want to feel proud of the product and what we’re delivering
The work isn’t structured to be delivered iteratively
There’s a complete lack of user perspective on upcoming work
Despite 4 managers, we have no leadership.
Brutal – but completely valid
For the last 6-12 months we’d been entirely focused on helping the team deliver what the business had been asking for. This was a good team. Sure they’d had a lot of technical problems on the previous project but the new one should have been different.
Both projects started with a technical requirement – rewrite X, finish Y.
This built some odd behaviour into the team. They were at the mercy of the backlog. They couldn’t self-organize, they couldn’t deliver iteratively, they couldn’t manage scope, they couldn’t even decide the best solution for our users. Everything was pre-defined.
I hadn’t spotted the problem for months. I’d been so focused on delivery of what we’d been given that I failed to identify the biggest issue was the incoming work itself!
After killing this second project, the next feature on the list was called “Implement Z” – where Z was a half-written product (again).
I now have two “stop” phrases added to my list of dangerous projects;
These are gross generalisation phrases much like “finish X” or “Implement Z”, they’re the smell of unbounded requirements. Taking them at face-value and accepting work into the team is a recipe for failure. Both failed features started out on the same two flawed premises but beyond that, take another look at the list of issues raised by the team.
We’d let the team down. We were managing poorly, not leading and the features were not user-facing. They might once have been but not by the time they reached the team.
How do things look now?
First we clarified who in the management team was responsible for what. Who was “in charge” from the business, who was “supporting” and who was “managing”.
Then we restructured our view and approach to incoming work into 4 balanced streams with clear constraints, policies and ownership for each.
This is probably best explained using the presentation I ran with the team…
We asked the team during the presentation who would be willing to take on the role of “feature lead” for upcoming features. Every hand in the room went up!
We’re just reaching the 2 month checkpoint and things are significantly better.
The fixes and achievable streams are generally running smoothly and shipping frequent product updates.
The team have learned what sizes of work and level of understanding do/don’t work well as “achievable” items.
The feature lead role and relationship with the business is successfully flushing out real user needs.
The analysis/discovery stream took longer than the team were hoping for but development of the first major feature off the back of the research work has just started.
The team completed their first user story mapping session to plan a genuinely iterative set of potential releases.
The first iteration of a new major feature is (optimistically) planned to finish in 2 sprints.
At any point after the initial release we could stop and move on to something else.
After the first release the team will have the ability to run UX sessions, gather feedback and adjust plans based on real user responses.
The team’s new project manager has been able to take his sabbatical and leave the running of the project to the team for 6 weeks! (with occasional check-ins and support from me)
Better still, Nick (my successor) has been gathering measurable feedback from the team since we started the reset. Here’s the data.
From Architectural Decoupling to a Complete Rewrite
For a lesson in what not to do, here’s the gory detail on the first project that was killed.
The project goal was to “make it significantly easier to change the UI in future” – delivering value to internal stakeholders in the form of efficiency and quality savings.
This was conceived as “3 month” project back in April but ran into trouble over the summer. The PMs involved in the start of the project had assembled enough project data to establish “This could take anything up to another 1-2 years to complete”.
The project scope had changed from architectural decoupling to a an internal technology refresh and from a technology refresh to a pilot for a new (unfinished) product style guide, a complete UI replacement, implementing a set of untried technologies with a team that had little or no experience of using those technologies and improving the user experience of the UI along the way.
Essentially they were going for a complete unbounded UI rewrite.
The initial idea was to introduce the new UI as an MVP alongside the old UI with limited functionality and support for end-to-end workflows.
Somewhere in the course of time the “end-to-end” part of this had been lost.
By the time I joined, the implementation approach had become replacing whole tabs of the UI, developing multiple tabs in parallel to improve and validate code reuse and completing these widget-by-widget rather than by workflow.
On the surface, technically this seemed sensible. It reduced rework and kept architectural quality high.
I made things worse by pointing out that an MVP wasn’t viable for a product whose broad set of features were in daily use by over 1,000 customers. Thus it became a “like for like” replacement project – of which so many end up doomed by a big bang approach and repeated delays as scope is discovered!
Shipping a new release with significantly cut-down functionality is a big gamble and usually only one you can take if you have something else of significant value to offer to the customer.
In fact, the most commonly requested feature from our customers – a rewrite of some particularly nasty functionality to support additional use cases and improve existing ones – wasn’t being worked on. We’d actually decided to remove the poorer version of that feature and not replace it until a future release.
In hindsight and from the outside this all seemed a bit nuts.
Before I joined the team, in my oversight role across all projects in the company I was aware of a nagging worry about the state of the project but I couldn’t put my finger on why. The reports and whiteboards seemed “ok” or “meh” but not fundamentally bad.
This is now another warning sign to us when reviewing project status. Month after month of “ok” (not “good”) status across an entire series of checks is a smell (see “Yellow is the new Green” )
I repeatedly asked about any deadlines, due dates and what was going on and received reassurance that everything was under control but there weren’t any real deadlines so everything was still “OK”.
As “Head of Project Management” I have to admit it’s pretty embarrassing to not have tackled this wave of trouble. I and others in the leadership teams saw it coming but trusted the feedback we were seeing. We continued to treat everyone as knowing what needed to be done and safe to be able to do so (“S4” in Situational Leadership terms) when actually they were in trouble and needed help. (“D2” level).
Things came to a head at the beginning of August with the project on the rocks, one PM left and another wanted to go back to a development role.
Determined not to face a “Netscape project” I joined the team to get things back on an even keel. Until now, I’d never had a project I couldn’t salvage quickly – usually in under a couple of weeks. This one was definitely much more of a challenge.
Fortunately the team had backlog and historic size & velocity data so we at least had a baseline to work from (the outgoing PM pointed out that the estimates were wildly inconsistent and hard to trust).
Step 1 – We visualised the entire scope of the UI and discovered some 30+ dialogues and little-used screens that could be postponed. We took a scythe to the scope listed in the backlog based on this visualisation and managed to improve the forecast delivery dates.
Step 2 – I was unhappy that the partial backlog on the wall didn’t offer easy traceability to the full digital backlog so I tagged up all the stories and reconciled the wall with the backlog in a spreadsheet and discovered another 50% of scope hiding in the walls.
At this point the project had ballooned beyond all recognition of the original. We’d dropped into sunk costs and the approach to delivery had removed our ability to deliver iteratively.
Every sprint that passed uncovered more technical problems and more missed scope. Some major performance issues with the new technology tripped the team up for weeks and even after shutting the project down, taking what was delivered as an Alpha in November, salvaging the most complete parts and getting it shippable involved 3 more months of stabilising the technology – insanity!
Fortunately, the salvaged tech finally paid off. After releasing the main work, the team were able to deliver a series of small enhancements near weekly for the next month.
They’re through the worst now and I’m moving onto another team.
The team did a great job staying sane in spite of the craziness around them and both they and “management” have learned a lot . What we have in place now goes beyond preventing a repeat of all these problems – it resets the whole way they work to focus on customer needs, customer problems and customer value. It gives them space and freedom to self-organise whilst balancing the needs of the business when necessary.
I’m sad it took nearly 9 months to get here but glad we’re finally on the right track.
We’re also using the lessons from here to ensure other projects that threaten a rewrite focus on incrementally delivering end user value by only rewriting areas that directly improve a team’s ability to deliver on the next customer-facing changes and features.
Update – 24th April 2015:
I had a chat with one of the team who reviewed this for me. Other than some tweaks to the early timeline he mentioned that as of a couple of weeks ago the team have now actually deleted about 2/3 of the UI code that was developed but never finished. That’s a really brave move given the effort they all put if but they found it was holding them back in adding new features (using the new UI technology in smaller increments).
He also mentioned that the first new feature to run through the updated team analysis process is flying – a way better start than the last!
They’re is still a great team and have really turned the corner by the looks of things now.
This cautionary tale came up a few times at Agile 2010 (yes I know this may be old news!) – including one of the keynote speeches.
“For a million dollars we’ll make you Agile.
Here’s a list of previous clients whom we helped achieve a 1,000% performance improvement. Lead times halved, profits doubled and everyone was AWESOME!.
Here’s some REAL DATA …”
Axis titles shamelessly acquired from a recent company presentation
What you’ve not been told is that those testimonies are statistical outliers!
These are the top 1-5% of companies that have successfully undergone a major transformation. (or the bar was set very low to start with) There are thousands of companies out there that don’t reach these lofty heights.
The journey is longer and harder than the marketing will ever tell you but that’s fine as long as you know what you’re investing in and why.
Your organization is unique. There are many factors about your organization that will make significant improvements hard to achieve and most of them will not be technical. The forces of resistance will be many and will be a mix of institutionalized and personal.
Let’s replay this with a more realistic conversation…
A consultant visits your site, and does a “free” one-day Agile Assessment of your teams.
“OK Boss. Right now you suck at developing software. Give us a million dollars, a year of your time and a willingness to drop productivity for a while and we can make you suck a bit less.”
Furthermore, they won’t actually be able to tell you just how much less you can suck and by when.
Back to all those forces of resistance – how many of those can you really prod, assess and quantify in a day or even a week?
Every company, organization and site differs.
The investment may still be worthwhile but it’s time to manage expectations better. Those assessments should highlight where unexpected limitations lie. Maybe they could offer alternative higher priority areas to tackle (rather than up-selling scope).
If product development and software engineering was like cutting coke cans, there really would be a “one true right way” of producing software products.
Moreover. There wouldn’t be a thousand consultancies promising you the moon on a stick, there wouldn’t be conferences on improving the state of the art and there wouldn’t be hundreds of books full of great ideas on how to do/be agile or perform software engineering a little bit better.
In fact chances are there wouldn’t be a competitive software industry.
Or would there?
• Maybe there really is a one true way and the entire software, consulting, authoring and conference industry is in on the joke.
• Maybe all those leading lights on their skiing trip a decade ago came up with one of the world’s greatest “Long Cons”
• Oh and they invited 3M to the party and agreed to promote stickies in the 21st century in exchange for a marketing budget.
There are no “best practices”. Stop looking for them, stop asking consultants and Scrum Masters to implement them.
There are only “best known practices for your current state, knowledge and context”.
When your state, knowledge and context change, it’s time to look at what’s next and more importantly – what’s beyond your current focus – what have you missed or not considered yet?
What did you discard previously because there were constraints or issues preventing them working? (I learned a great term for this from Chris Matts &/or Dan North – I can’t remember which – “Idea Wombling” – seeking out great ideas that were previously discarded)
You may reach a point where your organizational immune system (politics and process) blocks progress.
Sometimes you’ll need outside help to see what’s “better” or learn new ways of working. That outside help can often be more effective at delivering hard truths than you can yourself. It’s worth investing in “straight-talk” from strangers sometimes.
Figure out what is holding larger improvements back (and where) and determine who you could pair with either externally or in a different part of the organization to make a real difference!
Every time I return to the draft of this post I tweak, rewrite, remove and add. Today I decided it’s better to ship “good enough” and move my investment elsewhere. I hope this still delivers enough value to you as readers…
Even with a good investment, wouldn’t you want to increase your chances of success?
I’ve been involved in annual and bi-annual portfolio reviews of varying quality over the last 7 years and been in the fortunate position to change and improve how many of those have run. With every review I think about how we can do a better and quicker job.
If your portfolio is anything like those I’ve seen in both my current and last employer you’ll have a mix of “no-brainers” and speculative work. Often those are run through the same process with mixed results. You may even have the luxury of standing teams from some areas.
When a project or bid reaches review, on the surface it’s in the proposer’s personal interest to minimize the level of scrutiny their plans receive. They’re busy trying to deliver a business and want to get their project moving fast.
I regularly see plans proposed at portfolio review that magically change within 2 weeks of starting and again within a single quarter. I’ve even seen plans delivered “late” and miss large parts of a review cycle and yet still be approved.
A cynic might say that the owner has secured their funding, now they’re free to change! Less cynically, this is simply what happens when early-stage plans and new projects hit reality. With the review cycle over the same apparent due diligence no longer comes into play.
With peer reviews of code, good reviewers are well-practised in what to look for and may even have checklists and standards. What tools, checks and support do reviewers of your portfolio have? Do they really work?
I frequently see other portfolio “smells” too.
Where a project is a continued investment in an area already in progress, these are often a shoe-in for the next cycle. This can turn into sunk cost behaviours.
If a project is recently started, we may simply assume that because the project is only just under way that we need to “leave it alone” and that it may be unfair to re-scrutinize a business case and plan.
Large revenue increases are predicted against short time scales on the promise of a new release. The time for these increases to actually occur is much longer. Furthermore, these are usually single-point commitments without agreed tolerances or ranges.
I’ve had the luxury in the last couple of years of being in a company that has a track record of actually shutting down projects early (occasionally not early enough) when claimed benefits fail to materialize or overruns occur, Yet even when those projects were set up, bold forecasts were made and operating tolerances against those forecasts were not negotiated.
I’m torn. Proper due diligence on all investment decisions seems obviously right but the case for funding a standing team to continue driving a nascent or even successful product forward is strong. As my CEO said to me recently – “It’s very hard to resist the man with the plan”.
The “VC” Investment Philosophy
Consider your product development organization as a Venture Capital company.
Every year you review funding and invest in a variety of options. Like VC, a smart portfolio and balance of risk/reward is important. You make some investments to sustain cash flow, some for organic returns and you hope to find the occasional (and mythical) “hockey stick” growth curve.
Balanced risks and a mix of investment durations is needed for a sustainable business and a regular review of those decisions is vital to your operating health. You may choose a multi-year investment in a loss-making area believing it’ll come good in future or drive sales for other products. A year later the climate may have changed or the market may not have materialized. Do you kill the investment or ride it out? You can’t answer this question without understanding the context of the rest of your portfolio.
If it were lightweight enough and delivered better decision-making, I assert you’d want to your review your investment much more frequently. In my simplistic view of the world it’s like the different between a basic index tracker and the increased cost (for potential better return) of buying into a managed fund.
Treat your portfolio like a VC investor, consider taking every new product release (including new releases of existing products) like you’re going back to the dragon’s den to bid for another round of funding.
Don’t commit all your capital on an annual basis unless there is a truly sound reason to do so. Seed your investments with an option to increase investment in future or kill it early if it no longer looks valuable in your portfolio (Real Options anyone?)
Picture this rather extreme “Dilbert” conversation…
– I want to improve my existing product, can I have some money to take these improvements to market?
– Sure what’s your business case
– Well the product is already in the market and profitable, I don’t know who all my customers are but I have lots –
I can’t tell you exactly which of my products is bringing the money in because we package it all up together under a single product line but all those existing customers give me support money every year.
They raise enhancement requests that they never expect to get fixed because we take so long to deliver and all new features we deliver are never on their version anyway.
Most of them don’t have any immediate plans to upgrade so basically we’re making millions from fixing defects and taking support calls. But I’ll make you a personal commitment to 1000 new customers as long as I can get every “must have” feature in the backlog implemented in a year.
And if I don’t meet my numbers it’s because we didn’t get all the features implemented
– Ok, I’m feeling flush and can’t be bothered to visit the race track this week – here’s a year’s funding, I’ll see you in a year.
Let’s try again on some sounder (but still not perfect) footing…
– I want to improve my existing product, can I have some money to take these improvements to market?
– Sure what’s your business case
– Well, we have 200 customers on out of date releases. They have no immediate incentive to upgrade as what they have meets their needs but upcoming regulatory requirements plus opportunities in their markets mean that they will need to change and add the following 10 business processes.
Right now these are manual or slow and cost them on average $1M with a 3 month cycle time.
In order to take a viable upgrade to market we need to show a return on investment to our existing install base.
We believe with the right level of software support we can halve their costs and reduce cycle time to 4 weeks for the top 3 of these processes.
Providing this ROI to our existing install base will preserve M&S revenue.
Moreover, here’s the list of RFP responses we can’t compete on (and their value) due to these processes not being supported right now.
If we modularize these features and deliver them sequentially, here’s the first round of prospects we’ll be able to work with – plus we can charge an additional premium for some of these features.
Based on current team velocity and backlog, we’d like to focus on delivering a minimum marketable feature set by the end of next quarter to meet the top priority business processes however there’s some viable value-add work that we want to move forward on, can we have 6 month’s funding now and an option of a further year if we’re successful?
– Ok, I’ll tell you what. Here’s 4 months funding. Let’s have a checkpoint review in 3 months and assuming you’re within 20% of your target we’ll immediately release a further 3 months (7 in total).
If you can deliver on those, I’ll make sure you’re on the fast-track priority list for the next round but if things aren’t going well at the end of 6 months we’ll need to either shut the work down or push for a hard reset.
How soon can you have something to show me?
I’m grossly exaggerating and over-simplifying. It’s an extreme example to illustrate a point but which is a more compelling business case and conversation?
Which has clear priorities, measures of success and is believably achievable?
Which would you invest your personal million dollars in?
Which is more likely to be set up for success?
The impact of governance on portfolio and product management
Let’s assume your governance system is broken and you can “get away with” the first case. There’s more to explore!
Let’s imagine both of the above were products in the same active portfolio…
Scenario 1: – Portfolio resourcing and opportunity cost
I have a constrained resource – developers. I can choose to invest those staff across all products in parallel and deliver slowly or focus on a smaller set, deliver faster and then move onto the rest. I know I want them all but surely there’s an absolute priority. Considering the business cases above, which will allow me to prioritize better?
Scenario 2: – Portfolio investment in a downturn
We’re in a big economic downturn, our numbers are looking bad and we need to start cutting investment fast. Looking back at the business cases for our existing portfolio, which should I take out first and which should I keep?
Scenario 3: – Portfolio collaboration in a tight market
We’re in a business team where each of us owns a given product or number of products. Again, the market is tight. Ours was the first business case above.
We have a choice. Fight our corner for our product’s success or collaborate with our peers to set up the best selection of new product releases based on overall market trends, install base, requests etc.
If we fight our corner, we risk sub-optimizing around personal goals at the expense of the entire group’s success. If we collaborate, we won’t get a new release for our product this year.
Which is the “right” thing to do?
Scenario 4: – Improving product quality through your business case.
I’m a developer on a team. I’ve been working with the same product for 5 years adding new features continuously.
I like my job and want to stay. I don’t question why we’re adding new features, I’m sure there’s a good reason and I’m told that’s up to “the business” to decide.
I’m now halfway through writing a new feature and I don’t know what the user is actually trying to achieve.
Given the above business cases:
Which is most likely to have backup information that allows me to make a sound implementation decision quickly?
Which one has users I can talk to?
Which is the more motivating product to work on?
Which one am I likely to actually care about any deadlines or commitments for?
A sound business case** is just as vital to an agile team’s success as it is to our portfolio decisions. How many times are engineering teams really engaged in the business case for the product they’re working on – or are even able to challenge parts of it?
**When I say sound I don’t just mean it has great projections, I mean it stands up to truly solid testing – try the 200 questions technique for testing a proposal.
In your organization is it their place to do so or none of their business?
Is it more cost-effective to keep them out of the loop?
Really? How’s that working out for you?
We need to understand and explain what really meets our customer’s and our own business needs. Did the last release fail to solve their problems or do they have new issues? It’s not just what “features” (and how many) the next version of the product should have. It’s who do they impact & how and should we even have a new version of that product right now relative to other priorities.
Time to re-check your governance systems and start thinking like a VC. Set up your funded start-ups for success, actively manage the portfolio without wasting resources, engage experts in decision making and business support and where a real new opportunity is seen, don’t just fire a single shot, try a spread bet on that market.
Beyond the initial investment, good VCs offer support, mentoring and critical feedback. They don’t just hand over money and quietly hope their horse will come in at the end of the race.
If you’re on any kind of portfolio review, do a proper job, ask tough questions and make sure your business is making sound decisions.
All of us as reviewers are empowered to do so, don’t let precedent or pressure erode your business sense. It’s our responsibility to test, challenge, verify and make sure we’re investing in the right spread of activities.
If you can’t see the value in an investment or feel your money is better spent elsewhere, tackle those issues head-on.
Some thinking for you.
Are your portfolio reviews being gamed because of a poor periodic funding cycle?
What operating tolerances have you agreed for each of your investments?
What would need to happen for you to be able to easily review your portfolio more frequently?
In your current portfolio are there any investments that smell bad?
Is your portfolio a good balance of risk, reward and cycle time?
If you had to shut down 10% of your projects next week, which would they be and where would you reassign that investment?
What critical questions do you (or should you) be asking of any investment?
Do those questions change depending on the type and scale of investment you’re making?
Are you truly “empowered” to change any of these things?
Who do you need to discuss this with?
What would really happen if you tried?
If you could change or improve 3 things in the way you review your portfolio, what would they be?
What’s truly stopping you pushing for those changes?
The best software developers I’ve worked with so far (in the last 20 years) are a particular type of person. They tend to blend the creativeness of an artist with the technical adroitness of a scientist and the clear-thinking of a businessperson.
That’s quite a lot to personalities to fit in one head.
The trouble with creative types is that sometimes we’re a bit needy.
Almost gone are the days when you could have a great idea and code away with your mates on the next “killer app”. Those success stories and startups you read about are outliers, many markets are saturated. You need a genuine differentiator to succeed and it’s a long and painful road to get there.
In many successful (and unsuccessful) companies it’s a rare luxury to work on genuinely green field software development projects. Legacy code is the every day reality for most of us.
That new and shiny creativeness is a different beast in large-scale or legacy software development. Step back and take time to recognize that your needs are being fulfilled – but in ways you may not be open to seeing.
For those of you who live in the harsh reality of “normal”, when truly creative opportunities come up, relish them but don’t cling to them.
Embrace it, make it work for you and remember how every genuine improvement you make is one more step along the road to “better”. And that in itself is creative.
There is art in refactoring
There is love in fixing bugs
There is rare beauty in using coding standards
There is satisfaction in fixing broken tests
There is joy in shipping release 29.5.36 of your successful but old product.
Some old gems I picked up at Agile 2010 from “Agile Test Automation Strategy for the non-Technical” by Gerard Meszaros (Author of xUnit Test Patterns).
These are still as relevant today as they were 5 years ago.
First, there are 2 critical definitions to provide context…
Legacy code:Code that does not have automated unit tests.
Legacy products:Products that contain legacy code.
However – a legacy product does not have to contain only legacy code.
Now, considering the differences between legacy product & legacy code think about your work on “legacy products”.
If you are adding new code, where do you add it and what testing do you do?
If you are modifying legacy code, how do you treat it and what testing do you do?
Here’s a quick refresh of the testing pyramid concept…
The (grossly simplified) ideal world view is that you have a large number of automated (fast) unit tests as a safe platform upon which you build a suite of automated and manual functional tests and at the peak you have your broader system tests.
Most legacy products and systems have this pyramid inverted with heavy investment in system test and UI automation, insufficient small functional tests and few if any unit tests.
Great. So What?
Stop trying to “Correct” the testing pyramid.
Rather than wasting our constrained capacity trying to automate everything, we must understand that legacy products have a testing “sweet spot”.
Focus testing and test automation strategy and investment on the areas of highest risk and highest ROI.
Even at the uppermost level, automate the things that steal most of our time first. This frees us up to do more value-added work.
Do not try to retrofit unit testing into legacy code.
To paraphrase Gerard, I distinctly remember him saying “Don’t waste your time unit testing legacy code”.
Instead, invest in the middle layer – the small automated functional tests.
This is because for the volume of code you are working with and its maturity, small functional tests will provide you a far higher return on investment than the same effort spent unit testing legacy code.
Given this is from the guy who literally “wrote the book” on unit testing this surprised me at first. It all becomes clearer in the next 2 points he makes!
(This is where the difference between legacy code and legacy product is key)
If you’re adding new code to a legacy product, unit test it.
Add a procedure or method call from the legacy code to the new code and then treat the new code as independent new code.
This practice also encourages better coding & architecture practices with greater readability.
Whilst the ROI on retrofitting unit tests to legacy code is low, the unit test effort for new code is recovered in prevented defects, less review issues, faster build & test feedback and simplified code maintenance.
Legacy code modifications should be refactored into new code.
If you are modifying legacy code, use good refactoring patterns to turn it into new code.
Now it’s new code, it’s small, unit test it!
The “Extract Method” refactoring pattern is of particularly high value here. Essentially you have a fragment of code within a method to fulfil a particular purpose. Pull it out into its own short well-named method and treat it as new.
Avoid merciless or repeated refactoring.
This point came from Naresh Jain during his time working at Industrial Logic but fits here…
Refactoring is a discipline, don’t get carried away.
Understand what refactoring really means, study the patterns involved and use them with caution.
Wholesale rewrites are not refactoring.
I’d argue that all developers will work on legacy code in their careers so I strongly recommend a complete read of these…
When I first joined Red Gate nearly 4 years ago my first major project was working on a replacement system for managing marketing and transactional email for our customers.
Yes – email marketing – and boy did we suck!
Back in summer 2011 I kicked-off some team research, analysis and story activities on one major enhancement we wanted to develop – “Subscription Management”.
This sounds pretty straightforward right?
This is a sensitive area involving a series of tensions between international compliance & standards, good end-user experience, the working lives of a marketing team and ultimately the reputation of the company.
Let’s put this into context of a few major groups of users (this is just a subset)
1: The “Customer”
I bought your software once, I pay my annual support & maintenance, I’m reasonably happy with what I have. Let me know when there’s a free upgrade or my renewal is due but don’t sent me any more damn marketing or I’ll report you as a spammer.
2: The “Tourist”
I attended an event sponsored by you at some point in my life and since this I seem to sporadically receive mails from you or one of your affiliate companies. I’m interested in the community stuff and freebies so I might read some of the mails you send but if they’re not directly relevant to my interests I want to get off of whatever list/sub-list I’m on fast.
3: The “Fan”
I bitched about your company when you acquired a free product and turned it into a paying one but I’ll concede you guys have done an awesome job and I already used a few of your other tools anyway.
I’m always on the lookout for things that make my life easier and I’m not averse to finding out about the new developments you have in the area I work in.
If you send out something interesting I’ll even share it.
These people are worth looking after & listening to but they’re a fickle bunch – use with caution.
4: The “WTF”
How the hell did I get on this list? I’ve never heard of you, stop sending me spam!
It turns out they attended an event or downloaded some freebie from an affiliate site an their details found their way into the system or into the hands of a marketer who moved project, role or teams but kept their contact “because it was closely related”.
5: The “Prospect”
Yup, that’s right – you might actually be sending mails to real prospects!
What are they trying to achieve right now and what’s the most useful thing you could offer them?
If that’s not in the mails you’re sending, expect an unsubscribe. Staying in there inbox “to keep them aware” is probably not going to improve their opinion of your products.
6: The Marketer
I’ve been sending mails to this specific subset of the company-wide install base for years. I have my campaigns carefully planned with content, timings, massive target audience etc. I’d love to see the campaign convert into direct revenue but I’m more interested in getting it out of the door as one small part of my overall marketing strategy for the quarter.
This should be the easy bit, right?
What do you mean everyone unsubscribed because half a dozen other marketers hit them with irrelevant stuff over the last month as well?
Yup. That really used to happen here.
So the great thing is that with a little research, designing a great user experience around email subscriptions is surprisingly easy. There’s a basic set of users with competing and/or overlapping needs but more importantly. When someone doesn’t want your marketing mails, that doesn’t mean they hate you. It’s not a break-up, it’s not the end of a beautiful relationship, it’s simply someone asserting their needs.
So many companies manage to screw this up because they focus only on the needs of the marketer.
Make opting out a positive experience and chances are if a contact has a reason to talk to you (or someone like you) again they’ll be in touch. If they don’t – well, there’s no point in them being a contact still, right?
Spend a bit of time thinking about the more negative interfaces your customers, users and prospects have with your company.
What small steps can you take to raise the bar on these?
Offsite meetings have a checkered past. Back in 2006 a colleague shared this painful article with me. To quote from the main article:
“…only about 10 percent of executives consider offsites truly valuable. Half said they aren’t worth the time or money.”
Over the last 7 years I’ve seen the exceptionally high value offsite meetings can bring but also the stress and pain that often leads up to them.
At my last employer, the entire global executive and senior management team would have a 6-monthly operations review offsite that ran for 3 days. We booked out significant portions of a large hotel in the US and fly the entire team together from over a dozen sites around the world. Over 3 days, the leaders from each site would present their statuses, finances, roadmaps, portfolios, visions, strategies and plans to a group of about 50 of us. We’d put them under the grill and review their work in depth both as bosses and peers.
We’d take time to focus on specific strategic issues that benefitted from face-to-face focused collaboraton. (When your management team are distributed around the world. A few intense days of co-location often delivers far more decisions and clarifications than 6 months of project work and conference calls.)
The general approach for these was: Prepare, Pitch, Review, Collaborate, Revise, Re-pitch.
Sounds pretty sensible? – It is… … if you know what you’re in for.
The offsite workouts were usually brutal but exceptionally valuable. By the end of the week everyone’s plans were in far better shape, we all understood what each other were doing, found areas for shared benefit and made a lot of difficult decsions.
Better still, we spent a few days working, eating and drinking together. 3 days of deliberate convergence in order to stem the behaviour issues associated with extreme divergence kepts us functioning as a successful management team.
Whilst the outcomes were great, they took their toll. The pain behind the scenes generally involved a bunch of us working 2 weeks of long evenings and weekends to pull all the data, presentations and spreadsheets together. ($100m of software projects contain a lot of moving parts & data) Often we’d still be revising our work during the start of sessions and whoever went first bore the brunt of painful lessons and questions whilst everyone else frantically updated their slide decks to accomodate the new knowledge.
And here’s the thing – the lines of questioning…
Our Senior Exec at that employer was an exceptionally sharp, experienced, bright guy. After working through multiple iterations and having joined the operations team on the organizing side of the offsites I saw what made him tick. He had a brain full of powerful questions that he’d learned from experience and was constantly adding to them. He knew what to ask and when that would lift the lid on just the right barrel to find a body (if there were bodies to be found).
As a spectator, sometimes it’s satisfying seeing the blood on the floor when someone you think is an ass-kisser takes a roasting. It’s not so fun when it’s your own team.
I even helped develop a set of “difficult questions” for our Head of Operations in the weekend prior to one of these offsites. They were great questions to ask. The thought and effort put into developing them was a lot like spending an hour writing test cases up-front. We learned quickly what we really needed and wanted to know (and why). We were confident we could get under the covers of even the most prettied-up project report.
But we could have made the whole thing so much less brutal.
Much like writing test plans, if we’d just put the questions out for participants to review in advance they’d have adjusted their research and reporting to accommodate them rather than playing project question battleship on the day.
So here’s an action for you.
Next time you’re reviewing a project or a piece of work, capture the questions you regularly ask.
Share them with the people you’ll be asking beforehand.
Now they can answer those questions sensibly first and you’ll all have time to dig into more valuable insights.
I have a few more posts planned around “critical questioning”, plenty around running offsites and I’ll be sharing where I’ve been for most of the last year so keep your eyes open for more soon.
First of all, I’m flat-out at work right now on recruitment and running a challenging project so here’s a plug for some jobs…
We’re looking for 3 amazing Agile PMs here at Red Gate. If you think you’ve got what we need and fancy working with me, please dive in and apply.
 – actually we’re hiring for more than a dozen different roles you might expect in a software company! – Take a look at the rest too.
Second. I’m currently supporting part of the SQL Source Control development team here on a rather challenging technology refresh project. We’re planning to ship an Alpha soon to validate our technology decisions and then press ahead with completion and new features. I can’t say much more about this at the moment but despite being relatively small by most company standards, it’s one of the trickiest projects I’ve ever inherited. Once we’re successful, I hope to write more.
Finally – back to the title…
Forgive the Portal reference but for the last few months I’ve been using my spare time coding a game.
If you’re of a certain age (like me), you may fondly remember hours of bashing away trying to solve puzzles and guess verbs in black and white (or green screen) dungeon/text adventure games.
I grew up coding on a BBC micro and developed an early love for interactive fiction. It’s now a pretty niche area with some amazing games.
Whenever I want to learn something new, I need a goal – something concrete. So I decided to write my own but using slightly more modern technology. If you’re interested in why, see my last post.
Progress has gone really well and I think the results so far have some promise. There’s a few unique aspects of the game in comparison to the more old-fashioned equivalents. Essentially, the games characters and scoring are somewhat more dynamic and based much more on the actions of you as a player. I won’t spoil it but if you have the patience, give it a shot at: http://mvta.herokuapp.com/
Whilst not “finished”, it is live and that’s an achievement in itself.
I reconnected my developer empathy circuit, thoroughly enjoyed going back to coding and managed to learn some new skills on the way.
A few nerdy highlights…
I’m really impressed with Heroku as a hosting platform. I managed to go form zero knowledge to live and running in under 2 hours – including restructuring the codebase and packaging in order to support heroku deployment.
I’ve come to love Github for source control and got to grips with a modern IDE (Visual Studio) having last used Eclipse back in around 2007!
I’ve learned to value real user testing and usage reporting. A large part of the game usability came from observing the attempted and failed actions of users playing the game.
I still prefer writing business logic over coding the “plumbing” but did occasionally have to drop back to some old-school computer-sciency work to achieve what I wanted.
Most importantly, whilst it’s a well-tested and working game and engine, I built a nasty legacy codebase that forces me to resort to writing tests and reworking/refactoring whenever I want to add something new and facing the pain of missing tests and regressions as I go.
I’ll continue updating the content over time but please dive in, have a play and waste a few hours.
Oh… and post comments here if you need “guess the verb” type hints or clues!