Home > The Lean Post> What Do I Tell My Leaders When Experiments Fail?
The Lean Post
Sharing how the world is making things better through lean.

What Do I Tell My Leaders When Experiments Fail?

by John Drogosz
August 23, 2016

What Do I Tell My Leaders When Experiments Fail?

by John Drogosz
August 23, 2016 | Comments (2)

I’m dealing with the fallout of a failed experiment with set-based concurrent engineering (SBCE). As the product development operations specialist, I understand that LPPD experiments don’t always result in success – but my leadership team doesn't. How do I help them understand that a failure in LPPD isn’t a total loss?

The first step we always talk about is, “What was the purpose of the experiment and were we very clear with the management team about what we were trying to prove or disprove?” A lot of times at the start of a project we communicate, “We’re going to do an experiment on set-based design to see its benefits,” and we leave it very open-ended. And like anything that you leave open-ended, that leads to different expectations from different people in the organization – those who are more familiar with LPPD obviously will have a more realistic expectation of what will come out of set-based design; senior managers will certainly be more interested in the end result, which may happen in the short term or, in accordance with product development, may happen years later.

So from their perspective, if they’re not seeing instant results, they may see the experiment as a failure when the jury may still be out. Or this may be a failure in that there was an experiment conducted with a specific target outcome, and we didn’t achieve that.

In the latter case, it’s important to remember that there is ALWAYS a silver lining that occurs. Very rarely do we have a complete and utter lack of learning in an experiment. If we do, we probably didn’t do something or just stuck with the status quo.

The first thing to usually do is do a good reflection with the experiment team to analyze what went well and what did not.

Then you can hold a report-out with the management team to share your findings. And I think this is where we start getting some clarity that the glass is either half-empty or half-full.

I remember a time where we had a failed experiment that was viewed as a failure. But when the group went through the reflection, it wasn't quite as much of a failure as it was perceived. One team did their first set-based experiment on a given project and partway through, the regulatory requirements kind of changed up the project. And when the requirements changed, the parameters changed as well and the team was forced to change their design direction at that point.

Now the perception by the organization – the "talk around the water cooler," if you will – was that the team was really struggling with set-based design since they’d obviously just had to change directions. They thought that was a failure in set-based design.

Interestingly, when we sat down with the team to discuss this failure, the team did not call it a failure – they called it a hiccup! They said, “Sure, it’s true we changed design direction but only with two out of the five sets we were working with.” But the real perception from the organization on why they thought the team had failed was because the project’s cost metrics went up. People interpreted the result not being achieved as a reflection on how poorly the team executed set-based design. But once the team had set the record straight with the leadership team, the response was “Oh. That makes sense. No problem – keep working on the project.” One of the directors even repeated the team’s explanation to the rest of the organization.

It’s my experience you need at least three or four experiments before organization starts to deeply understand the nuances of SBCE. And they learn enough from their successes as well as their failures to be able to really define their own way of what SBCE looks like in their own product development community.

Now, let’s talk about what you can do to manage your leadership’s expectations while the experiment is going on. You don’t want to wait until the end of the experiment to tell management about your progress. If you do, you may be disappointed, and frankly it may be several months or years – and as we all know, management likes to see results quickly.

You want to have a series of certain touch points when you’re going through the process. When we’re doing an experiment in product development – or in lean in general – we should be just as interested in the process as the metrics. We should be checking in with management more periodically when we’re going through the process in order to inform them when the process is in control or deviating. And that’s certainly where it can be helpful to have a steering committee or some sort of leadership champion that you can go to, because if the team has a problem, there needs to be a specific organization or group they can escalate it to when needed. It is a very effective way to avoid telling management about your “failure” in retrospect.

The views expressed in this post do not necessarily represent the views or policies of The Lean Enterprise Institute.
Search Posts:
Improvement Kata / Coaching Kata
Elizabeth Carrington, Mike Rother & William Costantino
Kaizen Express
By Toshiko Narusawa and John Shook
Lean Product and Process Development, 2nd Edition
By Allen C. Ward and Durward K. Sobek II
Was this post... Click all that apply
8 people say YES
12 people say YES
6 people say YES
5 people say YES
Related Posts
2 Comments | Post a Comment
Mark Graban August 23, 2016
1 Person AGREES with this comment

This goes to show why it can be hard to transition to a Lean culture (or even use Lean methods) in a traditional organizational culture.

Does the hiccup or "failure" mean that the group just hasn't gotten good at SBCE or LPPD yet? Is a learning curve natural and to be expected? 

Is any baby expected to go from crawling to running a marathon?

Does a surgeon have mastery of a certain procedure after one try?

We can try to coach up to our leaders... to teach them about PDSA, the need to learn through "failure" and they might instinctive respond with platitudes like "failure is not an option!"

Sometimes the system and culture we're a part of swallows up our best Lean efforts and best Lean intentions. That's not your fault and it's not Lean's fault.

Why do attempts by organizations to embrace any new methodology or technique often fail?

Lean doesn't work in this company? Checklists don't work in our operating room? SPCE doesn't work here?

Or we haven't figured out how to make it work yet?

Reply »

Lev Ono September 04, 2016

This is a good example of how a highly structured process for experimenting and learning such as SBCE can help a team reach a different attitude toward "failure" after only two to four experiences. Very healthy they called the failure a "hiccup". Setting expectations for senior sponsors is always more difficult unless the are in on the "experiment" from the beginning. 

Reply »

Search Posts:
Improvement Kata / Coaching Kata
Elizabeth Carrington, Mike Rother & William Costantino
Kaizen Express
By Toshiko Narusawa and John Shook
Lean Product and Process Development, 2nd Edition
By Allen C. Ward and Durward K. Sobek II
Frontloading Product Development
The Designer's Dilemma
Developing Your Obeya, Stage-by-Stage
Please include links as plain text URLs only. Do not copy and paste directly from a web page or other document. Doing so may pick up additional HTML that will not function here.
URLs will be converted to functioning links when your comment is displayed on the site.
Here's an example:
See this article for more details: https://www.lean.org/whatslean