Home > The Lean Post> The Road to Production Hell is Paved with Lack of LPPD

The Road to Production Hell is Paved with Lack of LPPD

by Jim Morgan
April 27, 2018

The Road to Production Hell is Paved with Lack of LPPD

by Jim Morgan
April 27, 2018 | Comments (15)

By now most everyone is familiar with the quality and delivery problems that Tesla has had with its Model 3 product launch. Even CEO Elon Musk acknowledges its subpar performance, famously referring to the “production hell” of late parts, poorly performing processes, worker injuries and reworking builds as just a normal part of its product launch process. They are not. While launches are never easy, no company that I know has these kinds of problems. Not only is Tesla months behind schedule, but it is still experiencing the kind of major build quality issues that should have been resolved months, if not years ago. Build issues recently identified by automotive engineering and benchmarking firm Munro and Associates in an Auto Week article by Jay Ramey were astonishing. Munro’s CEO remarked while reviewing a recently delivered Model 3, “look over here I can barely get my fingernail in this gap and on the other side I can just about fit my thumb.  This is very unusual; the tolerance stackups on this car are just like nothing we’ve ever seen before.

I was recently reading a Bloomberg View article by Edward Niedermeyer in which he called out the automaker for poor quality, missed deadlines and large scale re-work. “Tesla is taking automotive manufacturing back to the dark ages,” he said. Then Niedermeyer correctly identified the introduction of the Toyota Production System (TPS) as the force that dramatically transformed performance in automotive manufacturing. What’s more he even correctly stated the Toyota way emphasizes fixing the root cause of problems: “rather than trying to more efficiently repair defects, which by their nature vary wildly and thus confound standardized processes, the Toyota way emphasizes fixing the causes of the defect.”

So, I’m reading along, nodding my head and thinking, “yeah, it’s about time someone called this out.” And then bam! The End. The author falls into the same trap that Tesla apparently has: thinking that you can cure these issues in production.  Just do TPS – problem solved. Not hardly. 

I did not write this to criticize Tesla. I hate when people pile on a company that is struggling. I spent many years at Ford during some of its most challenging and difficult times and I know exactly how it feels. It is not helpful. In fact, I almost did not write this piece for that very reason. However, I believe that there is an important lesson here for many companies:  That the time to address these issues is in development. All of the shop floor heroics going on at Tesla do little to get at the root of the problem, and they quickly burn people out. What’s even worse is that the engineers that are trapped in production hell are not available for the next program, almost guaranteeing that program will struggle thus beginning of a classic development death spiral.

Do not misunderstand me. TPS is an incredibly powerful manufacturing system. But once you are at launch, your tools, fixtures, processes, part designs, interfaces, and requirements - they are all done. The degrees of freedom remaining for the manufacturing folks are now just a small fraction of what they were during the earlier development process. Front loading your development process has long been a fundamental tenant of lean development. And the front end of the development process is where rapid learning cycles through targeted experimentation should take place. Not during the launch, let alone production. While problems may be rich learning opportunities, when it comes to launch the best problems are the ones you never have.  

Elon Hell

Development is an enterprise activity

Designing, building and delivering great products is an enterprise activity that involves engineering, quality, marketing, manufacturing as well as key suppliers. And not as some kind of “bolt-on” activity, but as full partners with real equity and responsibility. One of the characteristics that differentiates Lean Product and Process Development (LPPD) from traditional approaches is that it enables you to create new value streams, not just a product in isolation. One of the ways it does this is by engaging the enterprise early in the process in a structured and collaborative way through the obeya management system and a process that enables the principle of compatibility before completion (CbC). 

In this way companies are able to identify and work through cross-functional issues early in the process while the solution space is largest. Nowhere is this more important than with manufacturing where the ability to apply manufacturing know-how upstream effectively is a powerful competitive advantagein producing both great products and manufacturing processes.  Perhaps not surprisingly, Toyota excels in this capability. But so do companies like Apple who create “insanely great” products and equally phenomenal manufacturing capability.

Although it may not seem like it, I am actually rooting for Tesla. Their thinking and their products have challenged the status quo in the auto industry much as Toyota has done many times in the past. And that is good for the industry and the customer.  So I hope they continue. I know they are smart and there is evidence that they are learning. In a recent tweet, Tesla’s CEO said “excessive automation was a mistake. To be precise, my mistake. Humans are underrated.” This is encouraging and sounds like a person that is open to learning. But the time for urgency, the time for bringing your sleeping bag to work is during development – not in production. So we just have to get him and his team to look up stream they may begin to understand and embrace the principles and practices of LPPD. That will finally be the beginning of the end of Tesla’s production hell. 

So what about you? Are your launches a little hotter than they need to be?

(Jim Morgan will be speaking at and hosting the Designing the Future summit this June 19-20, featuring speakers from Ford, Toyota, Rivian, Menlo, MIT and more. See the full agenda: https://www.lean.org/designfuture2018 )

The views expressed in this post do not necessarily represent the views or policies of The Lean Enterprise Institute.
Search Posts:
Value-Stream Mapping: a Methodology for Sustainable System Improvement
Jim Luckman, Judy Worth, Karl Ohaus & Tom Shuker
Learning to See
By Mike Rother and John Shook
Was this post... Click all that apply
HELPFUL
23 people say YES
INTERESTING
36 people say YES
INSPIRING
19 people say YES
ACCURATE
28 people say YES
Related Posts
15 Comments | Post a Comment
Michael Ballé April 28, 2018
8 People AGREE with this comment

Hi Jim,

Thank you for this significant post. I have no doubt in my mind this is the most important topic in lean today. And yet, we fail to capture attention as everyone is still focused on solving operational issues to... well, solve operational issues rather than capture the learning and reinject it into product and service design in order to increase customer value at the root.

Completely agree that the anti-Tesla bandwaggon is not helping. They've made incredible breaktrhoughs in over-the-air technology, and, yes, have ignored the heritage of the auto industry at their peril. What remains is that they're pushing everyone very hard, and this will benefit all eventually.

How can we give the VA/VE dynamic greater visibility? How can we better explain that delivery excellence is key to... discovery excellence. Every problem solved in manufacturing has process and product design implications, because this is finally fact, not idea: where the rubber meets the road, so to speak.

Let me know if I can do anything to help?

Reply »

John Shook April 28, 2018
1 Person AGREES with this reply

Downloadable upgrades is a fun piece of technology, borrowed from the adjacent smart electronics industry, but is an easy add-on, not an innovation that gets to the heart of the matter.  

Reply »

Marco Castellanos April 28, 2018
1 Person AGREES with this reply

Mr. Shook, I really like your work and have been following what you do for some time now. However, on the topic of software updates over-the-air, I respectfully disagree. Working at one of the main automotive micro controller suppliers, I see that all OEMs are struggling massively with the introduction of this technology: it forces everyone in the value stream to significantly change their processes for software development, validation, and create new security structures to ensure this software upgrade ability does not create an open door for hackers.

This is one area where Tesla, having designed their electrical architecture from the ground up to support over-the-air updates, have an edge over other manufacturers who have to deal with immense hurdles to change a very complex supply stream and vehicle ECUs designed to be left alone once the car leaves the dealership. To this day, not even Toyota are able to update but a handful of their ECUs over the air. This subject is more complex than it looks, and is the subject of intense research and work at the moment and for the next few years.

Reply »

Jim Morgan April 28, 2018
4 People AGREE with this reply

Hi Michael,

Thanks for your thoughtful comment and all you do to support the continued growth of LPPD.  I completely agree that this is the most important issue facing the lean community today.  Fortunately, after many years of struggle, LPPD is finally gaining traction at an impressive rate.  Organizations in automotive, aerospace, consumer electronics, energy, healthcare and many more are finally moving upstream to take on these issues with encouraging results.  I hope you will continue your important work in this space and continue to help the lean community understand how critically important this truly is.

Best Regards

Jim

Reply »

John Shook April 28, 2018
11 People AGREE with this comment

What an excellent and timely post this is, Jim. Stories of Elon’s production hell are alarming, to say the least. Worse, I don’t know how unusual his situation really is. Your argument – that TPS on the plant floor can’t cure upstream evils – is a powerful insight for most companies I know.  

Taiichi Ohno’s TPS is a wonderful thing and zeal for it is an understandable and good thing. On the plant floor, Ohno and his team never talked much about “value.” They had plenty of waste to do something about and they knew what to do about it, which kept them plenty busy. Addressing the question “what value are we producing?” wasn’t really necessary; just eliminate the mountains of waste at its manufacturing source and things would get better and better. And so they did.

Ohno and team didn’t need to focus on value because someone had already done that for them. Value and the cost of producing that value was designed in by the product & process development team before the product and most of the processes even hit the plant floor. This entailed more than relatively simple DFM and 3P exercises.  

Interestingly, as the production team didn’t concern itself much with existential questions of value, when you go upstream and talk with Toyota’s development teams, they don’t bother themselves with worrying too much about waste. Design a great product with a great value proposition and the right cost equation versus price – and sell your product (don’t forget that little part) – and you will make a profit.  Durward Sobek, in the work he did with Al Ward to understand Toyota’s approach to product & process, noted 20 years ago that Toyota’s development engineers never even mentioned the word “muda” or waste.

Fast forward to today’s stories of Elon and his production hell. If you design a mess, you will be dealing with a mess. “Kaizen” as rework and all that. To your final question, launches I observe are indeed hot. Looks like lack of lean product and process is paving a lot of roads.  

Reply »

Jon Miller April 28, 2018
1 Person AGREES with this reply

Contrarian view here. I doubt that LPPD would have made any difference at Tesla, had they done it prior ot launching.

No amount of LPPD tools, workshops, and well-intentioned activities counteracts a strong-willed & brilliant, founder-CEO. Musk was hell-bent on the automation route, on "schooling Toyota". Not so much on doing the produent, rational, or proven (boring) thing.

Putting LPPD findings into practice only happens in the context of management by fact, where leaders put customer value first and have the humility to listen.

"Lean solves, because tools!" will continue to leave us scratching our heads as to why smart leaders don't call us to help them.

Reply »

Jim Morgan April 28, 2018
6 People AGREE with this reply

Hi Jon,

Thanks for your comment.  You may be right about Elon Musk - I don't know him personally.  However I suspect he is a pretty bright guy and wants Tesla to be successful. So I will continue to hope that he is able to learn from this experience and make the necessary changes to their development practices.  More and more brilliant leaders are indeed pulling on LPPD to make their organizations more competitive.

I believe the industry needs Tesla.  In fact it needs more companies to push boundaries and challenge the art of the possible.  

Best

Jim

Reply »

Michael Ballé April 29, 2018

Jon, great point, and great challenge

The jury is still out on Tesla, but whatever happens, do we agree that Musk's ballsy initiative has moved the lines?

But I'm not sure we get this right either, our default mode tends to be process, not people. In this case, Musk was really terribly upset when Toyota pulled out of the joint venture after the failure of the full electric Rav 4 - Anger is bound to have clouded his judgment about anything Toyota related (and the need to prove them wrong).

Secondly, the deeper lean issue with any engineer, is what they get and what they don't - which technical issues they do understand and what they don't. The tear downs available on-line of Tesla cars are an excellent demonstration of that.

The real issue is not so much listening - Musk has proven himself very capable of listening - but knowing who and what to listen to, and what to ignore with the current overwhelming level of noise (and imagine what it must be like for a guy like him).

The challenge I see for LPPD is not about leaders, but about becoming more listen-worthy: who of us should Musk have listened to? What would we have to say that would have made sense to him? How would he find out about it in the first place?

Your thoughts?

Reply »

Jon Miller April 29, 2018
1 Person AGREES with this reply

Agreed. Hope Musk succeeds in his Boring, Space X and other visionary ventures as well.

Brilliant intellect is useless if it's clouded by emotion, especially in a leader. Self-awarenes helps intellect learn and correct.

How to reach the learner, what to say, how to say it - good questions, but not the ones I would ask. It should not even have been necessary to reach the Musk about the message of LDDP. Why is the CEO getting so far into the weeds? Because he is an engineer? Because he wanted to "school" Toyota? Ego? A good CEO puts competent people in charge and stays out of the way, letting them do their jobs, finding the best practices in product launch and ramp up, or whatever area. LPPD is out there, not hard to find or to understand. The long-term countermeasures is not how we get Musk and other CEOs to notice LPPD, but how we teach leaders to set the vision, direction and expecations, empower and support but stay out of the weeds.

Jon Miller April 29, 2018

Agreed. Hope Musk succeeds in his Boring, Space X and other visionary ventures as well.

Brilliant intellect is useless if it's clouded by emotion, especially in a leader. Self-awarenes helps intellect learn and correct.

How to reach the learner, what to say, how to say it - good questions, but not the ones I would ask. It should not even have been necessary to reach the Musk about the message of LDDP. Why is the CEO getting so far into the weeds? Because he is an engineer? Because he wanted to "school" Toyota? Ego? A good CEO puts competent people in charge and stays out of the way, letting them do their jobs, finding the best practices in product launch and ramp up, or whatever area. LPPD is out there, not hard to find or to understand. The long-term countermeasures is not how we get Musk and other CEOs to notice LPPD, but how we teach leaders to set the vision, direction and expecations, empower and support but stay out of the weeds.

Jim Morgan April 28, 2018
5 People AGREE with this reply

John,

Thanks for your very insightful comment and all you have done to help us better understand LPPD.  You are right, the focus on creating new value underlies the shift in lean thinking up stream.  And to do this well organizations must learn to enroll the whole organization in creating new and successful value streams.  That capability will increasingly define exceptional lean organizations and represent both a major step toward a lean enterprise and a powerful competitive advantage.  

Best

Jim

Reply »

Michael Ballé April 29, 2018

Hi John, Jim,

Waste - fascinating issue. One I believe we should clarify for the LPPD community.

Yes completely agree, you don't hear much talk about waste in the design side at Toyota (the little I've come across on the dev side), BUT, as I understand it, there is a complete cost model of the project before the concept is even considered.

My understanding is that the expectactions for the car are set in terms of both sales and profitability - with a very detailed cost model - before design work proper begins. Which leads to testing critical assumptions early, in terms of both design and production technology. One of these assumptions being on which line the future car is going to be produced, with the adjacent constraints- hence muda.

Did I get this right?

I had a chat with a Honda engineer who considered Toyota engineers not that exceptional compared to Honda, but was impressed with their business agressiveness in terms of cost and line-up positioning.

I don't know how you feel about this, but my sentiment is that we're still very stuck on Ward's original insights of Set-Based Concurrent Engineering and trade-off curves. This is great, but my hunch is there is so much more to this, starting with the role of CE as you yourself has mentioned, the development path of great engineers, the kentou work of validating concept, the link to zero EC after tooling and production change and so for.

How do we continue the map the full range of the lean development, from the business approach to detailed design? 

Reply »

Francesco Culos June 04, 2018

Dear all,

First of all... thanks  for the great value I am receiving in reading your posts.

If I can remember well in  Ward's book the role of CE was described (simplified) as a kind of project manager with entrepreneural approach. Could this be the joint between the value adding and waste removing? I mean: if I am CE I am managing with entrepreneural approach so trying to design product and process with no waste, with a product satisfing customer and (so) a process that generates profit (main output).

The second important output will be usable knowledge (in terms of trade off curves or similar).

Thanks again for sharing

Francesco Culòs

Reply »

William Kaiser June 19, 2018

Jim, excellent post.  I have a question: Beacause lean on the floor has been around so long, do you find the many "A" (of of PDCA) collect into a pool of standards so constricting that lean process and product are difficult to initiate?  Meaning, standards that optimize manufacturing processes can often conflict with cutting-edge designs, the customer value?  e.g. the insaley great products that Apple delivers?

Reply »

Jim Morgan September 14, 2018

Sorry for my delayed response.  Excellent question - and a central challenge in achieving excellence in product development.  I think of the answer in terms of "fixed and flexible" elements of you product and your development system.  The sort of yin and yang of lean product development - two halves of the same whole.  The key lies in understanding how your product delivers value to your customer.  Where to apply hard earned knowledge often expressed as standards and where the product (or development system) will benefit most from innovation and/or customization.  Thanks for your question.

Reply »

Search Posts:
Value-Stream Mapping: a Methodology for Sustainable System Improvement
Jim Luckman, Judy Worth, Karl Ohaus & Tom Shuker
Learning to See
By Mike Rother and John Shook
The Designer's Dilemma
Building as Product