How to Design a Better Reliability Test Program
Contents
1. We tested a Neutrino Telescope for the National Science Foundation NSF This product could be tested with full functions only if all the pieces were working Rather than waiting for that we elected to start with some non operational testing on some fixtures we custom designed assembly and picture on right is the subassembly 4 CONCLUSION Reliability test plans are often generic or blindly following industry standards Test plans must be tailored to fit customer use profiles FMEA is a development tactic that can help solve the problem of testing too little by uncovering failure modes that require tailored test methods rather than only cookbook methods from industry standards Also reliability testing often occurs too late in the process There are many methods we can use to start the testing process earlier in order to get better results faster ERT is a development tactic that can enable earlier feedback smarter engineering less total cost and lower risk for reliability Thus ERT offers better final product and better reliability BIOGRAPHIES Mike Silverman CRE Ops A La Carte LLC 990 Richard Ave Suite 101 Santa Clara CA 95050 e mail mikes opsalacarte com Mike Silverman is Managing Partner of Ops A La Carte a Reliability Consulting firm He has over 25 years experience in reliability engineering reliability management and reliability training He is an experienced leader in reliability improvement throug2. Accelerant Bulten Pushing taira Duty Cycle Force Angle Touchacreen fale Duty Cycle Location Forca Headed Charger Connector failure Duty Cyce Force Angle Bathory Figure 9 FMEA on a Cell Phone As you can see by these examples we would have missed many of the potential failure modes had we not used FMEA to help drive our test plan program 3 EARLY RELIABILITY TESTING Early Reliability Test ERT is a development tactic that offers earlier feedback and thus enables e more lead time e smarter engineering e better reliability and quality e less total cost amp risk ERT needs to overcome potential challenges samples from immature manufacturing low test coverage too few samples immature designs parallel concurrent designs integrated can t test until This will show how to overcome these challenges in many cases 3 1 Overcoming Immature Manufacturing Early generation specimens typically are just a few specimens made with immature manufacturing process On so few specimens often we can afford to augment this test with simple examination e g naked eye or simple microscope or even Failure Analysis This enables us to identify the cause of each failure e to exclude failures probably restricted to immature mfg e to include failures probably significant for mature mfg e Distinguish probably relevant failures versus probably irrelevant failures Relevant failures found early may be li
3. we tried using alcohol and found that it caused the plastic to become brittle causing it to crack over time Is this a valid failure even though we told the user not to use alcohol ABSOLUTELY Therefore we changed the plastic to a different type one that could be cleaned with alcohol I like to think of an analogy to your cell phone or mobile phone or hand held phone or handy depending on where you are from How many people have every dropped their cell phone from over 3 feet in height Ill bet that most of us have at some point in the life of our cell phone And did the phone still work TIl bet that most of you answered yes If you look at the drop height specifications for a cell phone they are to be able to withstand a drop of 3 feet in a packaged container and just a few inches in an unpackaged situation So clearly we are out of specifications when we drop from higher than 3 feet So why did the phone still work Because the cell phone manufacturers have anticipated that users will drop the phone from these heights and they have subjected the phones to the process of Highly Accelerated Life Testing HALT whereby they test the product beyond the specifications to determine and expand product margins as much as possible This is what we should all do with all critical parameters of our products Anticipate what the user may do and then make sure our product can withstand that Forget about specifications Forget about liability Mak
4. How to Design a Better Reliability Test Program Mike Silverman Ops A La Carte LLC Key Words FMEA Failure Modes and Effects Analysis ERT Early Reliability Testing HALT Highly Accelerated Life Testing ALT Accelerated Life Testing SUMMARY amp CONCLUSIONS More and more industries are competing on reliability and companies need to develop more reliable products faster However reliability test plans are often generic or blindly following industry standards Test plans must be tailored to fit customer use profiles Also reliability testing often occurs too late in the process Tests and improvements often are performed when e time is short e development is nearly complete e engineering corrections are difficult and costly e the product is nearly frozen This paper will offer a solution to these two fundamental issues of e Testing too little solved with more robust test plans e Testing too late solved with Early Reliability Testing 1 INTRODUCTION In order to write better test plans we must first understand the use environment the key risks to the design The best tool for this is FMEA Once the risks have been identified and prioritized it is time to develop mitigations Often times the best mitigations are with reliability testing More and more companies are using FMEA as an investment which saves them money over the life of the product However despite its enormous benefits the FMEA can b
5. e sure the product will work to what the user will do Because if the phone does break and the user returns the phone he she will expect a refund If one is not given then you will lose that customer forever because there are plenty of cell phone manufacturers who manufacturer phones that will survive this type of abuse That is where reliability and competition intersect So many companies are making their products more reliability because their competitors are doing the same Pen Test What do we mean by a pen test Well this inhaler is small enough to fit in your pocket What else might you have in your pocket when you put this in your pocket a pen a set of keys coins A bit of background about this device The device operates by passing a liquid from a canister through a disk with tiny holes in it The size of the holes dictates the size of the particles being delivered This disk in fact was the majority of the intellectual property IP of the company and also the majority of the expense of the product If the disk gets damaged you basically must throw away the entire device So if a pen comes in contact with the disk this is a major issue We could design the product with a cover but who replaces the cover on their pens or markers Pens are made to be lost or forgotten They get in the way when we need to use something But what about if we tether the cap Will that solve the problem It certainly could Lipstick Test What abou
6. e very tedious time consuming and labor intensive Thus the method has become unpopular among many design engineers and it is viewed as a burden as oppose to a useful process To remedy this problem a generic FMEA methodology can be developed that overcomes problems such as development time and cost A library of generic FMEA has been created for various components that can be reused for different products We will provide a number of case studies and exercises on how to think out of the box in order to create more relevant and meaningful test plans We will review test plans created both with and without an FMEA to show the differences ERT is a development tactic that offers earlier feedback and thus enables more lead time smarter engineering better reliability and quality less total cost amp risk ERT needs to overcome potential challenges e samples from immature manufacturing e low test coverage e too few samples e immature designs e parallel concurrent designs can t test until integrated 2 DEVELOPING BETTER TEST PLANS 2 1 FMEA Failure Modes and Effects Analysis FMEA is the process by which we explore potential failure modes and then prioritize by key risks Once the risks have been identified and prioritized it is time to develop mitigations Often times the best mitigations are with reliability testing Stated another way we cannot know what to test for unless we understand t
7. ent use a few specimens from an early generation to develop test technology amp resources such as test apparatus test methods test analysis test acceleration techniques test monitoring methods ERT often can provide earlier understanding of causes and mechanisms for defects wear fatigue and failure that otherwise would degrade the final generation Also ERT test enables much more lead time for reliability work longer test runs milder acceleration easier extrapolation minimized schedule driven compromises easier and smarter follow on engineering 3 3 1 Case Study for Low Samples We tested a 100K cooling cabinet with multiple subunits These were separated and tested as individual subunits Spares were used only for a a few of the critical subunits rather than as second copy of the entire system oo gt k x Py gt Y AY ees a X lt 4 an ae tye pes late ae cc Figure 12 Testing on a Storage Cabinet separated into subunits 3 4 Overcoming Immature Designs For immature designs we can again use HALT for early discovery of qualitative design defects This will accelerate design maturation The goal of this reliability test is qualitative learning to uncover problems rather than quantitative learning to pass final generation The earlier we test and uncover defects the more time and money we will save which can be used partially to allow more time and money for later quantitative test
8. for Overcoming Immature Designs For an electro mechanical medical device during PO we knew that life test was premature Instead during Pl we tested margin and characterization to prove design repeatability During P1 this was more feasible and valuable than testing until manifest failure We set up a high speed camera on the mechanical assembly and tested during hundreds of runs on several products Thus we measured the repeatability of the mechanical design Figure 15 High Speed Camera Equipment used to Overcome Immature Designs 3 5 Overcoming Parallel Development Parallel or concurrent design amp development requires mating two or more subunits as prerequisite for meaningful test data This impedes test prior to integration Nevertheless we still can test earlier although not as less early as serial development of subunits Start test as soon as subunits are ready for integration Don t wait for SW or diagnostics to be complete Just make sure you have a way to functionally test unit Worst case is inability for functional test of two subunits Once subunits are ready for physical integration we can test these non operationally This less desirable than full functional test Nevertheless we still can perform vibration tests to find resonant frequencies This may point out many things including component interference issues mounting issues and board layout issues 3 5 1 Case Study for Overcoming Parallel Development
9. h analysis and testing Mike is also an expert in accelerated reliability techniques including HALT and HASS Through Ops A La Carte Mike has had extensive experience as a consultant to high tech companies and has consulted for over 500 companies in over 90 different industries Mike has authored and published 15 papers on reliability techniques and has presented these around the world He has also developed and currently teaches over 30 courses on reliability techniques Mike is a Certified Reliability Engineer CRE through American Society for Quality ASQ Mike is a member of ASQ IEEE SME ASME PATCA and IEEE Consulting Society Mike is currently the IEEE Reliability Society Santa Clara Valley Chapter Chair
10. he key risks Therefore FMEA is one of the best sources of input for a Reliability Test Plan FIGURE 1 Inhaler used during FMEA If we tried to develop a test plan without the use of FMEA what tests could we think of for this device We used the IEC standards and came up with a number of solid tests including e High Low Temperature e Temperature Cycling e Vibration e Drop e Shock e Crush e Humidity e Altitude Did we miss any Then we performed an FMEA and came up with the following 1 Different cleaning solutions 2 Pen test 3 Lipstick test 4 Motor Oil Test 5 Cap Tether Test 6 Battery life test Different Cleaning Solution The company was recommending that users use soap and water for their cleaning solution But what if someone decided to use alcohol instead Or what if they were to put the device in a dishwasher Well that went against what the company put in their users manual So can we still assume that the user may do this Absolutely The first rule of a user s manual is that users will not read them Do not put any information in a users manual that you expect and require someone to read in order to figure out the functionality of the product because they will not read the manual Make everything intuitive and anticipate what the user will do and design for that In this case we must assume the user will not read the manual and will use alternate cleaning solutions so in our early testing
11. nents The test coverage wasn t complete but the coverage was good enough for useful early feedback Figure 11 Testing on an Internet Appliance Product 3 3 Overcoming Few Samples In an early generation typically only a few specimens are available Therefore engineers previously avoided early testing Instead from even a few specimens we can use a test analyze fix method and gain early qualitative feedback During early development with few specimens it is quite feasible amp worthwhile to test for qualitative gross defects amp failures in order to gain early qualitative feedback and hence to stimulate early engineering changes During later development with numerous specimens it is also required to test for quantitative fine defects and failures in order to gain quantitative feedback and hence to prove product lifetime There is useful synergism between these two tests Typically samples will fail within a fairly tight distribution Therefore Highly Accelerated Life test HALT can be used to trade test margins for a size of specimen population Thus even with few samples HALT test of the outer edge of this distribution will tell about product performance Successive generations are usually strongly correlated in defects wear fatigue failures mechanisms and root causes Previously this correlation was not sufficiently appreciated These correlations enable smarter tactics In parallel with early developm
12. s During an early generation often it is more sensible to test the product margins rather than to test for manifest failure and product lifetime If we divide up the product life cycle into three phases P1 being the Feasibility Phase P2 being the Development Phase and P3 being the Qualification Phase then when should we test for reliability P1 P2 or P3 Product development is nearly complete Figure 13 Typical Development Phases For best results reliability testing is done at all three phases In P1 test for early reliability feedback In P2 test for cleaner specimens and better coverage In P3 test to validate the design We should perform experimental tests early and life tests later Do not try to run life tests on generation P1 because P1 typically is NOT built with final materials design process and thus P1 defects wear fatigue and failure may be NOT relevant to later generations Start Lifetime amp Reliability Demo during P2 so this is completed before the end of P3 For a project that develops subunits in parallel test each subunit as early as it is available P1 rather than waiting for final system test P3 when it is painfully late Perform Reliability Tes ting on more samples Theses amples will be closer to final product and functional tes ts will be more refined with higher test coverage Figure 14 Reliability Testing during product development 3 4 1 Case Study
13. t if you put in your purse What might it come in contact with pens and keys for sure but what about something like lipstick or eye shadow Could the cover have come off these products and come in contact with our inhaler Certainly So should we think about this during the design Absolutely Motor Oil Test Could our product come in contact with motor oil Well we probably won t have motor oil in our pocket or our purse but what about if we are doing repairs on our car and then pick up the product We could have motor oil on our hands or grease from a hamburger we just ate or all sorts of chemicals on our hands Can the plastic withstand these types of chemicals Could these possibly come in contact with the metal disk and clog it These are areas we need to consider during the design and during the testing Cap Tether Test We talked about tethering the cap If this is to become one of our mitigations against failure then the tether must work It cannot impede use of the device and cannot be such that the user wants to break off the tether to set the cap aside It must be an integral part of the design not an afterthought Battery life test Since the device is portable it will run off batteries Have we tested out all scenarios in which the battery could be depleted in the middle of an operation such as priming or dosing What happens if the battery has enough life left in it before dosing but runs out during dosing Does the
14. ttle gold nuggets These may forewarn what could happen with a mature process Thus overcome fear amp paralysis due to irrelevant failures and immature manufacturing 3 1 1 Case Study for Overcoming Immature Manufacturing Very early during design we tested a Gigabit Fiber Channel product and found a tolerance rubbing issue between the housing and a component near the edge of the board This early feedback facilitated early board re spin by moving this component Figure 10 Testing on a Gigabit Figerchannel Product 3 2 Overcoming Low Test Coverage During early development in house test coverage typically is low and specimens are few Therefore test for qualitative gross issues and postpone test for quantitative fine issues This early test may be very worthwhile because it may save considerable cost compared to late learning amp late engineering such as late board spin or late chassis changes Sometimes it is allowable to start with commercially available test equipment This temporarily bypasses custom test programs amp scripts that won t become available until later Often this is good enough for worthwhile early test even though complete coverage is postponed to later phase 3 2 1 Case Study for Overcoming Low Test Coverage We tested an internet appliance product well before the diagnostics were ready Therefore we just purchased some off the shelf software to exercise the memory hard drive and a few other compo
15. user know that the dosing did not occur Is it possible that the user may get no dosage or just as bad is it possible that the user got a dosage but thought they did not get a dosage so when they replace the battery the reapply the dosage These are all scenarios we must think about during design 2 2 A FEW MORE EXAMPLES OF FMEA Here are a few more examples of products in which we performed FMEA first and then developed a test plan Note that the pictures in the following section are samples of products we have worked on These are not the actual products to protect the proprietary nature of the products we work on Fallure Mode Test f Accelerant Fallure Mode Finck once febure Test Accelerant Failure Node Toat Accelerant Duty Cycle 8 peed Torque Backrossus Contaninaticn Figure 4 FMEA on a Fan Failure Mode Test Accelerant Duty Cycle Stari Stop Viiration Head Spinning taiure Contamination on Head Suntace Boar Deraling beue Tamperniure Voluge Seneca wear Por Dae Dy Ore Forca Ar Figure 5 FMEA on a Hard Drive Non Openaiional Vibnaiion Failure Mode Test f Accelerant Cycle Speed Torque parzire Alfude Cyle Figure 5 FMEA on a Robot reduced Duly Cycle Deap Diecharge Speocd of Chappe Fallure Mode Ballary Changing Touch aren flue Pumping failure Failure Mode Test Accelerant Duty Cycle Force Angle Failure Mode Test
Download Pdf Manuals
Related Search