5 - Deep Blue - University of Michigan
Contents
1. sessssssusenesenussnnnenreneununnnnnnnnnennnn 6 Data Reduction a ie 7 SAE J2365 Keying Time Estimates usensnnennnnonunnununnnnnnunnnununnnnnnunne 8 BESUETS Lesen east 9 Destination Entry Vehicle Parked zusssnnunenonunonnnununnnununnnunennunnnnn 9 el H Total Task TIME VS el 10 Total Task Time vs SAE en 11 SAE 2365 Operator Estim les a ee naia aaRS 12 Menu lem Selection eisen 13 Destination Entry While Driving uunsuununununnunnnnnnunnnnnnnnnunnnnnnnunen 14 OVE OW EE E REISEN 14 Total Task TIMES Ee 15 Entry While Parked vs Entry While Driving unuansnunnenennennennnnnnnnnn nun nnnnnn nun ann 16 CONCLUSION S seen 17 REFERENCE Sist 20 APPENDIX A Participant Consent FOrmM ssssssssssnunnsnsunnnnnnnunnnusvaannne 23 APPENDIX B List of Test Trials u 25 APPENDIX C M 14 Test Route uunsonunnnnnunnanonnnnnunanunannnununnnanenunen anne 27 APPENDIX D SAE J2365 Operator Elements 29 INTRODUCTION Overview Data from Japan indicate that using cell phones and navigation systems while driving can lead to an increased risk of crashes Green 2000 and because of task similarities other telematics applications could have the same result To minimize the crash risk there have been several efforts to provide recommendations and guidelines to improve the driver interface design of navigation systems See Green 1999a for a review These include the BSI2. SELVAS ADS SC Wr EE gt ie e s x Lae E 2 SE SE ARCA SE SE N IN SIT LEER LEER BR SIR ANOS ER Overall While Parked While Driving Number of Measured Task System Keying Measured Task Keystrokes Time s Delay s Time s Time s 13 20 7 07 6 13 Std Dev 4 55 3 18 1 87 Minimum 40 6 43 3 23 3 03 Maximum 13 0 lt 22 97 13 97 9 97 r2 SE Y 6 26 79 X 0 5 10 15 20 25 30 Keystrokes Measured Task Time s Entry While Parked Total Keying Time s While Parked Measured Task Time s Entry While Driving N RN HERNE Zem SE E u a on While Parked O jo E eg oO LL D 10 15 Estimated Task Time s Using the Revised SAE J2365 Elements Mean Keying Time s O d E L D E gt lt c o gt 0 0 SAE J2365 Task Time Estimate s Issue 4 SAE J2365 Operator Estimates Operator Elements SAE J2365 Estimate s Revised Estimate s Enter Keystrokes 1 Cursor Keystroke Additional Cursor Keystrokes TABLE OF CONTENTS INTRODUCTION us E 1 Ov rvieW erosius a a aaa 1 Selecting Tasks to Study sssesssesronennuesuunnnennuenuseensuonuonnnssnneuunouennnsnenon 2 EC PICO E E A E E E 3 TEST PLAN rinda daa 4 Test Participan ins 4 Test Materials and EquipMent occoccconnanonocencnrunnororonononcaroronarananananuncananas 4 Test Activities and Their Sequence
3. The destination entry while driving took place on an 8 mile stretch of M 14 just northeast of Ann Arbor Michigan Figure 17 This stretch of rural expressway contained 2 12 foot lanes in each direction separated by a grass infield The posted speed limit was 70 mph and the annual average daily traffic on this section was 47 003 vehicles according to the Michigan Department of Transportation 1996 The traffic density during the experiment was light as all vehicles were moving at the posted speed limit N o To Livonia gt 4 Miles Scale 3 4 inch per mile O OO O Man lt MO D amp he gt LL d lt J dl Fox Hills Country Club 7 Miles Exit 10 Figure 17 Graphical representation of the M 14 test route 28 APPENDIX D SAE J2365 Operator Elements An operator is a keystroke level subtask element Table 6 shows the operator times specified in SAE J2365 Green 1999b These values were based on the classical Keystroke Level Model operators described by Card Moran and Newell 1980 1983 that were obtained from a variety of office tasks These estimates were based on office data that were likely to underestimate the task times because navigation data entry tasks are not well learned Le not routine cognitive tasks and because automotive workstation design does not permit rapid keyboard entry as in an office even when the vehicle is stationary The original model opera
4. and Menu Structure Depth Versus Breadth Technical Report UMTRI 97 42 Ann Arbor MI The University of Michigan Transportation Research Institute Manes D Green P and Hunter D 1998 Prediction of Destination Entry and Retrieval Times Using Keystroke Level Models Technical Report UMTRI 96 37 also released as EECS ITS LAB FT97 077 Ann Arbor MI The University of Michigan Transportation Research Institute Miller D P 1980 Factors Affecting Item Acquisition Performance in Hierarchical Systems Depth vs Breadth Ph D dissertation Columbus OH Ohio State University Miller D P 1981 The Depth Breadth Tradeoff in Hierarchical Computer Menus Proceedings of the Human Factors Society 25 Annual Meeting Santa Monica CA Human Factors and Ergonomics Society 296 300 Norman K L 1991 The Psychology of Menu Selection Norwood Nu Ablex 21 Nowakowski C Utsui Y and Green P 2000 Navigation System Evaluation The Effects of Driver Workload and Input Devices on Destination Entry Time and Driving Performance and Their Implications to the SAE Recommended Practice Technical Report UMTRI 2000 20 Ann Arbor MI The University of Michigan Transportation Research Institute Olson J R and Nilsen E 1987 1988 Analysis of the Cognition Involved in Spreadsheet Software Interaction Human Computer Interaction 3 309 349 Paelke G and Green P 1993 Entry of Destinations into Route Guidance System
5. Figure 16 Predicting entry time while driving from entry time while parked 16 CONCLUSIONS 1 How do task times estimated using SAE J2365 compare with the task times obtained experimentally The SAE J2365 calculation method was developed to assist in predicting the total task time for navigation system tasks performed while the vehicle was stationary The experiment measured the task times of 16 navigation system entry tasks performed while the vehicle was stationary by 8 drivers between the ages of 20 and 30 The mean measured task time including system for the entry tasks was 13 2 seconds ranging from 6 43 to 22 97 seconds The SAE method does not however include system delays greater than 1 5 seconds Removing these delays from the measured task times the mean keying time was 6 13 seconds ranging from 3 03 seconds to 9 97 seconds The estimates based on SAE J2365 for those same tasks ranged from 4 4 to 9 6 seconds Averaging over the number of drivers there was a good correlation r 815 between the keying time and the estimated task time Overall the estimates were slightly lower than the measured task times by a mean of 0 68 seconds with the largest differences ranging up to 1 5 seconds Although the actual keystroke times could not be measured estimates were obtained for the first cursor keystroke additional cursor keystrokes and enter keystrokes using multiple linear regression as shown in Table 4 Overall the revi
6. guidelines British Standards Institution 1996 the JAMA guidelines Japan Automobile Manufacturers Association 2000 the EU guidleines European Union 1998 the HARDIE guidelines Ross Vaughn Engert Peters Burnett and May 1995 and several guidelines prepared for the U S Department of Transportation Green Levison Paelke and Serafin 1993 and Campbell Carney and Kantowitz 1997 Two recent efforts to provide design guidelines for in vehicle navigation systems include 1 the development of the In Vehicle Information Systems DEMAnD Behavioral Model Hankey Dingus Hanowski Wierwille and Andrews 2000a b and 2 the development of a recommended practice by the Society of Automotive Engineers SAE J2364 Farber Foley and Scott 2000 Green 1999c d and the Society of Automotive Engineers 2000 The IVIS DEMAnD model is a resource based model of driving and secondary task performance Using this model key components of the task such as the longest expected glance to the display the total task time and the expected number of glances required to complete the task are recorded or estimated Templates are provided for a limited number of predefined tasks that can be used to help estimate the key model parameters such as the total task time The model then uses these key parameters in conjunction with the visual auditory cognitive and manual resources used by the in vehicle task to estimate the amount of interference wit
7. keystrokes 0 4 seconds and enter keystrokes 1 2 seconds as the elements for the task No mental operations were used in the estimates for this task Because the measured task time did not include the reach to the navigation system or the time to make the first keypress these elements were not included in the total task time estimates which allowed for a direct comparison between the calculated estimates and the experimental findings Had the reach to the navigation system been included 0 45 seconds would have been added to each trial RESULTS Destination Entry Vehicle Parked Overview The experiment resulted in 128 trials or destinations entered while the vehicle was parked but only 107 trials were analyzed as the remaining 21 trials contained errors extra keystrokes As shown in Table 2 the mean measured task time was 13 20 seconds ranging from a low of 6 43 to a high of 22 97 seconds The mean for trials containing errors was 16 75 seconds or 27 percent higher due to the extra keystrokes These estimates include the system response time delays which averaged 7 07 seconds According to SAE J2364 if system response times were greater than 1 5 seconds and feedback regarding the delay was provided the system response time should not be included in the total task time With the system delay removed the mean keying time was 6 13 seconds ranging from a low of 3 03 to a high of 9 97 seconds As shown in Figure 7 the diff
8. of keystrokes ignoring the first keystroke since it was not timed as shown in Figure 9 There was however a great deal of variability across trials and test participants in the keystroke estimate which could be due to the fact that the duration of certain types of keystrokes can be double or triple that of others For example as suggested by SAE J2365 a second successive cursor keystroke should take 0 4 seconds or half the time required for the first cursor keystroke 0 8 seconds Thus 2 trials with the same number of keystrokes could result in different estimates if one trial contained more repetitive keystrokes than the other 15 L 1Y 1 75 56 X 12 zs Hi L al o 10 E 5 gt 10 X i e D E pan Cc y 0 0 5 10 15 Keystrokes Figure 9 Mean keying time while parked as a function of keystrokes 10 As noted in J2365 the times for older drivers are about 1 8 times that of younger drivers Thus the 15 second time limit of the older drivers should be completed in about 8 3 seconds by younger drivers According to Figure 9 for a cursor bound menu task a task of 11 keystrokes or less could be completed within the 15 second rule However other navigation systems or navigation tasks may not be as structured or completed as swiftly Total Task Time vs SAE J2365 Estimates Comparing the SAE J2365 task time estimates to the actual mean keying times Figure 10 a relatively good line
9. of matches First only 5 destination categories appeared in the same order for each trial A classic menu study would randomize the order of these categories to prevent memorization of 17 the menu order which would reduce the menu item selection time Second when a target destination that might fall into multiple categories was given the category name was also mentioned This protocol would change many of the trials in the category match to identity matches Because the number of keystrokes required to select a menu item was directly proportional to the desired item s position the time required to select a particular menu item was linear with a Y intercept of 95 seconds and a slope of 38 seconds per keystroke 3 How do the task times obtained for a single task entry while parked correlate with the task times obtained during dual task conditions entry while driving The experiment measured the task times of 24 navigation system entry tasks performed by eight drivers between the ages of 20 and 30 while the vehicle was travelling on an expressway The mean measured task time including all system delays for the entry tasks was 15 85 seconds ranging from 4 90 to 30 67 seconds Of these 24 trials 14 were comparable to the trials performed while the vehicle was parked Comparing the two conditions the measured task times while driving were approximately 1 2 times the measured task times while parked for the driving workload
10. 2 Marathon 5 10 1 society Bank 4 8 3 Wendy s 2 10 1 Your best friend s house 3 6 2 Amoco 1 6 2 Nissan Dealer 1 5 3 Cottage Inn Pizza 1 8 3 Burger King 4 12 1 Comerica Bank 3 7 5 Rite Aid 2 10 25 Trial kh zech i N 000 rd OO P Go Trial 13 14 15 16 17 18 19 20 21 22 23 24 Part 3 Test Trials While Driving East on M 14 List Restaurant Fast Food Gas Station Address Book Gas Station Grocery Store Restaurant Grocery Store Previous Fast Food ATM ATM Menu Position ch 0 WN OI GO 0N NY WW List Item Papa Romanos Wendy s Amoco The Lake Boggs Gas Busch s ValuLand Chili s Grill amp Bar MC Alley Pharmacy Matt s House Lee s Famous Chicken ATM Canton Bank One Item Total Position Keystrokes DO Y P P Y 01 09 Oz bi Ol Part 3 Test Trials While Driving West on M 14 List Fast Food Address Book ATM Gas Station Previous Restaurant Grocery Store Address Book ATM Grocery Store Grocery Store ATM Menu Position 3 1 1 2 2 3 5 1 1 5 5 4 List Item Taco Bell Home National City Bank Consolidated Stations Mom amp Dad Karl s Country Cabin Arbor Drugs Your best friend s house Comerica Arbor Drugs Merchant of Vino National City Bank 26 Item Position P GO AO COM AM OI zb A 12 10 6 8 8 13 10 12 9 11 7 6 Total Keystrokes 12 4 5 10 7 8 10 6 9 9 11 8 APPENDIX C M 14 Test Route
11. 365 predictions and static task times for tasks near the 15 second limit were desired Therefore in selecting tasks the following items were considered 1 Would access to the feature be desired while driving 2 How often would the feature be used 3 Was the feature already accessible while driving in current navigation systems 4 Could the task be accomplished within the boundaries of the 15 second rule The task of setting a nearby point of interest POI was chosen because it met all of the criteria specified above This particular entry task was also chosen because it was menu based and menu tasks while driving have not been examined extensively Given that many other current and future in vehicle tasks involve the use of menus examining this entry task should provide design information that might be generalized to other menu selection tasks The literature on menus is substantial and best covered in The Psychology of Menu Selection Norman 1991 However most of the menu selection work has focused only on the item selection time assuming that all physical responses required for any particular item were equal such as when using a touch screen The task being studied in the current experiment differs from the classic menu literature in that responses were made with cursor keys Thus selecting a given menu item required multiple keystrokes Given this limitation the results of the current experiment may not be comparable to the results
12. A A ARR TN ERT rennen rn ee Technical Report UMTRI 2000 49 January 2001 Prediction of Menu Selection Times Parked and While Driving Using the SAE J2365 Method Christopher Nowakowski Paul Green IT o UMTRI The Univ University of Michigan aS Transportation Research Institute Technical Report Documentation Page 1 Report No _ 2 Government Accession No 3 Recipient s Catalog No UMTRI 2000 49 4 litle and Subtitle Report Date Prediction of Menu Selection Times Parked and January 2001 While Driving Using the SAE J2365 Method EE account 377218 3 Performing Organization Report No Christopher Nowakowski and Paul Green UMTRI 2000 49 5 Performing Organization Name and Address 0 Work Unit no The University of Michigan Transportation Research Institute UMTRI 2901 Baxter Rd Ann Arbor Michigan 48109 2150 USA i ponsoring Agency Name and Address Nissan Motor Company ITS Development Nissan Technical Center North America 39001 Sunrise Drive P O Box 9200 Farmington Hills Michigan USA 48333 9200 oupplementary Notes Contract or Grant No Type of Report and Period Covered 6 2000 12 2000 Sponsoring Agency Code 16 Abstract Recent concern has been expressed over the use of cell phones and navigation systems while driving Several efforts including SAE Recommended Practice J2364 the 15 second rule and J2365 the associated calculation procedure have r
13. Food column When the list of grocery stores appeared the driver selected the target item which was third in the of items This operation under Item Select required a first cursor keystroke C1 0 8 a single additional cursor keystroke C2 0 4 and an enter keystroke E 1 2 Finally to complete the entry an enter keystroke was required E 1 2 under the OK column Adding all of these elements together provided an estimated task time of 8 0 seconds Table 7 SAE J2365 calculations for the destination entry trials while parked Tria Item Select C1 gt C2 E 1 0 8 1 2 2 0 8 1 2 3 0 8 1 2 4 0 8 1 2 5 0 8 1 2 6 0 8 1 2 7 0 8 1 2 8 0 8 1 2 9 0 8 1 2 10 0 8 1 2 11 1 2 12 1 2 13 1 2 1 2 1 2 1 2 30
14. able from the International Standards Organization as document ISO TC 22 SC Li 8 N 181 Farber E Foley J and Scott S 2000 Visual Attention Design Limits for ITS In Vehicle Systems The Society of Automotive Engineers Standard for Limiting Visual Distraction while Driving Transportation Research Board 79th Annual Meeting Foley J Greenberg J Farber G Blanco M Curry R and Serafin C 2000 Visual Demand While Driving Presentation to the SAE Safety and Human Factors Committee on March 24 Warrendale PA Society of Automotive Engineers Gould J D and Lewis C 1985 Designing for Usability Key Principles and What Designers Think Communications of the ACM March 28 3 300 311 Green P 1999a Estimating Compliance with the 15 Second Rule for Driver Interface Usability and Safety Proceedings of the Human Factors and Ergonomics Society 43rd Annual Meeting CD ROM Santa Monica CA Human Factors and Ergonomics Society Green P 1999b Navigation Systems Data Entry Estimation of Task Times Technical Report UMTRI 99 17 Ann Arbor MI University of Michigan Transportation Research Institute Green P 1999c The 15 Second Rule for Driver Information systems CD ROM ITS America Ninth Annual Meeting Conference Proceedings Washington D C ITS America 20 Green P 1999d Visual and Task Demands of Driver Information Systems Technical Report UMTRI 98 16 Ann Arbor MI The Universi
15. ar correlation r 815 was found In Figure 10 a perfect 1 1 correlation between the estimates and the actual times would follow a line with a slope of 1 but as depicted the SAE J2365 method consistently overestimates the keying times by a mean of 0 68 seconds However given that the SAE J2365 method was created to predict task times for a variety of controls and interfaces some differences are expected due to the nature of the task interface and controls 10 gt Y 68 79 X A Y r 815 P Mean Keying Time s While Parked 10 15 SAE J2365 Task Time Estimate s Figure 10 Comparison between the SAE J2365 estimates and the actual keying time Of particular interest are data points 11 and 13 in Figure 10 with predicted values of 8 8 and 9 6 seconds respectively These cases resulted in the worst errors a keying time estimate of 9 6 seconds with an actual keying time of 7 3 seconds a 31 percent margin of error These 2 trials asked the drivers to set a fast food restaurant as the destination and resulted in much lower actual keying times 6 7 and 7 3 seconds respectively than predicted Almost half of the overestimation for these 2 trials came from an intermediate step that asked the driver to select between all restaurants and fast food restaurants Because this was a novel step in the entry sequence for this system and it was highly anticipated by the drivers as they had seen the additional step during practice th
16. d thus removed from the model when estimating the keystroke operators Enter keystrokes were estimated at 1 06 seconds p lt 0001 First cursor keystrokes were estimated at 0 72 seconds and additional cursor keystrokes were estimated at 0 36 seconds p lt 0001 Using these values for the operator elements the fitted model resulted in an r of 0 77 See Figure 11 15 Mean Keying Time s While Parked 0 5 10 15 Estimated Task Time s Figure 11 Comparison between the SAE J2365 estimates using the revised operator times from the regression and the actual keying time 12 Menu Item Selection The menu item selection times for destination entry while parked were recorded during the video analysis Two types of menu selections as defined by Miller 1980 and 1981 occurred during each trial First the driver was required to select the correct menu category for the target destination referred to as a category match Second the driver was required to select the target item from the list of destinations referred to as an identity match Accordingly a repeated measures ANOVA was performed with menu selection type and item position as the within subject measures There was a marginally significant effect for the menu selection type F 1 7 4 97 p lt 0 06 indicating that the category match was performed slightly faster than the identity match Figure 12 The item position F 3 21 42 48 p lt 0 001 and the item
17. e responses during this step were much faster The SAE J2365 estimate predicted 0 8 seconds for a cursor keystroke and 1 2 seconds for an enter keystroke for a subtask time of 2 0 seconds yet the mean time to complete both keystrokes during this step was less than 1 3 seconds 11 SAE J2365 Operator Estimates Although the actual keystroke times have yet to be obtained from the videotapes of this study estimates for the SAE J2365 operators were obtained using multiple linear regression techniques Three elements first cursor keystrokes C additional cursor keystrokes C and enter keystrokes E were used to estimate the total task time For each trial an equation could be written of the form X C Y C Z E measured task time where X Y Z are the number of C C and E keystrokes respectively for the trial However in the above equation the variables first cursor keystroke and additional cursor keystrokes are not independent since an additional cursor keystroke cannot occur independently of a first cursor keystroke Rewriting the equation to account for this interdependency provides the following assuming that first cursor keystrokes will be twice the value of additional cursor keystrokes based upon the previous values given in SAE J2365 X 2C Y C Z E measured task time The resulting regression for the task while parked was significant F 2 13 67 42 p lt 0001 The intercept was not significant an
18. e stretch of the M 14 expressway north of Ann Arbor Michigan between Ford Road and Beck Road as shown in Appendix C While driving at 70 mph the experimenter read the target destinations at a rate of approximately 1 5 destinations per mile about 1 every 45 seconds A trial was not started until the driver was maintaining 70 mph with no potential interference from other nearby vehicles While the driver was engaged in the task the experimenter acted as a safety observer The experimenter allowed the driver at least 5 to 10 seconds of rest between trials Data Reduction Keystrokes and eye glances were recorded using a Panasonic Model AG 5700 VHS recorder at 30 frames per second from a Hitachi VM H38A camcorder split with a Panasonic GP KS152 lipstick camera See Figure 5 Though not apparent in this figure a portion of the road scene was visible in the upper right hand corner of the recorded image The recordings were then analyzed using a frame accurate VCR Panasonic Model AG DS550 and a 13 inch high quality color monitor Sony Trinitron to determine the measured task times The measured task time began when the first key was pressed and ended the moment that the last key required to enter the destination was pressed Because the system provided a tone each time a key was pressed there was no ambiguity regarding the start or end of a trial Figure 5 Split screen image captured from the in vehicle video recording system To simpl
19. ecently tried to provide guidelines for the design of in vehicle devices The purpose of this study was to examine a destination selection task in the model year 2000 Nissan Infiniti 130 navigation system which was currently allowed while driving in the context of SAE J2364 and J2365 Eight licensed drivers between the ages of 20 and 30 mean of 25 years old selected destinations using the in vehicle navigation system s address book and nearby points of interest features The tasks were performed both while the vehicle was parked and while driving on a 2 lane expressway during low volume traffic conditions at 70 mph The mean measured task time while the vehicle was parked was 13 20 seconds requiring an average of 8 8 keystrokes to complete the task The mean measured task time while driving on the expressway was 15 85 seconds or approximately 1 2 times the measured task time while parked Subtracting system delays greater than 1 5 seconds the mean keying time was 6 13 seconds when the vehicle was parked Consequently given that SAE J2364 specifies testing older drivers who typically take up to 1 8 time longer to complete in vehicle tasks a task of 11 keystrokes or less would pass the 15 second rule SAE J2365 was also used to estimated the task times while the vehicle was parked and it was found to slightly overestimate the task times by a mean of 0 68 seconds 17 KeyWords mmm 18 Distribution Statement ITS human factors ergonom
20. em Destination Entry Consent Form Most major automakers have been developing navigation systems to aid drivers in reaching their destinations In the past few years several manufacturers have offered navigation systems as options on new vehicles Though the potential benefits of these systems are great there are still unresolved safety concerns over which features should be accessible while driving In the experiment today you will spend about 45 minutes using the navigation system currently offered in the 2000 Nissan Infiniti Detailed instructions will be provided on how to use the system before the experiment begins During the first part of the experiment you will enter destinations from various lists such as the address book or the nearby restaurant list while parked at UMTRI Most of these tasks take less than 30 seconds each The second part of the experiment will involve entering similar destinations while driving on M 14 between Ford Road and Beck Road just north of Ann Arbor While performing the destination entry tasks your first priority is to drive safely The second priority is to complete the task both accurately and quickly Several cameras will be recording you as you perform the tasks If you feel unsafe at any time the trial or experiment can be stopped and you will be paid regardless of whether you complete the experiment For your time you will be paid 20 If you have any questions please do not hesitate to ask the
21. erence in mean keying time between female 6 18 seconds and male 6 08 seconds test participants was negligible Keep in mind that these times do not include the time to reach from the steering wheel to press the first button Had the measurement followed the 15 second rule SAE J2364 precisely the first reach would have added about 0 45 seconds to the measured task times Table 2 Task time summary for destination entry while parked Measured Task Time system Delay Keying Time Keystrokes s s s Mean 13 20 7 07 6 13 8 8 Std Dev 4 55 3 18 1 87 2 4 Minimum 6 43 3 23 3 03 5 0 Maximum 22 97 13 97 9 97 13 0 15 rn Female Male 2 E 10 O o co SS dE SC aD S 0 1 2 3 4 5 6 7 8 Test Participants Figure 7 Mean keying time while parked by test participant Note Error bars for all graphs denote 1 standard deviation The system response time or delay Figure 8 was largely due to the database search times which were not consistent between trials or between test participants within the same trial Therefore the delays could not be reliably predicted The longest system delays trials 1 5 9 14 and 16 were found when selecting an item from the grocery store list or the fast food list 15 system Delay s While Parked o 1 5 10 15 Trial Figure 8 Mean system delay while parked by trial Total Task Time vs Keystrokes There was a linear relationship between the keying time and the number
22. experimenter at any time Thank you for your participation It is OK to show segments of my test session in presentations to UMTRI visitors UMTRI papers and reports and conferences and meetings This is not required for participation in the study but is useful to have Your name will not be mentioned agree disagree have reviewed and understand the information presented above My participation in this study is entirely voluntary Subject Name PRINTED Date Subject Signature Witness experimenter Investigators Christopher Nowakowski 763 2485 Paul Green 763 3795 23 Trial O A OMN Trial EEE D STL TTS Has ENGE A tal List Grocery Store Address Book ATM Fast Food Previous Dest List Grocery Store Gas Station Restaurant ATM Grocery Store Previous Dest Gas Station ATM Fast Food Address Book Gas Station Previous Dest Restaurant Fast Food ATM Grocery Store APPENDIX B List of Test Trials Part 1 Practice Trials While Parked A ARO CAI AAA IEA UI IE AS ERA DOTA Menu List ltem Item Total Position Position Keystrokes 5 Rite Aid 2 10 1 The Mall 4 7 1 Society Bank 5 9 3 McDonald s 1 9 2 Just Jewelry 3 7 Part 2 Test Trials While Parked Menu List Item Item Total Position Position Keystrokes 5 Dong Yu s China Market 3 11 2 Sunoco 3 8 3 Y amp S Sandwich Shop 5 12 1 National City 2 6 5 Busch s ValuLand 4 12 2 Caf Java 5 9
23. explored daytime expressway 70 mph very light traffic smooth road surface As shown in Table 5 the results of this study compared reasonably well to the results of other studies Table 5 Relationship between dynamic and static task times Study Task Subjects Roads Task Time Ratio Ages Dynamic Static Paelke and Nav System 16 Low Fidelity 1 1 ES A ana Tijerina 1999 Nav System 10 Test Track 1 26 2 Cell Phone 55 65 E O Tsimhomi Yoo Map Reading 16 Simulator 1 55 O 0h 112 2 eee ent EE Foley et al Nav System 40 Simulator 1 7 2000 Cell Phone 45 65 and CD Changer Expressway O ee Nowakowski Nav System 16 Simulator 1 27 Utsui and Green Menu Selection 18 65 2000 18 4 Is this interface in compliance with SAE J2364 for nearby POI point of interest selection tasks The 15 second rule as specified in SAE J2364 calls for testing any feature that is to be accessible while driving to ensure that the task can be completed by older drivers in under 15 seconds while the vehicle is parked The method used in this study only tested younger drivers with the device however past studies have shown that the response times for older drivers are on average 1 8 times those of younger drivers Given this relationship it could be extrapolated that any task completed by younger drivers in under 8 3 seconds would pass the 15 second rule if the tests were rerun using older drivers Since the mean keying time for this g
24. h driving caused by the task The central focus of the SAE effort has been to clearly define which functions should and should not be accessible to the driver when the vehicle is moving and recently this has led to a significant level of interest in the topic of driver distraction SAE J2364 stipulates that Any navigation function that is accessible by the driver while a vehicle is in motion shall have a static total task time of less than 15 seconds Society of Automotive Engineers 2000 The timing starts when the driver moves his or her hand from the wheel to begin the task and ends when feedback from the last step of the task is received The vehicle or mockup is assumed to be parked during the testing The rule only applies to navigation systems with visual displays and manual controls The 15 second rule represents a compromise given the various views of those voting for the recommended practice the safety implications from recent research on the use of navigation and other systems and the boundaries of what is considered acceptable for conventional in vehicle controls and displays SAE J2364 includes a compliance procedure that involves testing a sample of drivers using a working device Compliance with J2364 does not assure that a task is safe to do while driving only that the most egregious tasks are not permitted It is possible that some tasks that should not be performed while driving could comply with SAE J2364 While t
25. he practice does not directly address all possible sources or mechanisms of distraction it represents a reasonable practical first cut at reducing the likelihood of system induced crashes As additional research is completed this recommended practice is likely to be enhanced A key lesson from the literature on human computer interaction is the importance of early evaluation of usability Gould and Lewis 1985 Thus to support J2364 a procedure was needed to estimate compliance early in design during the conceptual stage At this stage changes in the user interface can be made very quickly at no cost This need led to the development of SAE J2365 Green 1999a b SAE J2365 provides a hierarchical method for 1 describing user actions on a step by step basis 2 using look up tables of estimates for mental operations visual search and various keystrokes and 3 adding the operators to estimate the total task time The approach was based on the GOMS method goals operators methods and selection rules commonly used for evaluating the user interfaces of computer systems Card Moran and Newell 1980 1983 as well as for specific studies of automotive navigation systems Steinfeld Manes Green and Hunter 1996 Manes and Green 1997 Manes Green and Hunter 1998 and Green 1999d Selecting Tasks to Study There has been considerable debate concerning SAE J2364 and J2365 To provide a scientific basis for further discussion SAE J2
26. ics driving No restrictions This document is intelligent transportation systems available to the public through the navigation destination entry usability National Technical Information Service safety telematics Springfield Virginia 22161 19 Security Classify of this report 20 Security Classify of this page 21 No of pages 22 Price None None 36 orm DOT F 1700 7 8 72 Reproduction of completed page authorized Prediction of Menu Selection Times Parked and While Driving Using the SAE J2365 Method UMTRI January 2001 University of Michigan Christopher Nowakowski and Paul Green Ann Arbor Michigan USA 1 How does the menu selection task time vary with the number of keystrokes 2 How do the task times obtained for single task entry while parked compare to the task times obtained during dual task conditions entry while driving 3 How do estimates from SAE J2365 compare with the task times obtained experimentally 4 How could the operator elements in SAE J2365 be adjusted to obtain a better fit to the task times obtained experimentally Task Performed While Parked and During Expressway Driving Nav System Experience Experienced Never Seen Drivers Age 20 30 AA ATTE Control Layout 2000 Nissan Infiniti 130 Q45 SEA EC Setting ou owot Cancel Destination Selection Task Nearby Point of Interest Identity Match ste
27. ience with navigation systems Three of the test participants had participated in previous experiments involving navigation system usability or destination entry on systems from different manufacturers Test Materials and Equipment The test vehicle was a left hand drive Nissan Infiniti 130 model year 2000 with an automatic transmission The in vehicle navigation system was a manufacturer s option on this model The database CD used during this experiment covered Michigan Indiana and part of Ohio Illinois and Wisconsin As shown in Figure 1 the navigation system used a retractable 5 1 2 inch display shaded by a cover mounted on top of the center console The controls for the navigation system were located in the middle of the center console below the heating vents and above the radio controls AACA CANA YAA Figure 1 Infiniti 180 center console with the optional navigation system Figure 2 depicts the layout of the navigation system s controls The joystick on the left labeled Push Enter controlled all of the cursor and enter movements on the screen The button labeled previous was used to back up one menu level and the cancel button in the lower right hand corner was used to leave the menu system at any point and return to the map view The destination entry menus Figures 3 and 4 were activated by pushing the DEST or the Route button As each level of menu appeared the cursor highlight defaulted to the top item in the
28. ify the analysis of trials where the vehicle was in motion only the measured task time was analyzed The inter keystroke intervals and other data may be obtained in future analyses and reported in a subsequent report For the trials with a parked vehicle the duration of each step of the destination entry process such as the time to select an item from a menu and the system delays in excess 1 5 second as specified in SAE J2364 were also noted to allow for the computation of the keying time The keying time was defined as the time the driver actually spent engaged in the task or the measured task time minus the system delays See Figure 6 As mentioned above a limited view of the traffic was also available in the upper right hand corner of the recorded video but there were no lane departures or other obvious driving errors to analyze Total Task Time Measured Task Time Start k4 ko Ka Kg kr ke ke b Y b End System Busy Total Keying Time Measured Task Time b Figure 6 Definitions for total task measured task and keying time SAE J2365 Keying Time Estimates Keying time estimates for each of the 16 trials performed in a parked vehicle were computed using SAE J2365 The element operators used were taken from SAE J2365 and the details of the calculations for these estimates are listed in Appendix D The estimates for the interface used only the first cursor keystroke 0 8 seconds additional cursor
29. ime while the vehicle was parked ranged from an average of 17 5 to 21 7 seconds for drivers under the age of 30 Additionally the total task time increased by a factor of 1 27 when the task was performed while driving Issues As noted earlier given the history of discussions concerning the SAE recommended practices the main purpose of this project was to gather additional data to validate SAE J2365 Specifically the issues examined were as follows 1 How do task times estimated using SAE J2365 compare with the task times obtained experimentally 2 How does menu selection time vary with menu item position or the number of keystrokes 3 How do the task times obtained for single task entry while parked correlate with the task times obtained during dual task conditions entry while driving 4 Is this interface in compliance with SAE J2364 for nearby POI point of interest selection tasks TEST PLAN Test Participants Eight licensed drivers participated in this experiment 4 women and 4 men each between 20 and 30 years of age with a mean of 25 Funding constraints did not permit inclusion of an older driver sample as is typical practice at UMTRI for studies of this type Participants were recruited from the UMTRI subject database which was compiled from the respondents to past newspaper advertisements for previous experiments All were paid 20 for their participation Four of the test participants reported prior exper
30. menu ees On Off SC Figure 2 Diagram of the 130 s navigation system control panel Home Office Address Book Your best friend s house The Mall The Lake 1 Nissan Dealer 2 Mom Dad 3 Just Jewelry 4 Matt s House 5 Caf Java Figure 3 Destination menu tree Previous Destinations 1 Nearest 5 2 destinations 3 in the 4 selected 5 category E 1 Nearest 5 Gas 2 destinations Restaurant Fast Food 3 in the Hospital 4 selected Grocery Store 5 category ATM 1 Nearest 5 2 destinations 3 in the 4 selected 5 catego Figure 4 Route menu tree The address book and previous destinations list were preprogrammed before the start of the experiment The Quick Stop feature calculated the 5 nearest destinations in the selected category based on the vehicle s current location Given that this calculation was performed in real time there was often a delay of several seconds between the category selection and the appearance of the completed destination list Although the destination lists contained more than 5 destinations accessible by scrolling only the first 5 destinations were avallable for selection while the vehicle was in motion The navigation system provided a single tone as auditory feedback each time a control was activated After a destination was selected the system automatically calculated a route During the experiment the experimenter cancelled the r
31. of the traditional menu selection experiments In particular one study on menu design described 2 types of menu selection tasks category matches and identity matches that occurred when using hierarchical menus Miller 1980 1981 In the destination entry task the driver might be asked to set the nearest McDonald s as the destination The first part of this task would involve a category match such as selecting restaurant from a list of destination types The second part of the task would involve an identity match that entails selecting the exact item McDonald s in this case from the list of destinations Miller s work predicted that category matches should require more time to complete than identity matches At least one study has examined the application of SAE J2365 in the context of destination entry and menu item selection Nowakowski Utsui and Green 2000 In this driving simulator study destinations were selected from lists using 2 types of input devices The lists were organized such that there were 6 items per screen with 2 additional buttons on the screen for forward and back Each task required exactly 10 keystrokes and the task was performed both while parked and while driving The results of Nowakowski Utsui and Green 2000 suggested that the operator estimates in SAE J2365 could vary by as much as 20 percent depending on the input device For the particular interface and input devices studied the total task t
32. onds ranging from a low of 4 90 to a high of 30 67 seconds The mean measured task time for trials with errors was slightly higher 17 67 seconds due to the extra keystrokes caused by the error The mean measured task time included the system response times and estimates for these response times could not be obtained As shown in Figure 14 the difference in mean measured task time between female 15 50 seconds and male 16 18 seconds test participants was negligible Table 3 Task time summary for destination entry while driving Measured Task Time s Keystrokes Mean 15 85 8 8 Std Dev 5 79 2 4 Minimum 4 90 4 0 Maximum 30 67 13 0 14 Female Male N N oO o Mean Measured Task Time s While Driving o a o Om i 2 3 4 5 6 7 P Test Participants Figure 14 Mean total task time while driving by test participant Total Task Time vs Keystrokes There was a linear relationship between the measured task time including the system delays and the number of keystrokes ignoring the first keystroke since it was not timed as shown in Figure 15 The relationship between keystrokes and measured task time was not as strong for the dual task entry while driving condition r 0 863 as it was for the single task entry while parked condition Ir 0 95 As discussed previously some of the additional variability may stem from the system delay which could not be estimated for the dual task condition but which va
33. oute calculation by pressing the cancel button before the route calculation was completed All other auditory guidance was disabled during the experiment Test Activities and Their Sequence The experiment consisted of 3 parts in a fixed order as described in Table 1 The order was fixed for convenience Each subject began by completing a consent form Appendix A and displaying a valid driver s license for the United States Next during the practice session the experimenter gave the driver an overview of the system and then talked the driver through the first 5 practice trials The target destination for each trial was read aloud by the experimenter who was sitting in the passenger seat A list of the destinations available for each trial can be found in Appendix B Table 1 Overview of experimental sessions Part Task Vehicle Location Trials 1 Practice Parked 5 2 Destination Entry Parked 16 3 Destination Entry Expressway 24 The target destination name was read exactly as it appeared in the destination selection list The name of the category containing the target destination was also provided in the context of the instructions Three typical trials are listed below 1 Select Wendy s from the nearby fast food restaurants list 2 Select your best friend s house from the address book 3 Select the Nissan dealer from the previous destinations list During the third part of the experiment the drivers drove an 8 mil
34. position by menu selection type interaction F 3 21 4 05 p lt 0 02 were also significant Bee N O gt Item Selection Time s While Parked Item Position Figure 12 Menu item selection time as a function of the item position Given that the number of keystrokes required to select an item was directly proportional to the item position these results were not surprising However the interaction between item position and menu selection type which was seen only for selecting the second item during an identity match currently defies explanation According to the SAE J2365 predictions Figure 13 there should an increase and a departure from linearity for item 2 however the time to select each additional item should then continue to increase linearly The increase corresponded to the fact that the additional cursor keystrokes to select item 3 or above required 0 4 seconds each while the first cursor keystrokes to reach item 2 required 0 8 seconds 13 O1 oD SE e E Ze O 83 po e a TH Ba LLI O XI E aN g 0 0 1 2 3 4 5 6 Hem Position Figure 13 Item selection time based on SAE J2365 keystroke predictions Destination Entry While Driving Overview The experiment resulted in 192 trials or destinations entered while drivers were travelling on the expressway but only 174 trials were analyzed as the remaining 18 trials contained errors As shown in Table 3 the mean measured task time was 15 85 sec
35. ried greatly during the single task condition on both a trial by trial and subject by subject basis Additionally the task required multiple glances so the measured task time was affected by the frequency of glances back to the road which may have been related to the strategies chosen by the driver or the moment to moment workload of the road and traffic conditions S r Y 5 09 1 33 X bi D O 0 Sc O Mean Measured Task Time s While Driving o a a Keystrokes Figure 15 Mean measured task time while driving as a function of keystrokes 15 Entry While Parked vs Entry While Driving Fourteen similar trials were performed in both the single and dual task portions of the experiment Although the target destination was different between the single and dual task conditions the item location and total number of keystrokes remained constant allowing for a comparison of the measured task times including system delays and averaged across subjects between these conditions As shown in Figure 16 the single task time was a only a modest predictor of the dual task time r 0 51 keeping in mind that the study only examined 8 subjects Adjusting the model to remove the Y intercept the measured task time while driving was approximately 1 2 times the measured task time while parked 20 15 bh O Measured Task Time s Entry while Driving O1 0 5 10 15 20 25 30 Measured Task Time s Entry while Parked
36. roup of tasks was 6 13 seconds with a mean of 8 8 keystrokes it could be extrapolated that the point of interest selection task would be in compliance with SAE J2364 for the tested navigation system Furthermore based on a linear regression between the number of keystrokes and the mean keying time a similar task using the tested interface would likely comply with SAE J2364 as long as it required 11 keystrokes or less to complete 19 REFERENCES British Standards Institution 1996 Guide to In Vehicle Information Systems Draft Document DD235 1996 London U K British Standards Institution Campbell J L Carney C and Kantowitz B H 1997 Draft Human Factors Design Guidelines for Advanced Traveler Information Systems ATIS and Commercial Vehicle Operations CVO Washington D C U S Department of Transportation Federal Highway Administration Card S K Moran T P and Newell A 1980 The Keystroke Level Model for User Performance Time with Interactive Systems Communications of the ACM July 23 7 396 410 Card S K Moran T P and Newell A 1983 The Psychology of Human dida Interaction Hillsdale Ny Lawrence Erlbaum Associates European Commission 1998 European Statement of Principles on Human Machine Interface for In Vehicle Information and Communication Systems final version Brussels Belgium European Commission Telematics Applications for Transport and the Environment Task Force HMI avail
37. s A Human Factors Evaluation Technical Report UMTRI 93 45 Ann Arbor MI The University of Michigan Transportation Research Institute Ross T Vaughn G Engert A Peters H Burnett G and May A 1995 Human Factors Design Guidelines for Information Presentation by Route Guidance and Navigation Systems Deliverable 19 Workpackage L2 Luxembourg European Commission Host Organization Society of Automotive Engineers 2000 SAE draft Recommended Practice J2364 Navigation and Route Guidance Function Accessibility while Driving Revision of January 20 Warrendale PA Society of Automotive Engineers Steinfeld A Manes D Green P and Hunter D 1996 Destination Entry and Retrieval with the Ali Scout Navigation System Technical Report UMTRI 96 30 also released as EECS ITS LAB FT97 077 Ann Arbor MI The University of Michigan Transportation Research Institute Tijerina L 1999 A Test Track Evaluation of the 15 Second Rule Presentation to the SAE Safety and Human Factors Committee on February 26 Warrendale PA Society of Automotive Engineers Tsimhoni O Yoo H and Green P 1999 Effects of Visual Demand and In Vehicle Task Complexity on Driving and Task Performance as Assessed by Visual Occlusion Technical Report UMTRI 99 37 Ann Arbor MI The University of Michigan Transportation Research Institute 22 APPENDIX A Participant Consent Form PARTICIPANT CONSENT FORM Infiniti Navigation Syst
38. sed estimates were 10 to 15 percent lower than the estimates from SAE J2365 These differences may reflect such factors as amount of practice or task difficulty or factors such as the control design button size shape and location As an example in this study only 5 destination categories appeared in the same order for each trial This made them easy to memorize which could possibly account for some of the decrease in the keystroke times Table 4 Revised estimates for the SAE J2365 operators based on regression Keystroke Type SAE J2365 Estimate s Revised Estimates s 1 Cursor 0 80 0 72 Additional Cursors 0 40 0 36 Enter 1 20 1 06 IA ERAN PEA AA E A IA A SESS A A AAN A E ENE NI RP RAT ES 2 How does menu selection time vary with menu item position or the number of keystrokes Two types of menu item selection tasks have been identified in the literature category matches and identity matches The category match matching the destination to the facility type e g matching McDonald s to the fast food restaurants category has been shown to require more time than the identity match matching the target destination to the exact item on the screen The results of this experiment showed no significant difference between the two types of menu item selection tasks for this system Although this result appears contrary to the literature several attributes of the experiment could explain the lack of a difference between the two types
39. shown in Table 7 total task time estimates using J2365 were created for each of the 16 trials in Part 2 of the experiment test trials performed while the vehicle was parked Because the experimental timing of these tasks began after the first key either the Destination or the Route button was pressed the total task time estimates were also created starting at that point After the first key was pressed refer back to Figures 3 and 4 for a diagram of the entry tree there were 5 steps Select the entry type quickstop address book or previous destination Select the destination category if using the quickstop feature Select All or Fast Food if selecting a restaurant from quickstop Select the desired destination from the item list Press the OK button to confirm the destination and route preferences oh YD NY ch As an example for trial 1 the driver was instructed to select Dong Yu s China Market from the grocery store list After pressing the Route button the driver pressed Enter to select quickstop denoted by the 1 2 second enter element under Entry Type Next the driver selected the category grocery store which was the fifth item in the list This operation under Category Select required a first cursor keystroke C1 0 8 3 additional cursor keystrokes C2 at 0 4 seconds for a total of 1 2 seconds and an enter keystroke E 1 2 Since the category was not a restaurant no keystrokes were required under the All Fast
40. tors have thus been refined using data from Olson and Nilsen 1987 for spreadsheet use and data from Manes Green and Hunter 1998 for entering data into a Siemens Ali Scout Navigation System Table 6 SAE J2365 operator element times seconds Code Description Time Adjusted s Time s Note 1 Reach near From steering wheel to other parts of 0 56 the wheel stalks or pods each far From steering wheel to center console 0 81 1 ursor once Press a cursor key once 0 80 1 44 ursor 2 times or Time keystroke for the second and 0 40 0 72 each successive cursor keystroke Letter or space 1 Press a letter or space key once 3 OJO U Q D leg Letter or space 2 Time keystroke for the second and 0 50 0 90 times or more each successive cursor keystroke Number once Press the letter or space key once 0 90 1 44 Number 2 times or Time keystroke for the second and 0 45 0 81 more each successive number key Press the enter ke 1 20 2 16 Press the function keys or shift 1 20 2 16 shift Time mental operation 2 70 Search for something on the display 2 30 4 14 Response time of Time to scroll one line 0 00 0 00 system scroll m Response time of Time for new menu to be painted 0 50 0 50 E system new menu Note 1 The final column shows the data adjusted for the test user population 55 60 using the 1 7 multiplier where appropriate SAE J2365 Calculations for Part 2 Test Trials while Parked As
41. ty of Michigan Transportation Research Institute Green P 2000 Crashes Induced by Driver Information Systems and What Can Be Done to Reduce Them SAE paper 2000 01 C008 Convergence 2000 Conference Proceedings SAE publication P 360 Warrendale PA Society of Automotive Engineers 26 36 Green P Levison W Paelke G and Serafin C 1993 Preliminary Human Factors Guidelines for Driver Information Systems Technical Report UMTRI 93 21 Ann Arbor MI The University of Michigan Transportation Research Institute also published as FHWA RD 94 087 McLean VA U S Department of Transportation Federal Highway Administration December 1995 Hankey J M Dingus T A Hanowski R J Wierwille W W and Andrews C 2000a In Vehicle Information Systems Behavioral Model and Design Support Final Report FHWA RD 00 135 McLean VA Federal Highway Administration U S Department of Transportation Hankey J M Dingus T A Hanowski R J Wierwille W W and Andrews C 2000b In Vehicle Information Systems Behavioral Model and Design Support IVIS DEMAnD Prototype Software User s Manual FHWA RD 00 136 McLean VA Federal Highway Administration U S Department of Transportation Japan Automobile Manufacturers Association 2000 Guideline for In vehicle Display Systems Version 2 1 February 22 Japan Manes D and Green P 1997 Evaluation of a Driver Interfaces Effects of Control Type Knob Versus Buttons
Download Pdf Manuals
Related Search