Home

Roscoe - Aero Innovation Inc.

Contents

1. subdirectories FRENCH LAN lt BATSYMBO LS fm BATMSG files ha gt BATSYMBO LS ARABIC LAN Figure 6 WOMBAT CS File Structure as Delivered This figure repre sents the subdirectory structure and the enclosed files as delivered RESULTS DIR and COMPOSIT TXT are default names for the scores subdirectory and the composite soreadsheet data file respectively There are virtually no limitations on the number of language subdirectories you can create To be recognized by the program each language subdirectory must be located inside the WOMBATCS V43 directory and have its name end with the suffix LAN With the EDIT command you have access to the text of any BATMSG file and can make all the editorial changes you want provided that you don t add too many lines of text to the file and that each individual line ends with a carriage return CR Keep in mind the fact that WOMBAT CS uses fixed width characters and uses both the text mode of the terminal and the graphics fonts displayed by METAWINDOW If new lines are added to the file some lines from the top of the page could go out of sight once invoked by WOMBAT CS In this case just edit the same file again and reduce the number of lines of text The limit on the number of lines varies depending on the contents of the screen 69 If one or more lines of text in a BATMSG file is longer than 80 characters the WOMBAT CS will suffer from a fatal runtime error and crash
2. In the summer of 1945 the US Army Air Forces AAF Aviation Psychology Program included Colonels John Flanagan Frank Geldard J P Guilford and Arthur W Melton Flanagan 1947 By this time the program s personnel had grown to about 200 officers 750 enlisted men and 500 civilians Alluisi 1994 Their wartime work was documented in 1947 in a series of 19 publications that came to be known as the blue books Volume 19 edited by Paul Fitts 1947 and titled Psychological Research on Equipment Design was the first major publication on human factors engineering or simply human engineering as it was referred to in those times 19 In August of 1945 with the war about to end the AAF Aero Medical Laboratory at Wright Field near Dayton Ohio established a Psychology Branch The group under Lt Col Paul Fitts included 21 officers 25 enlisted men and 10 civilians that first year Fitts 1947 Prominent psychologists included Majors Judson S Brown Launor F Carter Albert P Johnson and Walter F Grether Captains Richard E Jones and H Richard Van Saun First Lieuten ants Julien Christensen John Cowles Robert Gagne John L Milton Melvin J Warrick and Wilse B Webb and civilian William O Jenkins Fitts was succeeded as Technical Director by Grether in 1949 Meanwhile Arthur W Melton and Charles W Bray were building the Air Force Personnel and Training Research Center commonly referred to as Afpatrick into a hug
3. duo instruction periods and breaks can be selected by the supervisor in the custom made scripts The 90 minute test shown in Figure 11 consists of two 30 minute phases of duo performance sandwiched between three 10 minute solo phases 10 30 10 30 10 The three solo phases pro vide a learning curve for each individual to serve as a basis against which the team s CRM performance is evaluated When Duois used as a team training device the supervisor might create a script calling only for a sequence of Duo phases and breaks Such sequences may in clude scheduled breaks for brief ing debriefing purposes Figure 12 The supervisor monitors the training session and intervenes when needed The same configuration is also used within CRM classrooms as a virtual yet demanding cock pit like environment allowing demonstration and practise of learned CRM skills before a group of trainees Objective evaluation of CRM training or periodic CRM assessment is also possible as Left Member Right Member Instructions Instructions Duo Phase Duo Phase Figure 12 Typical CRM Training Sce nario This figure represents a typical Duo only training scenario named STD_TNIG SPT as delivered with DUOWOMBAT CS 84 change in the individual crew member s behavior will result in altered performance on the DuoWOMBAT test beyond that attrib utable solely to taking and retaking the solo WOMBAT CS test Conversely continued pra
4. las Bond Joseph Rigney Laddie LaPorte William Grings L S Hoffman and S A Summers The reports published by this group during the 1950s e g Bryan Bond LaPorte amp Hoffman 1956 and Bryan Rigney Bond LaPorte Hoffman amp McAllister 1959 and Grings Rigney Bond amp Summers 1953 had a major impact on the subsequent efforts of the military to cope with the problems of maintaining electronic systems of ever increasing complexity The lessons learned from this early work were later set forth in Nick Bond s 1970 Human Factors article Some Persistent Myths about Military Electronics System Maintenance which won the Jerome H Ely Award of the Human Factors Society IN CONSULTING In parallel with these developments several small companies were organized to provide research design and consulting services to industry and the government Among the earliest of these were Jack Dunlap s Dunlap and Associates Bob Sleight s Applied Psy 26 chology Corporation Harry Older s Institute of Human Relations and John Flanagan s American Institutes for Research Alluisi 1994 p 16 Of these the American Institutes for Research and Dunlap and Associates expanded into fields other than engineering psychology Still Dunlap and Associates warrants extra attention here because of its predominant association with engineering over a long period and the nature of its contributions In 1946 Captain Jack Dunlap separated from t
5. Instructions phases Recording time spent in 73 Time lapsed store in Scores file 73 Intelligence 52 Interface card Installing and setting up 101 interface card unpacking 56 Intervals See WOMBAT CS Time Required Time Allowed Default number and duration 63 Definition 74 J Jerome H Ely Award 25 Johns Hopkins University 21 joystick check before each test 56 Joysticks 37 K kamikaze pigeons 18 Keyboard keys in MS DOS 53 Keypad on WOMBAT console 37 L Laboratory of Aviation Psychology 22 Lackland AFB 19 Language issues 53 67 71 Encapsulated PostScript Files 70 Launching WOMBAT CS_ 55 Learning curves 79 Example of a typical group learning curve 76 Group curves overview 43 in DUOWOMBAT CS_ 83 Life Achievement Award Roger Shepard 45 Limitations Number of test in a single Scores file 65 loading scores into a commercial spreadsheet 77 Lockheed Georgia Company 22 Long Island 19 Longitudinal study validation 50 125 Los Angeles 23 Lotus 1 2 3 using Lotus to analyze scores 76 Lotus 1 2 3 See Composite Spreadsheet Data File Lowry AFB 19 M Manikin Test 52 Manual dexterity See Psychomotor Marietta 22 Martin Company 24 Maryland University of 15 23 Mastering Asymptotic performance level 35 Quickly mastering a task 33 mastering a Quadrant Location Task sequence 42 Mather AFB 19 Maze See Quadrant Location Task See also Quadrant Location Task Origins Medal of Science 44 Melton Arthur W 16 18 19
6. President Clinton 1995 in awarding Shepard the National Medal of Science For his creative theoretical and experimental work elucidating how the human mind perceives and represents the physical world and why the human mind has evolved to represent objects as it does In 1968 the field of cognitive psychology was dominated by theo ries of artificial intelligence based on the assumption that all think ing involved the manipulation of discrete mental symbols But Shepard was convinced that some thought processes were non sym bolic that they are more like con tinuous simulations of external events Then Shepard hit upon a great idea Using solid block fig ures he and two new graduate students Jacqueline Metzler and Lynn Cooper embarked on a series of experiments on mental rotation later reported in a bomb shell paper in Science Shepard amp Metzler 1971 They had shown students pairs of pictures of objects in different spatial orientations Sometimes the objects were the same and some times not and they measured the time the students took to decide The greater the difference in the orientation of the two objects the greater the decision time It became apparent that the students were making comparisons by mentally rotating one of the two objects into the same 45 orientation as the other The time differences even provided an indication of the rate of mental rotation But once again there were wid
7. conceptual foundations for mission analysis and the experimental study of flight display and control design principles Williams 1947 1980 Soon a second major thrust was the pioneering measure ment of transfer of pilot training from simulators to airplanes Williams amp Flexman 1949 Flexman Roscoe Williams amp Williges 1972 And by 1951 experiments were underway on the world s first air traffic control simulator Johnson Williams amp Roscoe 1951 Also on January 1 1946 Alphonse Chapanis who had served as a psychologist but not as a pilot joined the Psychology Depart ment of Johns Hopkins University Initially Chapanis concentrated on writing rather than building up a large research program with many graduate students as Williams was doing at Illinois The result was the first textbook in the field Applied Experimental Psychology a monumental work for its time and still a useful reference Chapanis Garner amp Morgan 1949 With the book s publication and enthusiastic reception engineering psychology had come of age and aviation was to be its primary field of application in the years ahead Strong support for university research came from the Depart ment of Defense particularly from the Office of Naval Research and its Special Devices Center and from the Air Force s Wright Air Development Center and its Personnel and Training Research Center The Civil Aeronautics Administration provided funds for human engine
8. script above is a typical test script that calls for an instructions period limited to 60 minutes followed by an unlimited break waiting for a user input before launching the first 10 minute solo phase Short breaks separate duo and solo phases of different lenghts until the end of the script marked by the sign at the begining of the line modify the STANDARD SPT file but copy it at will and modify the copies to suit your specific needs Any text editor can be used to modify the contents of a script file You must make sure however not to insert formatting characters that will not be read by DUOWOMBAT 91 To create a new script 1 Choose one of the DuoWOMBAT computers for the script location 2 Copy an existing script and rename the copy The script name must comply with MS DOS format 8 characters maximum and must end with the suffix SPT Example copy standard spt new spt The above DOS command will create a copy of standard spt and name it new spt 3 Edit the copy with a text editor Your modifications must be located below the line start of script and above the line end of script for the computer program to be able to read your script commands and execute them You will use a special script ing language as described below INSTRUCTIONS PERIOD Instructions periods are used by the teammates to learn and practice the various tasks involved in all phases of DuoWOMBAT To schedule an instructions per
9. you will be asked to decide whether to erase it or to go back and reenter a different reference string to avoid duplicat ing the file name This check only guarantees that the current scores file will not overwrite any other scores file existing in the current scores subdirectory If you decide to erase an old scores file at this point the data collected and stored under that filename in the composite spreadsheet data file will remain thus creating two records or rows of numbers with the same scores filename see Composite Spreadsheet Data File It is therefore good practice not to erase an old scores file and to select another reference string if prompted to do so It is highly recommended that you periodically archive or backup the score files 99 SINGLE Data STORING COMPUTER A DuoWOMBAT session generates an impressive amount of data for further analysis That data is stored in the computer that runs the script in the results subdirectory as shown in Figure 18 If the desired script is present on both computers the surpervisor can choose which computer will store the data for both teammates by choosing which computer will run the script The other computer will not store any data related to the current session but will send it all over the connection cable to the computer chosen to store it RESULTS DIR lt SCR files COMPOSIT S1 COMPOSIT D1 COMPOSIT D2 COMPOSIT D3 COMPOSIT S2 COMPOSIT D4 COMPOSIT D5 COM
10. 16 AAF Aero Medical Laboratory 19 AAF Aviation Psychology Program the Blue Books 18 Aeromedical Research Laboratory FtRucker 80 Army Navy Instrumentation Program ANIP 24 26 US Navy 16 26 Army Navy Instrumentation Program ANIP 24 26 Human Engineering Division 19 Naval Training Devices Center 27 Special Devices Center 23 27 V Validation criterion problem 48 Difficulties 32 How Predictive is WOMBAT in My Operation 77 How predictive is WOMBAT in my operation 78 Predicting distant future success 52 reminiscence effect 80 Requirements 49 Stratified Experiment 50 Test retest 79 Variability known signals of 43 132 Variance in training success 32 33 Velocity control 35 37 See also Tracking Task Video game bias 52 WwW Walter Reed Performance Assessment Battery 51 warranty period 57 Washington 79 Weapon systems 31 wheels up after landing 16 Williams AFB 19 WOMBAT Parallel Port Interface 102 WOMBAT CS Bonus Tasks See Bonus Tasks Composite Spreadsheet Data File See Composite Spreadsheet Data File Computer Failures 65 Requirements 54 101 Runtime error 69 File Annotation 59 98 Graphics Environment 57 Instructions Period amp Phases BATMSG Files See BATMSG Files Language issues 67 71 Parameters See Parameters Program launching 55 Reference String 59 Retesting the same candidate See Retest Running WOMBAT CS 59 66 Scores See Scores Several tests in one single Scores file 65 Time Required Time Allow
11. 22902 19164 36 69622 37 48042 38 63922 61731 41 65268 42 83833 43 41705 120579 46 54651 47 35666 48 18841 11011 51 194893 52 0 53 0 2 4 50 33 minutes and 7 seconds Test Phase 1 Interval ms 2 Tracking Score 3 Tracking Performance 4 Figure Rotation Score 5 Quadrant Location Score 6 Sequences Mastered 7 Digit Canceling Score 8 Total Bonus Score 9 Overall Score 10 Predicted Score 600003 599997 600000 600003 599997 600000 600000 600000 600001 5400001 10 6 32 6 13 7 35 5 14 8 36 7 14 9 36 9 17 7 39 7 13 7 34 7 15 2 37 3 14 9 37 6 17 5 39 5 132 9 330 3 RU PWNWWUO RE WwIoVNAUNwVI 3S9SUr onunu RU Ww ROR WWONWAUW WwoorForRrOGO COURAUARTI AUD ONKHWUOP RAIL w A A Figure 7 WOMBAT CS Scores Sheet Actual scores sheet at the completion of a WOMBAT CS 5 0 test of 90 minutes To view a scores sheet simply use a word processor or text editor program This scores sheet is named 1234 SCR and is found in the RESULTS DIR subdirectory on the computer s hard drive in the WOMBATCS V50 subdirectory 76 SPREADSHEET DATA FILES Whenever the number of tests justifies it we recommend the use of a commercial spreadsheet or database program to help you analyze the scores of your candidates Such commercial packages include Excel Lotus 1 2 3 or DBase to name just a few Not only will it be easier to compare one candidate s performance against the group but you will be able to view the candidate s p
12. A survey of the current status of the elec tronic reliability problem RM 1131 PR Santa Monica CA Rand Corporation Chapanis A Garner W R amp Morgan C T 1949 Applied experimental psychology New York Wiley Clinton USA President Bill 1995 Presentation of the National Medal of Science to Roger Shepard Washington DC Office of the President Craik K J W 1940 The fatigue apparatus Cambridge cockpit Report 119 London British Air Ministry Flying Personnel Research Committee Craik K J W 1944 The psychological and physiological as pects of control mechanisms with special reference to tank gunnery Unpublished Report Cambridge UK Cambridge University Applied Psychology Unit Craik K J W amp Vince M A 1943 Psychological and physi ological aspects of control mechanisms with special reference to tank gunnery Part I London UK Medical Research Coun cil Military Personnel Research Committee Craik K J W amp Vince M A 1944 Psychological and physi ological aspects of control mechanisms Part II Report BPC 44 322 London UK Medical Research Council Military Personnel Research Committee Damos D L 1972 Cross adaptive measurement of residual attention to predict pilot performance TR ARL 72 25 AFOSR 72 12 Savoy University of Illinois at Urbana Champaign Aviation Research Laboratory Fitts P M 1947 Psychological research on equipment design Resear
13. Antonio By that time the air traffic control studies for Wright Air Development Center were under way and Conrad Kraft was developing his broad band blue lighting system for radar air traffic control centers Kraft amp Fitts 1954 Williams stayed at Illinois until 1955 when he joined Hughes Aircraft Company and fashioned a second career this time as a practicing engineering psychologist Roscoe 1980 1994 He was succeeded at Illinois by Robert C Houston for two years and then by Jack A Adams until 1965 when the laboratory was temporarily closed Fitts remained at Ohio State until 1958 when he rejoined his wartime friend Arthur Melton who had moved on to the University of Michigan when Afpatrick was being dismantled Pew 1994 Fitts was succeeded by another brilliant psychologist George Briggs Howell 1994 Williams Fitts and Briggs all died of heart attacks at early ages Williams and Briggs at 48 and Fitts at 53 The laboratories of Williams at Illinois Chapanis at Johns Hopkins and Fitts at Ohio State were by no means the only ones involved in the engineering psychology field in the 1940s and early 50s but they were the ones that produced the lion s share of the engineering psychologists during that period Other universities with outside support for graduate students doing human engineer ing research in aviation included Harvard MIT California at Berkeley UCLA Southern California Tufts Purdue Michigan 23
14. DuoWOMBAT giving the teammates a chance to rest discuss their strategy or receive advice from training personnel To schedule a break anywhere in the script insert a line with the character B at the beginning of the line Unlike the other phases breaks are not always time limited except that the maximum scriptable time for breaks is 127 minutes The minimum scriptable time is 1 minute The default duration is an unlimited break Any break can be terminated before the time elapses wiht an input from either one of the teammates Specify the time in minutes for the break with a positive integer after the B or leave blank or use a negative integer for an unlimited break 94 SCRIPTING CONSIDERATIONS Operators are invited to modify and duplicate scripts at will Here are a few considerations about scripting REPEATING PHASES Consider breaking down a long period of work into several shorter phases Each time a phase ends a phase end signal is exchanged between the computers and the current re sults from both computers are stored in several scores files For example ifyou plan to have 30 minutes of duo work you can script three consecutive duo phases of 10 minutes This sequence will produce three sets of duo scores in the scores files instead of a single set of scores Operators used to running the solo WOMBAT CS will notice the similarity of script phases used in this manner to score intervals Al though pointless you can
15. Group EXPERIMENT One study of this type has been performed with the WOMBAT CS test at the University of Otago in Dunedin New Zealand O Hare 1997 An unusual opportunity presented itself in January 1995 when pilots from around the world participated in the Omarama Cup soaring competition at Omarama New Zealand preceding the 24th World Gliding Championships David O Hare of the University of Otago recruited fourteen of the competing pilots to demonstrate their situational awareness on the WOMBAT CS Test Eight participants were classified as elite pilots on the basis of their consistently superior performances in gliding competitions at national and international levels Six of these were national champions competing at the World Championships The other two were highly successful soaring competitors with distinguished careers as professional pilots with both military and test flying experience Another six pilots were also highly experienced but without notable competitive honors A control group of twelve nonpilots were closely matched with the pilots on the basis of gender age and occupational status The elite pilots had higher WOMBAT CS scores than the other highly experienced but less successful pilots and both pilot groups scored higher than the demographically matched nonpilot controls A score of 200 was a fre quently used selection threshold for the version of the test used in 1995 by O Hare the usual threshold
16. In this case identify which BATMSG file is defective and insert a carriage return at or before the 80 character of that line A few message screens must be limited to 76 or fewer characters or else there will be conflict between graphics elements of the display and the text Remember not to delete any BATMSG files in a given language subdirectory THE BATSYMBO LS Fite In each language subdirectory there is a text file named BATSYMBO LS which contains a list of words or short sentences used by WOMBAT CS to display commands menus or comments in the program Some of the words of BATSYMBO LS will appear inside colored rectangles during the test like the words TRIGGER AUTOTRACK or BONUS See a printout of the American English version of BATSYMBO LS in Appendix 2 If you take a look at the BATSYMBO LS file using the EDIT command you will see that some lines begin with one or more key words followed either on the same line or on the next indented line by acomment to help you understand the meaning or context of the key words Sometimes a Maximum Number of Characters will be specified to indicate the maximum length of the words so they fit inside the colored rectangles when displayed in the test If the comment is the next line then the whole preceding line is read into a string and the program attemps to display the whole string Sometimes the string is centered in the available area and other times it starts at the left edge of that
17. N A Jr amp Summers S A 1953 A methodological study of electronics troubleshooting skill II Inter comparisons of the MASTS test a job sample test and ten reference tests administered to fleet electronics techni cians Los Angeles University of Southern California Elec tronics Personnel Research Group Helson H amp Howe W H 1943 Inertia friction and diameter in handwheel tracking OSRD Report 3454 Foxboro MA The Foxboro Company Hick W E 1945 Friction in manual controls Report 18 Cam bridge UK Medical Research Council Applied Psychology Unit Howell W C 1994 George Edward Briggs 1926 1974 In H L Taylor Ed Division 21 members who made distinguished contributions to engineering psychology Washington DC Division 21 of the American Psychological Association Jacobs R S 1976 Simulator cockpit motion and the transfer of initial flight training TR ARL 76 8 AFOSR 76 4 Savoy University of Illinois at Urbana Champaign Aviation Re search Laboratory Ph D dissertation University of Illinois at Urbana Champaign 108 Johnson B E Williams A C Jr amp Roscoe S N 1951 A simulator for studying human factors in air traffic control systems Report 11 Washington DC National Research Council Committee on Aviation Psychology Kay H 1953 Experimental studies in adult learning Ph D dissertation Cambridge University UK Kelley C R 1968 Manual and aut
18. Output space from 300 Hex to 304 Hex Other boards in your system that conflict with this such as modem scanner or sound cards will need to be removed If you do have to remove one or more cards from the PC Computer make sure you also disable their drivers in the CONFIG SYS or AUTOEXEC BAT files The base address switches on the WOMBAT board have been set for these values but you should check them before proceeding to anew installation The correct dip settings are shown in Figure 19 101 102 To install the WOMBAT interface board proceed as follows 1 Turn off the power to the computer and remove the cover 2 Remove one of the rear cover plates and insert the board l iili Figure 19 WOMBAT Interface Board Base address switches on the WOMBAT board have been set for the values shown You should check them before installation WOMBAT PC Interface Card c 1992 AERO INNOVATION firmly into the connector on the mother board 3 Refit the cover plate screw to hold the board in place 4 Fit the cable into the socket on the interface board Push it home firmly taking care not to bend any of the pins The plug is polarized so that it can have only one position Connect the other end to the socket on the WOMBAT console 5 Arrange the control box on a table in a comfortable position and switch on the computer THe WoPPI In 2001 a new device was introduced The
19. World War II at the Applied Psychology Unit of Cambridge University England under the leadership of Sir Frederick Bartlett In 1939 this group began work on problems in the design of aviation and armored force equipment Bartlett 1943 Craik 1940 Prominent among the early contributors to engineering psychology at APU were Norman Mackworth K J W Craik Margaret Vince and W E Hick Mackworth explored problems of human vigilance Craik Vince and Hick performed classic studies on the effects of system design variables on manual control performance e g Craik 1944 Craik amp Vince 1943 1944 Hick 1945 and on direction of motion relationships between controls and displays Vince 1945 Also in 1939 in the United States of America the National Research Council Committee on Aviation Psychology was estab lished This committee first chaired by Jack Jenkins of the Univer sity of Maryland and later by Morris Viteles of the University of Pennsylvania stimulated a wide range of research in aviation psychology primarily at universities With support from the NRC Alexander C Williams Jr working with Jenkins at the University of Maryland began flight research in 1939 on psychophysiological tension as a determinant of performance in flight training These experiments involving the first airborne polygraph also appear to have been the first in which pilot performance was measured and correlated with physiological responses in f
20. chapter by Paul Fitts 1951 in the Handbook of Experimental Psychology edited by S S Stevens the major source of inspiration for graduate students for years to come and The Magical Number Seven Plus or Minus Two Some Limits on Our Capacity for Processing Information in the Psychological Review by George A Miller 1956 which encouraged quantification of cognitive activity and shifted the psychological application of information theory into high gear HISTORICAL PERSPECTIVE Taken as a whole these key reports and articles and the earlier research on which they were based addressed not only pilot selection and training deficiencies and perceptual motor problems encountered by aviators with poorly designed aircraft instrumenta tion but also flight operations aircraft maintenance and air traffic control All of these problem areas have subsequently received serious experimental attention by engineering psychologists both in the United States and abroad There are now some established principles for the design organization maintenance and operation of aviation systems that have application beyond the immediate settings of the individual experiments on which they are based 29 The early educators in the field Alex Williams Al Chapanis Paul Fitts Ross McFarland Len Mead Lick Licklider Neil Warren John Lyman Jack Adams George Briggs and Ernest McCormick had in common a recognition of the importance ofa multidisc
21. for Version 5 0 in current use is about 50 higher None of the 51 WOMBAT PERFORMANCE 60 MINUTES A SITUATIONAL AWARENESS 0 and 10 5 ATTENTION 1 1 MANAGEMENT elite pilots scored below 200 and 62 5 of them v y scored above 250 Only lt 4 gt one of the less successful pilots scored above 250 and two scored below 200 No control subject scored above 250 and 75 of them scored below 200 O Hare concluded that despite the relatively small number of subjects involved the distribution of scores was significantly different from chance WORKING MEMORY VISUAL SEARCH SPATIAL RECOGNITION IMAGERY Figure 3 WOMBAT CS Performance Model David O Hare created this model based on evidence that the strong relationship between high scores and elite pilot performance is relatively independent of so called basic indi vidual abilities O Hare 1996 personal com munication Prior to testing the glider pilots O Hare had investigated the relationships between the WOMBAT CS scores of twenty four nonpilot males varying widely in age and occupational status and their scores on four tests from the Walter Reed Performance Assessment Battery Thorne Genser Sing amp Hegge 1985 The tests were selected to measure the individual abilities hypothesized to underlie performance on the component tasks in WOMBAT namely pattern recognition short term working memory visual search and recognition and spatia
22. having much in common with the WOMBAT family These tests involved multiple sources of information with shifting priorities calling for time sharing and frequent reallocation of attention In all cases the subtasks to be performed were unlike any subtasks in complex real world systems This was done both to guard against the possible transfer of skills acquired through prior training in operating specific devices or systems and to minimize any biasing effects from cultural differences due to race language physical activities such as sports or computer facility The other key feature of such tests is that they must extend the testee to full attention capacity saturation at which point situ ational awareness is taxed to the edge of breakdown To achieve this effect such tests need to be automatically adaptive increasing the flow of information and response demands until performance starts to deteriorate then backing off to keep from crossing the threshold that leads to breakdown The adaptive logic of the WOMBAT tests has been fine tuned over the past decade to achieve precisely this 49 necessary balance between stress and overload that keeps the testee working at his or her situational awareness limit For any organization involved in the training or hiring of complex system operators the first step in the selection process before large sums have been invested in medical examinations and or operational testing in system simulators
23. little that a testee can do to improve his or her rate of scoring Figure 10 Data typically show a constant flat learning curve after the first 30 to 40 minutes of the first test A study at George Mason University near Washington D C Bruce 1994 using WOMBAT CS showed however a small in crease of performance on the order of 6 of the overall score per First Test Second Test Overall Interval Score 12 3 4 5 6 7 89 1 2 3 Ten Minute Intervals Figure 10 A Typical Two Tests Group Learning Curve A typical WOMBAT CS group learning curve during a first administration of the test followed by the first 30 minutes of a retest a day or two later 80 interval between the first and second administration The amount of the performance increase known as a reminiscence effect is generally attributed to the testee s thinking about strategy and mental rehearsal during the time between the two tests Despite the reminiscence effect Bruce demonstrated a correlation of 0 88 between the first test s ranking and the second test s ranking This indicates that if a whole group of candidates were tested twice they would keep nearly the same relative ranking at the outcome of the second test In 1999 during a pilot fatigue experiment conducted by Dr Patricia Leduc the US Army Aeromedical Research Laboratory in Ft Rucker Alabama administered 9 consecutive 30 minute WOMBAT CS sessions on elite Apache helicopter pilots and foun
24. most candidates to approach a terminal performance level on an aptitude test its application would still be cost effective if only candidates of high aptitude were selected and the probable failures were rejected before large sums had been invested in their training While situational complexity is necessary to test situational awareness it is not sufficient To avoid confounding basic aptitude with the effect of prior training in specific tasks the elements that comprise the test must be unlike any real world activities such as operating computers or controlling specific vehicles Furthermore the individual subtasks must be sufficiently simple to allow their mastery in a short practice period before combining them in the test situation Sufficient situational complexity can be achieved by the manner in which the individually simple subtasks are combined in an adaptive scenario involving multiple sources of information and multiple response alternatives A complex system operator must search for evaluate and integrate information about all relevant events conditions and resources quickly assess changes in situational priorities and allocate attention accordingly To determine an individual s apti 34 tude for meeting these demands requires a complex test in which high scores depend on Finding out what s important now and in the long run and allocating priorities accordingly Perceiving a situation correctly by avoiding preco
25. open the WOMBAT CFG file Each parameter is self explanatory If you collect WOMBAT data on more than one WOMBAT station there is a question about where you will maintain a single archive of all the scores files WOMBAT CS s embedded safety features do not insure that duplicate names will not be on some other WOMBAT station Consequently copying all scores together from different stations could result in loss of some scores files with duplicate names For this reason we suggest that you use a unique scores subdirectory name on each of yourWOMBAT stations Then copying each subdirectory to the single archive will keep the files from different stations from ever mingling See the manual for advice on changing the parameters in this file 562 The code for a standard VGA display see README for others 60 is the whole number of minutes of instruction time 1 or more 10 minutes for each scoring interval 1 to 90 9 scoring intervals in the complete test 1 to 90 2 composite data file order 1 SCR file order 2 grouped scores RESULTS DIR is the name of the scores subdirectory COMPOSIT TXT is the name of the composite spreadsheet data file RESEARCH TXT is the name of the research data spreadsheet file language directory to be used Figure 4 The WOMBAT CFG File as Delivered 62 If you decide to change any of the following e the interval duration e the number of intervals or e the composite spreadsheet data file or
26. short a WOMBAT CS session 64 Testee Identification 59 Testing Strategy Solo WOMBAT CS_ 43 Tests batteries 33 comprehensive 47 culture free 47 Digit Recall 52 discriminative 47 duration of WOMBAT CS See WOMBAT CS Time Required Time Allowed Manikin 52 objective 47 Pattern Recognition 2 52 reaction time 33 reliable 47 short term memory 33 six qualities 47 Six Letter Search 52 spatial orientation 33 valid 47 thumb switch pressing the 84 Time Limitation Immediate Exit 64 Instructions Period See WOMBAT CS Time Required Time Allowed Instructions Period Restarting a WOMBAT session 64 Time limitation breaking instructions 64 using the ESC key 64 Time Required Time Allowed 62 Tinker AFB 19 TRACON 52 Tracking Task Acceleration Control 35 Autotrack 38 Translating the name 69 131 Controlling the Duo Tracking Task 85 Description 37 DuoWOMBAT 84 expanding hexagon 38 Overview 35 predictor circles 38 stereotypical motions 38 Tracking Performance explained 74 Tracking Score explained 74 Velocity Control 35 Translating 67 71 70 Trigger switch 37 69 troubleshooting system problems 24 Tufts College 16 20 22 27 Tyndall AFB 19 U UCLA 22 24 Uncontrolled Program Exit 65 University of Illinois 44 Graduate Students in the 1970 s 48 Unreliable See Autotrack US Air Force Personnel and Training Research Center 19 21 28 School of Aviation Medicine San Antonio 22 Wright Air Development Center 21 22 28 US Army
27. should be the admin istration of WOMBAT CS to screen out a majority of those who would never succeed as complex system operators Assuming the organization has a sufficient pool of applicants to draw from it can then select those with high WOMBAT CS scores who also satisfy the organization s other hiring criteria If selecting from a large pool of already trained operators with comparable levels of experience performance in system simulators may serve as the final screen VALIDATION REQUIREMENTS The difficulty of developing tests of high predictive validity for operational aptitude involves several factors the first of which is the usual clouding of operational performance criteria against which to validate any such test As discussed in Chapter 2 if measures of operational performance are unreliable there is no way that the true predictive validity of the test can be shown statistically The pass fail criterion is virtually useless when all operational personnel are given whatever amount of simulator refreshment is needed for periodic recertification and rating scales are no better when almost everyone receives the same grade The objective evaluation of a test of situational awareness requires a valid criterion of operational success one that is unlike any of our traditional validation criteria Given the fact that instruc tors ratings and pass fail tests do not discriminate among operators accurately where can the investigator tu
28. the TRACON task in a training paradigm involving the emphasis change strat egy advanced by Daniel Gopher 1993 Initial findings show a significant interaction p lt 0 05 between the pretraining and posttraining scores of high and low WOMBAT performers relative to those of a control group who receive no emphasis change training Those with already good attention awareness and management strategies high SA scorers benefit little from the training whereas those with low SA scores benefit more O Hare concludes that his studies offer preliminary support for the notion that the WOMBAT CS test measures an aspect of SA beyond basic intelligence and motor skills Roscoe et al 1997 p 11 Evidently this ability is related to the management and control of attention Additional fragmentary evidence of the near term predictive validity of WOMBAT CS was recorded by Gavan Lintern 1994 at the University of Illinois Correlations of 0 80 and 0 78 were found between WOMBAT CS scores and respectively the number of practice landings and the number of flight hours required before flying solo those with the higher WOMBAT CS scores required fewer practice landings Predicting performance in training is easier than predicting future operational performance but these results do sug gest that WOMBA CS measures an aspect of situational awaremess INSTALLATION OF WOMBAT CS The following chapters are intended to assist the supervisor in charg
29. the filename will bear the usual SCR extension Scores files are stored in the scores subdirectory specified in the WOMBAT CFG file RESULTS DIR is the default DuoWOMBAT also generates composite scores files that are easily loaded into commercial spreadsheet programs These composite scores files are generated by any instructions periods or solo or duo phases scheduled in a script using names determined by lines in the script They are stored in the scores subdirectory inside a subdirectory that uses the name of the script that created the files and a DIR exten sion Blank Page RUNNING DuiOoWOMBAT CS LAUNCHING DuoWOMBAT CS We will store the DuOoWOMBAT CS program and all the neces sary files in a subdirectory named after the version of the DuoWOMBAT CS program current at the time of shipment For example Version 1 5 of DuoWOMBAT CS will be stored on the hard disk of your computer in a directory named Duo CS V15 Subsequent updates of the program should be stored in distinct directories to insure a smooth transfer from old versions to new ones prior to deleting superseded versions To run DuoWOMBAT CS switch to the subdirectory that contains the DuoWOMBAT CS software The batch commands LEFT BAT or RIGHT BAT supplied with your DuOWOMBAT soft ware will load in sequence everything that is required for a DuoWOMBAT CS session On the left computer type LEFT lt Enter gt On the right computer type RIGHT lt Enter gt
30. will become shorter and the reward will increase If the testee is wrong or slow the reward decreases The testee must remember the digits as they briefly appear and must also remember the last two digits if an interruption becomes necessary to deal with the primary task The illustration shows the action of answering digit 6 after 5 and 2 have appeared 43 TESTING STRATEGY All four tasks one primary and three secondaries are rela tively culture free in that each is unlike anything called for in operating any complex real world system and each can be learned quickly by the apt testee As you will see later in this manual typical group learning curves on the WOMBAT CS test initially show a slight gradual improvement in the rate of scoring with relatively uniform scoring after the third to fifth 10 minute segment The attention demands of the test are increased by situational information presented by the peripheral indicators as seen earlier To score well the testee must monitor the indicators vigilantly to follow the shifting priorities of the various tasks as indicated by their potential scoring worths and current scoring rates and to detect indications of failure modes that may require immediate termina tion of one activity in favor of another The task structure places a high premium on a rational attention allocation strategy The background tracking task with its unreliable Autotrack generating a performance scor
31. worth A two back serial digit canceling task is both tediously boring and frustrating These tasks comprise the menu of scoring alternatives avail able to the testee on request Each is relatively culture free in that it has no real world counterpart and each can be learned quickly 35 36 by the apt testee The attention demands of the WOMBAT CS test are expanded by the ever changing information presented by peripheral indicators To score well the testee must monitor the peripheral indicators vigilantly to follow the shifting priorities of the various activities as indicated by their potential scoring worths and current scoring rates and to detect indications of failure modes that may require immediate termination of one activity in favor of another Tracking Bonus prediction of the final END score The testee s performance is moni tored continuously during the test Two Performance Worth indicators are always visible in the top corners of the display and serve as guides to the testee in making the recent proper choices establishing the best strat effectiveness egy and monitoring scoring progress The left indicator relates to the primary task target tracking and the right indicator a shows the secondary Bonus task worth total points now and performance level current points The product of the Performance P and Worth W represents the current scoring rate and is computed into an overall index of recent effectiven
32. 104 Blank Page FOREWORD The world has many highly effective complex system opera tors Beyond basic intelligence and motor skills operator perfor mance depends largely on situational awareness the overarching ability to attend to multiple information sources evaluate alternatives establish priorities estimate probable outcomes for different courses of action work on whatever has the highest momentary urgency without losing sight of the routine reorder priorities as situations deteriorate or improve e act decisively in the face of indecision by others The solo WOMBAT CS test is designed to measure situ ational awareness stress tolerance and the attention manage ment abilities of individual complex system operators When two or more operators are working in teams or crews their interactions add a social dimension to individual performance that is now addressed by training in crew resource management CRM The DuoWOMBAT CS addresses the abilities of flight crews and other teams to manage their collective resources 11 12 ABOUT THIS BOOK This book is intended to provide the basic operational informa tion about the WOMBAT CS Situational Awareness and Stress Tolerance Test The first five chapters present the historical and scientific backgrounds that led to today s WOMBAT CS system The subsequent chapters deal with the technical side of WOMBAT CS and DuoWOMBAT CS from their installation to o
33. 22 Memory See Tests short term working 51 Mental imagery studies 44 discrete mental symbols 44 Messages See BATMSG Files Metawindow delivered 55 97 Michigan University of 22 Microsoft Excel See Composite Spreadsheet Data File MIT 22 24 Morgan C T 21 MS DOS commands 53 Text Editor 61 87 text editor 67 N National Medal of Science 44 National Research Council Committee USA 15 Naval Electronics Laboratory 20 Naval Research Laboratory 19 New York City 26 New York University 20 New Zealand Dunedin 50 NRC See National Research Council Committee USA NYU Electronic Research Laboratories 20 126 O Objective test 47 Office of Naval Research 21 24 25 O Hare D Stratified Experiment 50 Ohio State University 20 Laboratory of Aviation Psychology 1949 22 Omarama Cup 50 Orientation See Tests Origins Bonus Tasks 44 Otago U of 50 Overall Score 74 P PageMaker 70 Parameters in WOMBAT CFG file 62 Interval duration 62 Number of intervals 62 Test duration 63 pass fail criterion for validation 49 Pattern Recognition 2 52 Patterns See Quadrant Location Task recognition 51 pause or break in DUOWOMBAT 93 Pennsylvania University of 15 Performance asymptotic level 35 Performance Assessment Battery 51 Performance Worth See indicators Peripheral indicators 36 See also indicators performance feedback 36 periscope project 23 Personality factors 31 tests 82 Phases See also Instructions Period Ph
34. C key is pressed anytime before the testing begins the program will stop when the testee has spent all the time allowed or when 9 and O are pressed The time spent in each of the instructions phases is then written to the scores file and the program is suspended The previously exited session of candidate 1234 can be re invoked by typing WOMBAT 1234 lt Enter gt initial start WOMBAT CS runs until instructions the next time to period store the scores and then ends the test WOMBAT CS runs until the end of the instructions period then stops Figure 5 The Use of the ESC Button 65 The system will search for the file 1234 SCR created during a previous session and stored in the current scores subdirectory If the file is not found the program stops with an error message Figure 5 If the file is found the program will append the new data to the end of the previous data in the file no matter whether the previous session was a complete test or a shortened test There is no limitation to the number of sessions that can be appended to a single scores file If the file 1234 SCR is present in the current scores subdirectory the instructions period is entered near its end and the user is given up to 10 minutes of additional practice During this practice the scheduled exit mechanism is disabled That is pressing ESC will have no effect After the 10 minutes are up the test begins and can be st
35. Columbia and Maryland Several prominent engineering psycholo gists were mentored by Ernest McCormick at Purdue in the late 1950s and early 60s IN THE AVIATION INDUSTRY The students of Williams and Fitts invaded the aviation indus try in the early 1950s The boom was on especially in southwest Los Angeles where one could park along Airport Boulevard at the east end of LAX Runway 25 Left and see new North American and Douglas planes being rolled out and tested every day Douglas El Segundo alone had five different production lines running simulta neously in 1952 From a small hill near the airport one could see the plants of Douglas North American Northrop and Hughes which were growing to enormous size and Lockheed was just over the Hollywood Hills in Burbank Strange planes like the Northrop flying wing flew low over the Fox Hills Golf Course Stanley N Roscoe was Williams first student at Illinois and received his Ph D in 1950 but stayed on at the lab for two years to complete a flight by periscope project for the Navy s Special Devices Center Roscoe Hasler amp Dougherty 1952 1966 Then in 1952 Roscoe was recruited by Hughes Aircraft Company to organize a Cockpit Research Group and went on to become manager of the Display Systems Department Earlier that year Walter Carel who had completed all but his dissertation at Columbia University was hired by General Electric to do research on flight displays and William B Kno
36. Interface Installation on Another Computer 101 System RequireMents yesss cee sees tes EE sevens esvevteree 101 Interface Installation cece eeeceeeeneeeeeeaeeeeeeaeeeneeeees 101 Appendix 2 The DuoWOMBAT CS Cable ee 103 Cable Configuration ce ccceecceeeccneceeseceeeeeeeesenaeeeaneees 103 REfEreNCES noron cto seston ea cdine Heston aac eine eedoedae dos Wastes eae goseedes 105 Authors Index 352 ccd de ccs ieneds Sevcdesoveds idveds covets ieds comets ives comedies 113 List OF FIGURES Figure 1 WOMBAT CS Tracking Task ccccecccescceeceeeeeeeeseeeeeeeees 37 Figure 2 WOMBAT CS Bonus Menu ssssssssesssseserseseesesersesesseserseseo 40 Figure 3 WOMBAT CS Performance Model cccessceeseeeeeeeeeeeees 51 Figure 4 WOMBAT CFG File as Delivered ccccccecceees nescence enone 61 Figure 5 Use of the ESC Button cece cee cece eres ee eeeeea ne eees 64 Figure 6 WOMBAT CS File Structure as Delivered eee 68 Figure 7 WOMBAT CS Scores Sheet cccccceecsseceeeeceeeeeeeeseeseeenees 75 Figure 8 Typical WOMBAT CS Group Learning Curve 0064 76 Figure 9 Normal Probability Distribution eee eee cence ee eees 78 Figure 10 Typical Two Tests Group Learning Curve ceeeee 79 Figure 11 Typical DuOoWOMBAT Test Scenario ccccec cece sees eens 83 Figure 12 Typical CRM Training Scenario
37. POSIT D6 COMPOSIT S3 COMPOSIT STO COMPOSIT DTO RESEARCH TXT STD_TEST DIR a ae COMPOSIT D1 COMPOSIT D2 COMPOSIT D3 COMPOSIT D4 COMPOSIT D5 COMPOSIT D6 COMPOSIT DTO RESEARCH TXT STD_TNIG DIR t Figure 18 DuoWOMBAT CS Scores Storage Structure as Delivered This figure represents the Duo CS V15 subdirectory structure where all the scores are stored RESULTS DIR is the default name for the scores subdirectory All SCR files are stored in this subdirectory along with folders named after the scripts that generated them In the figure above two scripts generated their respective folders The first script was the standard test script STD_TEST SPT described in Figure 11 The second script was the training script STD_TNIG SPT described in Figure 12 These folders contain all the spreadsheet data files generated by DUOWOMBAT CS 100 Blank Page INTERFACE INSTALLATION ON ANOTHER COMPUTER This section of the manual is provided in case you have to install the WOMBAT CS system in another computer SysTEM REQUIREMENTS The minimum configuration for WOMBAT CS is a PC compat ible system that includes a 486DX processor and 1 megabyte of RAM The graphics system must be color VGA Nowadays WOMBAT CS is delivered on fast Pentium PC Compatibles but this level of performance is far from being a requirement to run WOMBAT CS INTERFACE INSTALLATION The WOMBAT interface board occupies 5 locations in the Input
38. PSYCHOLOGISTS IN THE MILITARY None of the wartime human engineers had received formal training in engineering psychology indeed the term hadn t even been coined Those who became involved in the study of human factors in equipment design and its application came from various branches of psychology and engineering and simply invented the budding science on the job B F Skinner stretched the concept a bit by applying his expertise in animal learning to the design of an air to sea guidance system that employed three kamikaze pigeons who learned to recognize enemy ships and voted on which way to steer the bomb they were riding Skinner 1960 It worked fine and still would but there were moral objections After the war the field of engineering psychology quickly gained momentum The Applied Psychology Unit in Cambridge England was expanded under the leadership of Donald Broadbent who succeeded Sir Frederick Bartlett as Director Christopher Poulton s comprehensive work at APU on the dynamics of manual control integrated in his 1974 book stands as a major contribution as does his work in other areas The psychologists of the Royal Aircraft Establishment at Farnborough conducted a wide range of research under the direction of Air Marshal William Stewart with John Rolf leading the flight simulation work Alan Burrows who learned his trade under Stewart later headed a human factors unit at Douglas Aircraft in Long Beach California
39. STEM PROBLEMS In the late 1940s and early 50s an unanticipated technologi cal problem arose in the military community one that obviously had critical human components The new and complex electronics in 25 both ground and airborne weapon systems were not being main tained in dependable operating condition The weapon systems included radar and infrared guided missiles and airplanes with all weather flight navigation target detection and weapon delivery capabilities These systems had grown so complex that more often than not they were inoperable and even worse unfixable by ordinary technicians Few could get past the first step trouble shooting the failures It was becoming evident that something had to be done The first alert on the scale of the problem came from the Rand Corporation in the Carhart report which documented a host of people problems in the care of electronic equipment Carhart 1953 The technicians needed better training aiding by built in test circuits simulation facilities for practicing diagnoses critical infor mation for problem solving and objective performance evaluation To address these problems the Office of Naval Research in 1952 contracted with the University of Southern California to establish an Electronics Personnel Research Group with the mission of focusing on the people aspects of maintaining the new systems coming on line The original EPRG organized by Glenn Bryan included Nicho
40. Seeger C C Smith Alex Williams and Jay Wray Not all of the reports of this genre were the products of large committees Others written in academia usually under military sponsorship included Handbook of Human Engineering Data generally referred to as The Tufts Handbook produced at Tufts College under a program directed by Leonard Mead for the Navy s Special Devices Center and heavily contributed to by Dunlap and Associates Tufts College amp US Naval Training Devices Center 1949 followed by Vision in Military Aviation by Joseph Wulfeck Alexander Weisz and Margaret Raben 1958 for the Wright Air Devel opment Center Both were widely used in the aerospace industry 28 Some Considerations in Deciding about the Complexity of Flight Simulators by Alex Williams and Marvin Adelson 1954 at the University of Illinois for the USAF Personnel and Training Research Center A Program of Human Engineering Research on the Design of Aircraft Instrument Displays and Controls by Alex Williams Marvin Adelson and Malcolm Ritchie 1956 at the Univer sity of Illinois for the USAF Wright Air Development Center Adelson went on to form the first human factors group in the Ground Systems Division of Hughes Aircraft and Ritchie formed his own research and consulting company in Day ton Ohio Perhaps the two most influential articles in the field during the 1950s were Engineering Psychology and Equipment Design a
41. Shepard by B atrice Favereau Page 85 amp 93 cartoons by Fran ois Bougie All other illustrations and figures by Jean LaRoche Legal Deposit of the second edition March 2001 National Library of Canada Bibliotheque nationale du Qu bec ISBN 2 9803682 7 X Printed in Canada by A G M V Marquis inc Stanley N Roscoe Louis Corl Jean LaRoche Predicting Human Performance 5 0 Edition HELIO PRESS Blank Page CONTENTS List Of Figures oisin aen asia sesisdseeiseiaeseaseas cyiaedeeseaesavestvedee tes 9 List of Illustrations 3 sic iscscsccassacdessissaseasiecsdssiavascesnscdecstacesevas 9 POPE WOT 355 iis Bees i Rete aad anaoa Sons AAE o PECAR Seas d Leaded Cade addons 11 About this Dook airesin ideia aa ai Aa 12 Chapter 1 The Adolescence of Engineering Psychology 13 CONTE ies Sobactins cia lteoed E EE EAEE ete etocteetemnenityes 13 Principle and DeSign cc ccc cece g a a 13 Selection and Training seteriai eraten i anie 14 Application sssr Srey decisir riri iEn ees ys EE E EE Ea 15 The Trail Bla erS siosan ea a aa ea aE 15 Human Ensineering seee ai aaa daR e ae iee 16 PsyChoacousties sosise resarses ven dad dee ikan ETN iii 17 Enter the Engineering Psychologists eessseerssrerrerereee 18 Tr THO Military iio onenian aeni dene enee sen REEE 18 n Academiae era edu a Eei es Aa aR AS 21 In the Aviation Industry s sseesessesessrsreresrerreresreresse 23 In Troubleshooti
42. Stanley N Roscoe Louis Corl Predicting Human Performance 5 0 Edition Helio Press Canadian Cataloguing in Publication Data Roscoe Stanley Nelson 1920 Predicting human performance Includes bibliographical references and index ISBN 2 9803682 7 X 1 Human engineering Computer programs 2 WOMBAT CS Computer file 3 Performance Evaluation Computer programs 4 Employees Rating of Computer programs 5 Stress Psychology Evaluation Computer programs I Corl Louis 1944 II LaRoche Jean 1962 Title TA166 R67 2001 620 8 2 02855369 C2001 940258 9 This work is subject to copyright All rights are reserved whether the whole or part of the material is concerned specifically the rights of translation reprinting reuse of illustrations recitation broadcasting reproduction on microfilms or in any other ways and storage in data banks Quotations are authorized in scientific literature provided the source is mentioned Other unauthorized use or reproduction of any kind for advertisement journals brochures newspapers magazines internal documents company manuals printed electronic or photographic whether the sources are mentioned or not will be considered a breach of copyright and prosecuted accordingly We mean it 2001 Helio Press Inc 970 Mont e de Liesse suite 210 Saint Laurent QC H4T 1W7 Phone 514 336 9310 Fax 514 336 9400 Email info aero ca Page 44 portrait of Roger
43. WOMBAT Parallel Port Interface The WoPPI allows the connection of up to two standard WOMBAT consoles to a single computer via its parallel port This accessory sold separately from Aero Innovation is perfect for operating WOMBAT with laptop computers whether using the laptop s own LCD monitor or a separate plug in CRT monitor THE DuoWOMBAT Caste CABLE CONFIGURATION DuoWOMBAT CS requires a special null modem cable to link the two computers via their COM serial ports The cable is normally supplied by Aero Innovation Below is the configuration when DB 9 connectors are used and next page shows the configuration for DB 25 connectors 9 PIN Connector cable Female Female OOAnN OOK O N Oo ooNnNoOoOGOA ON S par gt x lt 103 104 25 PIN Connector cable Female Female 1 1 3 3 es ee 5 5 6 Ground 6 7 7 8 8 9 9 10 10 11 11 12 12 13 13 14 14 2 Transmit 15 15 A oe data 16 16 5TTS 17 17 6 DSR e a goea 19 19 20DCR 20 20 21 21 22 22 23 23 24 24 ine on ine on REFERENCES Adams J A 1995 Personal communication Alluisi E A 1994 Roots and rooters In H L Taylor Ed Division 21 members who made distinguished contributions to engineering psychology Washington DC Division 21 of the American Psychological Association Bartlett F C 1943 Instrument controls and display Efficient human manipulation Report No 565 London UK Medical Research C
44. an and military governmental agencies and aviation psychologists were often included on and sometimes headed such committees Three of the most influential committee reports each of which contained major contributions by Alex Williams included 27 Human Engineering for an Effective Air Navigation and Traf fic Control System Fitts et al 1951 Human Factors in the Operation and Maintenance of All Weather Interceptor Systems Licklider et al 1953 The USAF Human Factor Engineering Mission as Related to the Qualitative Superiority of Future Man Machine Weapon Systems Fitts et al 1957 The air navigation and traffic control study by the Fitts com mittee was of particular significance because in addition to its sound content it was a beautifully constructed piece that set the standard for such study reports The group Fitts assembled in cluded Alphonse Chapanis Fred Frick Wendell Garner Jack Gebhard Walter Grether Richard Henneman William Kappauf Edwin Newman and Alexander Williams The study of all weather interceptor operation and mainte nance by J C R Lick Licklider et al 1953 though not as widely known marked the recognition by the military and the aviation industry that engineering psychologists in the academic community had expertise applicable to equipment problems not available within the military at that time Licklider s committee included George Clementson Joe Doughty Bill Huggins Charles
45. an run at a time A menu of available scripts is shown on both displays following the successful use of the serial port specified in WOMBAT CFG to communicate with the other computer The operator must designate which computer will be scriptor by choosing the desired script The other computer will receive its orders from the scriptor throughout the session 89 HARDWARE EXE METASHEL EXE DUO CS EXE WOMBAT CFG FONT files Duo CS V15 E LERTE RIGHT BAT README j Scores folders are named after the scripts that generated them muon E e PROBLEMS DIR PROB files ae E uM RESULTS DIR SCRIPTS DIR SPT files AMERICAN LAN lt BATMSG files BATSYMBO LS _ BATMSG files FRENCHILAN BATSYMBO LS Figure 16 DuoWOMBAT CS File Structure as Delivered This figure represents the Duo CS V 15 subdirectory structure and the enclosed files as delivered RESULTS DIR is the default name for the scores subdirectory from WOMBAT CFG where all scores and composite scores files are stored Scripts files are stored inside the SCRIPTS DIR subdirectory and must end with SPT to be recognized as scripts by DuoWOMBAT CS The subdirectories of the scores subdirectory RESULTS DIR bu default are named for each script that is used to write data on that computer with the SPT extension replaced by DIR CREATING A New Script The easiest way to crea
46. and extension of a paper and pencil test used by Professor Donald Johnson in an experimental psychology class at the University of Illinois in 1946 We have no idea where that test came from and have lost touch with Professor Johnson but graduate students were mightily impressed with the wide individual differences in the scores of an already highly selected group of psychology students Our version of the task has some of the characteristics of a temporal as well as spatial maze and it is inherently motivating people like it The premium is on the quick recognition of each new pattern of numbers Digit CANCELING Running memory tasks have a long history with much of the early work done in England At Cambridge University Harry Kay 1953 systematically investigated delayed digit canceling with 1 back 2 back 3 back and 4 back responses called for Immediate and 1 back responses were almost without error From 1 back to 4 back error rates rose rapidly The 4 back task was impossible for many a few were able to develop rehearsal strategies that worked occasionally Later it has been found that a longer interstimulus interval than Kay used allows some to handle the 4 back responses consistently For his Ph D dissertation at the University of Illinois Robert North 1977 paired the 1 back task concurrently with each of three other tasks designed to measure immediate memory classification and tracking abilities respectively He also
47. and you will have HARDWARE METAWINDOW and Duo CS in the appropriate sequential order 97 98 REFERENCE STRINGS AND FILE ANNOTATIONS When DuOoWOMBAT CS is launched as a new start the user is asked to enter a reference string on the computer that is the source for the script and file annotations on both computers The refer ence string must be entered by the user for the program to be able to store the exercise results in one of the scores files It can also allow the user to match a team of candidates with their results in a coded manner If the string you type contains a period then the string as you enter it will be used to name a DOS file If there is no in the string then SCR for SCoRe file will be added to the end and used to name a DOS file In both of these cases an error will be generated if the string is not a legal DOS name It is recommended that you create and maintain a consistent system for determining a unique refer ence string assigned to each team tested or trained on DuOWOMBAT The file annotations are optional and can be any message up to 30 characters long The information is only copied by the program into the final results file for future reference If you do not wish to enter a file annotation just press lt Enter gt If DuoWOMBAT CS detects another file in the scores subdirectory named in WOMBAT CFG the current scores subdirectory with the same name as the one created from the reference string
48. ant to adapt the sequence of events to suit their specific requirements to assess teach or demonstrate effective team behavior The result is a DuoWOMBAT totally scriptable and quite flexible but slightly more complex to set up than the solo WOMBAT software This chapter is intended to guide the operator in configuring DuoWOMBAT to control where and how the data will be stored and to create custom made scripts The following chapter will describe how to run DuoWOMBAT In the next edition of this book we will describe how to retrieve and read the data produced by DuOWOMBAT Visit Aero Innovation s Web site to download updates of this book in PDF format Duo s WOMBAT CFG Fite Locate the file named WOMBAT CFG in the Duo CS V15 subdirectory This text file lists important parameters controlling a number of functions The file delivered with Version 1 5 is shown in Figure 16 WOMBAT CFG is a user modifiable text file You can use your favorite word processing package but be careful not to add any formatting codes to the text or a simpler application such as the DOS Editor to open and modify the contents of a text file For example at the DOS prompt type 87 88 EDIT WOMBAT CFG lt Enter gt Figure 15 shows what you can expect if you open the WOMBAT CFG file Each parameter is self explanatory If you collect WOMBAT data on more than one WOMBAT station there is a question about where you will maintain a single archive of all the scor
49. area If you change or translate some or all of the key words contained in the BATSYMBO LS file make sure that you change ONLY the key words nothing else The number of lines in the file and their order must remain unchanged for WOMBAT CS to run without problems 70 TRANSLATING WOMBAT CS WOMBAT CS is normally delivered in American English and Canadian French languages WOMBAT CS can also display several other languages if certain conditions are met Languages based on the Roman Cyrillic or Arabic alphabets can easily be used in the BATMSG files Other languages such as the ones based on Japa nese or Chinese characters may possibly be used with some restrictions Contact Aero Innovation s Technical Support for more details on translating WOMBAT CS On occasion it may be more cost effective to translate the printed documentation only such as the Candidate s Manual which follows step by step the WOMBAT CS instructions Encap sulated PostScript EPS graphics and Adobe PageMaker 6 5 templates are available free of charge through Aero Innovation for both MS Windows 98 and Apple MacIntosh platforms to help you publish your own version of the WOMBAT CS Candidate Manual If you want to add a new language to your WOMBAT CS software German in the following example proceed as follows e At the C WOMBATCS V50 gt prompt type XCOPY AMERICAN LAN GERMAN LAN V lt Enter gt Then type the following command CD GERMAN LAN lt En
50. ases pigeons 18 pilot error 16 Pilot Group Stratified Experiment 50 plotting the scores 76 polygraph airborne 15 Port Washington 19 Pouch See Bonus Tasks Practice landings vs WOMBAT 52 President Clinton 44 Primary Task See Tracking Task Controlling the Duo Tracking Task 85 Duo Tracking Task 84 127 principles of design 13 Priorities allocating priorities 34 responding to shifting 36 43 Procedural compliance 39 Project Cadillac 20 Protecting previous data 62 psychoacoustics 17 Psychomotor dexterity 33 Purdue 22 23 24 Q Quadrant Location Task description 41 Duet Quadrant Location Task 86 Origins 45 Overview 35 Quadrant Location Score explained 74 Race bias 47 Rand Corporation 22 25 Randolph AFB 19 Rate of scoring See Scoring rate Ray Lyman Wilbur Award 44 RCA 24 Reaction time See Tests Real World See culture free Real world No counterparts 33 35 43 Recent effectiveness 36 Reference String 59 Regression or correlation See Validation How Predictive Reliable test 47 reminiscence effect 80 Repeating Phases 94 Requirements Hardware 54 101 102 research the Research Data File 77 Restarting a WOMBAT CS session 64 Results See Scores Archiving 60 98 RESULTS DIR See Scores subdirectory Retest See Validation reminiscence effect 80 retyping data vs automatic spreadsheet loading 77 Riverside Research Institute 20 Roman alphabet 70 Roscoe S N 16 21 22 23 51 Rotation Task See Figur
51. ass fail criterion would be of value if approximately equal numbers of trainees passed and failed but when the ratio is four or five to one as in many training programs it is almost worthless Rating scales are no better when almost all trainees are given the same grade 33 Aside from the criterion problem development of effective aptitude tests has been crippled by the notion that performance of complex operations depends on a collection of individually simple abilities Consistent with this idea batteries have been developed to test reaction time manual dexterity short term memory spatial orientation and the like The fact that such batteries account for only about 25 percent of the variance in training success is also caused in part by the correlations among the so called basic abilities measured by the individual tests Any one or two of the tests provides almost as much predictive power as the entire battery Administering the rest of the battery is a waste THE SECRET The secret of operational aptitude testing is to recognize the complexity of what we are trying to predict and construct a measur ing instrument of similar complexity The fact that expanding a test battery adds little predictive validity does not mean that a selection test should be short to be cost effective It is wishful to expect situational awareness and stress tolerance to be revealed reliably in a short test If a day or even part of two days is required by
52. ate wartime fix a small rubber tired wheel was attached to the end of the wheel control and a small wedge shaped end was attached to the flap control on several types of airplanes and the pilots and copilots of the modified planes stopped retracting their wheels after landing When the war was over these mnemonically shape coded wheel and flap controls were standardized worldwide as were the tactually discriminable heads of the power control levers found in conven tional airplanes today PsYCHOACOUSTICS In the human engineering area of psychoacoustics the intel ligibility of speech transmitted over the noisy aircraft interphones of World War II presented serious problems for pilots and their crews At Harvard University s Psycho Acoustic Laboratory S S Stevens J C R Licklider and Karl D Kryter with help by a young George A Miller later the 77th president of the American Psychological Association conducted a series of articulation tests of standard and modified interphones at altitudes of 5 000 and 35 000 feet in a B 17 bomber Licklider amp Kryter 1944 Intelligibility was improved by peak clipping the powerful vowel sounds in human speech and then amplifying the remaining balanced mixture of vowels and conso nants Licklider amp Miller 1951 Incidentally the psychologists also showed that the B 17 could operate effectively at 35 000 feet which the Air Force had not yet fully realized 18 ENTER THE ENGINEERING
53. ation and with other psychometrists for the purpose of improving the test s scoring algorithms The data contained in both spreadsheet data files are saved in columns vertical delimited by tabs and rows horizontal delimited by carriage returns To open the file first launch your spreadsheet application then select OPEN from the FILE menu and locate the desired composite spreadsheet data file Once the loading is com pleted you will observe a number of long rows of data The first row on top contains all the column labels and each subsequent row represents one WOMBAT CS test The data numbers are exactly the same as in the SCR files they were just formatted so as to speed up spreadsheet loading operations and avoid error inducing retyping Scores FIiLE ORDER VS GROUPED SCORES ORDER There are two different orders for the data in a row as controlled by the composite data file order parameter in WOMBAT CFG In the scores file order the data shown in Figure 7 are written into the spreadsheet from left to right on each line and from the top to the bottom through all the lines In the grouped scores order the instruction phase times are written in the farthest right columns of the spreadsheet and the test data are written from top to bottom of each column of the table and from left to right through the table columns This results in interval times being in adjacent spreadsheet columns followed by all track ing scores and so on How Pr
54. c cece ee eee eee renee 83 Figure 13 Controlling the Duo Training Task ccccecceee cece enone 85 Figure 14 Duet Figure Rotation Task ccc cccceeceeecenees anes enone ees 86 Figure 15 Duo s WOMBAT CFG File as Delivered ccecceee seen 88 Figure 16 DuOWOMBAT CS File Structure as Delevered 89 Figure 17 A Standard Test Script as Delivered c cece eeee eee 90 Figure 18 DuoWOMBAT Scores Storage Structure cceec eee eeeees 99 Figure 19 WOMBAT Interface Board cece cee ees eeeee ee eeeeeaee 102 List OF ILLUSTRATIONS WOMBAT CS Performance and Worth Indicators ceeeeeeeeees 36 The BONUS OPUON enana cannsarapensaetsrapnpeevsencveswegete e I IE RR 39 Figure Rotation t ska eara ee E E EET a E iE 41 Quadrant Location task s esssesesessessseeseesesessesresesessesessesereesesseseeee 41 Digit Canceling task 3 secccdisccscccteuwitevatwsesersspcusterssecnectesvarvuvanvesier SEE 42 Portrait of Roger Shepard ssrdsn ei erinan resa ana Viia oian 44 Tolerating Frustration in DuoWOMBAT CS ssssssesssessessessessesseesess 81 Pressing and Holding the Thumb Switch cece eeee ence eees 86 DuoWOMBAT s Break Period 0 ceceeceececeeceeceeceeaeceeceeeeceeeeaeeueees 93 9 PIN Connector Cable Female Female scceceeceeeeceeeeeeeeees 103 25 PIN Connector Cable Female Female csececeeeeeeeceeeeeees
55. ch Report 19 Washington DC US Army Air Forces Aviation Psychology Program Fitts P M 1951 Engineering psychology and equipment de sign In S S Stevens Ed Handbook of experimental psy chology pp 1287 1340 New York Wiley Fitts P M Ed Chapanis A Frick F Garner W R 107 Gebhard J Grether W F Henneman R Kappauf W Newman E amp Williams A C Jr 1951 Human engineer ing for an effective air navigation and traffic control system Washington DC National Research Council Committee on Aviation Psychology Fitts P M Flood M M Garman R A amp Williams A C Jr 1957 The USAF human factor engineering mission as related to the qualitative superiority of future man machine weapon systems Washington DC US Air Force Scientific Advisory Board Working Group on Human Factor Engineering Social Science Panel Flanagan J C Ed 1947 The aviation psychology program in the Army Air Force Research Report 1 Washington DC US Army Air Forces Aviation Psychology Program Flexman R E Roscoe S N Williams A C Jr amp Williges B H 1972 Studies in pilot training The anatomy of transfer Aviation Research Monograph 2 1 Gopher D 1993 The skill of attention control Acquisition and execution of attention strategies In D E Mayer amp S Kornblum Eds Attention and performance XIV Cambridge MA MIT Press Grings W W Rigney J W Bond
56. contains a table of instruction phases and associated elapsed times in milliseconds WOMBAT CS keeps track of the total time in milliseconds the candidate devoted to each phase of the instruc tions and prints it next to the phase number At the present time there is little known benefit to an operator to record or use these times Such data may be found useful in scientific studies in the future The second part is a table of incremental scores collected for each scoring interval during the test and the last interval if the test was exited before a normal scoring interval The last line has the summary scores for the whole test Referring to Figure 7 on a later page you will find 73 74 The Interval ms is the duration of the data collection period in milliseconds The default interval is 10 minutes The default WOMBAT CS test consists of 9 scoring intervals for a total of 90 minutes The Tracking Score TS is calculated as the Tracking Worth multiplied by the Tracking Performance The TS depends not only on performance but also on the frequent playing of bonus tasks to maintain a high Tracking Worth The Tracking Performance TP figure reflects the efficiency of both the candidate and the Autotrack at tracking the moving targets 100 TS TP Perfect Tracking Score not shown on the scores sheet The Figure Rotation Score FRS is the amount of bonus points earned from the Figure Rotation task Bonus Performance multi plie
57. ctice on the DuoWOMBAT would be expected to develop team behavioral attitudes and strategies that would readily transfer to the operational situation VIEWING THE Duo TRACKING TASK During Duo phases the left hand Tracking Task appears only on the display of the teammate who presses and holds the left hand stick s thumb switch Similarly the right hand Tracking Task is only presented on the display of the teammate who presses and holds the right hand stick s thumb switch However only one teammate at a time can call up each of the Tracking Tasks Whenever both teammates attempt to view one of the Tracking Tasks by simultaneously pressing their corresponding thumb switches that task is not shown on either display and will remain invisible until one of the two teammates releases a thumb switch Pressing and Holding the Thumb Switch The situation just described is called a viewing conflict and until the conflict is resolved the Tracking Performance for that task will be zero Working out effective procedures for sharing the Tracking Tasks between teammates will help avoid costly periods of viewing conflict 85 CONTROLLING THE Duo TRACKING TASK Although only one teammate can view each of the Tracking Tasks at any given time during Duo phases both teammates can control the cursors then The four joysticks are active whether or not a Tracking Task is visible on a given teammate s display Conse quently just as there is the po
58. d by Bonus Worth The Quadrant Location Score QLS is the amount of bonus points earned from the Quadrant Location task Bonus Perfor mance multiplied by Bonus Worth The Sequences Mastered SM is the number of Quadrant Loca tion sequences that were mastered The Digit Canceling Score DCS is the amount of bonus points earned from the Two Back Digit Canceling task Bonus Perfor mance multiplied by Bonus Worth The Total Bonus Score TBS is just the bonus component of the overall score and is the sum of the QLS FRS and DCS The Overall Score OS is the sum of TS and TBS It is the Overall Final Score that should be used when selecting candidates 330 3 points in Figure 7 The other results are shown mostly for scientific research purposes The Predicted Final Score PFS is computed from the overall score to the present time and the scoring rate for the current interval extrapolated to the end of the test 75 Sir Winston Churchill WOMBAT CS Version 5 0 Initial Instruction Phase Date 1 16 1944 Using filename RESULTS DIR 1234 SCR And composite spreadsheet data file RESULTS DIR COMPOSIT TXT And research data spreadsheet file RESULTS DIR RESEARCH TXT Instruction phase and elapsed time in milliseconds 1 88523 2 56107 3 46862 18240 6 29688 7 154960 8 60591 31192 11 25652 12 27533 13 11465 23099 16 30613 2z 11317 18 30677 30409 21 3439 22 30971 108772 4179 26 30521 27 8022 28 45127 45169 31 46157 32 31530 33
59. d no evidence of improvement beyond the first administration In your own organization if you are particularly concerned about testees requests for additional testing sessions or if you believe some testees have received pretest training using a WOMBAT CS system other than yours just have everybody retested for 30 minutes a day or two later and use those scores to rank the testees This process can be aided by using two different WOMBAT CFG files possibly in two different subdirectories the first governing a 90 minute test and the second specifying a 30 minute partial retest When testing everybody twice you will short circuit any at tempt by testees to trick the system by training themselves on WOMBAT CS prior to the screening process If pretraining is not a potential factor in your organization you can probably limit the administration frequency to once per year thereby virtually elimi nating any past exposure advantage INTRODUCTION TO DuoWOMBAT CS ENHANCING AND PREDICTING TEAM PERFORMANCE The solo WOMBAT CS Situational Awareness and Stress Toler ance Test was designed to assess the inherent aptitude of individuals to operate complex systems without regard to their interactions with other individuals in a team or crew relationship The latter situation calls for additional personal attributes primarily social in nature but with a cognitive component that have gained the attention of airline management and governmen
60. d on the side where it was made Teammates must observe the current Bonus Worths on each display to decide which will earn more points for the team with a correct answer The Duet Quadrant Location Task displays identical num bers in the four quadrants to each teammate Either teammate can cancel numbers and earn points but close observation of the Bonus Worth and Bonus Performance indicators will allow teammates to decide who shold cancel which numbers to maximize the team score The Duet Digit Canceling Task will display the same se quence of digits to each teammate and either can respond Again close observation of the Bonus Worth and Bonus Performance indicators will allow teammates to decide who shold cancel which numbers to maximize the team score Figure 14 Duet Figure Rotation Task The two teammates above have agreed on doing a Duet Figure Rotation Task The left teammate controls the left hand 3 D figure while the right teammate controls the right hand 3 D figure Either teammate can respond but the program will accept only one answer The sooner a correct answer is given the sooner additional problems can be called up and solved To maximize the combined team score teammates must keep an eye on the changing Bonus Worths and determine who can earn more points by making correct re sponses ConricuriNG DUIOWOMBAT CS DuoWOMBAT CS was created with the idea that test supervi sors and CRM facilitators would w
61. der through their respective parameters in the WOMBAT CFG file you should also change the default name of the composite spreadsheet data file and or the name of the default scores subdirectory Either of these actions will produce a new composite spreadsheet data file with headers appropriate to the parameters That way the new data will not be appended to a composite spreadsheet data file that is only appropriate for the old parameters Time REQUIRED Time ALLOWED The WOMBAT CFG shown in Figure 4 specifies that the WOMBAT CS test is to run during 9 consecutive intervals of 10 minutes each for a total of 90 minutes Moving from one interval to the next is invisible and goes unnoticed by the candidate At the end of each interval WOMBAT CS records the scores in the scores file and the temporary composite spreadsheet data file WOMBATCS SPD At the end of the test the overall scores are also recorded in the scores file and the complete set of all scores is added to the end of the composite spreadsheet data file If the composite spreadsheet data file does not exist at this time it is created with a first record containing column headers that are appropriate to the kind of data being collected After writing the composite spreadsheet data file the temporary file WOMBATCS SPD is erased Research organizations may wish to vary the number and duration of intervals Modifications must be made carefully as they will affect the way scores are lab
62. e criterion of future success in complex system operations 47 48 The history of selection test validation has been a frustrating exercise yielding at best quite modest results The well known criterion problem discussed in the next section is not the only culprit Subjective measures such as interviews and peer ratings are notoriously unreliable and hence of low validity Batteries of individual basic abilities tests have some predictive power but still account for only about 25 percent of the variance in training success and have no documented correlation with operational performance The WOMBAT family has a theoretical basis supported by the experiments done at the University of Illinois in the 70s and its demonstrated operational validity is strong evidence that this approach brings selection testing to a new level During the 1970 s at the Institute of Aviation of the University of Illinois 44 graduate students earned masters degrees and 18 earned doctorates based on experiments done at the Institute s Aviation Research Laboratory Several of these experiments dealt directly with the predictive validity of various divided attention tests calling for high degrees of situational awareness In others the same or similar tests were used to extract individual differences among trainees through analyses of covariance and thereby increase the power of the experiments The tests that showed the highest predictive validities were ones
63. e Rotation Task 128 Routine maintaining the 39 Royal Aircraft Establishment 18 Running WOMBAT CS 59 66 Runtime error 69 S SAAB 24 San Antonio 22 San Diego 20 Sands Point 19 Santa Monica 22 26 Saturation point North R A studies 46 Science Pub 44 Scores Accessing analysing understanding 73 80 archiving 60 76 98 composite spreadsheet data file 76 Duo composite spreadsheet data file 98 Duo Script Names Scores Files 95 Example of a scores sheet 75 file test results 59 98 Final Predicting overview 36 Incomplete data collected 64 intervals See WOMBAT CS Time Required Time Allowed plotting on graph 76 Predicting the Final Score 74 Protecting previous data 62 Renaming subdirectory for security 62 research data spreadsheet file 77 Scores sheet explained 73 selection threshold 51 Spreadsheet Data Files 76 subdirectory 60 98 Scoring rate Overview 36 uniform after 30 minutes 43 Scripts Repeating Phases 94 Script Names Scores Files 95 Scripting Considerations 94 Unique Instructions Set in Duo 94 Scripts for DUOWOMBAT 88 Secondary Tasks See Digit Canceling Task Figure Rotation Task Quadrant Location Task Duet Bonus Tasks 85 Overview 39 Secondary tasks Digit Canceling 42 Figure Rotation 40 Quadrant Location 41 129 Secret The 33 Sequences Mastered 74 Serial Communication Duo Cable Configuration 103 Settings default 63 Short term memory See Tests signals of variability 43 S
64. e differences in the mental rotation speeds of individuals and these differences are believed to be directly related to how people perceive and interpret complex visual scenes in everyday life or in operating complex systems as well as in the laboratory Our Figure Rotation Task differs from Shepard s in that the computer animated figures can be rotated manually but there are still wide differences in decision times depending on how much individuals rotate the figures manually versus their facility in mental rotation On August 5th 2000 Roger N Shepard was awarded the American Psychological Foundation APF Gold Medal Award for Life Achievement in the Science of Psychology for rendering objective and quantitative what had seemed irremediably subjective and qualitative particularly through his invention of nonmetric multidimensional scaling and his introduction of methods of probing nonverbal internal processes and representations with external test stimuli as in his chronometric studies of imagery mental rotation and apparent motion His proposal that fundamental psychological principes such as his exponential law of generalization and his least path principle of mental transformation have arisen as adap tations to universal features of the world points toward a psychologi cal science partaking of the mathematical elegance and universality of physical science QUADRANT LOCATION This pattern recognition test is an adaptation
65. e of the WOMBAT CS implementation Described are different procedures regarding proper installation techniques how to run the software language issues accessing and understanding the scores running an in house validation program contacting the RO INNOVATION Technical Support De partment for supplementary information To benefit fully from the second part of this book the reader should know the following basic MS DOS commands and lt key board keys gt e lt Enter gt e DIR e COPY e lt ESC gt e DIR W e XCOPY e A B C e DIR P e lt CTRL BREAK gt e CD e DEL e AUTOEXEC BAT e CD e EDIT 53 54 INSTALLATION WOMBAT CS runs on most PC Compatible computers If you need to install WOMBAT in a computer other than the one supplied by Aero Innovation please refer to the annex at the end of the present manual You may contact Aero Innovation Technical Sup port for the proper installation procedures and the latest updates on the WOMBAT CS program The phone number is 1 514 336 9310 the email connection is info aero ca and technical information is also found on the Web at http www aero ca Each WOMBAT CS system requires e aPC Compatible computer equipped with a math coproces sor not required with Pentium equipped computers that also contains the following an internal hard disk a high density 3 5 floppy disk drive labeled A a VGA 512kb min graphics board anISA8 bit WOMBAT in
66. e on the appro priate indicator rewards the allocation of some portion of the testee s attention to the known signals of variability namely the changing worths of the different tasks and the running overall indices of current scoring and predicted final score How well the tracking and the bonus tasks are performed is important but only in proportion to their momentary worths To maintain high worths for all tasks requires that all be performed on aregular basis To maintain maximum target tracking worth bonus tasks must be performed frequently and conversely to maintain maximum bonus worth Autotrack failures must be attended to quickly Each time a bonus task is performed it loses an increment of worth and the others gain increments As a result the testee s scoring rate depends more on what task is chosen moment to moment than on how well it is performed The key to a high final score is effective management of the task worths 44 THE OriGINS or WOMBAT Bonus Tasks We are frequently asked why we chose these particular Bonus tasks Where did they come from What do they measure individually FIGURE ROTATION Mental imagery tasks have received far more experimental attention than any other type in recent years largely as a conse quence of the brilliant work of Roger Shepard the Ray Lyman Wilbur Professor of Social Science at Stanford University Our figure rotation task is an adaptation of the task used in the words of
67. e research organiza tion with laboratories at Mather Sted Williams Tinker Goodfellow Lowry Tyndall Randolph and Lackland Air Force Bases Promi nent psychologists included Edward Kemp at Mather Robert Gagne at Lackland and later at Lowry Lloyd Humphreys at Lackland Jack Adams at Tyndall and Bob French at Randolph In 1958 this farflung empire was dismantled by the Air Force Most of the psychologists returned to academia while others found civilian research positions in other laboratories The Navy was not to be outdone by the Air Force In late 1945 human engineering in the Navy was centered at the Naval Research Laboratory in Washington DC under Franklin V Taylor The stature of NRL was greatly enhanced by the originality of Henry Birmingham an engineer and the writing skills of Taylor a psy chologist Their remarkable 1954 work A Human Engineering Approach to the Design of Man Operated Continuous Control Sys tems had an unanticipated benefit to understand it psychologists had to learn about the electrical engineering concepts Birmingham had transfused into the psychology of manual control Another fortunate development in 1945 was the establishment of the Navy s Special Devices Center at Port Washington on Sands Point Long Island with Leonard C Mead heading its Human Engineering Division SDC invented and developed many ingenious training devices on site and monitored a vigorous university pro gram for the Office
68. e satisfied if the goal is to be achieved Next they predict on the basis of relevant validated theory or actual experimentation with simulated systems whether the functions associated with each subgoal can be satisfied more reliably and economically with automation or human participation Usually it turns out that the functions assigned to people are best performed with machine assistance in the form of sensing processing and displaying information and reducing the order of control Not only should automation unburden operators of routine calculation and intimate control but also it should protect them against rash decisions and blunders The disturbing notion that machines should monitor people rather than the converse is based on the common observation that people are poor watchkeepers and in addition tend to be forgetful This once radical notion is now a cornerstone of modern system design SELECTION AND TRAINING The selection and training of system operators enhance perfor mance within the limits inherent in the design of the system Traditional operator selection criteria have tended to emphasize general intelligence and various basic abilities believed to contrib ute to good psychomotor performance Although individuals with out reasonable intelligence and skill do not make effective opera tors it has become evident that these abilities are not sufficient To handle emergencies while maintaining routine operations calls fo
69. ed 62 Changing number amp duration of intervals 62 Immediate Exit 64 Restarting a WOMBAT CS session 64 Uncontrolled Program Exit 65 Validation See Validation WOMBAT CFG File Duo version 87 WOMBAT CFG File Solo version 61 WOMBAT FC Bonus Tasks Description Digit Canceling Task 42 Figure Rotation Task 40 Quadrant Location Task 41 instructions period amp phases 63 using the Candidate Manual 63 WOMBAT CFG File See WOMBAT CS WOMBAT CFG File Duo version 87 Solo version 76 WOMBAITCS SPD See Composite Spreadsheet Data File 133 WoPPI 102 World Gliding Championships 50 Worths maintaining high momentary worths 43 Wright Air Development Center 22 27 Wright Field 16 19 20 Y Y2K compliance 59
70. eled inside the composite spread sheet data file It is highly recommended that you change the scores subdirectory parameter each time you change one or more of the parameters mentioned above This will insure that tests adminis tered according to new parameters will have their scores saved into an independent subdirectory and consequently in a new spread sheet data file thereby protecting any previous data Feel free to 63 discuss this issue with an Aero Innovation representative before altering the number and duration of the intervals The following pages assume the default settings found in the WOMBAT CFG file upon delivery are being used namely the length and number of intervals and the names of the scores subdirectory and the composite spreadsheet data file RESULTS DIR and COMPOSIT TXT respectively Also assume that the reference number 1234 has been assigned to a candidate and is supplied to WOMBAT CS Version 5 0 at the startup screen INSTRUCTION TIME LIMITATION Candidates go through a succession of instruction pages and accompanying exercises each being known as an instructional phase These instructional phases constitute the complete instruc tions period The default time limitation for the instructions period is 60 minutes You can make the instructions period longer or shorter to suit your requirements by editing a new value into WOMBAT CFG Present users of WOMBAT CS tell us that the use of the Candidate Manual bef
71. epictive is WOMBAT CS in My Operation To answer this question you will have to conduct a comparison between the WOMBAT CS scores and ratings based on a recognized criterion such as a standardized performance test administered at a specified level of training or operational experience The criterion performance requirements for the test must be clearly understood 78 by the candidates and understood and agreed to by the evaluators who will rate the candidates To assure maximum objectivity at least two and preferably three isolated evaluators should make and record simultaneous independent assessments of the same test performance by each candidate Once both the performance ratings and the WOMBAT CS scores are completed a statistical correlation or regression analysis will show how well WOMBAT FC predicts the criterion For a meaningful validation study test 40 50 or more candidates at some specified level of training or operational experi ence Have each evaluator independently select the top and bottom 4 of the total number tested and assign them ratings of 5 best and 1 worst respectively Then have them assign ratings of 4 good to the next best 24 of the group and ratings of 2 poor to the next worse 24 of the group The middle 44 will receive a rating of 3 average Then take the means of the ratings of each candidate by the independent evaluators 44 1 2 3 4 5 Figure 9 Normal Probability Distribution Th
72. ering research via Morris Viteles and his NRC Com mittee on Aviation Psychology In 1950 that committee was com posed of Viteles as chairman N L Barr Dean R Brimhall Glen Finch Eric F Gardner Frank A Geldard Walter F Grether W E Kellum and S Smith Stevens The research sponsored by the CAA via the committee was performed mostly by universities and resulted in a series of studies 22 that became known as the gray cover reports Number 84 by A C Williams Jr and S N Roscoe 1949 described the first experi mental study of instrument displays designed for use with the new VOR DME radio navigation system Number 92 by S N Roscoe J F Smith B E Johnson P E Dittman and A C Williams Jr 1950 reported the first simulator evaluation of a map type VOR DME navigation display employing a CRT in the cockpit Number 122 described the previously mentioned first air traffic control simulator Johnson Williams amp Roscoe 1951 When Paul Fitts opened his Laboratory of Aviation Psychology at Ohio State in 1949 he attracted a flood of graduate students many of them veterans as Alex Williams had been doing since 1946 at Illinois Charles W Simon Oscar Adams and Bryce Hartman started the flow of Fitts doctorates in 1952 Simon joined the Rand Corporation in Santa Monica and Adams the Lockheed Georgia Company in Marietta Hartman embarked on his long career at the Air Force School of Aviation Medicine in San
73. es files DuoWOMBAT CS s embedded safety features do not insure that duplicate names will not be on some other DuoWOMBAT station Consequently copying all scores together from different stations could result in loss of some scores files with duplicate names For this reason we suggest that you use a unique scores subdirectory name on each of your WOMBAT CS stations Then copying each subdirectory to the single archive will keep the files from different stations from ever mingling See the manual for advice on changing the parameters in this file 562 The code for a standard VGA display see README for others 1 is the COM Port in this computer linked to the other computer 1 to 4 RESULTS DIR is the name of the scores subdirectory Figure 15 Duo s WOMBAT CFG File as Delivered Duo Scripts The different phases of DUOoWOMBAT are controlled and sequenced by the supervisor via custom made scripts or small programs Each script controls which language will be shown on the WOMBAT displays during the entire session how the data will be stored on the hard disk how long the different phases will last and in what order they will appear Like for the WOMBAT CFG file script files are user modifiable text files The files are stored inside the SCRIPTS DIR subdirectory within the Duo CS V15 directory Figure 16 All script files have a name ending with SPT Both DuoWOMBAT computers can store a number of scripts but only one script c
74. ess that is also continuously displayed for the testee shown at the right An indication of total points now current points and a prediction of the final score the E symbol based on current points plus current effectiveness extrapolated for the time remaining complete the thermometer shaped total scoring display While performing the WOMBAT test the testee receives constant perfor 37 mance feedback and extrapolated outcome based on his her previ ous choices The testee is expected to make good use of these indicators in determining the best course of action THE PRIMARY TASK TARGET TRACKING Testees interact with WOMBAT CS by means of a console containing two joysticks and a 13 button keypad The keypad consists of 10 numeric keys 0 9 left and right arrow keys anda key labeled Bonus The right hand joystick controls the position of a small cross on the screen and has a trigger switch that is used to engage an automatic tracking function Figure 1 The left hand stick controls the horizontal separation between two pairs of short vertical lines with fore and aft movements of the joystick Both sticks operate in either of two control modes In the velocity control mode the lines and the cross remain stationary until the sticks are displaced from their spring centered positions Right Hand Task Left Hand Task Left Hand Task Figure 1 WOMBAT CS Tracking Task The left hand task is to keep the two
75. ession of radar and sonar displays Report 671 San Diego US Naval Electronics Laboratory Williams A C Jr 1947 Preliminary analysis of information required by pilots for instrument flight Contract N6ori 71 Task Order XVI Interim Report 71 16 1 Port Washington NY Office of Naval Research Special Devices Center Williams A C Jr 1980 posthumously Discrimination and manipulation in goal directed instrument flight In S N Roscoe Aviation psychology pp 11 30 Ames The Iowa State University Press Williams A C Jr amp Adelson M 1954 Some considerations in deciding about the complexity of flight simulators Research Bulletin AFPTRC TR 54 106 Lackland Air Force Base Air Force Personnel and Training Research Center Williams A C Jr Adelson M amp Ritchie M L 1956 A pro gram of human engineering research on the design of aircraft instrument displays and controls WADC Technical Report 56 526 Wright Patterson Air Force Base Wright Air Develop ment Center Williams A C Jr amp Flexman R E 1949 An evaluation of the Link SNJ operational trainer as an aid in contact flight training Contract N6ori 71 Task Order XVI Technical Report 71 16 5 Port Washington NY Office of Naval Research Special Devices Center Williams A C Jr Macmillan J W amp Jenkins J G 1946 Preliminary experimental investigation of tension as a deter minant of performance in f
76. gation displays in a 1 CA 1 Link trainer Report No 92 Washington DC Civil Aeronautics Adminis tration Division of Research Shepard R N amp Metzler J 1971 Mental rotation of three dimensional objects Science 171 701 703 Skinner B F 1960 Pigeons in a pelican American Psycholo gist 15 28 37 Stroud J M 1955 The fine structure of psychological time In H Quastler Ed Information theory in psychology Glencoe IL Free Press Taylor H L Ed 1994 Division 21 members who made distin guished contributions to engineering psychology Washington DC Division 21 of the American Psychological Association Thorne D R Genser S G Sing H C amp Hegge F W 1985 The Walter Reed performance assessment battery Neurobehavioral Toxicology and Teratology 7 415 418 Tolcott M A 1994 Jerome Hirsch Ely 1924 1963 In H L Taylor Ed Division 21 members who made distinguished contributions to engineering psychology Washington DC Division 21 of the American Psychological Association Tufts College amp US Naval Training Devices Center 1949 Hand book of human engineering data Medford MA Author Vince M A 1945 Direction of movement of machine controls Report 233 London UK Medical Research Council Royal Naval Personnel Research Committee also Report 637 Cambridge UK Medical Research Council Flying Personnel Research Committee White C T 1956 Time compr
77. he US Navy joined The Psychological Corporation in New York City and imme diately established a bio mechanics division Orlansky 1994 Dunlap s initial recruits were Ralph C Channell John D Coakley Joseph Gallagher Jesse Orlansky and Martin A Tolcott Of this group all but Gallagher an accountant left The Psych Corp in 1947 to form what would become Dunlap and Associates in 1950 In addition to its main offices and laboratories in Stamford Con necticut until 1963 the company had a sizeable branch office in Santa Monica headed by Joseph Wulfeck In the 1950s Jesse Orlansky of D amp A played a key role in the forward looking Army Navy Instrumentation Program ANIP work ing closely with Douglas Aircraft the prime contractor and with Walter Carel of General Electric the originator of the contact analog concept Carel 1960 1961 Two of the best minds in the D amp A organization were those of Jerome H Ely and Charles R Kelley but in quite different ways A memorial plaque describes Ely who died at age 39 as a scholar scientist teacher and gentle man Tolcott 1994 Kelley on the other hand saw a perfect continuum between science and mysticism but his seminal research on predictor displays and his book Manual and Automatic Control 1968 were highly creative contributions IN Course SETTING During the 1950s blue ribbon committees were frequently called on to study specific problem areas for both civili
78. imulator Studying transfer of learning 31 simulator gun director tracking 16 Single Data Storing Computer 99 Situational awareness and stress tolerance 33 Defining 11 32 Social dimension 11 Six Letter Search Test 52 Smithsonian Institution 20 Soaring competition 50 Solo Phase in DUOWOMBAT 92 Southern California University of 22 24 Electronics Personnel Research Group 25 29 Spatial orientation See Tests mental imagery 51 Shepard s studies 44 Special Devices Center 19 21 Spreadsheet Data Files 76 Spreadsheet data files 77 DuoWOMBAT 98 Stamford CT 26 Stanford University CA 44 Sted AFB 19 Stereotypical motions See Tracking Task stereotypical motions Storage Duo Script Names Scores Files 95 Single Data Storing Computer 99 Strategy Testing 43 attention allocation 43 Stratified Pilot Group Experiment 50 Stress tolerance 33 String See WOMBAT CS Reference String Study Longitudinal Validation Study 50 Stratified Validation Experiment 50 Subdirectory changing 55 97 composite file 76 Language used in test amp instructions 67 Renaming for security 62 research data file 76 Scores 60 98 130 T Target tracking See Tracking Task Tasks See by proper names culture free 43 Digit Canceling 42 Duet Bonus Tasks 85 DuoWOMBAT Tracking Task 84 Figure Rotation 40 Origins of Bonus 44 Quadrant Location 41 Team Performance 81 Team relationship 81 Teammates Agreement 94 Technical Support How to contact 54 Terminating
79. iod insert a script line with the character I at the beginning An instructions period is always time limited but teammates may terminate a period at any time The time limit for an instructions period must be defined if you want it to be longer than 1 minute Define the time limit in minutes by writing an integer after the I For example I60 means the instructions period will be limited to a maximum of 60 minutes The maximum allowable duration is 127 minutes The default and minimum duration is 1 minute The time the candidate actually spends in each of the instructions periods called for in the script will be summa rized in separate columns of a spreadsheet file named with 92 the prefix from the second line of the script or the default COMPOSIT with a STO extension This spreadsheet file and other similar files will be located in the subdirectory named from the script name SOLO PHASE Solo phases are periods during which the two teammates work separately on their respective WOMBAT consoles at the same time To schedule a solo phase insert a line with the character S at the begining of the line Solo phases are always time limited The maximum scriptable time for solo phases is 127 minutes The default and minimum scriptable time is 1 minute Specify the time with an integer after the S Data for each solo phase in the script will be written to a composite spreadsheet file named with the prefix from the second
80. ion from an Aero Innovation Technician WOMBAT CS s Grapuics ENVIRONMENT The WOMBAT CS program needs a graphics environment named METAWINDOW to run properly Although the command to load METAWINDOW is normally stored in the batch command WOMBAT BAT you can manually load it by typing METASHEL I lt Enter gt The graphics environment will remain loaded until the command METASHEL K lt Enter gt is used to unload it or the power is turned off These two commands will only work if MS DOS can find the program METASHEL in the current directory or somewhere that the path environment variable points to Refer to the MS DOS User Manual for details on paths Blank Page RUNNING WOMBAT CS REFERENCE STRING AND FILE ANNOTATION When WOMBAT CS is launched as a new start the user is asked to enter a reference string and a file annotation and then to confirm the current date This Year 2000 protection allows you to run WOMBAT on non Y2K compliant computers without negative consequences on the date value saved in the test results The mandatory reference string is used by the program to create a DOS file for the storage of the test results If the string has a then the string as you enter it will be used to name a DOS file If there is no in the string then SCR for SCoRe file will be added to the end and used to name a DOS file In both of these cases an error will be generated if the string is not a legal DOS name such as a maxim
81. iplinary approach to aviation problems and their students were so trained The early giants on whose shoulders we walk could only be delighted by the extent to which all researchers and practitioners now have access to once unimagined information and technology to support creative designs based on sound behavioral engineering principles ACKNOWLEDGMENTS In preparing this historical review we have drawn on articles by Earl Alluisi 1994 Paul Fitts 1947 and Jefferson Koonce 1984 on the short biographies of George Briggs Jack Dunlap Paul Fitts Ross McFarland and Jerome Ely respectively by Bill Howell Jesse Orlansky Dick Pew Malcolm Ritchie and Marty Tolcott in the monograph titled Division 21 Members Who Made Distinguished Contributions to Engineering Psychology edited by Henry Taylor and published in 1994 by Division 21 of the American Psychological Association and on Mac Parsons 1972 book Man Machine System Experiments We also received valuable personal communications about Afpatrick from Jack Adams 1995 and about the USC Electronics Personnel Research Group and the strange planes flying low over the Fox Hills Golf Course from Nick Bond 1995 Blank Page PREDICTING HUMAN PERFORMANCE BACKGROUND While the early engineering psychologists were primarily con cerned with the human factors in equipment design many other psychologists were involved in personnel selection and the transfer of training in simu
82. is curve represents the normal prob ability distribution of human performance and the five scoring categories represent equal intervals based on the variability of scores Admittedly a 5 point rating scale does not provide as fine a discrimination as one would desire but evaluators will find it difficult enough to pick the best and worst 4 and the next best and worst 24 according to the distribution shown in Figure 9 Once the WOMBAT CS testing and the difficult task of rating candidates are completed keep the results confidential until the 79 statistical comparison is finished If you desire you can release all the figures later normally with the exception of the identities of the candidates To do the statistical tests refer to your spreadsheet Functions manual and calculate the Pearson coefficient of correlation between the two sets of scores using the WOMBAT CS scores on one side and the 5 4 3 2 1 ratings on the other side Feel free to contact Aero Innovation for support in your data analysis Make sure that your criterion based rating procedures closely followed the ones outlined above before you draw any conclusions about the predictive value of WOMBAT CS or of any other selection tests you may want to include What Ir Test A CANDIDATE More THAN ONCE According to the present WOMBAT CS users it appears that once a candidate has reached his her asymptotic performance typically at mid time during the test there is
83. it Research Group See Hughes Aircraft Company Collins Radio 24 Columbia University 20 23 Comparison See Validation How Predictive Complex system operators duties of a 33 qualities of a testing device 47 COMPOSIT TXT See Composite Spreadsheet Data File Composite Spreadsheet Data File 76 121 DuoWOMBAT 98 Renaming through WOMBAT CFG 61 Temporary WOMBAT SPD file 62 WOMBAT SPD during computer failures 65 Comprehensive test 47 CONFIG SYS 101 Configuration See WOMBAT CS WOMBAT CFG File Console 37 Control key to exit WOMBAT CS 64 Control Modes See Velocity or Acceleration Control Correlation or Regression See Validation How Predictive Cost effectiveness of any selection system 33 Crew Resource Management crew relationship 81 DuoWOMBAT CS_ 81 Measuring social dimension 11 32 criterion choosing a criterion for validation 77 criterion problem during validation 48 Criterion based rating See Validation How Predictive Culture free tasks 43 test 47 Curves See Learning curves Cyrillic alphabet 70 D Data Presentation order 77 Protecting 62 Data Files 76 Dayton OH 28 DBase See Composite Spreadsheet Data File Using DBase to analyze scores 76 Decision time in Shepard s study 44 Default settings 63 Delayed Digit Canceling See Digit Canceling Task Demographic matching See Stratified Pilot Group Experiment Department of Defense 21 Dexterity See Psychomotor Digit Recall Test 52 Digit Canceling Ta
84. ith if you want LANGUAGE IssuUES The text that appears during the instructions and during the test has been stored in text files inside language subdirectories on the computer s hard disk You can have as many language subdirectories as you wish see Figure 6 each one containing all the files of one language THe BATMSG Fires As seen in Chapter 7 the instructions period consists of instructional pages and accompanying exercises both of which are called phases Each instructions page comes from a separate file There are as many text files as pages of text in the instructions period Each file is named BATMSG x where x is a number between 000 and 999 You must not change the name of any BATMSG file Accompanying exercise phases have no text associated with them The BATMSG files are standard user modifiable ASCII files You can use your favorite word processing package or a simpler application such as the DOS Editor to change the text in any of the files For example to edit the text found at Phase 3 of the instruc tions using the DOS Editor at the DOS prompt type EDIT BATMSG 5 lt Enter gt 67 68 HARDWARE EXE METASHEL EXE WOMBATCS V50 lt _ WOMBATCS EXE WOMBAT CFG WOMBATCS SPD temporary file FONT files RESULTS DIR COMPOSIT TXT file lt RESEARCH TXT file XXXXXXXX SCR files fm PROBLEMS DIR PROB files ys AMERICAN LAN BATMSG files BATSYMBO LS Language BATMSG files
85. k Charles 24 Finch Glen 21 Fitts Paul 18 19 20 22 27 28 Flanagan John 16 18 26 Flexman R E 21 French Bob 19 Frick Fred 27 G Gagne Robert 19 Gallagher Joseph 26 Gardner Eric F 21 Garner Wendell 21 27 Gebhard Jack 27 Geldard Frank A 16 18 21 Genser S G 51 Gopher D 46 52 Grether Walter F 19 21 27 Grings Wiliam 25 Guilford J P 18 H Hartman Bryce 22 Hasler Scott G 23 24 Hegge F W 51 Helson Harry 16 Henneman Richard 27 Hick W E 15 Hoffman L S 25 Hopkins Charles 23 Horst Paul 16 Houston Robert C 22 Howe W H 16 Howell W C 22 Huggins Bill 27 Humphreys Lloyd 19 Hunter Walter S 16 J Jacobs R S 46 Jenkins Jack 15 Jenkins William O 19 Jerome H Ely Award 25 Johnson Albert P 19 Johnson B E 21 22 Johnson Donald 45 Jones Richard E 19 K Kappauf William 27 Kay Harry 46 Kelley Charles R 26 Kellum W E 21 Knowles William B 23 Koonce Jefferson 16 29 Kraft Conrad 22 Kryter KarlD 17 L LaPorte Laddie 25 Leduc Patricia 80 Licklider J C R 17 24 27 Lintern Gavan 52 Lund Max 20 Lyman John 24 M Mackworth Norman 15 McCormick Ernest 23 24 McFarland Ross 24 Mead Leonard C 16 19 27 Melton Arthur W 16 18 19 22 Metzler Jacqueline 44 Miller George A 17 28 Milton John L 19 Morgan C T 21 Muckler Fred 24 115 116 N Newman Edwin 27 Nicklas Douglass 24 Nordstrom Lenne
86. king skill and manual control New York amp London Academic Press Ritchie M L 1994 Ross A McFarland 1901 1976 In H L Taylor Ed Division 21 members who made distinguished contributions to engineering psychology Washington DC Division 21 of the American Psychological Association Roscoe S N 1980 Aviation psychology Ames The Iowa State University Press Roscoe S N 1992 From the roots to the branches of cockpit design Problems principles products Human Factors Soci ety Bulletin 35 12 1 2 Roscoe S N 1993 An aid in the selection process WOMBAT Civil Aviation Training 4 2 48 51 Roscoe S N 1994 Alexander Coxe Williams Jr 1914 1962 In H L Taylor Ed Division 21 members who made distin guished contributions to engineering psychology Washington DC Division 21 of the American Psychological Association Roscoe S N Hasler S G amp Dougherty D J 1952 1966 Flight by periscope II Making takeoffs and landings the influence of image magnification practice and various condi tions of flight Human Engineering Report SDC 71 16 9 Contract B6ori 71 Task Order XVI Port Washington NY Office of Naval Research Special Devices Center Eventually declassified and published in 1966 in Human Factors 8 13 40 110 Roscoe S N Smith J F Johnson B E Dittman P E amp Williams A C Jr 1950 Comparative evaluation of pictorial and symbolic VOR navi
87. l imagery Roscoe 1993 The tests used were Pattern Recognition 2 Digit Recall Six Letter Search and Manikin The only measure to correlate significantly with WOMBAT scores was pattern recognition indicating that WOMBAT measures something not measured by basic ability test batteries Figure 3 shows a model based on O Hare s findings 1996 personal commu nication which support the original premise that the individual tasks are relatively unimportant in the context of situational aware 52 ness What is important is how those tasks are managed to maximize the rate of scoring To be sure the ability to perform the tasks has an impact on scoring but its contribution is secondary to the management of the relative worths of the tasks In a recent study O Hare 2000 found that WOMBAT CS scores reliably predict early performance on TRACON an air traffic control task that requires high levels of SA He also found that 28 percent of the variance in scores was accounted for by general intelligence g thus leaving 72 percent to be attributed to other sources He found no evidence that WOMBAT scores depend on age over the range included on experience with conputers or computer games or on any specific underlying ability except for its sizeable overlap with general intelligence The remaining variance is attrib uted to SA O Hare is following with an investigation of the differences between high and low WOMBAT CS performers using
88. lators to the operation of complex systems mainly airplanes but also some fairly complex weapon systems The involvement of psychologists in pilot selection had started in the first World War and during World War II they became deeply involved The results of the psychomotor and other tests they developed and used were submitted to statistical analysis and the modest conclusion was that they could account for about 20 of the variance in pass fail numbers for pilot navigator and bombardier trainees THE Roots or WOMBAT CS THe NEED For most of this century psychologists have been developing precise measures of human intelligence and somewhat less precise but nonetheless useful instruments for describing human personality factors Unfortunately they have been less successful in assessing human aptitudes for operating nuclear reactors controlling air and 31 32 surface traffic directing civil disaster responses and providing emer gency medical services to name but a few of the many complex operations humans perform daily In recent years with the advent of high speed computers the military have invested heavily in the development and validation of selection batteries that now account for more than 25 of the variance in training success but still have no evident correlation with operational performance after training The need for valid tests of complex operational aptitude is increasing as the explosion in information techn
89. light The report of this 16 research was completed in 1941 but was not released until after the war Williams Macmillan amp Jenkins 1946 In 1940 John Flanagan was recruited to set up a large aviation psychology program for the US Army Several dozen leading psy chologists were commissioned starting with Arthur Melton Frank Geldard and Paul Horst Koonce 1984 With America s entry into the war Flanagan s original organization the Applied Psychology Panel of the National Defense Research Committee NDRC was greatly expanded and its work was extended into what was later to be known as the US Army Air Forces Aviation Psychology Program Flanagan 1947 The history of the NDRC Applied Psychology Panel was re corded by Charles W Bray 1948 who served as its chief succeed ing Walter S Hunter One of the projects started in 1942 was a study of Army antiaircraft artillery at Tufts College directed by Leonard Mead and William Biel which led to the development of a gun director tracking simulator Parsons 1972 Early efforts in the United States to study manual control problems systematically were stimulated by the experiments of Harry Helson and W H Howe 1943 on the effects of friction and inertia in controls HUMAN ENGINEERING While most of the psychologists in the British Royal Air Force and the United States Army and Navy were involved hands on in aviator selection and training others were occasionally called o
90. light training Report 54 Publica tion Bulletin L 503 25 Washington DC Civil Aeronautics Administration Division of Research Williams A C Jr amp Roscoe S N 1949 Evaluation of aircraft instrument displays for use with the omni directional radio range Report 84 Washington DC Civil Aeronautics Admin istration Division of Research Woodson W 1954 Human engineering guide for equipment designers Berkeley University of California Press Wulfeck J W Weisz A amp Raben M 1958 Vision in military aviation TR WADC 58 399 Wright Patterson Air Force Base OH Wright Air Development Center Blank Page AUTHORS INDEX A Adams Jack A 19 22 29 Adams Oscar 22 Adelson Marvin 28 Alluisi E A 18 26 29 Barr N L 21 Bartlett Frederick 15 18 Biel William 16 Birmingham Henry 19 Bond Nicholas 25 29 Bray Charles W 16 19 Briggs George 22 29 Brimhall Dean R 21 Broadbent Donald 18 Brown Judson S 19 Bruce D 79 Bryan Glenn 24 25 Burrows Alan 18 C Carel Walter 23 26 Carhart R R 25 Carter Launor F 19 Channell Ralph C 26 Chapanis Alphonse 16 21 27 Chapman Robert 20 Christensen Julien 19 20 Clementson George 27 Coakley John D 26 Cooper Lynn 44 Cowles John 19 Craik K J W 15 113 114 D Damos Diane 46 Dittman P E 22 Dougherty D J 23 24 Doughty Joe 27 Dunlap Jack 25 Elkind Jerome 24 Ely Jerome H 26 F Fenwic
91. line of the script or the default COMPOSIT and the extension Sn in the subdirectory named from the script name The n extension will be a 1 or 2 digit number giving the sequential position of the solo phase in the script or any instructions period Then a composit spreadsheet file named with the prefix from the second line of the script or the default COMPOSIT and with extension STO will be written to the same subdirectory with totals of all solo phases Duo PHASE Duo phases are periods of work for both teammates as a team in full interection To schedule a duo phase insert a line with the character D at the begining of the line Duo phases are always time limited The maximum scriptable time for duo phases is 127 minutes The default and minimum scriptable time is 1 minute Specify the time with an integer after the D Data for each duo phase in the script will be written to a composite spreadsheet file named with the prefix from the second line of the script or the default COMPOSIT and the 93 extension Dn in the subdirectory named from the script name The n extension will be a 1 or 2 digit number giving the sequential position of the duo phase in the script Then a composit spreadsheet file named with the prefix from the second line of the script or the default COMPOSIT and with extension DTO will be written to the same subdirectory with totals of all duo phases BREAK OR PAUSE A break is a halt in the execution of
92. lly archive or backup the score files If you are currently in the WOMBATCS V50 subdirectory the command to archive scores files onto a floppy disk in drive A is XCOPY RESULTS DIR A RESULTS DIR lt Enter gt This command will send all the files contained in RESULTS DIR to the floppy disk in drive A It will even create the directory RESULTS DIR on the drive ifit doesn t already exist Of course you can always revert to a MS Windows interface and use the mouse to copy whole directories onto floppy disks To delete the specific scores file 1234 SCR from the RESULTS DIR directory the DOS commands are CD RESULTS DIR lt Enter gt DEL 1234 SCR lt Enter gt or just DEL RESULTS DIR 1234 SCR lt Enter gt Refer to MS DOS manuals for more information on the use of the COPY and DEL commands as well as the use of wildcards and in filenames 61 THE WOMBAT CFG Fite Locate the file named WOMBAT CFG in the WOMBATCS V50 subdirectory This text file lists important parameters controlling a number of functions The file delivered with Version 5 0 is shown in Figure 4 WOMBAT CFG is a user modifiable text file You can use your favorite word processing package but be careful not to add any formatting codes to the text or a simpler application such as the DOS Editor to open and modify the contents of a text file For example at the DOS prompt type EDIT WOMBAT CFG lt Enter gt Figure 4 shows what you can expect if you
93. loss of points in the event of a complete Autotrack failure or to continue performing another task of sufficient worth to offset the points lost from a minor deterioration of tracking performance When the failed Autotrack mode is repaired after a brief delay it can be reengaged provided the manual tracking errors in all three dimensions are within the indicated target limits SECONDARY TASKS THE Bonus PoucH There is another source of activity for the testee Known as WOMBAT s Bonus pouch it is the source of three side tasks each of one minute duration the performance of which can yield rewards and penalties in various forms Any bonus task can be requested whenever the testee elects to move away from the primary task into this secondary level of activity The tracking task is primary in the sense that it cannot be ignored without serious penalty the routine must be maintained and the bonus tasks are secondary in that the testee may at any time suspend them and return to the tracking task without any penalty in the active bonus task when its play is resumed The secondary tasks provide the problems and opportunities for the testee to demonstrate not only situational awareness and procedural compliance but also spatial orientation with mental rotation temporal and spatial pattern recognition and short term working memory The three selectable bonus tasks are e a3 D Figure Rotation and matching task requires spa
94. n to deal directly with the subtle problems aviators were having in operating their newly developed machines During the war the term pilot error started appearing with increasing frequency in training and combat accident reports It is a reasonably safe guess that the first time anyone intentionally or unknowingly applied a psychologi cal principle to solve a design problem in airplanes occurred during the war and it is possible that the frequent wheels up after landing mishap in certain airplanes was the first such case Roscoe 1992 It happened this way In 1943 Lt Alphonse Chapanis was called on to figure out why pilots and copilots of P 47s B 17s and B 25s frequently retracted the wheels instead of the flaps after landing Chapanis who was the only psychologist at Wright Field until the end 17 of the war was not familiar with the ongoing studies of human factors in equipment design Still he immediately noticed that the side by side wheel and flap controls in most cases identical toggle switches or nearly identical levers could easily be confused He also noted that the corresponding controls on the C 47 were not adjacent and their methods of actuation were quite different hence C 47 copilots never pulled up the wheels after landing Chapanis realized that the so called pilot errors were really cockpit design errors and that by coding the shapes and modes of operation of controls the problem could be solved As an immedi
95. nceived assumptions and subjective biases and being vigilant Discovering rules that are not explicit through induction and deduction Recognizing serendipitous opportunities quickly and seiz ing them before they pass Ignoring irrelevant distractions and tolerating frustration when things are going badly Coping with the stress of high workload periods and poor performance indications and finally Coping with the boredom of routine tasks and resisting complacency during periods of low workload WOMBAT CS THE SCENARIO OVERVIEW The PC based WOMBAT CS Situational Awareness and Stress Tolerance Test is designed to embody all the demands and constraints described in Chapter 2 The individual tasks involve target tracking spatial orientation pattern recognition and short term working memory and on each a testee can reach his or her asymptotic performance level after a short practice period The target tracking involves velocity or acceleration control of two display cursors to match generous error windows on the targets making this test appropriate for assessing situational awareness rather than placing undue emphasis on motor skills In a 3 D figure rotation task two figures have to be rotated manually and or mentally to reveal whether they are the same mirror images or different in some other way In a quadrant location task as each pattern of numbers is learned it is replaced by a different pattern of greater scoring
96. ng System Problems seeeeseereseererees 24 IN CONSUMING i351 ernir ina i TTE A a 25 In Course Setin 8 yaranar r NEAR NAANA 26 Historical Perspective asirini res a a aa aaa NaS 28 Ackhowl edgments sisiran a a E a cnbenvscwecsoondvedn 29 Chapter 2 Predicting Human Performance 31 Back sroun es eterra vant coietee cnet nan A GebbetwelGed ehes 31 The Roots of WOMBAT CS ccceeceeeccsececseceneeeeneceaneeeaes 31 The Need ner coeds ivcti coves NEE E ERY 31 The Diffictalties naiinis a a E a OnE 32 THE GECE aie R Bs ois vin OE EE 33 Chapter 3 WOMBAT CS eeseseeresrsrerrrerrrerrerrerrrerrreseereree 35 THE SCenariO aea T A ee ae ee ae 35 OV ET VIEW eiei O sees ii E EEE EAEE a 35 The Primary Task Target Tracking seeeseererererrreee 37 WOMBAT CS s Autotrack Mode cece cece eeeeeeeee 38 Secondary Tasks The Bonus Pouch cccceeeeeees 39 Testing Strate Gy eerie iaar PEAKE EEANN ES 43 The Origins of WOMBAT Bonus Tasks 44 Figure Rotation esee eins aea a aia ES ia aS 44 Quadrant Location 2 0 0 ce cecececeeeeceececeeeececeeeeeneaeeeenens 45 Digit Canceling occcie csss sgt o cece ecensensvecvedeuersees 46 Chapter 4 Validation 20 00 0000 ceeceee rece eeeeeeeeeeeeeeeeeeeees 47 In Quest of the Id alis caie n ee aio i 47 Validation Requirements s ssssssessseesesrerersesresesrerreseesens 49 A Stratified Pilot Group Experiment eeeserrererrrerr
97. nt is an indication that there is a problem with the associated joystick and that the test should not be administered Contact Aero Innovation for a replacement joystick 2 Activate each of the joysticks thumb and trigger switches and observe the respective red cursors appearing and disappearing on the scales 3 Press each button on the WOMBAT keypad and observe the change of color of the respective keys on the screen HARDWARE will verify the integrity of the circuitry of each control you use You can access its HELP by pressing the F1 key on the computer keyboard Once the hardware test is satisfactory press the ESC key on the computer keyboard to exit the program HARDWARE You are then ready to enter the WOMBAT CS test If one or more connections do not test properly please make sure that the supplied DB 25 cable is properly connected and fastened to both the console and the WOMBAT computer interface connector located at the back of the computer The interface card is marked WOMBAT just below the DB 25 connector Be careful not to connect the cable from the console to the parallel printer connection on the back of the computer which uses the same connector type as the WOMBAT interface card If the cable doesn t seem to be the problem something might have happened during shipping Please call Aero Innovation s 57 Technical Support Department During the warranty period do not open the WOMBAT console without prior authorizat
98. of Naval Research including the original contract with the University of Illinois Aviation Psychology Laboratory Task 20 Order XVI as it was known was renewed for 20 consecutive years Mead went on to head an engineering psychology program at Tufts College and from there to the upper management of the college and eventually of the Smithsonian Institution Project Cadillac the first complex manned system simulation study was conducted at the Sands Point facility from 1948 until 1955 with experiments actually getting underway in 1951 Par sons 1972 The project initially directed by New York University grew out of the Navy s early problems with airborne combat informa tion centers CICs designed to perform surveillance functions and later interception control Robert Chapman Vince Sharkey and James Regan were prominent contributors H M Mac Parsons cut his human engineering teeth on Project Cadillac in 1950 while still a graduate student at Columbia University He stayed with the project when the NYU Electronic Research Laboratories split off as the Riverside Research Institute in 1952 In 1946 a Human Engineering Division was formed at the Naval Electronics Laboratory in San Diego under Arnold Small whose first criterion for hiring it seemed was that an applicant could play the violin in the San Diego Symphony Small who had majored in music and psychoacoustics and played in the symphony himself hired several musicians a
99. ology and associ ated automation makes more complex operations possible and the cost of placing the wrong person in charge greater than ever Increasing the information available gives the operator more to attend to and automation makes it all the more important and difficult to keep track of everything that is going on and decide when some intervention is critical This is now called situational aware ness and this ability is also centrally involved in crew resource management CRM The costs of haphazard personnel selection are not limited to those resulting from bad judgment and mismanagement of critical operations It is also costly to invest in the training of individu als who fail to reach criterion performance levels after training or worse yet pass all training tests but then are unable to stand up under operational stress As so often happens with trainees the individual may have all of the skills and knowledge normally required but be unable to put them together in the confusion of a complex incident THe DIFFICULTIES The failure to develop tests of high predictive validity for complex operational aptitude has been caused by several factors the first of which is the usual clouding of operational perfor mance criteria against which to validate any such test If measures of complex job performance are unreliable as they typically are there is no way that the high predictive validity of atest can be shown statistically The p
100. omatic control New York Wiley Koonce J M 1984 A brief history of aviation psychology Human Factors 26 499 508 Kraft C L amp Fitts P M 1954 A broad band blue lighting system for radar air traffic control centers Technical Report TR 53 416 Wright Patterson Air Force Base OH Wright Air Development Center Licklider J C R amp Kryter K D 1944 Articulation tests of standard and modified interphones conducted during flight at 5 000 and 35 000 feet OSRD Report 1976 Cambridge MA Harvard University Psycho Acoustics Laboratory Licklider J C R amp Miller G A 1951 The perception of speech In S S Stevens Ed Handbook of experimental psychology pp 1040 1074 New York Wiley Licklider J C R Chair Clementson G C Doughty J M Huggins W H Seeger C M Smith C C Williams A C Jr amp Wray J 1953 Human factors in the operation and maintenance of all weather interceptor systems Conclusions and recommendations of Project Jay Ray a study group on human factors in all weather interception HFORL Memoran dum 41 Bolling Air Force Base DC Human Factors Opera tions Research Laboratories Lintern G 1994 Personal communication Miller G A 1956 The magical number seven plus or minus two Some limits on our capacity for processing information Psychological Review 63 81 97 North R A 1977 Task components and demands as factors in dual task
101. on 77 What If I Test a Candidate More Than Once 79 Chapter 9 Introduction to DuoWOMBAT CS 0 81 Enhancing and Predicting TeamPerformance 81 DUOWOMBATHCS e tes seneskisg tssutvesteds Maeesetdes ven E E 82 Viewing the Duo Tracking Task ccccc cece cece eee eee eee 84 Controlling the Duo Tracking Task cccceccceee eee eeeee 85 The Dual B nus Tasks 2 53 annyira i 85 Chapter 10 Configuring DuoWOMBAT CS 0 eee 87 Duo s WOMBAT CFG File cccc ccc aneis naaie 87 DO SCFIPts eses e EAE EE eee 88 Creating a N w Script ccsieccsevevesersancds ASIE LEENE 89 INStFUCHONS Period iee a eE E EN 91 Solo Phases aoi erie eE KE RAEE aaa S 92 DUO PHASE ec nee eer a a a e a 92 Break Or Pause inr A NEAT A og vga aN 93 Scripting Considerations c cee eec cece eeeeeeee aia 94 Repeating Phases 2 ci occ ccctuesteahvectecesdeevetaves sheesh secsenes 94 Teammates AQreeMeNt 2 0 0 0 cee cee cee ceceeeeeeeeneeeeeeeeeeees 94 Unique Instructions Set cece eecceceeseee e 94 Script Names Scores Files ccccc cscs cece eee ee seen ees 95 Chapter 11 Running DuOoWOMBAT GS 0 0 0 eee 97 Launching DUOWOMBAT CS 1 0 cece eee ceec eee eeeeeeeeeeen ones 97 Reference Strings and File Annotations cee 98 Single Data Storing Computer cece e eee eeee eee eeeee 99 Appendix 1
102. on and in acceleration control stick displacement increases or decreases the speed of symbol motion a relatively difficult dynamic relationship that requires concentra tion patience and a delicate touch The direction of motion relations for the right hand are stereo typical normally expected However the relationship for the left hand is ambiguous not normally expected and must be learned forward for lateral expansion and backward for lateral contrac tion Attached to the cursors are small predictor circles that show the directions and amounts of control inputs WOMBAT CS s Autotrack Mope If the targets are tracked within the indicated error limits an automatic tracking function can be engaged with the right hand joystick trigger to free the testee to pursue other methods for earning points Autotrack however is prone to failures that can vary in severity Thus it must be monitored continually even though the testee is working on another task When Autotrack fails it does not disengage but results in loss of tracking performance and a flashing of the tracking performance display One possible Autotrack failure produces tracking performance only a little less than can be achieved manually in the other Autotrack fails completely and the cursors motions continue without any regard for the locations of the targets The testee will have to decide whether 39 it is necessary to take manual control to avoid a serious
103. opped by the second escape mechanism to be described below SHORTENING THE WOMBAT CS Testne TIME A second mechanism for scheduling an escape has been included in WOMBAT CS During the test it is possible to termi nate early by pressing the ESC key on the keyboard see Figure 5 The exit is queued for the next point when scores would be stored on the disk up to the maximum time of the test i e the end of the current interval A message will appear at the bottom of the screen giving the exit time that has been queued UNCONTROLLED PROGRAM Exit If a program problem a hardware failure or a power failure results in an unscheduled exit from the program the normal exit process for saving the latest data to the scores file or the composite spreadsheet data file will not take place If there are any on screen messages they should be copied before taking any futher actions The directory of the WOMBAT CS program should be examined at the earliest opportunity for the temporary file WOMBATCS SPD If it is found it contains a record of the last scores saved before the unscheduled exit and should be copied somewhere possibly with a new name before the program is run again to collect WOMBAT data otherwise it will be replaced with the new data 66 If you save the temporary file left by an unscheduled exit there are ways to incorporate the data into the composite spreadsheet data file which an Aero Innovation representative can help you w
104. ore the test shortens the time spent reading the on screen instructions We recommend that you distribute the Candidate Manual one week ahead of time to allow the testees to read it carefully Test DURATION By default the test duration is 90 minutes divided in 9 intervals of 10 minutes each This duration does not include the time spent in the instructions period The interval duration param eter and the number of intervals parameter are both found in the WOMBAT CFG file Version 4 3 amp up We recommend that you DO NOT shorten the total test duration determined by the number of intervals times the duration of each until you have acquired sufficient data to assess the consequences of administering a shorter WOMBAT CS test Please contact an Aero Innovation repre sentative to discuss the possibilities of shortening the test 64 IMMEDIATE Exit Pressing and holding the Control key CTRL and then pressing the Break key on the computer keyboard will immediately terminate a WOMBAT CS session at any time in the instructions period or during the test The incomplete data collected to that point will be stored in the scores file and the composite spreadsheet data file in the current scores subdirectory EXITING AND RESTARTING A WOMBAT CS Session To accommodate the possible need to break the instructions and testing into two separate periods a mechanism for exiting and then restarting a session has been included see Figure 5 Ifthe ES
105. ouncil Flying Personnel Research Committee Birmingham H P amp Taylor F V 1954 A human engineering approach to the design of man operated continuous control systems Report NRL 4333 Washington DC Naval Research Laboratory Engineering Psychology Branch Bond N A Jr 1970 Some persistent myths about military electronics maintenance Human Factors 12 241 252 Bond N A Jr 1995 Personal communication Bray C W 1948 Psychology and military proficiency A history of the Applied Psychology Panel of the National Defense Re search Committee Princeton NJ Princeton University Press Bruce D 1994 Personal communication Bryan G L Bond N A Jr LaPorte H R Jr amp Hoffman L S 1956 Electronics troubleshooting A behavioral analysis Los Angeles University of Southern California Electronics Personnel Research Group Bryan G L Rigney J W Bond N A Jr LaPorte H R Jr Hoffman L S amp McAllister B N 1959 The role of humans 105 106 in complex computer maintenance Los Angeles University of Southern California Electronics Personnel Research Group Carel W L 1960 Human factors in ANIP Contract Nonr 1076 00 Ithaca NY General Electric Advanced Electronics Center Carel W L 1961 Visual factors in the contact analog Report R61E1C60 Contract Nonr 1076 00 Ithaca NY General Electric Advanced Electronics Center Carhart R R 1953
106. pairs of vertical lines on both sides of the vertical segments targets of an expanding and contracting hexagon drawn in the center of the display The right hand task is to keep the cross center inside the moving circle Attached to the cursors are small predictor circles that show the directions and amounts of control inputs The combined performances of the left and right hands multiplied by the current Worth of the task produces a Scoring Rate that is indicated by the area shown in the Performance Worth indicator at the top left of the computer display as shown on Page 36 38 and the rates of movement of these symbols are proportional to the amounts of control displacement In the acceleration mode the symbols move at constant rates when the controls are centered and the stick displacements determine the changes accelerations and decelerations in their existing rates of movement In either mode the composite task is to track the vertical sides of an expanding and contracting hexagon with two pairs of short vertical lines controlled by the left stick while simultaneously tracking a target circle with the cross controlled by the right hand stick The control mode velocity or acceleration changes automatically At any time the testee may encounter either of the two modes of control and must quickly diagnose the situation and respond accordingly To recap in velocity control stick displacement controls the speed of the symbol moti
107. peration and score interpretation THE ADOLESCENCE OF ENGINEERING PSYCHOLOGY 4 This retrospective account of the emergence of engineering psychologists in the military in academia in the aviation indus try in troubleshooting system problems in consulting and in course setting for civil and military agencies is based largely on the recollections of the senior author and many years of correspondence with others of similar vintage or older CONTEXT Engineering psychology is the science of human behavior in the operation of systems Consequently engineering psychologists are concerned with anything that affects the performance of system operators whether hardware software or liveware They are in volved both in the study and application of principles of ergonomic design of equipment and operating procedures and in the scientific selection and training of operators The goal of ergonomics is to optimize machine design for human operation and the goal of selection and training is to produce people who get the best performance possible within machine design limitations PRINCIPLES OF DESIGN Engineering psychologists are concerned first with the distri bution of system functions among people and machines System functions are identified through the analysis of system operations 13 14 Engineering psychologists typically work backward from the goal or desired output of the system to determine the conditions that must b
108. performance TR ARL 77 2 AFOSR 77 2 Savoy University of Illinois Aviation Research Laboratory Ph D dissertation University of Illinois at Urbana Champaign North R A amp Gopher D 1974 Basic attention measures as predictors of success in flight training Proceedings of the Human Factors Society 18th Annual Meeting Santa Monica CA Human Factors Society O Hare D 1996 Personal communication 109 O Hare D 1997 Cognitive ability determinants of elite pilot performance In Proceedings of the Ninth International Sympo sium on Aviation Psychology Columbus The Ohio State University Department of Aviation O Hare D amp O Brien K 2000 Individual differences in situational awareness and training for complex tasks In A R Lowe amp B J Hayward Eds Aviation resource management Vol 2 pp 369 377 Aldershot Ashgate Orlansky J 1994 Jack W Dunlap 1902 1977 In H L Taylor Ed Division 21 members who made distinguished contribu tions to engineering psychology Washington DC Division 21 of the American Psychological Association Parsons H M 1972 Man machine systems experiments Balti more amp London The Johns Hopkins Press Pew R W 1994 Paul Morris Fitts 1912 1965 In H L Taylor Ed Division 21 members who made distinguished contribu tions to engineering psychology Washington DC Division 21 of the American Psychological Association Poulton E C 1974 Trac
109. r breadth and rapid selectivity of attention and flexibility in reorder ing priorities The more obstinate a system is to operate and the poorer the operator selection criteria the greater the burden on training Modern training technology is dominated by computer based teach ing programs part task training devices and full mission simula tors Engineering psychologists pioneered the measurement of the transfer of training in synthetic devices to pilot performance in airplanes starting in the late 1940s and demonstrated the effective ness of these relatively crude machines More importantly some general principles were discovered that can guide the design of training programs for systems other than airplanes 15 APPLICATION Fortunately improved human performance in system opera tions can come from all directions Ergonomic design can make the greatest and most abrupt differences in performance but improve ments in selection and training can be made more readily by operational management More immediate though usually less dramatic improvements in system effectiveness can be made through the redesign of the operational procedures used with existing systems A brief history of how all this got started during and immediately following World War II is best told by focusing on the pioneers who made it happen THE TRAIL BLAZERS Among the earliest experimental studies of the human factors in equipment design were those made during
110. raying out in the centrifuge Harry Wolbers a 1955 Warren graduate was the first engi neering psychologist hired by the Douglas Aircraft Company Wolbers was the human factors leader for Douglas in their prime contract for the Army Navy Instrumentation Program ANIP Another Warren product was Glenn Bryan who became the first director of the Electronics Personnel Research Group at the University of Southern California in 1952 and went on to head the Psychological Sciences Program at the Office of Naval Research for more than 20 years Gerald Slocum who joined Hughes Aircraft in 1953 and later earned his master s degree with Lyman at UCLA would rise to be a Vice President of the company and eventually of General Motors In the east Jerome Elkind a student of J C R Licklider at MIT formed the original human factors engineering group at RCA in the late 1950s Lennert Nordstrom a student of Ross McFarland Ritchie 1994 at Harvard organized the human factors program at SAAB in Sweden in the late 1950s Thomas Payne Douglass Nicklas Dora Dougherty Fred Muckler and Scott Hasler all students of Alex Williams brought aviation psychology to The Martin Company in the mid 1950s And Charles Fenwick a student of Ernest McCormick at Purdue became the guru of display design at Collins Radio in the early 1960s Managers in industry were gradually recognizing that aviation psychology was more than just common sense IN TROUBLESHOOTING SY
111. rerreree 50 Chapter 5 Installation of WOMBAT CS e 53 Tristalla tion rnn a E E E E a 54 Launching WOMBAT CS ienien ianari E ESS 55 The HARDWARE Program sesseeseseseeseeseeseseeseereseeseesese 56 WOMBAT CS s Graphics Environment eseeseseereseereererees 57 Chapter 6 Running WOMBAT CS 00 cece eee cece 59 Reference String and File Annotation eeeeereerrerrerrrereeree 59 The WOMBAT CFG File saristi ekpo iasi ioei ese ees 61 Time Required Time Allowed c ccs ec cece eeee eee e eens 62 Instruction Time Limitation cece ee ee eee eens 63 Test Duration esscr niEes ETERIK DCE ved vee Codes RERS 63 Immediate Exit 2sesses es lice coved caw tas des EE ETENE EIES 64 Exiting and Restarting a WOMBAT CS Session 64 Shortening the WOMBAT CS Testing Time 65 Uncontrolled Program Exit cccccce ec cec eee eee esse eeeee 65 Chapter 7 Language ISSUES cece ec cee cece ence ee eeneees 67 The BATMSG Piles vieii tiene EE ved E E 67 The BATSYMBO LS Pile sciseiesc ciiecitvetectcorveseesgeventertentves 69 Translating WOMBAT CS neposre siasii sinnekin 70 Chapter 8 The WOMBAT CS Scores ce ceeceeeeeeeeeeeeee ee 73 Reading the Scores File ccccccccccsceccesseecceesensceesenseneeees 73 Spreadsheet Data PileS sss siure iverna Te aS E 76 Scores File Order vs Grouped Scores Order 77 How Predictive is WOMBAT CS in My Operati
112. rn Surely measures of performance during training no matter how objective are not ideal criteria because the ultimate purpose of aptitude tests is not to predict immediate success but distantfuture success as an operator of any complex system In an ideal validation study a large number of applicants for complex system training would be tested all would be trained and all who completed training whether certificated or not would be assigned to specific jobs and their performances observed and 50 evaluated objectively over an extended period In addition all would be retested on the original selection test and a control group that received no training in the interim would be tested and retested to assess the effect of the first administration on the second In the real world none of these conditions save the first is practical Because a longitudinal study of all the same people over several years is not a feasible approach an alternative plan may become necessary one that will still address the predictive validity question in a realistic manner One such approach is a stratified experimental plan in which independent groups of individuals representing the various stages in the sequence of training and increasingly complex operations are tested The mean test scores for the successive groups are then compared statistically to assess the reliability of the anticipated successive increases in the group means A STRATIFIED PiLtot
113. rogression in the test by plotting the score intervals onto a graph as shown below Inside the scores subdirectory default RESULTS DIR you will find the composite spreadsheet data file default COMPOSIT TXT and the research spreadsheet data file default RESEARCH TXT The composite spreadsheet data file contains the scores from each test conducted from a WOMBAT CS subdirectory pro vided WOMBAT CFG called for adding spreadsheet data to a file with this name Each time a test ends a line of data is appended to the composite spreadsheet data file Ifyou rename or delete this file from the scores subdirectory WOMBAT CS will create a new file auto matically using whatever name is specified in WOMBAT CFG this may be required if you modify some parameters in the WOMBAT CFG file While you are archiving all the files containing results consider renaming the composite spreadsheet data file after a relocation to avoid overwriting it with another file of the same name later a Overall Interval Score 1 2 3 4 5 6 789 Ten Minute Intervals Figure 8 Typical WOMBAT CS Group Learning Curve 77 The research data spreadsheet file is similar in format to the composite file but it contains data to be used by psychometrists during scientific studies No data contained in this spreadsheet file should be used in isolation for selection purposes The test admin istrators usually share the data contained in this research file with Aero Innov
114. rt 24 North Robert 46 O O Hare D 50 Older Harry 26 Orlansky J 26 P Parsons H M 16 20 29 Payne Thomas 24 Poulton Christopher 18 Raben Margaret 27 Regan James 20 Rigney Joseph 25 Ritchie Malcolm L 24 28 Rolf John 18 S Seeger Charles 27 Sharkey Vince 20 Shepard Roger 44 Simon Charles W 22 23 Sing H C 51 Skinner B F 18 Sleight Bob 25 Slocum Gerald 24 Small Arnold 20 Smith C C 27 Smith J F 22 Stevens S S 17 21 28 Stewart William 18 Stroud John 20 Summers S A 25 T Taylor Franklin V 19 Taylor Henry 29 Thorne D R 51 Tolcott Martin A 26 V Van Saun H Richard 19 Vince Margaret 15 Viteles Morris 15 21 WwW Warren Neil 24 Warrick Melvin J 19 20 Webb Wilse B 19 Weisz Alexander 27 White Carroll 20 Williams Alexander C Jr 15 21 22 24 26 27 28 Williges B H 21 Wolbers Harry 24 Woodson Wesley 20 Wray Jay 27 Wright Field 16 19 20 Wulfeck Joseph 26 27 117 Blank Page INDEX A Acceleration control 35 38 See also Tracking Task Adobe PageMaker 70 Aero Innovation How to contact Technical Support 54 Aero Medical Laboratory Wright Field 20 Afpatrick 19 22 29 Agreement between teammates 94 airborne polygraph 15 Alphabets Arabic 70 Cyrillic 70 Roman 70 American Psychological Association 17 American Psychological Foundation APF 45 Annotation See WOMBAT CS File Annotation Applied Psycholog
115. s to the upper left quadrant the 4 button to the upper right quadrant the 7 button to the lower left quadrant and the 8 button to the lower right quadrant The illustration shows the display quadrants and their respective keypad buttons When all 32 numbers have been canceled with few errors before the end ofa 60 second trial the testee is offered the option of beginning another Quadrant Location problem while there is still some time left 42 The computer records and displays in the scores sheet the number of sequences mastered by the testee Until a sequence is canceled efficiently it reappears each time the Quadrant Location task is chosen from the Bonus Menu When a given sequence is mastered a different one appears the next time the task is selected The Two Back Digit Canceling task briefly displays a single digit from 1 to 8 inside a square drawn in the center of the display Starting with the third digit displayed as soon as each new digit appears the testee is required to press the key on the keypad matching the digit displayed two back in the sequence Once the answer has been given right or wrong another digit appears inviting the testee to match the next two back digit in the se quence Each time an answer is given the interval between the digits is adjusted according to how fast and how accurate the answers are If the testee is accurate and fast the interval
116. schedule consecutive breaks TEAMMATES AGREEMENT Only input from one teammate is required to terminate an instructions period or a break before it times out Such an input throws both teammates into the next phase whatever the level of readiness of the other teammate This potential for conflict resulting from a failure of communication is intentional However the sudden end of an instructions period that reaches its time limit while both teammates are engaged in study and practice could be quite disruptive for their performance if a solo or duo phase follows immediately in the script It is therefore recommended that a break period be inserted after an instructions period to prepare the teammates for the following work phase UNIQUE INSTRUCTIONS SET There is only one set of instructions The set contains all the screen pages of text and all the practice sessions for DuoWOMBAT thus convering all topics Sequencing more than one instructions period into a single script will simply 95 make the complete set of instructions available again for another limited time A future version of DuOWOMBAT may discriminate between solo oriented instructions and duo oriented instruction so that they could be scrited sepa rately Script NAMES Scores FILES The DuoWOMBAT program uses the reference string en tered by the test administrator to create a summary scores file If the administrator does not specify a filename exten sion by default
117. side a solid square and the other figure will be inside a dashed square The solid square means that the figure can be rotated using the two sticks on the WOMBAT console itis the active figure The testee rotates the active figure until he she sees all the details of its construction Then the testee transfers control to the other figure by pressing the right or left arrow key on the keypad 41 as appropriate and studies it as well The goal is to find out as soon as possible whether the two figures are identical mirror images or otherwise different Maximum points are earned by selecting and entering the correct answer as quickly as possible swer is given the testee is offered the option of be ginning another 3 D prob i lemwhile there stillis some l timeleft Thetesteeshould n 777777 try to solve as many prob lems as possible during the one minute trial When an incorrect answer is given no other problem is offered for the remainder of the minute l l l l l When a correct an l l l l l In the Quadrant Location task the numbers 1 to 32 are placed in groups of eight in the four quadrants on the display The task is to find each number in ascending order and press the button on the keypad that corresponds to the quadrant in which it lies thereby canceling the numbers in sequence 1 through 32 The quadrant designation number appears in a box in the center of each quadrant The 3 button correspond
118. sk description 42 Digit Canceling Score explained 74 Duet Digit Canceling Task 86 Overview 35 Discriminant test 47 discrimination 78 Distractions ignoring 34 122 Division 21 of the APA 29 DOS See MS DOS Douglas Aircraft Company 18 24 26 Douglas El Segundo 23 DTO Extension 93 Duets Digit Canceling Task 86 Figure Rotation Task 86 Quadrant Location Task 86 Dunedin NZ 50 Duo Phase Scripting a 92 Duo Scripts 88 DuoWOMBAT CS 11 82 Cable Configuration 103 Controlling the Duo Tracking Task 85 Duet Bonus Tasks 85 Introduction 81 Typical Test Configuration 83 Typical Training Configuration 83 Viewing the Duo Tracking Task 84 Duplication of file names 60 98 Duration See WOMBAT CS Time Required Time Allowed Electronic Research Laboratories 20 Electronics Personnel Research Group See Southern California U of Elite pilots See Stratified Pilot Group Experiment Encapsulated PostScript 70 Environment Metawindow Graphics Environment 57 ergonomic design 13 Error See pilot error limits in Tracking Task See Tracking Task Runtime error 69 ESC key 64 Excel See Composite Spreadsheet Data File Using Microsoft Excel to analyze scores 76 Exiting the WOMBAT test Uncontrolled Program Exit 65 Using the ESC key 64 Experiment stratified 50 F Failures of computer hardware 65 Farnborough 18 Feedback to the testee See indicators Figure Rotation Task description 40 Duet Figure Rotation Task 86 Figure Rota
119. t NEL including Max Lund who later moved on to the Office of Naval Research in Washington and Wesley Woodson who published his Human Engineering Guide for Equipment Designers in 1954 Outstanding contributions were also made by John Stroud 1955 known for his psychological moment concept and Carroll White 1956 who discovered and validated the phenomenal effect of visual time compression on noisy radar and sonar displays Similar to the pattern after the First World War some psy chologists remained in uniform but more including Grether Melton Bray Kemp Gagne Humphreys Adams French Taylor Mead and Small stayed on as civil servants for varying tenures as did Julien Christensen and Melvin Warrick who had long careers at the Aero Medical Laboratory at Wright Field Colonel Paul Fitts wore his uniform until 1949 then joined academia and opened his Laboratory of Aviation Psychology at Ohio State University Many who had not completed their doctorates went back to graduate school on the GI Bill A few who had earned Ph D s before the war 21 joined universities where they could apply their wartime experi ences to the training of a new breed of psychologists In ACADEMIA On January 1 1946 Alexander Williams who had served both as a selection and training psychologist and as a naval aviator opened his Aviation Psychology Laboratory at the University of Illinois Roscoe 1994 The laboratory initially focused on the
120. t regulators leading to worldwide formal training in Crew Resource Management CRM 81 82 Although certain so called personality tests are believed by some to reflect traits conducive to effective and harmonious inter actions with other team or crew members until recently there was no test other than flight simulator exercises specifically designed to call for the working exercise of those traits As the WOMBAT test came into use by airlines and air and surface traffic control agencies it soon became apparent that the higher order cognitive demands it imposes on individuals could be extended to encompass the social aspects of team performance To measure how well crew resources are managed the solo WOMBAT CS was expanded into the DuoWOMBAT Crew Resource Management Test DuoWOMBAIT CS DuoWOMBAT CS is a software program that runs on two WOMBAT CS stations linked by a special cable It challenges a pair s ability to work as team member managing a complex system The program requires teammates to share various tasks making sure that each member works on every task frequently In DuoWOMBAT teammates work together they must coop erate not compete To perform well as a team both members must maintain a high degree of awareness of the total situation including their teammate s performance as well as their own to manage this complex exercise effectively High scores for team performance depend on how well teammates perform individ
121. te a new script is to copy an existing script and modify the copy The process of copying and editing a script is similar to the process of translating files described in Chapter 7 Figure 16 shows where the script files are stored on the DuoWOMBAT computers Figure 11 shows a typical test script as delivered with DuoWOMBAT Version 1 5 This script is similar to the phases represented in Figure 17 It is recommended that you don t 90 AMERICAN LANis the name of the language directory COMPOSITis the prefix name for all the composite scores files RESEARCH TXTis the research data file name Bn creates a break n is absent or not a number gives an unlimited break n lt 0 gives an unlimited break n from 0 to 127 sets the length of the break in minutes n gt 127 gives a 127 minute break Dn creates a duo phase see below for n In creates an instruction phase see below for n Sn creates a solo phase see below for n n is absent or not a number gives a 1 minute phase n lt 0 gives a 1 minute phase n from 0 to 127 sets the length of the phase in minutes n gt 127 sets a 127 minute phase start of script I60 B S10 B1 D10 D10 D10 B1 S10 B1 D10 D10 D10 B1 S10 end of script Anything written after the line above is ignored Figure 17 A Standard Test Script as Delivered This figure shows the STD_TEST SPT file as delivered It is found inside the SCRIPTS DIR subdirectory of DUOWOMBAT CS The
122. tential for viewing conflict there is also the potential for control conflict with teammates fighting over the control of the cursors Teammates must agree on how to share Figure 13 Controlling the Duo Tracking Task Throughout the DUOWOMBAT exercise teammates sit side by side in front of the two WOMBAT displays and consoles Task sharing is a big part of a successful DUOWOMBAT performance To avoid fighting over the tracking controls teammates must establish efficient communication proce dures keep assessing their progress and use all their team s available resources to maximize the combined score the Tracking Task to avoid interfering with each other They have to use good team resource management to achieve the optimum combined team score THe Duet Bonus TASKS Chapter 3 described the Solo Bonus Tasks of DuoWOMBAT CS The Duet versions of these tasks found in DUOWOMBAT are similar to their solo versions but require the participation of both teammates for maximum performance scores 86 The Duet Figure Rotation Task shows one 3 D figure on each display The teammate sitting on the left hand side controls the figure that would appear on the left side of a solo WOMBAT display Similarly the teammate on the right controls the figure that would appear on the right side of a solo WOMBAT display Figure 14 Either teammate can answer the problem however only the first response will be accepted by the program and score
123. ter gt This action will create a whole new set of instructions located inside a new subdirectory labeled GERMAN LAN e Translate one by one all the BATMSG files in German using the EDIT command Just overwrite the BATMSG files don t remove any BATMSG file e Translate desired words in the BATSYMBO LS file again using the EDIT command e Once a few files are translated and once the BATSYMBO LS file is translated and saved launch WOMBATCS You will see at the top of the monitor the new language menu showing the German addition Choose it and start the program to check the translation done to that point You can have different versions of the same language to suit 71 different needs clients or populations Just follow the procedure above and give each subdirectory a unique name with the LAN suffix such names are restricted to 8 charac ters before the dot If you intend to edit the original distribution files in AMERICAN LAN we suggest that you make a copy of that subdirectory with some other name so you don t lose the originals They are your reference to the ideas we have thought appropriate to cover in that instruction phase Blank Page THE WOMBAT CS Scores READING THE Scores FILE You can access the desired scores file either with a word processor program or if you are in the current scores subdirectory by typing EDIT 1234 SCR lt Enter gt Each scores file is divided into two sections The first part
124. terface card supplied equipped with one DB 25 socket connector a keyboard a VGA color graphic monitor a WOMBAT console supplied a 2 meter long DB 25 connector cable supplied an optional WOMBAT Parallel Port Interface WoPPI that replaces the ISA 8 bit interface card and connects to the parallel port of the computer such as a laptop If you unpack the computer from Aero Innovation check that it has all the internal components listed above installed Choose a quiet environment for WOMBAT CS where the candidate can be left in isolation while he she takes the test Arrange the console on a table in a comfortable position Connect the color monitor and the computer keyboard then plug the DB 25 cable into the connector on the interface board Push it home firmly taking care not to bend any of the pins The plugs are polarized so that each end can have only one position Connect the other end to the socket on the console At this point the computer and monitor can be turned on 55 LAUNCHING WOMBAT CS We have stored the WOMBAT CS program and all the neces sary files in a subdirectory named after the version of the WOMBAT CS program current at the time of shipment For example Version 5 0 of WOMBAT CS will be stored on the hard disk of your computer in a directory named WOMBATCS V50 Subsequent updates of the program can be stored in different directories to insure a smooth transfer from old versions to new ones prior to dele
125. tial orientation and mental rotation to discover similarities and differences in geometric figures labeled 1 in Figure 2 e a sequential Quadrant Location task involves graphically presented temporal mazes in which a pattern of numbers recurs on successive trials until learned and then when learned each pattern is replaced by a different pattern to be learned labeled 5 in Figure 2 and e a Two Back Digit Canceling task of short term memory labeled 9 in Figure 2 40 9 Figure 2 WOMBAT CS Bonus Menu Each of the three Bonus tasks is identified by a number 1 5 or 9 and by a pictogram to minimize the use of language and reduce the risk of culture based biases The widths of the horizontal bars to the right of the pictograms indicate the current relative Worths of the three tasks Choosing and performing one task will reduce its subsequent Worth slightly and increase the Worths of the other two tasks accordingly Vigilance is an important aspect of situational awareness in the operation of any complex system and vigilant time sharing of attention is required during performance of any bonus task to catch and respond to Autotrack failures Let s have a quick look at each bonus task The Figure Rotation task displays two 3 D figures side by side One figure will be in
126. ting superseded versions To run WOMBAT CS switch to the appropriate subdirectory the one that contains the WOMBAT CS software by typing CD WOMBATCS V50 lt Enter gt The screen will then show C WOMBATCS V50 gt If you received your computer from Aero Innovation or if you installed new WOMBAT CS software using the supplied installation routine then the batch command WOMBAT BAT will load in sequence everything that is required for the WOMBAT CS test Just type WOMBAT lt Enter gt and you will have HARDWARE METAWINDOW and WOMBAT CS in the appropriate sequential order You can also invoke a fresh WOMBAT CS session by typing the same command stored in the WOMBAT BAT batch command WOMBATCS lt Enter gt 56 THE HARDWARE ProGram The procedures described in this manual will ensure that each time you invoke the WOMBAT CS program the diagnostics software named HARDWARE will appear on the monitor You can also invoke this program whenever you want by typing the command HARDWARE lt Enter gt This self explanatory program is used to verify the integrity of all the cable connections and hardware assembly before you begin the WOMBAT CS test While watching the screen 1 Move systematically each joystick and observe that full deflection causes the associated red cursor to be displayed full scale Also check that each motion has no effect on any other axis A noticeable delay between your hand motion and the cursor moveme
127. tion Score explained 74 123 Overview 35 solid block figures Shepard 44 File Annotation 59 98 Fox Hills Golf Course 29 Ft Rucker Alabama USAARL Pilot Fatigue Study 80 G Gender bias 47 See also Bruce D General Electric 23 26 General Motors 24 George Mason University 79 Gliding Championships 50 Gold Medal Award 45 Goodfellow AFB 19 Graphics Environment 57 gun director tracking simulator 16 H Hardware computer requirements 54 101 Failures 65 The program 56 Harvard University 17 22 24 help help function in HARDWARE EXE 56 Higher order cognitive demands 82 historical perspective 28 Hughes Aircraft Company 22 28 Cockpit Research Group 23 Human Engineering Division US Navy 19 human factors human engineering 16 psychophysiological tension 15 Identification Candidate 59 Illinois U of 45 46 52 Illinois University of 19 21 22 28 Imagery studies See Mental imagery studies Immediate Exit 64 immediate vs distant success 49 Indicators 36 Performance Worth overview 36 peripheral information 43 Installation Interface card into another computer 101 of WOMBAT CS 53 Institute of Aviation See also Illinois U of U of Illinois 48 124 Instructions Period In DUOWOMBAT 91 Language issues 67 71 Teammates Agreement in DUOWOMBAT 94 Unique Instructions Set in DUOWOMBAT 94 instructions period phases 63 67 time limitation breaking instructions period in two 64 using the Candidate Manual 63
128. ual and duet tasks but even more on making good decisions as to what tasks each player should be working on at any given moment during the exercise DuoWOMBAT CS can be either a testing or a training device As a test DUOWOMBAT CS is used to evaluate a team s combined performance relative to the sum of the individual performance levels of the team members Depending on the scenario designed by the supervisor this may require each team member to be tested during solo phases for a few minutes before between and after two longer sessions of team performance as shown in Figure 11 Duo phases include duet versions of the primary tracking task and of each of the three secondary bonus tasks along with other features that require highly efficient task sharing Left Member Instructions Solo Phase Right Member Instructions Solo Phase Duo Phase Solo Phase Solo Phase Duo Phase Solo Phase Solo Phase 83 Figure 11 Typical DUOWOMBAT Test Sce nario This figure represents a typical test scenario named STD_TEST SPT Here each team member goes through the timedinstructions period separately then begins the sequence with a solo phase of 10 minutes This is followed by a duo phase of 30 minutes then another solo phase and so on The alternation serves to establish a baseline for team mem ber to identify the expected relative contribution of each to the combined team score Any combination of solo
129. um of 8 characters It is recommended that you create and maintain a consistent system for determining a unique reference string assigned to each person tested One example of such a system is using secret codes while keeping track of the relation between these codes and the names of the applicants in a separate confidential file the key The file annotation is optional and can be any message up to 30 characters long The information is only copied by the program into the final results file for future reference If you do not wish to enter a file annotation just press lt Enter gt 59 60 Within a given scores subdirectory if WOMBAT CS detects another file with the same name as the one created from the reference string you will be asked to decide whether to erase it or to go back and reenter a different reference string to avoid duplicat ing the file name This check only guarantees that the current scores file will not overwrite any other scores file existing in the current scores subdirectory If you decide to erase an old scores file at this point the data collected and stored under that filename in the composite spreadsheet data file will remain thus creating two records or rows of numbers with the same scores filename see Composite Spreadsheet Data File It is therefore good practice not to erase an old scores file and to select another reference string if prompted to do so It is highly recommended that you periodica
130. varied the priorities of the tasks and the difficulty of the tracking task to elevate the attention demands to the individual testee s saturation point North s dual task measures and others developed by Diane Damos 1972 proved effective in predicting pilot performance in primary flight training notably in experiments by North amp Gopher 1974 and Jacobs 1976 The WOMBAT CS test is an extension of their concept VALIDATION IN QUEST OF THE IDEAL The perfect system for selecting complex system operators would have several qualities It would be COMPREHENSIVE meaning that the system would not depend on any single attribute of successful complex system operators but would address cognitive psychomo tor and medical factors as well as situational awareness RELIABLE meaning that any test used would yield test retest scores that are highly correlated DISCRIMINANT meaning that any test used would yield a wide range of scores normally distributed over the spectrum of human performance OBJECTIVE meaning that the scores of individual candi dates would not be biased by the subjective evaluations of individual examiners CULTURE FREE meaning that performance on any test used would not be biased or otherwise affected by race gender or prior training and experience in operating spe cific complex systems VALID meaning that any test used in the selection system would measure what it is intended to measure namely th
131. wles joined GE soon thereafter In 1955 Charles Hopkins and Charles Simon joined Williams and Roscoe at Hughes and Knowles and Carel soon followed Starting in 1953 several of the airplane and aviation electron ics companies hired psychologists but few of these had training in engineering psychology and fewer yet had specialized in aviation As the graduates of the universities with aviation programs started to appear they were snapped up by industry and by military laboratories as it became painfully apparent that not all psycholo gists were alike In a few cases groups bearing such identities as cockpit research human factors or human factors engineering were established In other cases the new hires were assigned to the Interiors Group traditionally responsible for cockpit layouts seating galleys carpeting and rest rooms 24 In this environment Neil Warren in the Psychology Depart ment at the University of Southern California and John Lyman in the Engineering Department at UCLA introduced advanced degree programs for many who would distinguish themselves in the aerospace field Starting in the late 1940s Warren had used the human centrifuge on the University of Southern California campus at that time the only one on the west coast to do display research It was in Warren s facility where it was first demonstrated that a single drag on a cigarette would measurably reduce the number of G s a pilot could withstand before g
132. y Panel 16 Arabic alphabet 70 Archiving WOMBAT results 60 98 archiving WOMBAT results scores 76 ASCII 67 Asymptotic performance 35 79 Attention allocation strategy 43 optimum allocation O Hare 52 AUTOEXEC BAT 101 automation 14 Autotrack See Tracking Task Aviation Stratified Experiment 50 Aviation Psychology Laboratory 21 Awards APF Gold Medal Award 45 Jerome H Ely 25 National Medal of Science 44 Ray Lyman Wilbur 44 World Gliding Championships 50 119 120 Backing up WOMBAT results 60 98 BATMSG Files 67 BATSYMBO LS 69 Batteries See Tests Berkeley U of California at 22 Biases computer experience 47 gender 47 prior training 47 race 47 video games O Hare 52 biases effects of pretest training 80 Bonus Tasks See Digit Canceling Task Figure Rotation Task Quadrant Location Task Actual Bonus Score explained 74 Duet Bonus Tasks 85 explained Digit Canceling Task 42 Figure Rotation Task 40 Quadrant Location Task 41 must play frequently 43 Origins 44 Overview 39 Break or pause in DUOWOMBAT 93 Break key to exit WOMBAT CS 64 British Royal Air Force 16 C Cable Configuration for DuoWOMBAT 103 Cadillac See Project Cadillac Cambridge University 46 Candidate See Testee Candidate Manual 63 Translating 70 Circles See Tracking Task predictor circles circuitry checks with HARDWARE EXE 56 Civil Aeronautics Administration 21 Gray Cover Reports 22 Clinton USA President Bill See Shepard Roger Cockp

Download Pdf Manuals

Related Search

Related Contents

LG BG5200ER Air Conditioner  Manual de usuario - CNET Content Solutions  取扱説明書    M-02N 取扱説明書ダウンロード(1.66MB  SITOP PSU300S 24 V/10 A 6EP1434-2BA10  OpenVPN WatchDog v6.0 User Guide  Serie LTC 8781  14 主な故障状況と対処方法  installation - Lincoln Electric  

Copyright © All rights reserved. Failed to retrieve file