Installation and Use Instructions
Contents
1. Select the subject from the list and press OK to remove that subject List Subjects To see all of the subjects select List all from the Subjects menu A dialog with a list of all of the subjects will be displayed 26 Running Experiments Remotely Configuring the Collection Station Install MTE on the experiment workstation as well as on the researcher s control station Installation is done in the same way on both On the remote station the one where the subject will perform the trials choose Launch Remote Server from the Tools menu The experiment canvas will appear and wait for an experiment to be sent to it by the researcher Running Remotely You will need to add the IP address of the experiment workstation the remote station to the servers xml file in the data folder of the MTE installation folder Alternatively you can type it into the Choose Server field in the Run dialog To run an experiment remotely choose Run from the Experiment menu In the Choose Server field simply select the remote workstation or type in its IP address or domain name The experiment will then start on the remote machine assuming that its remote server was launched as explained in the section above Note that you can select more than one experiment from the Run dialog in which case all of the experiments are run in sequence The subject can rest before each experiment by2. Value Description Reference Row The row number needed for deleting a row when scrubbing the data SID The subject identifier MT Movement time trial completion time ST Starting time in Java ticks ET End time in Java ticks A Amplitude of movement distance from home center to target center A Alternative amplitude of movement distance from starting point to end point A mm Amplitude as A except in millimeters D Distance traveled along the cursor path for indirect input devices Angle Angle between center of home region and center of target TW Target width TH Target height ID The Fitts Index of Difficulty according to MacKenzie s Fitts 1954 MacKenzie formulation 1991 Errors Number of selections outside the target MV Movement variability the average least square distance between MacKenzie et al 2001 the ideal path to target versus the actual path traveled measures the jigginess of the movement Model Evaluation The Model Evaluation tab displays correlation and regression results for a number of different performance models including various formulations of Fitts law Some of the models can be configured by pressing the header column in the model table In addition various width and amplitude values can be applied The regression and correlation results can be calculated on the raw and on averaged binned data similar to the sca
3. Subject second step which is running the experiment The researcher can recall a previously recorded configuration from a list of stored setups Before each experimental session input device type screen dimension probe characteristics gain settings for the input device as well as any other relevant ad hoc information are recorded For each subject demographic information is collected including age gender height and handedness MTE can be extended by adding new static as well as dynamic moving shape types target position distributions movement time models and data export formats Evaluation and Analysis To assist in the rapid evaluation and interactive exploration of movement models and input devices basic statistical analysis is built into the platform The recorded data can be easily exported to many statistical packages for more sophisticated analysis MTE supports correlation analysis and linear regression as well as configurable scatter plots and distribution graphs In addition the raw data and the trajectory of the individual acquisition movements during a session can be viewed so that the movement patterns for different input devices can be studied Lastly a table comparing the correlation coefficients of various movement time models including Fitts law is displayed The linear regression coefficients and the correlation coefficients can be computed either on the raw observations or a
4. lt ELEMENT server PCDATA gt lt ATTLIST server ip CDATA REQUIRED gt j gt lt addresses gt lt server ip 192 168 102 176 gt Tablet PC lt server gt lt server ip 192 168 102 170 gt Compaq iPAQ Elo lt server gt lt addresses gt 8 When the program first runs it looks for subjects xml and registry xml If it does not find them it creates default files The subjects xml file contains subjects for the experiment configurable through the Subjects menu in MTE and registry xml contains program settings configurable through Tools Options in MTE Overview Experiments are interactively configurable and the configurations are saved in a sharable XML format The experiments can be carried out via a network allowing the research and subject to be separated by any distance Its data sets are saved in either XML or CSV which simplifies importing into customized programs and off the shelf plotting and statistical packages such as Microsoft Excel and R Furthermore MTE contains interactive plotting R is an open source statistical package available from http Avww r project org and basic statistical analysis allowing student researchers to collect data and play with the data in a single environment The tool supports visualization of a single data set several data sets side by side or overlaid cursor paths spatial variability of cursor selection end points a
5. Northeastern University College of Computer and Information Science MTE Movement Time Evaluator Tutorial amp User Manual Input Device Usability Evaluation Education amp Research Workbench Martin Schedibauer Ph D mschedibanen neu edu mjs01776 yahoo com Table of Contents Introduction 5 Availability 5 Support 6 Installation 6 Windows 6 Linux 6 MacIntosh Error Bookmark not defined Overview 7 Configurability 8 Evaluation and Analysis 8 Exporting to Other Tools 9 Feedback 9 Process 10 Configure an Experiment 10 Target Shapes 12 Target Placement Strategies 13 Collect Data 13 Analyze Results 15 Merging Result Files 15 Loading the Data 16 Statistics 17 Raw Data 18 Model Evaluation 20 Width Models 21 Model Functions 22 Performance Models 22 Trajectory Plot Trajectory Analysis Ad Hoc Analysis Managing Subjects Add Subject Delete Subject List Subjects Running Experiments Remotely Configuring the Collection Station Running Remotely Managing Experimental Data Configurable Options Searching for Specific Experiments Performing Ad Hoc Analysis Adding New Models Exporting Data Exporting to R and Excel Exporting as XML 23 24 25 26 26 26 26 27 27 27 27 27 28 28 28 29 29 29 Overview MTE as a research and education workbench Introduction While many specialized utilities have been developed to capture data for evaluating the usability of input devices few of these tool
6. result file into MTE for analysis In addition you may want to export the data via CSV to R or Excel for additional statistical analysis and plotting Configure an Experiment Start the configuration process by creating or modifying an experiment configuration Select Configure from the Experiment menu The following dialog will appear amp Experiment Configurator X Experiments General Home Region Target Region ci configs gt EJ CHI 2007 gt Ef field Title gt ci lab Number of Trials 1H E Click and Drag Task gt Ej test Description Information Recorded C Display Trajectory V Trace Motion V Record Errors _ Defer Target Display Date Last Modified April 2 2005 13 25 33 re In the above dialog click on the folder in which you wish to store the configuration then select New To create additional folders for storing experiments you need to use your file system s explorer e g Windows Explorer The configurations are stored in lt MTE root gt data configs Ona standard Windows installation the lt MTE root gt folder is C Program Files MTE but on Windows 7 and Vista it should be changed during installation 10 Once you click on New the following dialog appears asking you to provide a name for the configuration e g simple fitts New Experiment eg Enter experiment name simp
7. trajectory for a small data set Showing the trajectory for large data sets is computationally slow and visually cluttered 23 Experiment Subjects Tools Hel Pett fp SS tee rt isa ns Bee orire Se ips Deacon o Te 3 Subject Path of the cursor from home to target only appropriate for indirect input devices F El mergecsete amp GI archive LD chimouse n 19 ser LD chismouce mechanical n 7 ser D LD chimuttitouch n 19 ser elostanding finger m8 ser D tujitsu standing stylus n 11 ser LD tujiteu standing stylus n 12 0 D tujiteuswatking stytus ne t1 ser D tujitsu mating styius n 12 ser keypass finger crea ser D keypactingerss n 8 ser H LD keypac finger0 nee ser LD keypa fingerrep nap ser D keypadstyius n 8 ser i Dy keypacetrackbalt n 8 ser Dy secondarystanding finger n 8 se LD secondanpstanding styius n 8 ser LD cecondarystanding trackball n LD tabletsitting tinger n 1 1 ser D tabletsitting finger 15 n 11 ser LD tabletsitting tingerso n 11 ser H Dy tabletsitting ringer 45 n 11 ser Dy tabletsitting finger 45 60 n 14 LD tabletsitting tingero0 n 11 ser D tabletsitting stytus n 11 ser at mouse optical n 12 ser 01 11 07 13 41 30 2 525 371 a File Type ser Summary Statistics Raw Data Model Evalulation tej Aot Trajectory Analysis _ Ad Hoc Analysis Trajectory Analysis Here you can visualize t
8. Accot J amp Zhai S 2003 Refining Fitts law models for bivariate pointing In Proceedings of the ACM CHI 2003 Conference on Human Factors in Computing Systems April 2003 Ft Lauderdale FL ACM 193 200 Fitts P M 1954 The information capacity of the human motor system in controlling the amplitude of movement Journal of Experimental Psychology 47 381 391 Kvalseth T 1980 An alternative to Fitts law Bulletin of the Psychonomic Society 16 5 371 3 MacKenzie S 1991 Fitts law as a performance model in human computer interaction Unpublished Doctoral Dissertation University of Toronto Department of Computer Science Toronto Canada MacKenzie S amp Buxton W 1992 Extending Fitts law to two dimensional tasks Proceedings of the ACM Conference on Human Factors in Computing Systems CHI 92 219 226 New York ACM MacKenzie S 1995 Movement time predictions in human computer interfaces In Readings in Human Computer Interaction 2nd Edition Morgan Kaufman Los Altos CA 483 493 MacKenzie I S Kauppinen T amp Silfverberg M 2001 Accuracy measures for evaluating computer pointing devices In Proceedings of the ACM Conference on Human Factors in Computing Systems CHI 2001 9 16 New York ACM Meyer D Abrams R Kornblum S Wright C amp Smith J 1988 Optimality in human motor performance Ideal control of rapid aimed movements Psychological Review 95 3 340
9. Means Only _ Omit Error Runs _ Omit Error Batteries Targdt Width Trial Number Frequlyicy Amplitude Merge Step1 Select the folder that contains the files If the folder contains subfolders all files will be merged recursively Step2 Enter a file name and folder for the merged data Step3 Specify merge controls You can omit errors selections average all results and save the data as XML not recommended Once you have selected the files to merge and have entered a filename for the merged data press Merge The process may take a minute or more depending on the number of files and whether the files contain trajectory information Once the merge is complete we recommend that you select Refresh Experiment List from the Tools menu so that the merged file will appear in the Run Files file tree Loading the Data Once the merged file has been prepared you can load the data in three ways One you can choose Load from the Experiment menu and then pick the file from the dialog Second you double click on the file in the file tree Third you can single click on the file in the file tree and then open the pop up menu with the right unless your mouse is configured for left handed use mouse button and pick Load Regardless which approach you use the data is loaded and a summary is displayed in the
10. is the information specified when the experiment was run File Properties File Name e mechanical n 7 ser Last Modified 01 11 07 13 40 51 File Size 1 380 350 File Type ser Update Summary Statistics Raw Data Model Evalulation Trajectory Piot Trajectory Analysis Ad Hoc Analysis Unless the loaded file contains data collected from a single subject the subject information is generic You can change the subject by pressing Change Subject However this only makes sense if the data is from a single subject It is important to update the Target Shape to any of the valid names shown in Table 1 If not updated the Trajectory Plot will not display the correct targets The next sections will explain the different tabs in more detail However it is difficult to show everything and we recommend that you explore and play with some sample data Sample data can be downloaded from our Yahoo forum http tech groups yahoo com group uml_mte Statistics The Statistics tab displays summative statistical information about the data set In addition it contains scatter and distribution plots as well as linear regression analysis In MTE amplitude refers to the Pythagorean distance between the movement starting and end points whereas distance is the actual distance traveled along the cursor path Distance is only valid for indirect input devices such as mouse or tra
11. simply waiting to click OK on the instruction dialog Managing Experimental Data ZAMTE UMass Lowell chi mouse Experiment Subjects Tools Help Data Filter eal Target Width Input Device unknown vl Ru Subject ID Tu Subject Name Configuration Apply Reset Ta Move cut copy rename and delete files Add new folders DDODDDODDODDRSBDRRBDRDBDBDRDD File Properties FileName mouse optical n 12 ser Last Modified 01 11 1 30 File Size 2 File Type ser 9 Configurable Options This section needs to be written 27 Searching for Specific Experiments This section needs to be written Performing Ad Hoc Analysis This section needs to be written Adding New Models This section needs to be written 28 Exporting Data Export your data to other programs for analysis and plotting Exporting to R and Excel To be written Exporting as XML The data files are savable in two formats All data collected during an individual run of an experiment is saved as an XML file When files are merged for performance reasons the default format is a Java serialized object However when you merge experiment files you can select Save as XML in the Data Options section of the merge dialog accessed via Tools Merge Data Sets 29 References Papers and reports containing background theory and results obtained with MTE
12. 370 Meyer D E Smith J E K Kornblum S Abrams R A amp Wright C E 1990 Speed accuracy tradeoffs in aimed movements Toward a theory of rapid voluntary action In M Jeannerod Ed Attention and performance XIII Hillsdale NJ Lawrence Erlbaum 173 226 Schedlbauer M 2010 Effects of Design on the Completion Time and Accuracy of Input Tasks on Soft Keypads using Trackball and Touch Input Submitted to Journal of Usability Studies Holzinger A H ller M Schedlbauer M amp Urlesberger B 2008 Fitts Law in Real Life Medical Scenarios Performance of Finger versus Stylus In Proceedings of Information Technology Interfaces ITI 2008 Dubrovnik Croatia July 2008 30 Schedlbauer M amp Heines J 2007 Selecting While Walking An Investigation of Aiming Performance in a Mobile Work Context In Proceedings of the 13th Americas Conference on Information Systems AMCIS Keystone Colorado August 9 12 2007 Recipient of Best HCI Paper Best Conference Paper Honorable Mention Schedlbauer M 2007 Completion Time Predictions of Touch Screen Interactions in Dual Task Situations In Proceedings of the 29 Annual Conference on Information Technology Interfaces ITI 2007 Dubrovnic Croatia June 2007 Micire M Schedlbauer M amp Yanco H 2007 Horizontal Selection An Evaluation of a Digital Tabletop Device In Proceedings of the 13th Americas Conference on Information Systems AMC
13. 6419 108 00 294 81 263 79 6419 188 00 515 64 70 91 6419 66 00 161 68 73 41 6419 216 00 696 18 317 09 6419 116 00 277 47 64 83 6419 44 00 104 53 227 56 6419 253 00 558 81 187 78 6419 274 00 687 70 179 40 6419 113 00 256 14 317 33 6419 115 00 259 03 335 22 6419 269 00 574 83 8 35 6419 252 00 538 55 169 88 6419 269 00 670 69 6 82 IZE 241 00 538 83 221 88 6419 95 00 213 67 220 24 6419 124 00 277 69 330 6419 296 00 a 708 28 197 59 6419 142 00 321 12 164 11 6419 123 00 265 58 161 63 6419 143 00 365 07 249 78 298 00 682 26 26 28 6419 festa 185 00 392 06 5 75 6419 246 00 536 28 6419 199 00 540 07 149 17 Joris stslststsislststelsiststsjsisisiststsisisistsisislstststslststststsjstst5 Row to remove from data set Save the data set back to the same or a different file name after rows have been removed Export the entire table as a comma separated values CSV file suitable for import into R or Excel 19 The raw data table displays a number of collected data scores The meaning of each value is explained in Table 3 Table 3 Collected data values
14. 7 763 12 827 791 60 63 26 15 31 15 46 secondarystandingfinger n 8 se 959 798 12 966 806 jis 1810 4 38 4 42 LD secondary standing stylus n 8 ser 546 047 12 554 654 15 17 75 4 30 450 Ly secondarystanding trachoalt ne6 351 384 12 366 400 30 26 13 6 32 6 38 D 760 518 12 792 545 60 58 10 14 06 14 52 tablebeiting inga Get Deu 621 658 12 647 687 60 49 14 11 89 12 81 LD tabletsitting ingert8 n 11 ser 27 228 12 50 251 45 41 60 10 07 1018 C tabietsitting finger30 n 11 ser 139 517 12 171 541 Jeo 74 88 L tabietsitting fingera5 n 11 ser 864 639 12 896 669 60 49 55 E biata aaien 1022 428 12 1038 440 fo 2270 etsitting finger empe oera 642 306 12 669 332 60 58 06 LD tabletsitting inger60 n 11 ser 703 46 12 734 71 45 35 65 O tabletsitting styius n 11 s0r 111 547 12 119 553 15 13 70 ff 121 371 12 134 386 30 40 18 Di 77 516 12 85 522 jis 17 66 File Properties 645 536 12 575 566 60 47 78 File Name moure optioal n 12 ser a a 2 Last Modified 01 11 07 13 41 30 File Size 2 525 371 File Type ser Width Models Export the effective width and SD data to a CSV file for import into R and Excel MTE supports a variety of width calculation approaches There is generally agreed upon way on how to calculate width for a bivariate pointing task Table 4 Width calculation approaches Width Model Description Reference 2D Width Widt
15. IS Keystone Colorado August 9 12 2007 Schedlbauer M 2007 Effects of Key Size and Spacing on the Completion Time and Accuracy of Input Tasks on Soft Keypads using Trackball and Touch Input In Proceedings of the Human Factors amp Ergonomics Society 51st Annual Meeting Baltimore MD October 2007 Pastel R Champlin H Harper M Paul N Helton W Schedlbauer M amp Heines J 2007 The Difficulty of Remotely Negotiating Corners In Proceedings of the Human Factors amp Ergonomics Society 51st Annual Meeting Baltimore MD October 2007 Schedlbauer M 2007 A Configurable Platform for the Interactive Exploration of Fitts Law and Related Movement Time Models Extended Abstracts of the ACM Conference on Human Factors in Computing Systems CHI 2007 San Jose CA April 2007 Schedlbauer M Pastel R amp Heines J 2006 Effect of Posture on Target Acquisition with a Trackball and Touch Screen In Proceedings of 28 Annual Conference on Information Technology Interfaces ITI 2006 Dubrovnic Schedlbauer M 2007 A Survey of Manual Input Devices Technical Report 2007 002 Computer Science Department University of Massachusetts Lowell MA Schedlbauer M 2007 A Survey of Human Cognitive and Motor Performance Models Technical Report 2007 001 Computer Science Department University of Massachusetts Lowell MA Schedlbauer M 2006 An Empirically Derived Model for Predicting Completion Tim
16. Summary tab An annotated example is shown below Data files with an extension of ser are Java serialized objects 16 A MTE UMass Lowell chi mouse mechanical n 7 mex Data Filter Target Width 7 Input Device unknown x Subject ID ill Subject Name Configuration Apply Reset Gal menpedsets aj amp GJ archive entmouse netopser C chimouse mechanical n 7 s chimouse optioal n 12 ser _ Ly cimutitoucn etoyser D elostanaingstinger n M D tujtrestandingsttus om1 Dy tujtrrstandingsttus ne 123 54 Ly tutu nahing sytus ve 12 se1 D taypacstnger ey sar Dy taypad nngerss reo ser e Experiment Subjects Tools Help Experiment Run Information Current File chi mouse mechanical Experiment Title Experiment Information Number of Trials 560 Input Device unknown Probe Size 0 0 Target Shape Subject Information Name wa Trial Environment unknown Date of Experiment Thu Jan 11 13 40 43 ES TargeNextent med Parameters Gender UNKNOWN Age nla The input device is unknown in merged files Any of the fields can be edited To save the information back to the merged file press Posture unknown Configuration Used Arm Body Angle 0 0 Distance unknown Screen Size mm 0 0 Parameters Height na Handedness UNKNOWN This
17. ata set Min MT The minimum MT in the data set Intercept The intercept value of the linear regression equation for the model Fitts MacKenzie Slope The slope value of the linear regression equation for the model Fitts Performance Models MTE calculates the above values for the following movement time models Clicking on the model name in the table header will display the model equation reference and any adjustable parameters In the table below A is the amplitude and W is the width according to the calculation approaches that were selected Table 6 Fitts models supported by MTE Model Description Reference Simple A W Fitts log 2A W Fitts 1954 Welford logo A W 0 5 Welford 1960 Shannon logo A W 1 MacKenzie 1991 Meyer sqrt A W simplified version of the generalized model Meyer et al 1988 Kvalseth A wy Kvalseth 1980 Accot Zhai_ see paper Accot amp Zhai 2003 ID 2 log A W 1 22 Trajectory Plot This tab displays the actual selection end points optionally with target outlines and the actual cursor path Display direct path Display target from home to target outlines as well as actual path Zoom slider traveled Experiment Subjects Tools Help Data Filter Target Width tpt Dee aae Subject ID Subject Name Configuration 9 CI merged sets o I atohive LD chimouse n 19 ser LD chi moure mechanioal n 7 ser O chimu
18. ckball It does not make any sense for touch input 17 Data Filter Target Width Descriptive Statistics Sample Size 500 MEAN MT 900 07 MEANA 370 47 MEAN 425 14 STDDEVMT 278 67 STDDEV A 16328 STDDEV 199 74 MEAN ER 0 03 Input Device Distribution of ID amplitude MT or distance over the specified number of bins Subject ID Subject Name Configuration Data Plot onfigure model parameters porenannen els X and Y axis parameters Dterraesingerzo D rerpacringerrep meyer D terrae D revas tisaoan moser 1D Shannon i File Properties 10 Shannon vs mt File Name se mechanical jn 7 ser Refresh Clear Regression Analysis t 0 00 r2 0 64 y 221 56x 120 90 Last Modified 01 11 07 13 40 51 File Size 1 380 350 I L Statistics File Type ser fodel Evalulation Trajpotory Piot Trajectory Analysis Ad Hoo Press Add to add the Clear the plotting Correlation R and R data to the scatter plot area and regression equation To attenuate the effect of outliers Y axis parameters can be averaged across X axis parameters For example many studies average MT over a fixed range of ID values to obtain better regression and correlation results The screen shot below shows the effect of averaging or bi
19. d in automatically named lt MTE root gt data runs To create additional folders for storing and organizing experiments or configurations you need to use your file system s explorer e g s Di On a standard Windows installation the lt MTE root gt folder is C Program root gt data configs files in On Vista and Windows 7 change the default installation folder to c mte otherwise you will not be able to run MTE As on Vista and 7 the Program Files directory is Linux and Mac OS X 1 Download the MTE Java jar file mte jar from http research cathris com mte 2 Launch a terminal window 3 In your home directory create a directory called MTE generally using cd mkdir MTE 4 Move into that directory cd MTE and create a subdirectory in MTE called data mkdir data Move into the data directory cd data and create three additional subdirectories within data called configs export and runs mkdir configs export runs 5 Move back into the main MTE directory and run MTE from the command line with java cp mte jar edu uml mte ui Main 6 Ifyou want to perform remote experiments you also need the XML file servers xml that contains the IP addresses or remote machines on which experiments are conducted It must be placed into the data directory 7 Here is a sample servers xml file lt xml version 1 0 encoding UTF 8 gt lt DOCTYPE addresses lt ELEMENT addresses servert gt
20. e of Cursor Positioning Tasks in Dual Task Environments Doctoral Dissertation Department of Computer Science University of Massachusetts Lowell April 2006 Schedlbauer M Heines J 2005 An Extensible and Interactive Research Platform For Exploring Fitts Law Technical Report 2005 015 Computer Science Department University of Massachusetts Lowell MA Soukoreff W and MacKenzie I S 1995 Generalized Fitts Law Model Builder In Proceedings of the ACM Conference on Human Factors in Computing Systems CHI 1995 ACM Press 113 114 University of Oregon HCI Research Laboratory WinFitts Two dimensional Fitts Experiments on Win32 http www cs uoregon edu research hci research winfitts html 31 Welford A 1960 The measurement of sensory motor performance Survey and reappraisal of twelve years progress Ergonomics 3 189 230 32
21. folders for each subject named 1 2 etc You could also use the subject s id or the subject s initials amp Run Experiment File Name CAProgram Files MTEdatatrunstab fujitsutlab standingtSTYLUSAHB Files of Type Persistence Save Trial Data Select to save experiment results Click to configure folder placed J Ip in which result file is Run cancer _ Create new folder Select folder in which result file is save then press OK 14 Once everything is configured press Run to start the experiment The following advisory dialog will appear in the experiment window showing the experiment ID which will also be the name of the experiment s result file Start Experiment X The experiment will begin once you press OK Please complete each task as quickly as you can Be sure to hit each target only once If a Home region is displayed you must first click on that region before the target appears or is selectable Experiment ID 1176162695609 Upon selecting OK the experiment starts and the home region if one has been configured and the target unless target deferment has been configured are displayed Trial timing starts upon selection of the home region If no home region has been configured trial timing starts as soon as the target is displayed The title of experiment window displays the number of trials remainin
22. g Repeat this process for each experiment configuration and each subject Upon collecting all of the data the individual experiment results will need to be merged into a single file for analysis This process is explained in the next section Analyze Results Merging Result Files The first step in analysis is the merging of all of the individual result files for each subject into a single file To start the process select Merge Data Sets from the Tools menu This will bring up the following dialog 15 amp Merge Data Set X Experiment Files amp Open Eg Look In 3 lab sitting il iat gl E an Look In C merged sets Iz ajja El Ham joystick Ci styks archive fujitsu standing stylus n 11 ser Ci tough D chi mouse n 19 ser D fujitsu standing stylus n 12 ser Ei tra kball D chi mouse mechanical n 7 ser D fujitsu walking stylus n 11 ser D chi mouse optical n 12 ser D chi multitouch n 19 ser D elo standing finger n 8 ser h fujitsu walking stylus n 12 ser D keypad finger n 8 ser D keypad finger 35 n 8 ser 4 il I E File Name new merg filel File Name joystick Files of Type All Files Type All Files Options Cancel As XML Average Summary Fields
23. g environment Keypad Numeric keypad with 10 numeric gap between buttons string to be buttons and two additional buttons entered visual feedback time in ms In a future release the font type and font size will be configurable 12 Target Placement Strategies MTE supports several placement or distribution strategies for targets Table 2 Target placement strategies Distribution Strategy Description Configurable Parameters FixedRandomDistribution Places targets at random locations on the screen Every time the experiment is run targets will appear in the same locations random seed configurable through Tools Options RandomDistribution Places targets at random locations that vary each time the experiment is run none ReciprocalDistribution Places targets along the horizontal oscillating between the left and right side of center of the screen Simulates the classic Fitts tapping experiment none StaticDistribution Places targets at pre programmed positions Use the to configure the target locations distance and angle from center of screen Save the configuration by selecting Save Once you are satisfied with your configuration settings you can test it by selecting Test The results are not recorded You can interrupt a test by simply dismissing the testing windows When done choose Close Pres
24. h along the approach vector MacKenzie Horizontal Extent Target size along the horizontal width MacKenzie Fitts Smaller of W H Smaller of target width or height MacKenzie Area Geometric area of the target MacKenzie Schedlbauer Sum of W H The sum of width and height MacKenzie We Effective width MacKenzie 21 Model Functions For each model the following functions are calculated and displayed in the table Table 5 Functions calculated for each performance model Function Description Reference R A W Correlation coefficient R based on the amplitude and the selected width model R D W Correlation coefficient R based on the distance actual cursor path and the selected width model R A W Coefficient of determination R based on the amplitude and the selected width model R D W Coefficient of determination R based on the distance actual cursor path and the selected width model 1 b Throughput based on the inverse of the regression slope Zhai MacKenzie TP A Throughput based in the ratio of averaged MT ID where ID is MacKenzie ISO calculated based on the amplitude TP D Throughput based in the ratio of averaged MT ID where ID is calculated based on the distance Mean ID The average ID in the data set Max ID The maximum ID in the data set Min ID The minimum ID in the data set Mean MT The average MT in the data set Max MT The maximum MT in the d
25. he speed time graph for a set or an individual target acquisition You need to enter the run number which you can get from the raw data table or the Trajectory Plot when displaying the ideal path 24 Experiment Subjects Tools Help Data Filter Run Number 1 5 Max Y Min X Clear a eet eet eee ieee eee me A Show Run Curves Mean Curve C Smooth Curves Subject ID Trajectory Analysis Subject Name Configuration SI Reset _ panene Speed vs time graph for a range of archive runs the range must be DD chimoure n 19 20r Dm consecutive and specified as a b ehhmouse optical n 12 se1 LD chi muttitouch neta ser D elo standing tinger n 8 ser LD tujiteustanding styius net ser LD tujitsu standing stylus n 12 ser D tujiteuwaning stylus net t zer po p a LD tujitsu aing stytus n 12 ser LD reypadsfinger na ser D keypad nngerss n 8 ser Dy keypadsfinger50 n 8 ser D tevpac nngersep crest eovpad stylus n ser D keypad trackbait nea ser N 3 8 ds dt units ms DD secondarystancing finger n8 sei O cecondarpstanding stylus n B ser LD secondar stancing trackball n 8 LD tabletsitting finger n tt zer C tadtetsitting singert5 n 11 500 Dy tablet sitting finger30 n 11 20 O tavtetsitting tingeras cn 11 ser LD tadtet sitting tinger a5 00 rm1134 O tavietsitting tingero0 rat tp ser O tablet sittingstytus n 11 00 A q
26. i a75 1000 1125 File Properties File Name mouse optical n 12 ser LastModified 01 11 07 1341 30 File Size 2 525 371 File Type ser Summary Statistics Raw Data Model Evalulation Selecting Show Mean Curve will display a curve that represents the average values of a set of runs Smoothing the curve attenuates the peaks and valleys of the curve Ad Hoc Analysis This tab displays output from custom Java code For more information see the next chapter 25 Advanced Topics Additional configuration mechanisms and commands Managing Subjects MTE allows individual tracking of subjects so that experimental factors such as age height gender or handedness can be included in the analysis All subjects are stored in an XML file in the data folder of the M7E installation Add Subject To add a new subject select Add New from the Subjects menu This will display the following dialog amp Add New Subject X Subject Information Subject Name I s Subject Age or Age Range Gender MALE Handedness RIGHT KI KIK Height 5 5 58 add Cancel Delete Subject To remove a subject select Remove from the Subjects menu This will cause the following dialog to appear Subject Selection X Select subject to remove from the list Warning This is permanent once you press OK MJS 35 40 4892
27. le fitts Cancel The next step is to configure the general parameters as shown in the figure below p Experiment Con Experiments i configs GI CHI 2007 Gi tieid gt ci lab Gitest C dtxmi D atarxmi D distractor testxm D labettestxmi D random extent testxm D static testxmi Dtestami D test2xmi General Home Region Target Region Title Classic Fitts Task Number of Trials Click and Drag Task Description Classic Fitts experiment in which stNgct move back and forth between two targets Free form text description and experiment title Information Recorded K Display Trajectory C Deter Target Display Record any error selections selections outside target region Display cursor trace only applied to indirect input devices Defer target display until home region is clicked on Click and Drag ask if selected or simple point and click task Number of repetitions i e selections subject has to make Record cursor path useful for kinematic and motion analysis Once the general parameters are configured the home region settings are made Select the Home Region tab to display those settings Experiments i configs E3 CHI 2007 gt ci field gt cJ lab c test O atx C afatxmi B distractor testxml Dy labe testxmi D random extent testxml Home Region Target Region Shape Config
28. mmary Experiment Configuration E Program Files MTE data runs merged sets chi fao Subject Data n a n a SiD MT Angle 9 CI merge sets O G archive j D cnimouse cnet9 ser C chi mouse mechanioal n 7 ser D ch mouse optical ne t2 ser LD cht muttitouen n 19 ser LD elostandingtinger n 6 201 O tujtteustanding stytus n 11 s0r L tulitsu standing stytus n 12 ser LD tujitsuaiking stytus n 41 ser DD tujteuwathing stylus ne 12 ser LD keypad tinger n s ser H C keypad tinger 35 n 8 ser LD keypad tinger s0 n 8 ser LD keypas tingerrep ne8 ser O keypad stytus nea ser LD keypad trackbail n 3 s0r LI secondary standing tinger n 8 se LD secondanystanding stylus n 8 ser LD secondanpstanding trackball rm O tabietsitting fingerin t4 s0r C tabletsitting tinger 15 n 11 ser D tabletsitting fingera0 nett ser tablet sitting ringeras nett ser 6419 244 51 IZE 265 00 A m 227 00 579 66 579 25 184 94 6419 217 00 507 95 237 15 f sta 292 00 1016 26 32 6419 115 00 246 41 9 73 6419 116 00 282 18 227 89 6419 99 00 218 70 140 16 6419 189 00 488 56 191 31 6419 85 00 182 54 191 94 6419 256 00 594 81 358 08 6419 329 00 803 54 33 49 6419 89 00 207 00 244 09 6419 147 00 384 26 53 99
29. nd cursor kinematics Lastly its internal object oriented architecture makes extensions to the tool relatively simple MTE is based upon a distributed architecture in which a researcher controls experiments from one workstation while the subjects interact with the software on different workstations This is particularly useful for students conducting experiments and collecting data remotely The overall platform capabilities are summarized in the UML Unified Modeling Language use case model to the left The papers listed in the references section at the end of the manual demonstrate the use of MTE Configurability Experiments are configured in two steps The researcher first creates an experiment setup which specifies session invariant parameters including target extent target shape type oval rectangular moving or soft keypad home region placement center upper left corner of screen or none target position distribution reciprocal pre programmable or random auditory and visual feedback preferences number of repetitions type of movement point and click versus drag and drop and the type of information to be recorded for each aiming task cursor path and errors The saved and sharable configuration files are used in the Load D Conti D on include ss Gs etoclucee Gs Information Perform Statistical Analysis includes Comms Experiment K Export Data includes Tabulate g Regression Models
30. nduct standardized experiments and to directly compare device characteristics performance and usability Silfverberg MacKenzie and Kauppinen lament the fact that between study comparisons are not addressed by ISO9241 and that unified test conditions only provide high level comparisons of research results Douglas Kirkpatrick and MacKenzie adamantly assert caution in comparing results across experiments They argue that it is critical that exactly the same experimental design task environment instructions and data analysis be given and that given these limitations it is useful to have standardized software Sharable experimental configurations and results are required to carry out between study statistical analyses MTE provides a framework upon which to build a database of reference conditions configurations and eventually experimental results Availability The MTE platform is available under the GNU Public License GPL as open source software An executable version with an installation wizard that runs under Java 5 or later as well as the full source code can be downloaded from the web allowing for modification and extension by students and researchers The software has been tested on Windows 98 Windows 2000 NT XP Vista 7 32 7 64 Linux and Mac OS X running Sun Java 5 and later Its source code and compiled versions are available for download from http research cathris com mte The web site has a self installi
31. ng kit for Microsoft Windows XP NT 2000 98 Vista 7 in the form of an MSI file Support Support is available directly from the author as well as through a forum hosted by Yahoo You can contact the tool s author Martin Schedlbauer via e mail at m schedlbauer neu edu or mjs01776 yahoo com The Yahoo group that hosts a discussion forum mailing list downloadable experiment configuration add ons and user contributions is located at http tech groups yahoo com group uml_mte Installation Windows 1 Download MTE from research cathris com mte The web site contains a self extracting MSI file that installs under Windows 95 98 NT 2000 XP Vista 7 It has been tested on 32 bit and 64 bit installations 2 Run the downloaded installer and follow the wizard s directions On Vista and Windows 7 change the default installation folder to c mte otherwise you will not be able to run MTE as on Vista and 7 the Program Files directory is write protected 3 Open your start menu then select Programs followed by MTE 4 When first run MTE will warn you about not finding a registry file Click OK to proceed A default registry file will be created in which MTE configuration parameters will be stored 5 You are now ready to configure execute and analyze experiments Windows Explorer write protected Files MTE The configurations are stored in lt MTE Experiment results are store
32. nning MT values over a range of IDs Display the regression line Data Plot le g X Axis ID Shannon Y oe Max X 0 Max Y 0 Experiment in Legend y Show Regression Line Configure the binning Asi i v Bin Data Yass e x Minx fo marp Led parameters Scatterplot 930 Binning Parameters Number of Bins bo 775 620 m Low Bin Value 0 0 amp 465 EE 2 High Bin Value 8 0 310 7 Use Min and Max If as range 155 o 00 0 53 1 06 1 59 212 265 10 Shann l Add Refresh Clear Regression Analysis r 0 99 r2 0 9 y 207 49x 116 56 t ie Clear any scatter plot The number of bins is first then select Add generally the number of different ID values that the experiment setup contained Raw Data This tab displays the actual experiment data raw data in a tabular format The table headers can be selected which causes the table to be sorted accordingly This is useful for detecting 18 outliers Outliers can be removed from the table by entering the row number at the bottom of the table The reduced table can be saved back to the same or a different file The data set can be exported to a CSV file if further analysis is to be carried out in another program such as R or Excel Raw Data Average data for matching trials across subjects Parameter Su
33. s are general purpose experimental and educational platforms This guide describes the Movement Time Evaluator MTE an interactive software platform for designing executing and analyzing Fitts type experiments It is an extensible tool which allows students to focus on discovery and exploration rather than programming In addition it provides students insight into graduate research which may encourage them to continue their education We believe that interactive experimentation will make theoretical concepts more accessible to the students and we hope will make them aware of scientific exploration in HCI MTE is a configurable tool for exploring input device characteristics as well as rapidly evaluating performance models It is written in Java and is constructed on an extensible object oriented and pattern based framework MTE has a comprehensive graphical user interface that allows researchers to configure their experiments and interpret results interactively It allows researchers to compare their own models immediately to Fitts law and the variations defined by Accot and Zhai Kvalseth MacKenzie and Meyer et al MTE extends Soukoreff and MacKenzie s platform by adding bivariate pointing tasks probe corrections additional performance models non stationary targets soft keypads dynamic configurability and movement microstructure evaluation As a result MTE presents a new research platform which allows input device investigators to co
34. sing Close will dismiss the dialog and not save any changes So be sure to press Save if you want your changes to be persistent Collect Data To run as experiment choose Run from the Experiment menu The following dialog will appear allowing you to select a previously stored experiment configuration 13 Configurations E configs c CHI 2007 gt El field gt cj lab Eltest Ci dfxm C afatxmt DB distractor testxml D label testxmi D random extent test Persistence El Save Trial Data Folder Settings Choose Subject Generic Male 16 65 3176 Choose Server frun locally gt v Input Device nown v Portus fas i Environment frationary v ariable X Distance Screen Size fin mm 0 xf Probe Width 0 Gain 0 Description Experiment to run Subject that will complete the experiment previously set up through Subjects Add New Demographic and environment information that will be stored for future reference Normally experiment results are saved in a file However you may not wish to save warm up trials To save the experiment results select Save Trial Data and then select a folder in which to save the results by pressing the button In the file dialog you can create new folders We suggest that you save the trials for each subject in a separate folder For example we created
35. t ci lab Set the target E test Distribution FixedRandomDistribution placement strategy C dtxmi Width 40 C Randomize Width LY afatxmt distractor test xml label testxml Keep Aspect Ratio random extenttestxml Options C Bimpie nitts xmi m E EA O Target width and static testxmi V Beep on Acquisition K Beep on Error height in pixels If Papia RERNA the randomize check boxes are set then the width and height are varied between 10 and the set extent Height 40 O Randomize Height Number of Distractor Shapes 5 Determines how many Target selection decoy targets are parameters displayed in addition to the actual target Target Shapes MTE supports several shapes for the target any distracter shapes are displayed in the same shape type as the target Table 1 Target shape types and associated description Shape Description Configurable Parameters RectangularRegion Rectangular area width height foreground background LabelRegion Rectangular area with a string width height text string displayed within the area OvalRegion Oval or circular area width height foreground background VibratingRegion Rectangular area that moves randomly width height amplitude in two dimensions to simulate a frequency on x and y axis movin
36. tter plot in the Statistics tab A second table shows statistical data for each target including the effective target width W and the standard deviation of the target end points from the mean and the target center The effective width is an indicator of how subjects actually perform the experiment Some subjects move more deliberately and slower therefore they have a lower error rate while others move faster but with less accuracy See the papers by MacKenzie and Schedlbauer for additional background information 20 Determine if amplitude is from starting point to end point of movement or from home center to target center Add the probe width in touch input to the width Omit outliers from the analysis Perform a log transform on MT to normalize the Z UMass Lo mouse op dis i J Ges Experiment Subjects Tool erage the ID to Data Filt i miie Model Fasmustiss attenuate outliers Target Width General Pasqgeters Input Device unknown StarttoEnd O Center to Center Ada Probe Width C 1ogMT Drop 1Ds lt 10_ C Average IDs Subject ID Width C alculation Model Subject Name 2D Width Horizontal Extent Smaller of W H Area Sum of W H We escore 2 066 C Incl
37. ttitouch re19 20r LD elo standing tinger n 8 ser LD tuiiteu standing stylus ne14 zer LD tulitsu standing stylus n 12 ser LD tujiteu matking stytus net tp zer LD tulitsu svathing styius n 12 ser keypas finger naa zer LD keypac tinger35 n 8 ser keypasifingerso rea ser LD keypac tingerrep n 8 ser LD keypadstytus rm8 ser LD keypad trackbail n 8 ser LD seconcarpstanding finger ns se secondarystanding stylus n 8 ser LD secondan standingtrackbal n8 LD tabletsitting finger n 11 ser LD tavletsitting ringer 18 nett ser LD tabletsitting finger30 n 11 ser tadtetsitting fingers cnet ser LD tabletsitting finger 45 60 n 11 5 LD tabletsitting ingereo nett ser LD tablet sitting stytus n 11 ser X Dm File Properties File Name mouse optical n 12 ser Last Modified 01 11 07 13 41 30 File Size 2 525 371 FileType ser Show error selections open circl Load another data set and displa along with the current data set Useful for comparing relative accuracy of input devices p Legend G Program Files MTE data runs merged sets chi mouse optical n 12 ser Summary Statistics Raw Data Model Evalulation Trajectory Flot Trajectory Analysis Ad Hoc Analysis Below is an example of a zoomed display showing the cursor path
38. ude Errors Configuratio z a Model Comparison Simple Fitts Welford __Shannon __ Meyer __AccotZhai_ _ID 2 _____ JRaw 0 70 074 0 74 0 74 0 74 0 74 0 70 P E mergedsets ROW 0 72 0 78 0 78 0 78 0 78 0 78 0 73 gt E archive RE A W 0 50 0 55 0 55 joss 055 0 55 0 49 chirmouse n 19 ser R D W 0 52 0 60 0 61 0 61 061 0 61 1053 L chismouse mechanical n 7 ser 1h 34 62 5 29 480 444 5 08 451 15 32 DREE TP A bps 1110 4 55 3 52 3 65 341 3 84 1 86 TP D bps 13 12 4 79 3 74 3 86 3 67 2 01 O chimuttouen cre 19 201 Mean ID 9 32 3 82 295 3 07 2 86 1 56 LD elo standing finger n 8 ser Max ID 33 85 6 08 510 512 5 82 4 09 LD tujitsu standing stytus n 11 ser Min 1D 0 70 0 48 _ 0 26 _ 0 76 _ 0 83 10 11 Mean MT 839 56 839 56 639 56 839 56 639 56 i 839 56 D sujtsostancing ys cre t2pcu Max MT 2 391 00 2 391 00 2 391 00 2 391 00 2 391 00 2 391 00 2 391 00 D tuiitsvmatingstyuscrettiser f Min oT 250 00 250 00 250 00 250 00 250 00 250 00 250 00 250 00 LD tujitsuwatking styus n 12 ser f Intercept 570 117 224 148 276 570 126 546 L heypadstinger nm set Slope 29 189 208 225 197 29 222 188 LD keypad tinger 35 n 8 ser se l Ly keypad tingers0 n 8 ser Effective width Binned Data keypadstingenrep n 8 ser Effective Width 7 D kevpad stytus n 3 ser u E D bain Minishb ath di _ Target Position N Mean End Point Nominal Width Effective Width SD Mean SD Center apne naewel Oot set W 79
39. uration Sane o Lebel Text homs e 4 Text displayed Home Position Center Width 50 Height 50 random exent Options D simpte fitts xt Beep OnStart V Invert On Start K Hide OPGtart D static test xml O testxmi D test2 xml Test New Save Clos Help Emits a sound Inverts the Hides the home upon selection of the home region foreground and background colors the home region upon selection region once it is selected and the trial timing starts inside the home region Location where home region is displayed Choices are Center centered on the screen Random randomly placed Origin upper left corner of screen None no home region is displayed Geometry of the home region If no home region display is chosen the trial timing starts immediately rather than when the home region is selected Not using a home region allows capturing reaction time rather than only movement time 11 Once the home region parameters are configured the target region settings are made Select the Target Region tab to display those settings Select the shape amp Experiment Configurator simple fitts type Use the Experiments General Home Region Target Regioa to configure the ci configs ass gt EI CHI 2007 Shape Configuration p gt Ei field Shape Type LebelRegion lam g
40. veraged MT values across ranges of D attenuating the effect of outliers A sortable table containing collected trial data can be drawn on to identify outliers Exporting to Other Tools The tool records the experiment configuration and the data for each movement trial in a persistent and sharable XML document While the statistical mechanisms built into MTE are certainly useful they are limited but easily augmented by specialized statistics packages The data can be copied to the clipboard or saved as CSV files for import into Microsoft Excel and statistical analysis packages such as R Although the plotting capabilities of MTE are useful for interactive exploration they are not configurable enough for publication Both R and Microsoft Excel offer better support in that area Feedback This manual is a continuous work in progress Your feedback and additions are welcome If you would like to edit the manual please let us know and we will send you the Word file for modification You can contact us at mschedlbauer neu edu Quick Start Tutorial A step by step tutorial Process Experimentation in MTE is a three step process 1 Configure the experiment parameters and store the configuration settings in a file 2 Load a stored experiment configuration and run the experiment either locally or remotely requires installation of M7E on the remote computer 3 Merge the results from multiple subjects into a single file and then load the
Download Pdf Manuals
Related Search