Bachelor thesis - Department of Cybernetics
Contents
1. stimul struct area of interest struct scanpath 1xN 1 0 sequence of areas of interest time 1xN double in milliseconds pupil center x 1xN double in pixels pupil center y 1xN double in pixels pupil width 1xN double in millimetres pupil height 1xN double in millimetres raw position x 1xN double in pixels raw position y 1xN double in pixels filtered position x 1xN double in pixels filtered position y 1xN double in pixels raw pixel velocity x 1xN double in pixels per second raw pixel velocity y 1xN double in pixels per second raw pixel velocity 1xN double in pixels per second fillered pixel velocity x 1xN double in pixels per second filtered pixel velocity y 1xN double in pixels per second filtered pixel velocity 1xN double in pixels per second raw degree velocity x 1xN double in degrees per second raw degree velocity y 1xN double in degrees per second raw degree velocity 1xN double in degrees per second fillered degree velocity x 1xN double in degrees per second filtered degree velocity y 1xN double in degrees per second filtered degree velocity 1xN double in degrees per second raw pixel acceleration x 1xN double in pixels per second 2 raw pixel acceleration y 1xN double in pixels per second 2 raw pixel acceleration 1xN double in pixels per second 2 filtered pixel acceleration x 1xN double in pixels per second 2 fillered pixel acceleration y 1xN double2. U N FP O tn tA A WN Application for Feature Extraction LIST OF TABLES from Eye Movement Signal Analysis List of tables Table 1 This table shows different types of eye movements their length in milliseconds amplitude and velocity adapted from Holmqvist 2011 5 Table 2 extractors 18 Table 3 file names 19 Table 4 results 19 List of figures Figure 1 A block diagram of the proposed method the part highlighted in yellow is concerned with this bachelor thesis 2 Figure 2 EMSA structure 10 Figure 3 33 446 of signal data are out of screen 16 Figure 4 dots2 17 Figure 5 Task time of different subjects 18 List of pictures Picture 1 I4Tracking measuring device 4 Picture 2 EyeMove Toolbox 4 Picture 3 Saccades and fixations during reading task picture adapted from Rayner 2007 6 Picture 4 Application window 13 Software Matlab Version 2012b The MathWorks Inc I4Tracking Medicton s r o EMSA toolbox ii Application for Feature Extraction 1 INTRODUCTION from Eye Movement Signal Analysis Chapter 1 Introduction Human eye is an extraordinary organ The ability to see helped the development of human and other species In last few decades it happened to be not only our main tool to see but also a powerful tool in diagnose of serious human conditions The measurement of human eye movements in last few decades led to important discoveries about psychologica
3. Vladim r Ma k DrSc vedouc katedry V Praze dne 10 1 2013 Prohl en Prohla uji e jsem p edlo enou pr ci vypracoval samostatn a e jsem uvedl ve ker pou it informa n zdroje v souladu s Metodick m pokynem o dodr ov n etick ch princip p i p prav vysoko kolsk ch z v re n ch prac V Praze dne 24 5 2043 Acknowledgements First and foremost I would like to thank my supervisor Ing Martin Maca for many valuable suggestions moral support and productive conversations My gratitude also goes to Ing Daniel Nov k who introduced me to Ing Maca Finally let me thank my whole family for their great support and great patience Abstrakt Hlavn m c lem t to pr ce bylo navrhnout aplikaci s grafick m u ivatelsk m rozhran m pro hromadnou extrakci p znak z anal z sign l o n ch pohyb v prost ed Matlab a struktury soubor p znakov ch extraktor Tato aplikace v znamn pom e v budouc m studiu o n ch pohyb Pomoc videookulografick metody VOG byla nam ena data pro otestov n hromadn extrakce p znak Bylo vyhodnoceno celkem 15 z znam poch zej c ch z m en o n ch pohyb zdrav ch subjekt p i e en verb ln ch 1 neverb ln ch sekven n ch loh Na tomto vzorku dat byla otestov na funk nost automatick hromadn extrakce p znak Abstract The main goal if this work is to propose the applicati
4. calculation failure or missing input data in try block handle the error in catch block and provide the right output or NaN The feature extractor always counts with only one EMSA file or any other Matlab data structure and it is always the first input argument If the feature extractor has special input arguments they must be specified in arguments cell array The special input arguments must have numeric value use 1 or 0 for Boolean values Output of the feature extractor is 1xN cell array This cell array can contain numbers strings or other Matlab structures Count N must be always same for one feature extractor the same feature extractor cannot give for one data file 1x3 cell array and for another data file 1x5 cell array 2 3 Graphical User Interface Because feature extractor has as input only one analysis file the Eye Movements Feature Extraction Tool EMFET was made as tool for mass extraction This application Picture 4 was made as Matlab Graphical User Interface Matlab GUI and needs Matlab 2009 or newer in order to run properly EMFET supports as input data any mat Matlab structure but the feature extractors must be adapted on this structure 12 28 Application for Feature Extraction 2 IMPLEMENTATION from Eye Movement Signal Analysis Eye Movements Feature Extraction Tool Files to analyze 0 files loaded Extractor selection CAEMFET extractors Search Available extractors Selected
5. compensation of the head movement the higher this number is the more distracted was the subject during the measurement and this measurement can be considered as faulty 16 28 Application for Feature Extraction 3 VALIDATION from Eye Movement Signal Analysis The out of screen feature extractor considers all tasks in file Instead out of screen2 consider only one task in file the task number is given as argument and is 1 in default 3 1 2 Stimuli type stimuli type In comparison of feature extraction from two and more data files is often essential that the extracted features are from the same stimuli This feature extractor lists the stimuli type of task number given as argument default 1 3 1 3 Task time task time In most cases the stimuli have not specified projection duration and the subjects tell the supervisor when they are done with the task The time they needed for task is feature that can help in diagnose and assessment if the measurement is successful The example how task time can differ by subjects is plotted in Figure 5 All these subjects had to make the same task called dots2 Figure 4 The stimulus for task dots2 Look through the dots one by one and knock by hand when you will be over The results are in milliseconds Figure 4 dots2 17 28 Application for Feature Extraction 3 VALIDATION from Eye Movement Signal Analysis Task 2 dots2 15000 10000 Time ms 5000 Subject
6. event in the whole recorded 6 28 Application for Feature Extraction 1 INTRODUCTION from Eye Movement Signal Analysis data Limiting the counting process on specified tasks is necessary As exception is for example the blink rate In reading tasks is frequent counting the number of fixations per line or word Number of fixations in the area of interest is called fixation density J M Henderson 1999 According to review in Jacob and Karn 2003 number of fixations is one of the most used metrics in usability research In study of Rubino and Minden 1973 authors found that children with learning disabilities made significantly more fixations but there were no significant differences in fixation s duration Also the number of regressions during reading is one of the most major factors in diagnose of genetic and developmental disorders Gilbert 1953 Pavlidis 1985 Biscaldi 1998 With still images stimuli the number of saccades should be equal to the number of fixations For stimuli that elicit smooth pursuit we can count the saccadic rate it would be a measure of prevalence of catch up saccades Studies reviewed by O Driscoll and Callahan 2008 tend to show that participants with schizophrenia have the saccadic rate much higher than control group subjects Number of undesirable fixations during smooth pursuit is also a common feature 1 5 3 Latency and distance based features Latency is a measure of time delay between event
7. extractors out_of_screen2 stimuli_type task_time Arguments Default Values A 7 Argument Value Picture 4 Application window EMFET support multiple analysis data files extraction by multiple feature extractors The application window is separated into 3 panels in the first panel Files to analyze are analysis files selected in the second panel Extractor selection is specified the folder with feature extractors and then selected which feature extractors will be used for feature extraction the third panel Arguments is there to set the individual arguments for each feature extractor The selected feature extractors and their arguments settings can be saved or load 13 28 Application for Feature Extraction 2 IMPLEMENTATION from Eye Movement Signal Analysis Everything about the feature extraction and EMFET application is described in Application Manual which is in Appendix 2 3 1 Output The application output is 3 cell arrays Result Extractors and Data Result is MxN cell array M is total number of files to analyze N is sum of selected feature extractors outputs containing the feature extraction results Extractors is 1xN cell array containing the feature extractors names input arguments if feature extractor has any Data is Mx1 cell array containing the names of data files There are 3 options how to save the computed results and use them in M
8. in pixels per second 2 filtered pixel acceleration 1xN double in pixels per second 2 raw degree acceleration x 1xN double in degrees per second 2 raw degree acceleration y 1xN double in degrees per second 2 raw degree acceleration 1xN double in degrees per second 2 fillered degree acceleration x 1xN double in degrees per second 2 fillered degree acceleration y 1xN double in degrees second 2 filtered degree acceleration 1xN double in degrees per second 2 artifacts 1xN 1 0 vector of ones and zeros fixations struct saccades struct jpg signal dynamic content pic data sig data dyn function unknown start 1xL double data i time end 1xL double data i time middle 1xL double data i time Figure 2b EMSA structure 11 28 Application for Feature Extraction 2 IMPLEMENTATION from Eye Movement Signal Analysis 2 2 Feature extractors In fact the feature extractor is Matlab function M File that has specific structure The structure is well described in the file extractor sample m stored on attached CD This function can have multiple numeric input arguments used for internal calculations This values can specify offset sampling frequency accuracy etc It is important to abide the internal structure of feature extractor for the smooth running of feature extraction It is highly recommended to provide the calculation part of feature extractor with try catch blocks In case of some
9. matlab plottools J C Program Files MATLAB R2012b toolbox matlab scribe J C Program Files MATLAB R2012b toolbox matlab specgraph EL J C Program Files MATLAB R2012b toolbox matlab uitools MovetoTop JE C Program Files MATLAB R2012b toolbox local J Files MATLAB R2012b toolbox matlab optimfun J C Program Files MATLAB R2012b toolbox matlab codetools Move Down J C Program Files MATLAB R2012b toolbox matlab datafun ae eet J C Program Files MATLAB R2012b toolbox matlab datamanager Move to Bottom J C Program Files MATLAB R2012b toolbox matlab datatypes J C Program Files MATLAB R2012b toolbox matlab elfun J C Program Files MATLAB R2012b toolbox matlab elmat k CAProgram Files MATLAB R2012b toolbox matlab funfun d C Program Files MATLAB R2012b toolbox matlab general amp Files MATI ARXR2012hitonlhodmatlabtanide lt Add Folder Save Revert Default Picture 1 Matlab Set Path dialog box Start the application Start the EMFET application from the Matlab command window the syntax is gt gt emfet gt gt result emfet gt gt result extractors emfet result extractors data emfet result extractors data emfet saves the last computed results of application into Matlab workspace when you quit application There are several other ways how to import the result into Matlab work
10. 0 mat 0 0 Kompsm2 10837 7487 result20121213 164358 mat 0 0 Kompsm2 9178 6038 result20121213 165218 mat 3 71 15 95 Kompsm2 9179 8703 result20121213 170206 mat 0 0 Kompsm2 7429 5390 result20121213 171534 mat 0 0 Kompsm2 8203 12281 result20121213 173252 mat 0 0 Kompsm2 6641 5955 result20121213 174750 mat 0 0 Kompsm2 6936 5289 result20121213 175426 mat 0 02 0 Kompsm2 6385 6472 result20121213 175844 mat 0 70 0 Kompsm2 6654 6171 Table 3 file names Table 4 results Table 3 saved as variable file names is where the input data filenames are stored Table 4 is the results table into this table is stored the results of extraction If Table 2 has N cells and Table 3 has M cells the size of Table 4 is MxN in other words the size of the results table is number of extractors results times number of input files 19 28 Application for Feature Extraction 4 CONCLUSION from Eye Movement Signal Analysis Chapter 4 Conclusion Application for feature extraction from eye movement signal analysis was proposed in this work This application manages mass feature extraction from one or more input data files in Matlab format The format of input data is independent on Eye Movements Feature Extraction Tool EMFET application Output files from this application are in Matlab file format that can be used for classification diagnose or in data mining User manual for this application is attached in Append
11. CZECH TECHNICAL UNIVERSITY IN PRAGUE Faculty of Electrical Engineering Bachelor thesis Michal Zeman Application for Feature Extraction from Eye Movement Signal Analysis 2013 Department of Cybernetics Thesis supervisor Ing Martin Maca Ph D Czech Technical University in Prague Faculty of Electrical Engineering Department of Cybernetics BACHELOR PROJECT ASSIGNMENT Student Michal Zeman Study programme Cybernetics and Robotics Specialisation Robotics Title of Bachelor Project Application for Feature Extraction from Eye Movement Signal Analysis Guidelines 1 Develop an application with GUI for automatic feature extraction from results of analysis of eye movement signals The application will enable a mass feature extraction for multiple analysis files b basic statistical analysis and visualization of features c simple extendibility in terms of additional feature extractors 2 Validate the application on provided analysis files a Apply the mass feature extraction b Compare the results with the sample results provided by the supervisor Bibliography Sources 1 Jakub Snopek Metody anal zy z znam o n ch pohyb p i ten a v sekven n ch loh ch Diplomov pr ce katedra kybernetiky CVUT 2003 2 Martin Maca Dyslexia detection using artificial neural networks Katedra kybernetiky CVUT 2005 Bachelor Project Supervisor Ing Martin Maca Ph D Valid until the end of the
12. Figure 5 Task time of different subjects 3 2 Mass feature extraction In this section is the mass feature extraction tested For the capability of mass feature extraction the application Eye Movements Feature Extraction Tool EMFET was created As the input were used the 15 data files in EMSA structure measured by I4Tracking system The output of the test feature extraction is shown below in Tables 2 4 Table 2 saved as variable extractors is where the feature extractors names and arguments are stored The structure of the cell content is feature extractor name argument values index Feature extractor name is always used argument values are used only if feature extractor support variable arguments index is used only if one feature extractor returns more than one result in that case each result is in one cell out of screen out of screen2 6 stimuli type 6 task time 6 task time 2 Table 2 extractors 18 28 3 VALIDATION Application for Feature Extraction from Eye Movement Signal Analysis result20121213 152458 mat 0 19 0 Kompsm2 21316 11266 result20121213 153917 mat 7 03 33 37 Kompsm2 16289 14663 result20121213 154811 mat 0 01 0 Kompsm2 9735 10416 result20121213 155653 mat 1 48 0 Kompsm2 8661 6606 result20121213 161159 mat 0 20 0 Kompsm2 8661 7387 result20121213 161924 mat 0 02 0 Kompsm2 5820 3791 result20121213 16334
13. atlab for more information see Application Manual chapter Saving the result which is in Appendix 14 28 Application for Feature Extraction 3 VALIDATION from Eye Movement Signal Analysis Chapter 3 Validation To test the application it was necessary to create feature extractors Because the EMSA structure is still in development stage and the methods for filtering the signal and feature detection are not available at this time the extractors for feature extraction had to be made for the raw data from the tracking system These feature extractors can be used for checking the accuracy of the measurements and calibration of the camera also can be used in data mining The records of eye movement signals measured by I4Tracking system were used for feature extraction in this work They were partially processed by EMSA toolbox and saved as Matlab mat files Each file corresponds to one measured subject whose eye movements have been recorded and analyzed these fifteen subjects were without eye disorders and data serves as test data for EMSA toolbox and feature extraction creation 3 1 Feature extractors description 3 1 1 Out of screen proportion out of screen out of screen2 The recorded data raw position x raw position y are taken from the tracking system How accurate the data are depends on the conditions of measurement and tracking system abilities Mostly the head of the subject is not firmly attached to be n
14. button Matlab automatically calculate the result cell arrays and save dialog will appear This is one of three options how to save computed results If you cancel this save dialog you can still save the result using the other two methods Saving the result As mentioned before there are 3 options how to save computed results and use them in Matlab 1 Save variables using save dialog that appears with Extraction Variables are stored in mat file which can be easily load into Matlab workspace see Matlab load function 2 Start the application using syntax mentioned in chapter Start the application Then when you quit the application Matlab will store the last computed results into variables you defined 3 After every extraction there is automatically saved the results into file last result mat in the folder with EMFET Application for Feature Extraction BIBLIOGRAPHY from Eye Movement Signal Analysis Bibliography Biscaldi M Gezeck S Stuhr V 1998 Poor saccadic control correlates with dyslexia Neuropsychologia 36 11 1189 1202 Gilbert C L 1953 Functional motor efficiency of the eyes and its relation to reading University of California Publications in Educations 11 159 231 Henderson J M Hollingworth A 1999 The role of fixation position in detecting scene changes across saccades Psychological Science 10 438 443 Holmqvist K Nystr m M Anderson R Dewhurst R Jarodzka H van de We
15. ction 1 INTRODUCTION from Eye Movement Signal Analysis different approach to this people can mean alleviation of disease symptoms and better quality of life This possibility can be discovered in studies of eye movements Applications of statistical pattern recognition methods are still not common Classic statistic methods are mostly used e g hypotheses testing Statistical pattern recognition methods can bring us different approach to the eye movements studies To be able to use these methods we need a tool for automatic mass feature extraction from eye movement signal analysis However there is no application that can be used for feature extraction Main objective of this thesis is to create an user application that will support mass feature extraction As environment for this application was chosen Matlab developed by MathWorks It is fourth generation programming language widely used on Department of Cybernetics Czech Technical University The important property of this application is its modulation character which means that is possible to add new feature extractors and be able to process any data files if the feature extractors are made to process this data type 1 3 Recording of eye movements The technology most widely used in the current designs of eye trackers is videooculography It is video based eye tracking technology using a camera with high sampling frequency The camera focuses on one or both eyes and records thei
16. e direction from one fixation to another They are the fastest movement the body can produce The fixation happens when the eye remains still over a period of time for example when eye stops on a word when reading Picture 3 Glissade is a post saccadic movement of the eye when the eye wobbles a little before coming to a stop Smooth pursuit is when the eye follows a moving object It is driven by different part of brain than saccades Moving stimuli is required for smooth pursuit Microsaccades are very short movements of the eye that are trying to bring the eye back to the center of fixation for example after drift Tremor is a small movement of frequency around 90Hz whose exact role is unclear it can be imprecise muscle control Drifts are slow movements taking eye away from the center of fixation 5 28 Application for Feature Extraction 1 INTRODUCTION from Eye Movement Signal Analysis When a person is reading a sentence silently the eye movements show that not every Word is fixated Every once in a while a regression movement that goes back in the text is made to re examine a word that may have not been f lly understood the first time This only happens with about 10 of the fixations depending on difficult the textis The More difficult the higher the likelihood that regressions are made Picture 3 Saccades and fixations during reading task picture adapted from Rayner 2007 1 5 Feature ext
17. fication The feature extractors based on features described in Chapter 1 5 would be appropriate to create for EMSA files The EMFET application can be redesigned as server client application or web application This would cause end of problems with Matlab licensing Matlab would be only on server machine with single Matlab license the clients will use the numerous possibilities of server machine Improving the EMSA toolbox internal methods this is related to EMSA structure is also possible The current status is not sufficient for studying of eye movements and there need to be a lot of improvements Figure 1 shows the system of eye movement data acquisition and their processing the blocks are separated as well as the software for analysis of eye movement In one day this could be all connected in one big application which will do everything 21 28 Eye Movements keat re Extraction Version 1 0 SOFTWARE MANUAL Table of Contents INTRODUC ON Rn A A A 24 STARTING THE APPLICATION sonda Adk kakaa nada A a AU KA 24 Set Matlab search path enna Piece Cu Rune dane Koa nune 24 Start the mec na 25 EXTRACTION CONFIGURATION ccce EES SEES nnn 26 Select the extraction datas 26 Load and select feature eene nenne
18. hic user interface for automatic mass feature extraction from results of analysis of eye movement signal This application should be able to provide mass feature extraction for multiple analysis files basic statistical analysis and visualization of features For this application is important to be easily extendible in terms of additional feature extractors 2 Design the internal structure of feature extractor and document it for facilitation of further feature extractors creation 3 Create the mass feature extraction output files structure These files in Matlab mat file format contain the data used for classification 4 Validate the application on provided analysis files Apply the feature extraction from multiple input analysis files 5 Contribute in development of Eye Movement Signal Analysis EMSA data structure This data structure is developed concurrently with the application for mass feature extraction and serves as analyzed data file format 1 2 Motivation Modern technologies of videooculography allow recording the movement of human eye with the precision needed for detail examination of these movements Several studies point the possibility to diagnose serious human conditions from the analysis of eye movement Imagine the option of diagnose dyslexia among pre school children even before they start to read diagnose schizophrenia faster than symptoms develop etc The early treatment or 2 28 Application for Feature Extra
19. ijer J 2011 Eye tracking A comprehensive Guide to Methods and Measures Jacob R J K Karn K S 2003 Commentary on Section 4 Eye Tracking in Human Computer Interaction and Usability Research Ready to Deliver the Promises The Mind s Eye Cognitive and Applied Aspects of Eye Movement Research 573 605 Maca M 2005 Dyslexia Detection Using Artificial Neural Networks Diploma thesis O Driscol G A Callahan B L 2003 Smooth pursuit in schizophrenia A meta analytic review of research since 1993 Brain and Cognition 68 3 359 370 Pavlidis G Th 1985 Eye movements in dyslexia their diagnostic significance Journal of Learning Disabilities 18 1 42 49 Rayner K Castelhano M 2007 Eye movements Scholarpedia 2 10 3649 Rubino A C Minden A H 1973 An analysis of eye movements in children with reading disability Cortex 9 217 220 Schmeisser E T McDonoug J M Bond M Hislop P D Epstein A D 2001 Fractal analysis of eye movements during reading Optometry and Vision Science 78 11 805 814 Snopek J 2003 Metody anal zy z znamu o n ch pohyb p i ten a v sekven n ch loh ch Diploma thesis
20. ix The important part of this thesis is design of the feature extractor structure Feature extractors are Matlab single function M Files automatically recognized by EMFET application with designated structure structure is described in Chapter 2 2 and in file extractor sample mat attached on CD They have one mandatory argument which is the single input analyzed data From this is evident the 1 N relation between the input data structure and the feature extractor The feature extractors can have extra variable arguments that are set before each feature extraction in the graphical user interface EMFET is capable of saving and loading the settings of selected feature extractors and their variable arguments For testing purposes were created 4 feature extractors they are described in Chapter 3 1 for EMSA files The complete structure of EMSA file is described in Chapter 2 1 1 for further feature extractor creation As input data served 15 measurements of healthy subjects measured via I4Tracking system They were partially processed by Eye Movement Signal Analysis toolbox EMFET application were used for the mass feature extraction from these measurements thereby was verified the functionality in practice 20 28 Application for Feature Extraction 4 CONCLUSION from Eye Movement Signal Analysis 4 1 Future work There is a lot of space for the future work At first creating more feature extractors is necessary for a successful classi
21. l processes that occur during reading visual search and scene perception There are several studies describing the relationship between eye movements and genetic or developmental disorders such as dyslexia Pavlidis 1985 Biscaldi 1998 Maca 2005 sexual deviance schizophrenia O Driscoll and Callahan 2008 etc However these studies do not provide any clear unified conclusions In a study of eye movements is a lot of space for additional research This work focus on providing user application that can help in further research of eye movements using the methods of artificial intelligence and statistical analysis The system of eye movement data acquisition and their processing used in this work consist of four separated main blocks Figure 1 The connections between them tell us that output of one block serves as input into another block 1 28 Application for Feature Extraction 1 INTRODUCTION from Eye Movement Signal Analysis Recording Raw data Analyzed data Signal analysis of eye movements dat csv xml etc mat iView EyeMove toolbox I4 Tracking EMSA toolbox etc etc Analyzed data Data for classification p TB Feature extraction Tat Classification EMFET k NN Bayes etc Figure 1 A block diagram of the proposed method the part highlighted in yellow is concerned with this bachelor thesis 1 1 Main goals The main objectives of this work are 1 Develop an application with grap
22. n additional eye tracking system is also possible ch todd ze oti Picture 2 EyeMove Toolbox 4 28 Application for Feature Extraction 1 INTRODUCTION from Eye Movement Signal Analysis Task of the signal analysis toolbox is to process the raw data from eye tracker This data contain a lot of artifacts the most significant ones are those made by the eye blink The artifacts like this needs to be found and excluded from following analysis After the artifacts removal it is possible to analyze the parameters of eye movement this is primary the detection of fixations and saccades also other eye movement components if the VOG system is fast enough to record this components The output from signal analysis is Matlab file mat consisting of raw measured data and analyzed signal data For each measured subject is exactly one data file 1 4 1 Classification of eye movements Human eye movements consist of several types of movements The most common types and their typical values are in Table 1 Recognition of these types is fundamental for feature extraction Type Duration ms Amplitude Velocity mmm 39 39 Smooth pursuit sl tremor e Table 1 This table shows different types of eye movements their length in milliseconds amplitude and velocity adapted from Holmqvist 2011 Saccades are guick simultaneous movements of both eyes in the sam
23. nnne nnn rnnt innen 26 Set parameters of feature extractors esee eeeeeeeeee 27 Save Load the 27 EXTRACTION 27 Savin o EO V ena rw SN on RSS 28 Introduction This application the Eye Movements Feature Extraction Tool EMFET was created to simplify the extraction of eye movements symptoms To run this application you need Matlab version 2009 or newer Starting the application Set Matlab search path There are two ways to add the application folder to a Matlab search path use one of these 1 Set your current Matlab folder to folder with EMFET To check your current Matlab folder type into Matlab command window C NEMFET 2 Add EMFET folder to Matlab search path You can do it temporarily by typing into Matlab command window path path folder Note change folder to EMFET folder for example C EMFET or permanently using the Matlab Set Path dialog box use the Add Folder button as you see on Picture 1 All changes take effect immediately MATLAB search path J C Program Files MATLAB R2012b toolbox matlab graph2d Add with Subfolders J C Program Files MATLAB R2012b toolbox matlab graph3d J C Program Files MATLAB R2012b toolbox matlab graphics J C Program Files MATLAB R2012b toolbox
24. o another If you want to use one extractor twice or more for example with another parameter setting simply add more same extractors in the Selected extractors listbox Set parameters of feature extractors If the feature extractor supports variable parameters values you can set them in panel Arguments In the listbox Selected extractors highlight the feature extractor you want to modify uncheck Default Values checkbox and modify the parameters This can be done for all selected feature extractors Save Load the configuration Application allows saving the selection of extractors and their parameter settings Once you have selected the feature extractors and modified the parameters press the Save button the Save dialog appears Picture 3 enter the settings name for example My settings 18 4 2013 and confirm the save with OK button The saves will remain even if you close the program and Matlab Enter settings name Cancel Picture 3 Save dialog Before loading the configuration settings make sure that you have feature extractors you want to load in Available extractors listbox if not see Load and select feature extractors Press the Load button the Load dialog appears Picture 4 choose the one you want to load and press Load button Moje 24 4 2013 test3 Picture 4 Load dialog Extraction Once you have selected the files to analyze and extractors you want to use press the Extract
25. on with graphic user interface for mass feature extraction from eye movement signal analysis in Matlab environment and structure of feature extractors files This application will significantly help in future studies of eye movements Using the videooculography method VOG were measured the signal data for mass feature extraction In total signal data of 15 healthy subjects in dealing with verbal and non verbal seguential tasks were processed On this data sample was tested the functionality of automatic mass feature extraction CONTENTS Application for Feature Extraction Contents Introduction l l Main goals 1 2 Motivation 1 3 Recording of eye movements 1 4 Signal analysis 1 4 1 Classification of eye movements 1 5 Feature extraction 1 5 1 Position based features 1 5 2 Numerosity based features 1 5 3 Latency and distance based features 1 5 4 Frequency based features 1 6 Classification Implementation 2 Signal data 2 1 1 EMSA structure 2 2 Feature extractors 2 3 Graphical User Interface 2 3 1 Output Validation 3 Feature extractors description 3 1 Out of screen proportion out of screen out of screen2 3 1 2 Stimuli type stimuli type 3 1 3 Task time task time 3 2 Mass extraction Conclusion 4 1 Future work A Eye Movements Feature Extraction Tool manual A 1 Table of contents from Eye Movement Signal Analysis O OQ ON tA K U U NY e NO N N E O KF
26. ot disturbed and feel more naturally When the subject moves the head and there is no compensation from the side of the tracking system then the data move off the screen into negative values 15 28 Application for Feature Extraction 3 VALIDATION from Eye Movement Signal Analysis or values higher then screen resolution The perfect example is on Figure 3 the tracking system in this case I4Tracking does not compensate the movement of the head and part of data is out of screen Task 6 kompsm2 Screen Raw data ES CN 22 pts Screen height ja m 200 400 600 800 1000 1200 1400 1600 1800 2000 Screen width Figure 3 33 4 of signal data are out of screen This can cause issues in further analysis of data or feature extraction Also the stimuli picture when given under the data as background does not correspond with the data which can be very confusing If the tracking system automatically compensate the head movement or the compensation is done by EMSA toolbox analysis there can still be some out of screen data For example when the subject just look outside the screen e g on measurement supervisor or some distraction The feature extractors out_of_screen and out_of_screen2 gives the percentage of data outside of screen in range 0 100 If there is no compensation of the head movement the higher this number is the more movement of head this subject made But if there is the
27. r movement The eye tracking system subsequently processes the images into raw data This data consist of raw pupil position in coordinates pixels or angle pupil size and measurement details subject details stimuli details etc Prior to this work and recent state there was a SMI http www smivision com infra red videooculographical device iView 3 0 used to track eye movements of 76 children at the Department of Neurology 2 Medical Faculty Charles University Czech Republic back in year 2003 Current measurements are executed by using I4Tracking tracking system provided by Medicton Group s r o http www medicton com The measuring device consists of high speed camera attached on glasses Picture 1 LCD screen and computer which displays stimuli and process the data 3 28 Application for Feature Extraction 1 INTRODUCTION from Eye Movement Signal Analysis 9 Picture 1 I4Tracking measuring device 1 4 Signal analysis Data measured by iView system were originally analyzed with Eyemove Toolbox Picture 2 developed by Ing Jakub Snopek in his diploma thesis 2003 The new data cannot be analyzed by Eyemove Toolbox and redesign of this toolbox for the new data was declined for various reasons The source code of this toolbox was used in creating the Eye Movement Signal Analysis toolbox EMSA toolbox for Matlab EMSA toolbox support data input from iView system and also I4Tracking system Support of a
28. raction The feature extraction is a way to statistically evaluate the recorded data and compare the results between the tested subjects The methods can be divided into four main groups position based features numerosity based features latency and distance based features and frequency based features The result of feature extraction can be numerical value string vector of values or strings For the mass feature extraction are the results stored in matrixes for simplification of further classification or data mining 1 5 1 Position based features The most used feature extractors are based on position of eye movement events The position duration measures and input output directions are significant Some of the position based feature extractions are prior to numerosity based feature extractions for example determining the regressions quick backward saccades The mean durations of eye movement events are mostly used in all researches The mean fixation duration is considered as an indication of visual information processing time This duration can refer to depth of understanding the text 1 5 2 Numerosity based features To count the eye movement events is one of the methods to quantify them It can be expressed in absolute numbers e g how many times saccades occurred during certain task in proportion to the total number of events or as rate over time There is often little point to calculate the total number of saccades or any other
29. ributed on the development of EMSA structure Figure 2a and Figure 2b shows the complete structure of EMSA file as is to date 2013 05 21 Further development of EMSA structure is possible and there can be major structure changes For the latest form of EMSA structure please refer to development coordinator Ing Martin Maca 9 28 2 IMPLEMENTATION Application for Feature Extraction from Eye Movement Signal Analysis subject recorded configuration origin data i i 1 2 3 struct struct struct string struct name surname gender education subject identification number LEGEND structure description variable name variable type variable description subject information string string string char double double string double male female or M F highest achieved string in string at string string date format yy mm dd town institute measuring person screen width screen height framerate distance notes screen resolution x screen resolution y in pixels in pixels in millimetres in millimetres frames per second millimetres subject to display noise and luminary condition notes double double double double double Figure 2a EMSA structure 10 28 Application for Feature Extraction 2 IMPLEMENTATION from Eye Movement Signal Analysis
30. s often can be used as reaction time based features Most of latency based features are used in dynamic tasks when there are new objects flashing on the screen and the time to saccadic start is measured Also eye voice latency is measured or pupil dilation latency after an event that start the dilation e g bright light Distance based features are not so much often The eye mouse distance is used in some tracking systems measuring the coordination between hand and eye If tracking system support tracking of left and right eye simultaneously the distance between points of gaze of each eye can be measured 1 5 4 Frequency based features Study of the power spectral density of eye movement signal Schmeisser 2001 indicates that frequency based features has significant meaning during reading tasks Eye movements have two components horizontal and vertical Frequency analysis can be used to find some significant properties of both signals Maca 2005 7 28 Application for Feature Extraction 1 INTRODUCTION from Eye Movement Signal Analysis 1 6 Classification Classification is based on the output of feature extraction There are many types of statistical classification methods In many cases there are two stages of classification the learning stage and the classification stage In the learning stage are training sets of data presented to classifier based on this data classifier can decide during the classification stage
31. space see chapter Saving the result The application will start in new window as you see on Picture 2 a 2 Eye Movements Feature Extraction Tool Files to analyze 0 files loaded Extractor selection CAEMFET extractors Available extractors Selected extractors a out of screen2 stimuli type task time Arguments Default Values Argument Picture 2 Application window Extraction configuration Select the extraction data Press the Search button in panel Files to analyze and locate EMSA files or any other Matlab structure mat files in your computer You can choose multiple files in one folder use the mouse selection or CTRL click to select more than one file Load and select feature extractors The default folder for feature extractors is extractors subfolder in EMFET folder If you want to use other feature extractors then copy the feature extractor M files to this folder and restart the application or use the Search button in panel Extractor selection and locate the folder with your feature extractors M files in your computer Feature extractors then appear in Available extractors listbox Files in the chosen folder that are not extractors are displayed on Matlab Command Window Using the arrow buttons select feature extractors you want to use on selected data files The double arrow button adds all feature extractors from one listbox t
32. to which set of categories a new observation belongs Generally the more of the training data the more accurate 15 the result of classification Classification of eye movements usually deals with subgroups of people with some disorders dyslexia schizophrenia etc and healthy test group As classifiers are mostly used k Nearest Neighbor Bayes classifier or neural networks 8 28 Application for Feature Extraction 2 IMPLEMENTATION from Eye Movement Signal Analysis Chapter 2 Implementation In this chapter is described the eye movement signal input data their conversion by Eye Movement Signal Analysis EMSA toolbox for Matlab into EMSA files and structures of EMSA and feature extractors For mass feature extraction was created the application Eye Movements Feature Extraction Tool EMFET which can be used for mass feature extraction of any Matlab structure file with adapted feature extractors 2 1 Signal data Analyzed signal data by EMSA toolbox were provided in mat files readable in Matlab These data contains raw measured data calibration configuration subject information processed signal data etc Because understanding of EMSA structure is required for feature extractor creation and no one yet described it description of EMSA structure is necessary 2 1 1 EMSA structure The EMSA files are Matlab variables of type STRUCT This variable type is similar to Java Object but has different syntax I personally cont
33. winter semester of academic year 2013 2014 sb prof Ing Vladim r Ma k DrSc Head of Department C prof Ing Pavel Ripka CSc Dean Prague January 10 2013 esk vysok u en technick v Praze Fakulta elektrotechnick Katedra kybernetiky ZAD N BAKAL SK PR CE Student Michal Zeman Studijn program Kybernetika a robotika bakal sk Obor Robotika N zev t matu Aplikace pro extrakci p znak z anal z sign l o n ch pohyb Pokyny pro vypracov n 1 Vytvo te aplikaci s GUI pro automatickou extrakci p znak z v sledk anal z sign l o n ch pohyb Aplikace bude umo ovat a hromadnou extrakci p znak pro v ce soubor anal z b z kladn statistickou anal zu a vizualizaci p znak c snadnou roz i itelnost o dal extraktory 2 Validujte aplikaci na souborech anal z dodan ch vedouc m pr ce a Prove te hromadnou extrakci p znak b Porovnejte s dodan m vzorov m v sledkem Seznam odborn literatury 1 Jakub Snopek Metody anal zy z znam o n ch pohyb p i ten a v sekven n ch loh ch Diplomov pr ce katedra kybernetiky VUT 2003 2 Martin Maca Dyslexia detection using artificial neural networks Katedra kybernetiky VUT 2005 Vedouc bakal sk pr ce Ing Martin Maca Ph D Platnost zad n do konce zimn ho semestru 2013 2014 A c 2 S AN prof Ing
Download Pdf Manuals
Related Search