Home

MCSc- CSCI- June- 2013 - DalSpace Home

1. Ups 1 uonewoyul S14 Bunsnu L Odd JNOA Jo UOReWJIUO9 SE a UO ewa siy UE aseajd 7107 6T Jaquiaydas ajep sAepo Jo se Sie Uu Aew yuawpuawe Jno 7107 et Jaquaydas aaay pueog UD youeasay ayy Aq paaosdde uaaq sey Guiusea puewap uo uag jeujsnpuy ue ui saoaaq SIN UO sjenuepy a3ueuayuiepy umay 104 sayepow yndyng se xa opn pue 3xa oipny Burn pagua yoaloud Joy 770727 Jaquaydas pajep yuawpuawe noA pasiape aq aseajd eweyg ug s 1230 ELOTTOT UCI R OTOT 61 60 7107 20 JF 1epasawobuoseys erep s Aaymey 3 7 rbiao ummg o JJ epasawobruoveys uo1 4 gt 40 1 abessay EL9T TTOT eAosdde yuawupuawy 29 76 APPENDIX I STUDY 2 EMAIL RECRUITMENT NOTICE We are recruiting participants to take part in a research study evaluating audio text and audio text as output modalities for viewing training manuals on mobile devices in on demand learning scenario We are looking for Dalhousie University students faculty or staff members Some previous experience with mobile devices like smart phones tablets is required Both expert and novice people in performing computer hardware assembly disassembly tasks or other mechanical installation of parts etc jobs are welcome We are looking for people who are not colour blind have good hearing abilities and have good English comprehension skills The study will be conducted in The Mona Campbell Building at Dalhousie University and will take about 80 90 minutes to compl
2. 8 mentioned that trainees generally remember more of what they see than of what they read or hear and more of what they hear see and do than what they hear and see This statement is illustrated in the form of a diagram refer figure 1 There is no doubt about the fact that e learning has emerged as a substitute to classroom learning hence supporting on demand learning Numerous universities across North America offer online courses Instructional materials can be provided over the web and may contain interactive video tutorials In their paper Zhang et al concluded that video tutorials enabled effective individual learning as the content could be accessed randomly i e the user can play or pause go back and forth to a segment of the video tutorial with minimized lookup time 9 It is easier and faster to browse through an interactive video tutorial when looking for a particular set of instruction as compared to browsing through a textbook 9 Figure 1 Learner s remembering capacities according to Baboo and Lobo 8 2 2 MOBILE LEARNING IT IS DIFFERENT FROM CLASSROOM LEARNING Mobile learning or M learning enables individualized learning which can be delivered to the user anytime and anywhere The use of portable devices like tablets and smart phones has enabled the end user to take the learning from a fixed location such as a classroom to wherever the learner wants to go Thus the context in which mobile learning happe
3. Do you pe Agree Undecided Disagree E 5 Which approach did you follow Learning the whole task then performing Learning the task as you were performing 70 6 The task was easy to perform Do you Strongly Disagree Strongly AGS Agree Undecided Disagree 7 You would be able to perform the task without the manual Do you Strongly Disagree Strongly Agree Agree Undecided Disagree 8 When you were performing the tasks with different interfaces your main focus was on Mention Ranks 1 being the highest priority 3 being the least Text Only Interface Priority Rank Diagrams Text Audio Only Interface Priority Rank Diagrams Audio Text Audio Interface Priority Rank Diagrams Text Audio 71 APPENDIX E INTERVIEW QUESTIONS 1 Which One was the most comprehensive instruction mechanism Why Ll Text Audio Text Audio 2 In which condition was processing the instruction more difficult Why U Text Audio L Audio Text 3 Were the diagrams self explanatory Were you being able to carry out the tasks by just looking at the diagrams Or would you need the text or audio instructions 72 4 Were you confused or found it difficult to switch your attention between the diagrams audio and text 5 How
4. P values gt 0 05 indicate no significant difference 95 Confidence Interval selected Although there were no significant differences found 45 8 participants felt that their attention was more focussed on the mobile maintenance manual than the actual mechanical tasks in the text only and audio only conditions whereas in the multimodal condition only 29 2 participants agreed to this statement A mixed factorial Analysis of Variance ANOVA was conducted on the time data e time data shown in Table 3 as we had 2 repeated measures factors and one between groups factor We looked at the 3 two way interactions Modalities vs Length Length vs Expertise amp Modalities vs Expertise and the 1 three way interaction Modalities vs Length vs Expertise The test of within subjects effects indicated that there is a significant effect of the instruction length on efficiency Dy 0 004 but no significant effect of the modalities on efficiency Pm 0 894 The lack of an interaction between instruction length amp expertise Prg 0 344 and instruction length amp modality Pim 35 0 801 indicates that this effect is consistent for both experts and novices as well as for all 3 modality conditions i e text audio and text audio In other words the almost parallel lines of figure 22 reflect that the difference in the level between the two lines green and blue indicate a main effect of expertise a difference in level for b
5. 18 were male and 6 were females see table 2 The age range of the participants was 18 to 64 The target population was Dalhousie University students faculty and staff members We had 12 expert and 12 novice participants The categorisation was based on the expertise level of the participants in performing LEGO Mindstorm robots assembly tasks or any other mechanical installation tasks Participants 19 were assessed verbally for their self reported expertise level All the expert participants were from the course MECH 6905 Autonomous Robotics thus confirming that they have previous experience in assembling the LEGO Mindstorm robot kits Table 2 Participant s Demographics for study 1 Experts Novices Total 12 12 Male 10 8 Female 2 4 Average Age 34 39 All the participants had prior experience of operating smartphones or tablets We pre screened the participants to ensure they were fluent English listeners and speakers Also we pre screened the participants to make sure that they have no hearing disabilities and were not color blind as the manuals used had some visual color cues as well We used the Ishihara Color Test to check the participants for color blindness The study was conducted in the GV Lab located in the Computer Science section of the Mona Campbell Building 4th floor 3 3 PROTOTYPE DESIGN The literature reviewed in the previous section was used to inform the design of the mobile
6. 19 16 17 13 16 E4 22 23 14 16 17 17 E5 27 24 12 16 17 17 E6 21 18 17 20 16 18 E7 23 19 17 22 18 20 E8 21 26 13 19 19 19 E9 26 24 17 14 12 15 E10 22 22 16 17 12 13 Bil 23 21 14 14 14 17 E12 21 26 14 14 15 18 Mean 23 08 22 41 14 83 17 16 15 16 16 91 NI 25 31 21 25 26 23 N2 26 30 22 21 23 20 N3 22 25 20 23 23 19 N4 29 27 23 22 21 18 N5 27 29 26 24 19 22 N6 31 28 21 26 18 15 N7 22 21 23 23 20 19 N8 28 23 21 18 24 21 N9 26 24 19 22 20 20 N10 25 27 18 25 21 23 N11 27 28 21 26 26 21 N12 22 26 20 24 19 20 Mean 25 83 26 58 21 25 23 25 21 66 20 08 52 30 E Expert E Novice Figure 34 Mean values of number of attention head switches between the mobile maintenance manual and the disassembly task for experts and novices N Novice E Expert A Audio T Text B Both L Long Instruction S Short Instruction 4 4 SUMMARY For this study we changed the mechanical tasks to laptop disassembling The participants agreed that they needed explicit instructions in addition to the diagrams to avoid any confusion related to how to remove a particular part and in which direction The addition of auditory instructions increased the efficiency of the participants for the mechanical tasks The participants completed the mechanical jobs fastest when they had both audio and textual instructions in addition to the diagrams i e the multimodal maintenance manual The participants felt that due to the presence of audio they wer
7. Audio text and multimodal conditions comparison for question 2 i e Your attention was more focused on the manual than the real world task given to you Do you strongly agree agree undecided disagree or strongly disagree Study 2 18 16 14 12 S E Audio 8 E Text 6 E Multimodal 4 2 0 Strongly Agree Undecided Disagree Strongly Agree Disagree Figure 29 Audio text and multimodal conditions comparison for question 3 i e processing the short instruction was difficult Do you strongly agree agree undecided disagree or strongly disagree Study 2 18 16 14 12 10 E Audio 8 E Text 6 E Multimodal 4 2 0 Strongly Agree Undecided Disagree Strongly Agree Disagree Figure 30 Audio text and multimodal conditions comparison for question 4 i e Processing the long instruction was difficult Do you strongly agree agree undecided disagree or strongly disagree Study 2 16 14 12 10 E Audio E Text E Multimodal Strongly Agree Undecided Disagree Strongly Agree Disagree Figure 31 Audio text and multimodal conditions comparison for question 6 i e The task was easy to perform Do you strongly agree agree undecided disagree or strongly disagree Study 2 14 12 10 E Audio E Text E Multimodal Strongly Agree Undecided Disagree Strongly Agree Disagree Figure 32 Audio text and multimodal conditions comparison for question 7 i e You would be able to perform the task without the
8. affect the overall efficiency of a user carrying out a mechanical task while referring to the mobile manual 2 Does the introduction of audio as an output modality option decrease the amount of time spent by the user looking at the screen visual attention when referring to a training manual and carrying out a mechanical task simultaneously 3 Does the expertise of the user impact the efficiency and the amount of time spent referring to a training manual secondary task while carrying out the mechanical task primary task 4 Does the instruction length impact the efficiency and the amount of time spent by the user looking at the screen visual attention when referring to a training manual and carrying out a mechanical task simultaneously 3 2 RECRUITMENT AND STUDY POPULATION The investigators obtained an approval from the Social Sciences amp Humanities Research Ethics Board at Dalhousie University to meet the ethical and safety requirements for research involving human participants see Appendix G Twenty four participants were recruited as substantial feedback was required to get a strong qualitative sense of user s efficiency and accuracy patterns All participants were recruited by email announcements through Notice Digest and through the Faculty of Computer Science mailing list see Appendix A Recruitment posters were also distributed on Dalhousie University s campus see Appendix B Out of the 24 participants
9. anywhere on the bar or by dragging the track bar control where desired 25 Now Insert the 3 7 Dark Stone Gray Angle Beam from the side as shown in fig 8 Figure 12 Action diagram with short textual instruction Insert the cross axle into the center hole of the orange section amp then slide the busing over the cross axle as shown in the figure Figure 13 Action diagram with long textual instruction 26 3 4 STUDY DESIGN All participants involved in the study signed an informed consent form see Appendix C This was administered by the principal investigator at the beginning of the experiment The informed consent form outlined the risks and benefits associated with the study a description of the study the participant s right to withdraw without consequence and assurances of confidentiality and anonymity of personal data At the beginning of the experiment the principal investigator gave a demonstration to every participant on how to interact with the mobile user manual After this training the participants were given some time to try out the interface to make them familiar with the controls Figure 14 A participant performing the assembly task Participants in the controlled laboratory study performed 6 different mechanical tasks using LEGO Mindstorm robots 27 e Creating a motor subassembly Long Instruction e Creating a sensor subassembly Short Instruction e Creating a tire subassembly Lo
10. grease Mobile instructional designers who are responsible for the design and development of learning material for the aforementioned industries can utilize the findings of this research to provide a better end user experience 58 BIBLIOGRAPHY 1 K Hawkey 2011 Adapting to the changing needs of aircraft mechanics Online Available http www is umbc edu DynamicAccessibility W orkshop participants Hawkey DynamicAccessibilityW orkshop CHI201 1 pdf 2 T Nicolai et al Case study of wearable computing for aircraft maintenance in Proc of 2nd International Forum on Applied Wearable Computing Zurich Switzerland 2005 pp 97 110 3 T Capin et al The State of the Art in Mobile Graphics Research in Computer Graphics and Applications IEEE vol 28 no 4 pp 74 84 July Aug 2008 doi 10 1109 MCG 2008 83 4 R C Clark and R E Mayer Applying the Modality Principle Present Words as Audio Narration Rather Than On Screen Text in E Learning and the Science of Instruction Proven Guidelines for Consumers and Designers of Multimedia Learning 2 ed San Francisco USA John Wiley amp Sons 2011 ch 5 pp 99 113 5 D P Brumby et al Fast or safe How performance objectives determine modality output choices while interacting on the move in Proc of the 2011 annual conference on Human factors in computing systems Vancouver BC Canada 2011 pp 473 482 6 C Kamlaskar Multimedia
11. instruct the manual to play and pause by using voice commands It would be even better to have gesture controls i e if I move my head away from the manual to focus on the mechanical task the manual should automatically pause and then when I go back it should play from where I left We think that this would be a great enhancement to the interface and would help minimize the interactions between with the mobile manual so that the users can concentrate more on the real world task 57 5 2 6 ACCURACY All the 24 participants in our studies were able to complete the tasks They made minor errors for instance inserting the wrong screw while performing the task but they corrected them as soon as they realized that they committed an error We neglected this error count as the participants quickly corrected them as they progressed further in the task These errors were correctable and they had no critical affect on the final subassembly But in industries like aviation the stakes are high and the scope for errors is very limited Thus error count cannot be ignored when conducting such a study on the factory floor of an aircraft maintenance facility 5 3 CONTRIBUTION Based on our findings we can conclude that the power of audio could be leveraged in supporting maintenance and repair personnel in industrial domains such as aviation automobile oil and gas plants etc via mobile devices as they get more robust and resistant to shock oil and
12. instructional manual We constructed an interactive prototype of a reference manual to be used in a just in time learning context The users were able to control the content so that they could adjust the pace of the video manual to their own learning pace This was implemented by standard media control features such as play pause rewind forward and skip 3 3 1 STATIC AND DYNAMIC PICTURES ChanLin 34 conducted a study to investigate the differences between static and dynamic animated pictures when used to provide step by step instructions He also mentioned 20 that animations may cause excessive cognitive load as compared to static images due to the fact that an animated visual keeps moving and may confuse the learner when presented for instructional purposes The study found out that sequential still images are better than complex animations for procedural instructions when presented to novice users However he also found that expert users who already have the knowledge of the area merely gets benefitted from the animations Hence for this study it was decided to use still images instead of animated video clips But there was another question to be answered how to make the sequence of still images intuitive enough As it would have been really difficult to figure out how the allocated task has to be performed only with the images It is critical to visually communicate the functional information needed so that a user could p
13. is much less effort to listen as compared to read especially the longer text Expert Preference Novice Preference E Audio E Audio E Text E Text Figure 23 Preference between text and audio multimodal 3 7 2 RESPONSES COLLECTED THROUGH THE INTERVIEW In the interview we asked the participants if they would have been able to carry out the tasks by just looking at the diagrams with no help from the audio or textual instructions 9 out of the 12 participants in the expert group and 7 out of the 12 participants in the novice group agreed to this statement They felt that the action diagrams were self explanatory This turned out to be a marked flaw in the instruction manuals chosen for the tasks The original LEGO mindstorm paper manuals never had any associated text with the diagrams They were designed to be descriptive enough via the diagrammatic illustrations As a result our ability to draw conclusions about the impact of modality were limited Therefore we designed a second study described in Chapter 4 37 In the post session interview we asked participants How useful the audio was when presented with the diagrams when the text was there already We present some of the responses on how the participants felt Participant E1 said Audio is a better reference I mean to read the textual instructions you have to look at the screen whereas with audio you can keep listening and building the sub assembly without constantly l
14. learning in our task domain as the chance of accomplishing the primary task successfully depends upon how well you have learned the secondary task i e learning how to do the job Our task domain differs from the case of GPS navigation systems as a driver knows the primary task already i e he knows how to drive he just wants the navigation instructions Therefore we anticipate a more extensive switch in visual attention when learning is underway 2 11 SUMMARY Based on the related work as described in this chapter we believe that the following factors might impact the effectiveness of the output modality text audio and a combination of both for just in time training while accomplishing a mechanical task e The length of the instructions for both the mediums textual and auditory e The mechanical expertise level of the person carrying out the mechanical task e The work setting in which the person is carrying out the mechanical task 17 CHAPTER3 STUDY ONE LEGO ROBOT KIT ASSEMBLING This research study is part of a larger project which is investigating how to support aircraft mechanics with mobile devices while carrying out maintenance and repair operations We ran two user studies that investigated the tradeoffs of using text audio and a combination of both modalities on the learning experience in a just in time mobile learning environment when the end users are managing two tasks at the same time i e primary task of
15. not have any textual instructions Therefore in order to get answers to our initial research questions we had to design tasks which are complex and for which the textual instructions were required in addition to the action diagrams We requested an amendment for the project 2012 2673 R 1011073 in which we proposed to change the tasks to computer hardware disassembly tasks 4 1 CHANGING THE TASK Two laptops were used to pilot the disassembly tasks the Gateway LT31 and the HP 8710w notebook The associated textual instructions included in the digital mobile manual were exactly the same as those mentioned in the paper service guides from the manufacturers The 6 tasks were as follows e Removal of the hard disk drive Gateway Laptop Long Instruction e Removal of the memory module Gateway Laptop Short Instruction e Removal of the wireless card Gateway Laptop Long Instruction e Removal of the hard disk drive HP Laptop Short Instruction e Removal of the primary memory module HP Laptop Long Instruction e Removal of the optical drive HP Laptop Short Short Instruction The mechanism for disassembling the parts for the laptops was different For instance to remove the HDD from the gateway laptop the participants had to remove the back cover whereas for the HP laptop the access to the HDD was from the front panel For the new tasks the target population was changed to include people who have prior experience in comp
16. of lost like where and what I have to work oni N10 mentioned I wasn t comfortable without the audio Without the audio I wasn t able to follow without looking at the screen It made me slow 4 3 VIDEO DATA ANALYSIS VISUAL ATTENTION SWITCHES We analyzed the video recordings of the participants performing the task We counted the number of attention head switches between the mobile reference manual and the laptop disassembly tasks for the 12 experts and 12 novice participants to get a sense of how many times the participants had to go back and refer to the manual in order to complete the task see Table 6 We found that the frequency of attention head switches for both experts and novice participants was less when presented with the audio and multimodal interfaces as compared to the text only mobile reference manual interface see Figure 34 This further supports the fact that the introduction of audio decreased the amount of time spent by the participants looking at the screen visual attention while carrying out the mechanical tasks as discussed in section 4 2 2 51 Table 6 Number of attention head switches between the mobile maintenance manual and the disassembly task N Novice E Expert A Audio T Text M Multimodal L Long Instruction S Short Instruction TS IL AS AL MS ML El 24 26 15 18 14 19 E2 24 21 13 19 15 14 E3 23
17. read direct quotes prior to their use in reports and I understand that the anonymity of textual data will be preserved by using pseudonyms If this option is chosen please include a contact email address Participant Researcher Name Name Signature Signature Date Date If you are interested in seeing the results of this study please check below and provide your email address We will contact you with publication details that describe the results T would like to be notified by email when results are available via a publication If this option is chosen please include a contact email address 68 Participant Researcher Name Name Signature Signature Date Date 69 APPENDIX D POST TASK QUESTIONNAIRES Participant ID Gender Male J Female Age Please answer the following answers based on the research experiment 1 The interaction with the mobile maintenance manual was easy Do you Strongly Agree Agree Undecided Disagree Strongly Disagree 2 Your attention was more focused on the manual than the real world task given to you Do you ES Agree Undecided Disagree SE 3 Processing the short instruction was difficult Do you pied Agree Undecided Disagree ae 4 Processing the long instruction was difficult
18. the results of the study could be generalized to a variety of contexts and populations Our primary motivation was to support aircraft maintenance technicians by discovering the best possible ways to deliver multimedia maintenance manual content over mobile devices in an industrial setting Ideally we would have used an existing maintenance manual digitized it created versions with the text changed to audio alone or a combination of audio and text and then evaluated it with aircraft mechanics doing real world installation tasks But we did not have access to an aircraft maintenance facility to their actual tasks or to their staff therefore we decided to use an abstraction and came up with complex LEGO Mindstorm robot kit installation task study 1 and computer hardware repair task study 2 so that we could conduct the study in a lab environment While developing the prototype we took into account the small screen size of the mobile devices and the limited availability of visual attention cognitive capabilities and hands while multitasking 1 5 CONTRIBUTION This work is built upon the principles of cognitive load theory We took three new factors into account that we studied in this work the mobile platform just in time scenario and the learning amp performing a mechanical task context Although the Cognitive Load Theory principles apply to the desktop form factor we were focused on investigating if the effects such as th
19. the text as the audio was there Reading text is difficult for me I am impatient to read it said E2 Whereas E7 quoted I never read the full long text it was too much and also makes you kind of slow Audio makes it easier to figure out which part goes where without looking at the manual again as I can listen For the question Which one was the most comprehensive instruction mechanism Why 10 out of 12 novices and 9 out of 12 experts said all three together Interestingly E8 said When I had all three I felt very relaxed Although I preferred not to read the text it didn t feel necessary Diagrams and audio were the priority as I felt that audio is kind of an ongoing crosscheck and gave me a confirmation that if I was doing it right But certainly having text is a reaffirmation and I would sometimes just skim through the short instructions For the question In which condition was processing the instruction more difficult Why 10 participants each of the 12 novices and 12 experts agreed that diagrams with textual instructions were most difficult Participant E9 said Reading is dull One the other hand it was easier to follow along with the audio at a steady pace it felt like someone was doing the task with me EI stated Reading text was an interruption as I had to look back on the screen Audio was beneficial as I was able to hear and capture the information as I was doing the task it acted
20. useful the audio was when presented with the diagrams 6 How useful the audio was when presented with the diagrams when the text was there already 73 APPENDIX F PARTICIPANT PAYMENT RECEIPT My signature below confirms that I received an amount of 15 CAD from Ankur Sharma as an honorarium payment for participating in the Evaluating audio text and audio text as output modalities for viewing maintenance manuals on mobile devices in an industrial setting on demand learning research project I understand this honorarium is taxable income and it is my responsibility to claim it on my income tax as Dalhousie University will not be issuing a T4A for this payment Name please print Signature Date 74 APPENDIX G RESEARCH ETHICS BOARD LETTER OF APPROVAL DALHOUSIE UNIVERSITY Inspiring Minds Social Sciences and Humanities Research Ethics Board Letter of Approval Date May 7 2012 To Ankur Sharma Computer Science Dr Kirstie Hawkey Computer Science The Social Sciences Research Ethics Board has examined the following application for research involving humans Project 2012 2673 R 1011073 Title Evaluating Audio Text and Audio Text as Output Modalities for Viewing Maintenance Manuals on Mobile Devices in an Industrial Setting on demand learning and found the proposed research involving human participants to be in accordance with Dalhousie Guidelines and t
21. will take about 80 90 minutes to complete You will first meet with a researcher to go over an orientation session to get the study details and give consent to do the study You will then complete a set of computer hardware assembly disassembly tasks with guidance through user manuals on a mobile device You will be answering questions about your experience after the tasks and at the end of the session Compensation is a 15 for participation in the study Do I have to sign a consent form Yes in order to participate in this study you will have to sign an informed consent form as it is part of a research project Who can I contact for more information You can contact Ankur Sharma at ankur cs dal ca if you have any questions 78 APPENDIX K STUDY 2 INFORMED CONSENT Evaluating audio text and audio text as output modalities for viewing maintenance manuals on mobile devices in an industrial setting on demand learning Principal Investigator Ankur Sharma MCS Thesis Student Faculty of Computer Science ankur cs dal ca Co Investigators Dr Kirstie Hawkey Faculty of Computer Science hawkey cs dal ca Sukhveer Dhillon MCS Thesis Student Faculty of Computer Science sdhillon cs dal ca Contact Person Ankur Sharma MCS Thesis Student Faculty of Computer Science We invite you to take part in a research study being conducted by Ankur Sharma and Dr Kirstie Hawkey at Dalhousie University Your participation in thi
22. 10 12 2 16433 24 1 03902 12 1 98130 12 1 85703 24 1 38286 12 1 55167 12 1 62782 24 1 47428 12 2 36628 12 2 14883 24 1 21624 12 1 88368 1 94665 24 31 The participants were asked to provide ratings based on their experience on a likert scale of 1 to 5 1 strongly agree 5 strongly disagree for each output modality situation as shown in figures 16 17 18 19 20 and 21 For each question we used the Friedman test to determine whether there was an overall statistical significance The results of the Friedman test revealed that there was no statistical difference found between the mean ranks for the different modalities i e text audio and multimodal 18 16 14 12 10 E Audio 8 E Text 6 E Multimodal 4 2 0 Strongly Agree Undecided Disagree Strongly Agree Disagree Figure 16 Audio text and multimodal conditions comparison for question 1 i e The interaction with the mobile maintenance manual was easy Do you Strongly agree agree undecided disagree or strongly disagree Study 1 32 12 10 8 6 P Audio E Text 4 E Multimodal 2 0 Strongly Agree Undecided Disagree Strongly Agree Disagree Figure 17 Audio text and multimodal conditions comparison for question 2 i e Your attention was more focused on the manual than the real world task given to you Do you strongly agree agree undecided disagree or strongly disagree Study 1 16 14 12 10 E Audio E Text E Multimodal Strongly
23. Agree Undecided Disagree Strongly Agree Disagree Figure 18 Audio text and multimodal conditions comparison for question 3 i e processing the short instruction was difficult Do you strongly agree agree undecided disagree or strongly disagree Study 1 14 12 10 E Audio E Text E Multimodal Strongly Agree Undecided Disagree Strongly Agree Disagree Figure 19 Audio text and multimodal conditions comparison for question 4 i e Processing the long instruction was difficult Do you strongly agree agree undecided disagree or strongly disagree Study 1 18 16 SS 19 m Audio om E Text m Multimodal Strongly Agree Undecided Disagree Strongly Agree Disagree O N A Q Figure 20 Audio text and multimodal conditions comparison for question 6 1 e The task was easy to perform Do you strongly agree agree undecided disagree or strongly disagree Study 1 34 16 14 12 10 E Audio E Text E Multimodal Strongly Agree Undecided Disagree Strongly Agree Disagree Figure 21 Audio text and multimodal conditions comparison for question 7 i e You would be able to perform the task without the manual Do you strongly agree agree undecided disagree or strongly disagree Study 1 The P values for questions 1 2 3 4 6 and 7 see figures 16 17 18 19 20 and 21 obtained from the Friedman test were P 0 749 Pa 0 270 P3 0 850 P4 0 850 P6 0 779 and P7 0 085
24. ETTEN Participant s Demographics for study 2 veeeg E ge BENENNEN dE NEEN EN Time data for task completion study 2 2e geg edd dE Number of attention head switches between the mobile maintenance manual and the disassembly task N Novice E Expert A Audio T Text M Multimodal L Long Instruction S Short Instruction 0e eee viii Figure 1 Figure 2 Figure 3 Figure 4 Figure 5 Figure 6 Figure 7 Figure 8 Figure 9 Figure 10 Figure 11 Figure 12 Figure 13 Figure 14 Figure 15 Figure 16 Figure 17 Figure 18 Figure 19 LIST OF FIGURES Learner s remembering capacities according to Baboo and Lobo Depicting a scenario for an aircraft maintenance mechanic An arm worn mobile device SEA NEE ENEE EE NEEN Inspectors running a series of checks on an aircraft cockpit using a tablet DEVICE dE Eege Different screen form factors desktop tablet and smartphone Cognitive channels Visual and Auditory eceeeee eee ees The working memory model eege Comparison of magnitude of spatial attention split Desktop Vs Tablet Structural diagram of a LEGO mindstorm robert Action diagram assembly instructions for LEGO mindstorm The Manual developed using Adobe Premiere Pro CS 5 5 Action diagram with short textual instruction cceeeee ence eee Action diagram with long textual instruction cceeeeeee eee ee eee A participa
25. G MECHER ege 2 7 SPLIT ATTENTION EFFECT canscvsvcisvisesasiecrsvesesaviwssevieesevtueswtvests 2 8 IS ADDITIONAL INFORMATION ALWAYS REDUNDANT 2 9 EXPERTISE LEVEL OF THE LEARNER ege eege Seege 2 10 THE CONCEPT OF MOTOR CONTROL PRIMARY AND SECONDARY TASKS sscsccxiinecssesccustionstenvapsccee tous versecouivenesseeeuxs 241 SUMMARY EE CHAPTER 3 STUDY ONE LEGO ROBOT KIT ASSEMBLING 3 1 RESEARCH QUESTIONS snsscccsseecsnsveevarour ost eteresrcvonstecressesen ranean NY Ch BA A A U W N jk ke ba 12 15 3 2 RECRUITMENT AND STUDY POPULATION oseere 3 2 PROTOTYPE DESIGN E 3 3 1 STATIC AND DYNAMIC PICTURES eege 3 3 2 STRUCTURAL VS ACTION DIAGRAMS eseu EE EEN Ehe 3 3 3 SEQUENCE OF DIAGRAMS OVER A SINGLE DIAGRAM WITH INFORMATION OVERFLOW E 3 3 4 PROVIDING SPATIAL CONTEXT IN EACH DIAGRAM 3 3 5 INTERFACE DESIGN AND IMPLEMENTATION 3 3 6 INTERACTION WITH THE MOBILE MANUAL eessen 34A STUDY DESIGN getrei ee ee 3 5 COUNTERBALANCING gek gEeNeERe ENNER ek E 36 DAT COLLECTION eege eet Age 3 7 RESULTS scesssccnssysicess cvareavenardacvereqecnsstesosvatessGetens REENEN EEN 3 7 1 TIME DATA amp POST TASK QUESTIONNAIRE FEEDBACK 3 7 2 RESPONSES COLLECTED THROUGH THE INTERVIEW SE EEN CHAPTER A STUDY TWO COMPUTER HARDWARE DISSASSEMBLY 41 CHANGING THE TASK ere elt eg ege 42 RESULT E 4 2 1 TIME DATA amp POST TASK QUESTIONNAIRE FEEDBACK 4 2 2 RESPONSES COLLECTED TH
26. I can But juggling in between the two tasks was a bit difficult N3 had similar views and said Not between the mediums of instruction but in switching attention between the task and the manual For the question Which One was the most comprehensive instruction mechanism Why 8 out of 12 novices and 7 out of 12 experts said all three together N1 replied I was too focused on finding the parts and assembling thus wasn t reading text just listening to the audio But if due to my negligence if I missed some instruction I would go and read the text which is why it was good to have all three 38 For the question In which condition was processing the instruction more difficult Why 10 out of 12 novices and 8 out of 12 experts agreed that diagrams with textual instructions were most difficult When probed N2 replied Without audio it was difficult for me to figure out which part to insert and where audio provided me a sequential flow Participant N10 replied Audio was good for short instructions but even better for the lengthier ones I would not read so many words when I can just listen If I did something wrong due to the audio playing back I was able to immediately correct myself Audio gives you an order of the steps In addition ES mentioned To read textual instructions you have to stop what you are doing I mean it is an interruption whereas audio isn t Interestingly E6 mentioned I
27. INTRODUCTION 1 1 MOTIVATION It is becoming customary to digitize the bulky paper maintenance manuals and assembly instruction documents used in aircraft maintenance and manufacturing facilities This digitization also provides an opportunity for the integration of video simulations for training As this research is a part of the Dalhousie Boeing Mobile Graphics project the interest was to dynamically adapt the content of the maintenance manuals to various devices including large displays desktop displays and mobile devices 1 In particular with the use of mobile devices technicians can access this detailed digital data Ge maintenance manuals while engaged in on demand learning as they perform their tasks on site A number of applications e g Autodesk Inventor Publisher Mobile Viewer are available for mobile platforms such as Android and iOS Wearable computing and the use of augmented reality and virtual reality based applications has been proposed for classroom training programs at aircraft maintenance training schools ep 2 However when on demand learning is required e a quick refresher for how to do a specific job during the everyday maintenance cycle the use of Head Mounted Displays HMDs will not be as practical as mobile devices The main drawback of using mobile devices is that they have small display size so as to maintain the portability of the device 3 Hence including a lot of textual instructions wi
28. Integrating textual and pictorial information via popup windows An experimental study in Behavior amp Information Technology UK Taylor amp Francis Group Vol 17 Issue 5 1998 pp 263 273 30 J Sweller et al The Redundancy Effect in Cognitive Load Theory New York USA Springer Vol 1 2011 pp 111 128 31 S Kalyuga et al The expertise reversal effect in Educational Psychologist UK Taylor amp Francis Group Vol 38 Issue 1 2003 pp 23 31 32 P Reimann and M T H Chi Human Expertise in Human and Machine Problem Solving New York USA Springer 1989 pp 161 191 33 B S Jensen et al Studying driver attention and behavior for three configurations of GPS navigation in real traffic driving in Proc of the 28th International Conference on Human Factors in Computing Systems Atlanta GA USA 2010 pp 1271 1280 34 L ChanLin Formats and prior knowledge on learning in a computer based lesson in Journal of Computer Assisted Learning Hoboken NJ USA Wiley Vol 17 Issue 4 2001 pp 409 419 35 M Agrawala et al Designing effective step by step assembly instructions in ACM Transactions on Graphics New York NY USA ACM Vol 22 Issue 3 2003 pp 828 837 36 J Heiser and B Tversky 2002 Diagrams and Descriptions in Acquiring Complex Systems Online Available http www researchgate net publication 2541507 Diagrams and Des criptions in Acqui
29. Modalities vs Expertise and the 1 three way interaction Modalities vs Length vs Expertise The test of within subjects effects indicated that there is a significant effect of the output modality on the efficiency of the learners implementing a mechanical task while learning Pm 0 000 The participants were fastest when provided with a multimodal interface for the mobile manual But no significant effect of the length of the instruction on efficiency was found Py 0 156 The lack of an interaction between modality amp expertise Pme 0 250 and instruction length amp modality P m 0 747 indicates that this effect is consistent for both experts and novices as well as for all the 2 instruction length conditions i e short and long Also there were no significant interactions found between instruction length amp expertise Piss 0 381 48 Similar to study 1 most of the participants agreed that the interaction with the maintenance manual was easy enough regardless of the modality output condition Also despite the fact that no statistical significance was found the results reflected that the participants found that the task was easy although they wouldn t have been able to complete it without the manual Again all of the participants followed the approach where they were learning the task as they were performing We also asked the participants to rank the modalities based on their main focus for the 3 different mobile man
30. ROUGH THE INTERVIEW 4 3 VIDEO DATA ANALYSIS VISUAL ATTENTION SWITCHES 44 SUMMARY E CHAPTERS CONCLUSION sesdasessccsnsncanssesessancninndcessanaansaencesswnncasiecios Sul SUMMARY OF FINDINGS eenig ege eebe heen 5 2 LIMITATIONS AND FUTURE WORK tegt tre EE DEE 3 2 1 IN SITU EVALUATION RE vi 5 2 2 DEVICES HAD NO VALUE TO THE PARTICIPANTS 56 5 2 3 PRECARIOUS WORK SETTINGS sscansasdewsaeaensahoysuaganiiebond 56 5 2 4 WORKPLACE VS HOME SETTING eu eege EA 57 5 2 5 ENHANCEMENT OF THE USER INTERFACE osoen 57 526 ACCURACY eegene E E sei 58 5 3 CONTRIBUTION TEE 58 BIBLIOGRAPHY egenen eegenen 59 APPENDIX A EMAIL RECRUITMENT NOTICE oseere 64 APPENDIX B POSTER RECRUITMENT NOTICE oseere 65 APPENDIX GT ET RE 66 APPENDIX D POST TASK QUESTIONNAIRES 2 sssaedesseccsevaniaeecerviecaseee 70 APPENDIXE INTERVIEW QUESTIONS 2ccicecuddenesetopeneddanddnsheeaswneastchens 12 APPENDIX F PARTICIPANT PAYMENT RECEIPT kg RENE NEEN ec 74 APPENDIX G RESEARCH ETHICS BOARD LETTER OF APPROVAL 75 APPENDIX H RESEARCH ETHICS BOARD AMENDMENT APPROVAL 76 APPENDIX STUDY 2 EMAIL RECRUITMENT NOTICE oss a APPENDIX J STUDY 2 POSTER RECRUITMENT NOTICE 78 APPENDIX K STUDY INFORMED CONSENT oseere 79 vii Table 1 Table 2 Table 3 Table 4 Table 5 Table 6 LIST OF TABLES Advantages and disadvantages of instruction modalities cceeeeee ees Participant s Dementi gege kd dE Re eg R
31. Simulation for Electronics Laboratory Activity in India in Asian Journal of Distance Education vol 5 3 ASODE 2007 pp 33 45 7 S D Sorden A cognitive approach to instructional design for multimedia learning in Informing Science Journal Santa Rosa USA Informing Science Institute 2005 vol 8 pp 263 279 8 S Baboo and N Lobo Improving Effectiveness of ELearning in Maintenance Using Interactive 3D in International Journal of Computer Science and Information Security USA IJCSIS 2009 vol 5 1 pp 21 24 59 9 D Zhang et al Instructional video in e learning Assessing the impact of interactive video on learning effectiveness in Information amp Management Amsterdam The Netherlands Elsevier 2006 Vol 43 Issue 1 pp 15 27 10 M Sharples et al Mobile Learning Small devices Big Issues in Technology Enhanced Learning Principles and Products The Netherlands Springer 2009 ch 14 pp 233 249 11 L Chittaro Distinctive aspects of mobile interaction and their implications for the design of multimodal interfaces in Journal on Multimodal User Interfaces The Netherlands Springer Verlag 2010 Vol 3 Issue 3 pp 157 165 12 D Fallman Wear point and tilt designing support for mobile service and maintenance in industrial settings in Proc of the 4th conference on Designing interactive systems processes practices methods and techniques Londo
32. UNDERSTANDING THE CONTEXTUAL ROLE THAT MODALITIES PLAY IN JUST IN TIME MOBILE LEARNING WHILE CARRYING OUT MECHANICAL TASKS by Ankur Sharma Submitted in partial fulfilment of the requirements for the degree of Master of Computer Science at Dalhousie University Halifax Nova Scotia June 2013 Copyright by Ankur Sharma 2013 DALHOUSIE UNIVERSITY FACULTY OF COMPUTER SCIENCE The undersigned hereby certify that they have read and recommend to the Faculty of Graduate Studies for acceptance a thesis entitled UNDERSTANDING THE CONTEXTUAL ROLE THAT MODALITIES PLAY IN JUST IN TIME MOBILE LEARNING WHILE CARRYING OUT MECHANICAL TASKS by Ankur Sharma in partial fulfilment of the requirements for the degree of Master of Computer Science Dated June 20 2013 Supervisor Readers il DALHOUSIE UNIVERSITY DATE June 20 2013 AUTHOR Ankur Sharma TITLE UNDERSTANDING THE CONTEXTUAL ROLE THAT MODALITIES PLAY IN JUST IN TIME MOBILE LEARNING WHILE CARRYING OUT MECHANICAL TASKS DEPARTMENT OR SCHOOL Faculty of Computer Science DEGREE MCSc CONVOCATION October YEAR 2013 Permission is herewith granted to Dalhousie University to circulate and to have copied for non commercial purposes at its discretion the above title upon the request of individuals or institutions I understand that my thesis will be electronically available to the public The author reserves other publication rights and neither the thesi
33. ails and give consent to do the study You will then complete a set of Lego robots assembling tasks with guidance through user manuals on a mobile device You will be answering questions about your experience after the tasks and at the end of the session Compensation is a 15 for participation in the study If you are interested in participating please contact Ankur Sharma ankur cs dal ca 64 APPENDIX RB POSTER RECRUITMENT NOTICE RECRUITMENT NOTICE Evaluating audio text and audio text as output modalities for viewing maintenance manuals on mobile devices in an industrial setting On demand learning We are recruiting participants to take part in a research study evaluating audio text and audio text as output modalities for viewing training manuals on mobile devices in on demand learning scenario We are looking for Dalhousie University students faculty or staff members Some previous experience with mobile devices like smart phones tablets is required Both expert and novice people in performing Lego robot assembly tasks or other mechanical installation of parts etc jobs are welcome We are looking for people who are not colour blind have good hearing abilities and have good English comprehension skills The study will be conducted in The Mona Campbell Building at Dalhousie University and will take about 60 minutes to complete You will first meet with a researcher to go over an orientation session to get the study detail
34. arning called the split attention effect When the learners are provided with two or more related sources of information spatially they have to make a switch from one source to the other so that they could maintain a link between the two sources 28 For instance when a picture is described with associated text the learner has to constantly split his or her attention in between the visual and the related text so as to learn what the visual represents This may result in hindered learning if the instructional material is not spatially integrated However this theory may not hold true for mobile devices as the screen size for smart phones and tablets is already very small as compared to desktop screens Thus there is very minimal spatial separation between the diagram and the associated text and integration is thus achieved refer figure 8 Placing the text under the visual instead embedding it with the diagram helps reduce onscreen clutter 29 lal Wd Green represents data X Green represents data X Figure 8 Comparison of magnitude of spatial attention split Desktop Vs Tablet 2 8 IS ADDITIONAL INFORMATION ALWAYS REDUNDANT Sweller et al 30 mentioned that if different modalities are used to convey the same information then that information may become redundant and might not be used to learn But this might not hold true when the information to be learned is very complex For instance a complex diagram illustrati
35. carrying out the maintenance job the secondary task of learning just in time about how to perform the job As we did not have access to the aircraft mechanic population we created an abstraction of an assembly task using LEGO Mindstorm robots 3 1 RESEARCH QUESTIONS As discussed in the related work section of this document there are several pros and cons of using text audio and a combination of both modalities in learning new material Table 1 summarizes the major advantages and disadvantages of using various modalities Table 1 Advantages and disadvantages of instruction modalities Text Only Voice Only Multimodal Advantages Text stays on screen long instruction can be processed Advantages Cognitive load gets divided No visual attention Advantages Choice between voice and text If miss voice have text and vice versa Disadvantages Disadvantages Disadvantages May cause on screen clutter and occlusion Demands visual attention Confusion or mishearing Processing long voice instruction may be difficult Competing modalities More controls to deal with 18 With digital multimedia content consisting of step by step action diagrams being provided over mobile devices in a just in time learning context the goal of our study was to get answers for the following research questions 1 Does the chosen form of output modality text audio and text audio
36. e able to focus more on the mechanical task Also they felt that longer instructions were easier to understand when delivered as voice For the participants text was the last resort to capture any missing instructions 53 CHAPTER5 CONCLUSION In this chapter we summarize the conclusions we derived from the two studies we conducted Then we talk about some of the limitations of the studies we conducted and describe some of the ways in which this research could be taken further 5 1 SUMMARY OF FINDINGS This work was motivated by the desire to support aircraft maintenance engineers and provide them with multimedia instructional content over mobile devices when needed just in time For this purpose we conducted controlled lab studies to obtain feedback on the modalities of instruction used 1 e diagrams text and audio Based on our findings we conclude the following statements for our research questions 1 Does the chosen form of output modality text audio and text audio affect the overall efficiency of a user carrying out a mechanical task while referring to the mobile manual Yes The participants were most efficient in carrying out the mechanical tasks when they had both audio and text as learning support 2 Does the introduction of audio as an output modality option decrease the amount of time spent by the user looking at the screen visual attention when referring to a training manual and carrying out a mechan
37. e load theory and mobile interface design The existing research describes how multimedia content helps in the learning process In this chapter I present how just in time mobile learning is different than classroom training as the context varies the difference between the two scenarios learning amp problem solving and learning amp performing a mechanical task I also discuss why there is a need to evaluate the effect of the modality of audio on learning in a just in time mobile learning environment e when the learner is performing an actual real world mechanical job as he is learning 2 1 MULTIMEDIA INSTRUCTIONAL DESIGN Multimedia De presenting the information using a combination of content delivery formats such as text graphics video and audio etc has been an integral part of education Research supports the fact that multimedia content when presented with appropriate media selection results in learning gains 4 6 7 amp 8 The most common examples of multimedia instructions that we see in practice are online learning and computer based training CBT The inclusion of multiple modes of instruction delivery increases the level of enthusiasm of the learner hence engaging the learner at a much greater magnitude 7 Therefore it is justified that most of the instructors make use of PowerPoint slides as they can include images animations and related videos in the instruction material In their paper Baboo and Lobo
38. e spatial split attention effect are valid for mobile devices as well An initial poster publication of this work was presented at Graphics Interface 2012 it highlighted the need to investigate the effect of text voice and a combination of both modalities on the learning experience in on demand learning 38 The attendees provided us with their valuable feedback which helped us refine our prototype with the inclusion of action diagrams i e a diagram that provides the functional information about the object via directional arrows 1 6 THESIS ORGANIZATION This thesis is divided into five chapters Chapter 2 Related Work discusses the related work in the areas of multimedia instructional design cognitive load theory and mobile interface design Chapter 3 Study One LEGO Robot Kit Assembling presents the research questions study design development of the digitized instructional content data collection methods and the outcomes of the first controlled lab study Chapter 4 Study Two Computer Hardware Disassembly presents the modified mechanical tasks and summarizes the results of the second study Chapter 5 Conclusion Limitations and Future Work summarizes the conclusions derived from the user studies some limitations of the user studies potential future work and the contributions of this research CHAPTER 2 RELATED WORK This chapter reviews the previous research done in the areas of multimedia instructional design cognitiv
39. erform the assembly task effectively When communicating visual instructions for an assembly task the spatial orientation of the object to be worked on and the order in which the parts get assembled or disassembled play a key role In their paper Agrawala et al 35 acknowledged that there are two vital tasks that must be taken care of when designing assembly instructions planning and presentation By planning they meant choosing the most appropriate order as there could be many of the assembly or disassembly process which could be easily understood by the learner By presentation they meant properly reflecting the order via successive diagrams 3 3 2 STRUCTURAL VS ACTION DIAGRAMS For instructional purpose diagrams can be presented in two ways structural or action 35 A structural diagram is a rigid portrayal of an object An action diagram on the other hand provides functional information about the object by the addition of directional arrows Heiser and Tversky 36 conducted controlled experiments where they compared the effectiveness of information communication via structural and action diagrams for mechanical systems such as car brakes and pulley systems They found that structural diagrams are only suited to describe the anatomy of the mechanical object with accompanying text The action diagrams conveyed the spatial operations to be carried out one after another in a more effective way The participants preferred action diagram
40. esign of agent based Microworlds in Journal of Educational Psychology Washington DC USA APA Vol 95 Issue 4 2003 pp 806 813 21 The Cambridge Handbook of Multimedia Learning Cambridge University Press New York 2005 22 K W Hartley Media Overload in Instructional Web Pages and the Impact on Learning in Educational Media International UK Taylor amp Francis Group Vol 36 Issue 2 1999 pp 145 150 23 J Sweller et al Cognitive architecture and instructional design in Educational Psychology Review New York USA Springer Vol 10 No 3 1998 pp 251 296 24 G A Miller The magical number seven plus or minus two Some limits on our capacity for processing information in Psychological Review Washington DC USA APA Vol 63 1956 pp 81 97 25 A R Artino 2008 Cognitive load theory and the role of learner experience An abbreviated review for educational practitioners in Association for the Advancement of Computing in Education Journal Chesapeake VA USA AACE Vol 16 No 4 2008 pp 425 439 26 A Baddeley Human Memory Theory and Practice Boston MA USA Allyn amp Bacon 1998 61 27 P L Smith and T J Ragan Instructional design 3 ed Indianapolis IN USA Wiley 2004 28 J Sweller et al The Split Attention Effect in Cognitive Load Theory New York USA Springer Vol 1 2011 pp 111 128 29 M Betrancourt and A Bisseret
41. ete You will first meet with a researcher to go over an orientation session to get the study details and give consent to do the study You will then complete a set computer hardware assembly disassembly tasks with guidance through user manuals on a mobile device You will be answering questions about your experience after the tasks and at the end of the session Compensation is a 15 for participation in the study If you are interested in participating please contact Ankur Sharma ankur cs dal ca 77 APPENDIX J STUDY 2 POSTER RECRUITMENT NOTICE RECRUITMENT NOTICE Evaluating audio text and audio text as output modalities for viewing maintenance manuals on mobile devices in an industrial setting On demand learning We are recruiting participants to take part in a research study evaluating audio text and audio text as output modalities for viewing training manuals on mobile devices in on demand learning scenario We are looking for Dalhousie University students faculty or staff members Some previous experience with mobile devices like smart phones tablets is required Both expert and novice people in performing computer hardware assembly disassembly tasks or other mechanical installation of parts etc jobs are welcome We are looking for people who are not colour blind have good hearing abilities and have good English comprehension skills The study will be conducted in The Mona Campbell Building at Dalhousie University and
42. extual data will be preserved by using pseudonyms Participant Researcher Name Name Signature Signature Date Date Or I want to read direct quotes prior to their use in reports and I understand that the anonymity of textual data will be preserved by using pseudonyms If this option is chosen please include a contact email address Participant Researcher Name Name Signature Signature Date Date If you are interested in seeing the results of this study please check below and provide your email address We will contact you with publication details that describe the results I would like to be notified by email when results are available via a publication If this option is chosen please include a contact email address 81 Participant Researcher Name Name Signature Signature Date Date 82
43. felt that audio is in sync with my pace of doing the task I was able to follow the instructions 3 8 SUMMARY Due to the flaw described above in our experimental task we can only say that if the supporting instructions aren t necessary to complete the mechanical tasks it doesn t matter in which modality they are presented to the learner Although we weren t able to find a statistical significance we did gain a rich understanding of the tradeoffs with each modality in this learning context Overall the participants both experts and novices felt that having audio made them faster as they were still engaged with the assembly task while paying attention to the audio for specific cues on the orientation and the in which the parts have to be installed The participants felt that the process becomes livelier when audio is there Also participants were selectively switching between text and audio instructions when they felt a need thus preferred the multimodal interface over the text or audio alone as they were able to look for the required information from various sources 39 CHAPTER4 STUDY TWO COMPUTER HARDWARE DISSASSEMBLY After running the first study we discovered that for the LEGO robot assembly tasks participants felt that the textual and auditory instructions provided weren t necessary to complete the tasks This is likely because the paper based user guide included in the LEGO Mindstorm kits used for the tasks did
44. g context the goal of our study was to get answers for the following research questions 1 Does the chosen form of output modality text audio and text audio affect the overall efficiency of a user carrying out a mechanical task while referring to the mobile manual 2 Does the introduction of audio as an output modality option decrease the amount of time spent by the user looking at the screen visual attention when referring to a training manual and carrying out a mechanical task simultaneously 3 Does the expertise of the user impact the efficiency and the amount of time spent referring to a training manual secondary task while carrying out the mechanical task primary task 4 Does the instruction length impact the efficiency and the amount of time spent by the user looking at the screen visual attention when referring to a training manual and carrying out a mechanical task simultaneously 1 4 RESEARCH CHALLENGES The objective of this research was to find out the best possible modality or a combination of modalities to support learners in a mobile just in time learning scenario while they are carrying out mechanical tasks such as assembly or disassembly of parts While working towards this objective we faced a lot of challenges For instance we had to come up with a combination of complex learning and implementation tasks which required considerable visual attention so that we can test the impact of modalities and
45. he Tricouncil Policy Statement on Ethical Conduct in Research Using Humans This approval will be in effect for 12 months from the date indicated below and is subject to the following conditions 1 Prior to the expiry date of this approval an annual report must be submitted and approved 2 Any significant changes to either the research methodology or the consent form used must be submitted for ethics review and approval prior to their implementation 3 You must also notify Research Ethics when the project is completed or terminated at which time a final report should be completed 4 Any adverse events involving study participants are reported immediately to the REB Effective Date May 4 2012 signed Expiry Date May 4 2013 Dr Lori Turnbull Chair SSHREB IMPORTANT FUNDING INFORMATION Do not ignore To ensure that funding for this project is available for use you must provide the following information and FAX this page to RESEARCH SERVICES at 494 1595 Name of grant contract holder Dept Signature of grant contract holder Funding agency Award Number Dal Account if known Dalhousie Research Services Research Ethics 6299 South Street 2 Floor Suite 231 PO Box 15000 Halifax NS Canada B3H 4R2 Tel 902 494 3423 Fax 902 494 1595 Email ethics dal ca www dal ca research 75 APPENDIX H RESEARCH ETHICS BOARD AMENDMENT APPROVAL ND Sawo UOUDUE EZrC P6P 206 SUD D sen sawog uoJeys
46. hey wanted to use text as the last resort just as a reference if they weren t able to make out what to do from the diagram and the audio instructions Audio played a key role when the participants wanted to get a sense of the orientation and the direction in which a part has to be removed or installed as they thought that the action diagrams weren t sufficient enough for that kind of information Audio also made the process more engaging and lively The participants felt that they weren t alone and as if somebody is actually instructing them how to perform the task Participants did not find it difficult to switch their attention between the visual sources i e the diagram and text but they found it difficult to go back and forth between the task and the mobile manual Hence we can say that there was no spatial split attention effect as we hypothesised 55 5 2 LIMITATIONS AND FUTURE WORK Every study has limitations and the studies conducted in this work are no exception There were some factors we wanted to consider but due to the lack of resources and to make sure that we did not introduce too many variables to be measured in our study we restricted to the aforementioned study design and context But we would like to present some suggestions that can be considered as a future development of this research 5 2 1 IN SITU EVALUATION Initially we wanted to conduct this study on the factory floor of an aircraft ma
47. iation N Expert Novice Total Expert Novice Total Expert Novice Total Expert Novice Total Expert Novice Total Expert Novice Total The participants were asked to provide ratings based on their experience on a likert scale of 1 to 5 1 strongly agree 5 strongly disagree for each output modality situation As shown in figures 27 28 29 30 31 and 32 For each question we used the Friedman test to determine whether there was an overall statistical significance The results of the Friedman test revealed that there was a statistical difference between the mean ranks for the different modalities i e text audio and multimodal for questions 2 i e Your attention was more focused on the manual than the real world task given to you Do you strongly agree agree undecided disagree or strongly disagree see figure 28 and 4 i e Processing the long instruction was difficult Do you strongly agree agree undecided disagree or strongly disagree see figure 30 44 14 12 E Audio E Text E Multimodal Strongly Agree Undecided Disagree Strongly Agree Disagree Figure 27 Audio text and multimodal conditions comparison for question 1 i e The interaction with the mobile maintenance manual was easy Do you Strongly agree agree undecided disagree or strongly disagree Study 2 16 14 12 10 E Audio E Text E Multimodal Strongly Agree Undecided Disagree Strongly Agree Disagree Figure 28
48. ical task simultaneously The participants felt that the introduction of audio decreased the amount of time spent by them looking at the screen visual attention while carrying out the mechanical tasks This holds for both novice and expert users 3 Does the expertise of the user impact the efficiency and the amount of time spent referring to a training manual secondary task while carrying out the mechanical task primary task 54 Yes The experts spent less amount of time referring to a training manual secondary task while carrying out the mechanical task primary task as compared to the novices 4 Does the instruction length impact the efficiency and the amount of time spent by the user looking at the screen visual attention when referring to a training manual and carrying out a mechanical task simultaneously There wasn t any significant impact of the instruction length on the efficiency and the amount of time spent by the user looking at the mobile manual while performing the tasks We also state the following based on or findings Participants preferred having both supporting auditory and textual instructions with the diagrams This was because when they were juggling in between the learning and the mechanical task they were looking for the information in the quickest possible way So they were switching their attention rapidly between the 3 sources of information De diagrams text and audio T
49. instruction modalities This was done to omit any learning or order effects Figure 4 1 illustrates the counterbalancing chart used 28 Novice ona nmz nm na ns No nz ns no mio nu n2 Figure 15 The Counterbalancing Chart N Novice E Expert A Audio T Text B Both L Long Instruction S Short Instruction 29 3 6 DATA COLLECTION The principal investigator observed the participants and took notes throughout the study The sessions were also video recorded so that any interesting behaviours were captured The time taken by each participant to complete each task was recorded using a digital stop watch and was analysed to measure efficiency Expressed confusion by the participants was noted while they carried out the allocated tasks After completing the allocated two tasks for each output modality condition the participants answered a brief post task questionnaire reflecting their experience with that interface and the learning see Appendix D Once the participant finished all six tasks the principal investigator conducted a brief interview with the participant in order to get ratings for the interfaces used for each of the tasks completed in the conditions provided and to elicit explanation and reflection about interesting behaviours that were noted during the condition see Appendix E 3 7 RESULTS As discussed previously participants were asked to fill a post task questionnaire for each output modalit
50. intenance facility Since we did not have access to one we created the task abstractions and did a laboratory evaluation Hence we weren t able to include factors such as changing levels of illumination distraction in form of background noise coming out of other machines or from the workers on the factory floor vibration and motion of the user or the industrial workbench itself It would be really interesting to see how these factors impact the learning and efficiency of the users and the results could be further generalized to various settings as the context changes 5 2 2 DEVICES HAD NO VALUE TO THE PARTICIPANTS The LEGO kits and the laptops used for the study were of no value to the participants Whereas in an industrial setting the mechanics are responsible for the proper use and maintenance of the equipment Also if the user owns the equipment they may even become more cautious while handling it They may become slow while carrying out some assembling operations on such equipment It certainly makes sense to evaluate and see how this factor impacts the efficiency of the users while carrying out some assembling operations 5 2 3 PRECARIOUS WORK SETTINGS In the aircraft maintenance facilities the maintenance mechanics often have to climb on a ladder or stool to carry out a repair job Thus they are exposed to safety hazards of falling down or hurting themselves with the tools Audio as an output modality when delivered over po
51. like a checklist like I did this then the next step and so forth N4 said Long text is like noise on the screen short text was still fine I listened to the audio throughout It facilitates the whole process I was confused and slower if with the diagrams and text said N5 50 For the question How useful the audio was when presented with the diagrams Participants expressed their views in the following statements E10 said When I am performing the task I knew what part to do next because of the audio even before looking at the screen It made me organized E11 specified I felt more relaxed when audio was there It felt another person is there talking to me and explaining the instructions to me N11 mentioned I was less nervous when audio was present I felt it made me go fast on the task N3 interestingly stated I guess there is a particular tone in the audio which is absent in textual instruction When you listen you can capture the key steps distinctly like release the latches at an angle So you are already aware of the fact that it has to be at an angle N6 replied With audio I was better able to handle the controls it gave me a clue on when to pause N7 expressed Audio is more conveying than text For me it was a better explanation e g it asked me to lift the part etc It makes you fast I guess N8 also had similar views and stated When audio wasn t there I felt kind
52. load on the learner Multimedia Content Working Memory Long Term Memory Verbal Model 5 p Prior Knowledge Existing Schema ee re f eevee T Figure 7 The Working Memory Model by Mayer 21 The human brain utilizes the working memory to store a limited amount of temporal information for a short duration to be processed refer figure 7 Also the Long term memory of a human is capable of storing comparatively larger amount of information for a lengthier time 22 Sweller et al 23 mentioned that the learner can actively be aware of only the information currently being processed by the working memory Also the working memory can only handle a small piece of information for a short duration of time 24 Apart from information processing working memory is also responsible for comparing and organizing information slices 25 According to the cognitive architecture of the working memory as explained by Baddeley 26 the pictorial amp textual information gets processed by the visual channel and the audio instructions get processed by the auditory channel Thus the instructional material when presented as a combination of both modalities can avoid reaching the working memory threshold of the human brain hence avoiding cognitive overload Smith et al 27 referred to it as off loading the extraneous information from the visual path to the auditory 14 2 7 SPLIT ATTENTION EFFECT Another phenomenon exists in multimedia le
53. manual Do you strongly agree agree undecided disagree or strongly disagree Study 2 The P values for questions 1 2 3 4 6 and 7 see figures 27 28 29 30 31 and 32 obtained from the Friedman test were P 0 318 P2 0 01 P3 0 327 P4 0 017 Po 0 420 and P7 0 786 P values gt 0 05 indicate no significant difference The results of the Friedman test only tell that there are differences somewhere between the related groups for question 2 and 4 see figures 28 and 30 To examine where the differences actually occur we did a post hoc analysis and ran the Wilcoxon signed rank tests on the different combinations for question 2 and 4 i e combinations text audio audio multimodal and text multimodal The results of the Wilcoxon signed rank tests showed that there is a significant difference between the combinations text audio amp text multimodal with the Asymp Sig 2 tailed values being Pra 0 01 amp Prm 0 00 respectively for question 2 and being Pra 0 013 amp Pru 0 004 respectively for question 4 But no significant difference was found between the audio multimodal combinations for both the questions Pam 0 712 for question 2 and Pay 0 134 for question 4 A mixed factorial Analysis of Variance ANOVA was conducted as we had 2 repeated measures factors and one between groups factor We looked at the 3 two way interactions Modalities vs Length Length vs Expertise amp
54. mplex much more detailed information is needed to be conveyed In such a case just using audio may not suffice A multimodal approach i e using both text and audio may be more appropriate as the user can refer to the text instructions on the screen if he or she wasn t able to understand the audio 1 2 RESEARCH PROCESS Two controlled lab studies were conducted to evaluate audio text and audio text as output modalities for viewing multimedia content on mobile devices in a just in time learning situation where participants were asked to perform mechanical tasks based on the knowledge that they gathered via referring to the digital manual provided After running the first study we analyzed the data collected and found that for the LEGO robot assembly tasks participants felt that the textual and auditory instructions provided weren t necessary to complete the tasks Therefore in order to complete the work we devised computer hardware disassembly tasks study 2 for which the textual instructions were necessary in addition to the action diagrams to carry out the tasks successfully For both the studies at the end of each output modality condition the participants answered a brief questionnaire After the session each participant was interviewed so as to collect feedback 1 3 RESEARCH QUESTIONS With digital multimedia content consisting of step by step action diagrams being provided over mobile devices in a just in time learnin
55. n England 2002 pp 293 302 13 D A Norman The Invisible Computer Cambridge MA MIT Press 1998 14 D Clark Instructional Design Just In Time Learning Online Available http www nwlink com donclark hrd media jit html 15 M Boronowsky et al Winspect a case study for wearable computing supported inspection tasks in Proc of 5 International Symposium on Wearable Computers Zurich Switzerland 2001 pp 163 164 16 F Drews et al Development and Evaluation of Just in Time Training to Provide Cardio Pulmonary Resuscitation In Proc of Human Factors and Ergonomics Society Annual Meeting Denver USA 2003 vol 47 no 12 pp 1507 1510 17 H Pinson 2005 Indepth Learning How to Build Effective Just In Time Web based Training Online Available http www indepthl com 60 18 R C Clark and R E Mayer Applying the Modality Principle Present Words as Audio Narration Rather Than On Screen Text in E Learning and the Science of Instruction Proven Guidelines for Consumers and Designers of Multimedia Learning 3 ed San Francisco USA John Wiley amp Sons 2011 ch 6 pp 115 129 19 R E Mayer and R Moreno Aids to computer based multimedia learning in Learning and Instruction Amsterdam The Netherlands Elsevier Vol 12 Issue 1 2002 pp 107 119 20 RE Mayer et al Multimedia learning in an interactive self explaining environment What works in the d
56. nditions comparison for question 6 i e The task was easy to perform Do you strongly agree agree undecided disagree or strongly disagree Study 21 Audio text and multimodal conditions comparison for question 7 i e You would be able to perform the task without the manual Do you strongly agree agree undecided disagree or strongly disagree Study 34 35 36 37 42 42 43 45 45 46 46 47 Figure 33 Figure 34 Preference between text and audio multimodal for study 2 49 Mean values of number of attention head switches between the mobile maintenance manual and the disassembly task for experts and novices N Novice E Expert A Audio T Text B Both L Long Instruction S Short Instruction ccccece eee eeneeeeeeeeseeeneees 53 x1 ABSTRACT Paper based user manuals that provide assembly and disassembly instructions often do so with a combination of diagrams supported with textual information that clarifies how to perform the tasks Mobile devices are emerging as a multimedia platform for providing on demand training due to their portability Mobile devices have limited screen size as a result the text instructions associated with the diagrams can produce clutter and occlusion on the screen Also too much information if fed through a single sensory channel visual may result in excessive cognitive load on the working memory of the human brain thus hindering the learning p
57. ng Instruction e Creating a leg subassembly Short Instruction e Creating an arm subassembly Long Instruction e Creating a motor extension subassembly Short Instruction All six tasks were of comparable complexity This comparable complexity level was achieved by pilot testing the tasks by the principal investigator and the co investigator Each task was divided into subtasks of part identification and assembling Participants were provided with digitized manuals that instructed them how to perform the tasks These manuals had multimedia content and were provided over a mobile device The study had a 3 2 2 mixed factorial design The modality of the instructions associated with the action diagrams text voice only and audio text and instruction length short and long were the within subject factors The mechanical expertise level novice and expert of the participants was the between subject factor All the participants were provided with the same mobile device running the same operating system and performed the same tasks hence inducing control The participants performed 2 tasks under each of the three output modalities audio text and audio text i e one each for the short and long instruction length conditions 3 5 COUNTERBALANCING The sequence in which the participants performed the tasks was varied This was achieved by changing the arrangement in which they received the varying conditions of instruction length and the
58. ng how to perform a mechanical disassembly task is 15 presented to the learner textual or audio instructions elaborating the diagram has to be presented as the information can only be understood conjunctively It is thus recommended to have associated information in form of text or audio as the diagram alone is not sufficient to tell the whole story It may also enable the learners to selectively switch their attention in between the three available sources of information diagram text and audio as per their convenience and they may grasp different pieces of information from different modalities 2 9 EXPERTISE LEVEL OF THE LEARNER The ability of the learner to grasp the information presented in the instructional material also depends on the learner s experience in that domain 31 Reimann and Chi 32 explained a schema based approach They mentioned that learners who have expertise in a particular area are able to find patterns and can easily connect related information from different sources of information as they become familiar with the schematics Thus they can overcome the limitations of the limited working memory available But it has not been investigated that what would be the modality preference for the expert learner in a just in time learning scenario 2 10 THE CONCEPT OF MOTOR CONTROL PRIMARY AND SECONDARY TASKS We can define motor control as the ability of a person to perform coordinated movements using their body pa
59. ns is unpredictable Figure 2 Depicting a scenario for an aircraft maintenance mechanic For instance in an industrial setting the user could be in an awkward posture refer figure 2 He may be on a ladder or crouched lighting conditions may vary and there could be background noise as well 10 Also the user may be present in different social settings e g with a group of peers or alone The context can also change the extent of user s visual attention on the mobile device A good example would be looking for directions on Google map rendered on one s smartphone while on the go One would not like to constantly gaze at the mobile map as they need their visual attention to be on the path so that they know where they are going Luca Chittaro points out in his paper that when designing mobile interfaces for such peculiar contexts the designers should try to minimize the interactions between the mobile device and the end user 11 Figure 3 An arm worn mobile device 12 Figure 3 illustrates how an ABB technician uses an arm worn mobile device while carrying out day to day maintenance operations This allows the service technician to make use of both his hands when not interacting physically with the device as opposed to if he holds it in one of his hands 12 The context has a direct impact on the available motor and cognitive abilities that the end user can utilize to carry out a particular task Thus it is idealistic t
60. nt performing the assembly task ccc ce eeeee ene ee es The Counterbalancing Chart ss0ccdsneiernedeeenssacennnecnnnsesaninesueesex Audio text and multimodal conditions comparison for question 1 i e The interaction with the mobile maintenance manual was easy Do you Strongly agree agree undecided disagree or strongly disagree Sudy heee E eer eer E rn erieeseeie Audio text and multimodal conditions comparison for question 2 i e Your attention was more focused on the manual than the real world task given to you Do you strongly agree agree undecided disagree or strongly disagree Study 1 E 18 Audio text and multimodal conditions comparison for question 3 i e processing the short instruction was difficult Do you strongly agree agree undecided disagree or strongly disagree Study 1 Audio text and multimodal conditions comparison for question 4 i e Processing the long instruction was difficult Do you strongly agree agree undecided disagree or strongly disagree Study 1 ix 10 11 12 14 15 22 23 24 26 26 27 29 32 33 33 34 Figure 20 Figure 21 Figure 22 Figure 23 Figure 24 Figure 25 Figure 26 Figure 27 Figure 28 Figure 29 Figure 30 Figure 31 Figure 32 Audio text and multimodal conditions comparison for question 6 i e The task was easy to perform Do you strongly agree agree undecided disagree or strongly disag
61. o design multimodal interfaces for mobile devices so as to reduce the cognitive work load and the attention and distraction issues Multimodal interfaces go beyond the conventional way of providing input and receiving output from a computer system Interactions could be touch based gesture based voice based etc Thus a voice based input mechanism may reduce the use of hands to command the system and a voice based output mechanism may considerably reduce the amount of visual attention needed on the interface Figure 4 Inspectors running a series of checks on an aircraft cockpit using a tablet device 2 3 DESKTOP VS MOBILE Apart from the fact that mobile devices have limited processing power and less available screen real estate there is a significant difference between how a user interacts with a mobile device as compared to the desktop systems For instance tablets and smart phones do not have physical mouse and keyboard instead input is usually provided via touch sensitive screen and an on screen keyboard The development of multimedia content and multimodal interfaces can help overcome the hardware limitations that mobile devices have Also combining different modalities can be beneficial when designing mobile interfaces for certain contexts For example if a user is at a meeting or at a conference message notifications received on a smart phone can be conveyed via a haptic stimuli vibration alert instead of a beeping sound which
62. ob The technicians refer to the manuals as they carry out the repair procedure 15 Researchers have studied the effect of JIT Just in time training in other situations as well Drews et al 16 used JIT graphical training in an emergency situation where inexperienced participants had to perform cardiopulmonary resuscitation CPR They found out that JIT is more efficient as compared to conventional paper based classroom 11 training Also in his work Pinson 17 mentioned that JIT is most effective in situations where the task description is clearly defined JIT training delivered using mobile platforms and services may also save time and money as it eliminates the need of human instructors and other resources e g classroom teaching material which is a must in an in class training methodology Visual Channel Auditory Channel Figure 6 Cognitive channels Visual and Auditory 2 5 COGNITIVE LOAD AND THE MODALITY OF AUDIO Cognitive load is defined as the measure of the amount of information that has to be processed by the working memory of the human brain during learning activities Clark and Mayer discussed in their book that learning is increased whenever audio is used instead text when presenting the instructional information related to a particular graphical representation 18 Thus narrating the instructions divides the cognitive load into two separate cognitive channels i e diagram through the vis
63. ooking at the screen I guess I saved time as I tried not to read the whole instruction and listen instead Participant E3 mentioned I could easily figure out which part goes where by just hearing and without looking at the screen I completely discarded text as I hate to read lengthy instructions N12 mentioned Diagram and text were boring audio made the process alive When asked How useful the audio was when presented with the diagrams participant E2 replied Audio helped me as the information that I wasn t able to perceive from the diagram I was able to understand by listening to the audio like from which side to put the axle into the motor Participant E4 stated Not only the audio helped me to focus on the mechanical task it also helped me to gauge how quickly I have to move It gave me a sense of when the instruction is going to move to the next step thus helped me to decide when to pause or adjust my pace at which I am performing the task When asked Were you confused or found it difficult to switch your attention between the diagrams audio and text participant E3 said I could process audio and diagrams at the same time but not all three of them together Participant E12 stated For me it was easy to go back and forth between the diagrams text and audio as I was not devoting my full attention instead I was selectively looking for the information from the 3 sources in the easiest way
64. oth lines between short and long instruction lengths indicate a main effect of the instruction length Also there were no significant interactions found between modalities amp expertise Pme 0 679 and modalities amp instruction length Pwr 0 803 Estimated Marginal Means of MEASURE_1 6 50 Expertise Expert Novice 6 00 5 50 4 50 4 00 ee 3 50 Estimated Marginal Means 2 Short Length Long Figure 22 Instruction Length Expertise Most of the participants agreed that the interaction with the maintenance manual was easy enough regardless of the modality output condition Interestingly all the and expert participants followed the approach where they were learning the task as they were performing Nobody followed the approach of actually watching the whole video and then trying to assemble the subassemblies We also asked the participants to rank the modalities based on their main focus for the three different mobile manual interfaces For the diagram with text and the diagram with audio interfaces every participant gave the 1 36 priority to the diagram For the multimodal interface the diagram was the unanimous 1 priority But when it came to prioritize between audio and textual instructions provided as a support 9 out of the 12 novice participants amp 8 of the 12 experts preferred listening to the audio see figure 23 When asked why in the interview the participants replied that it
65. re location under confidentiality in accordance to University policy for 5 years post publication In the event that you have any difficulties with or wish to voice concern about any aspect of your participation in this study you may contact Catherine Connors Director Office of Research Ethics Administration at Dalhousie University s Office of Human Research Ethics for assistance phone 902 494 1462 email Catherine connors dal ca T have read the explanation about this study I have been given the opportunity to discuss it and my questions have been answered to my satisfaction I hereby consent to take part in the study However I understand that my participation is voluntary and that I am free to withdraw from the study at any time Participant Researcher Name Name Signature Signature Date Date I understand and consent that my participation in the experiments will be video recorded for the purpose of analysis I understand that this is a condition of participation in the study and I understand that this video record may be used in publication or presentation of results 80 Participant Researcher Name Name Signature Signature Date Date Please select one of the options below T agree to let you directly quote any comments or statements made in any written reports without viewing the quotes prior to their use and I understand that the anonymity of t
66. ree Study 1 Audio text and multimodal conditions comparison for question 7 i e You would be able to perform the task without the manual Do you strongly agree agree undecided disagree or strongly disagree Study KEEN Instruction Length REENEN NENNEN REENEN e Preference between text and audio multimodal 000e0e Memory module removal action diagram with long textual instruction Wireless card removal action diagrams with short textual instruction A participant performing the laptop disassembly ask Audio text and multimodal conditions comparison for question i e The interaction with the mobile maintenance manual was easy Do you Strongly agree agree undecided disagree or strongly disagree SIVGY EE Audio text and multimodal conditions comparison for question 2 i e Your attention was more focused on the manual than the real world task given to you Do you strongly agree agree undecided disagree or strongly disagree Study 2 EE 18 Audio text and multimodal conditions comparison for question 3 i e processing the short instruction was difficult Do you strongly agree agree undecided disagree or strongly disagree Study 2 Audio text and multimodal conditions comparison for question 4 i e Processing the long instruction was difficult Do you strongly agree agree undecided disagree or strongly disagree Study 2 Audio text and multimodal co
67. ring Complex Systems 62 37 L R Novick and D L Morse Folding a fish making a mushroom the role of diagrams in executing assembly procedures in Memory amp Cognition New York USA Springer Verlag Vol 28 Issue 7 2000 pp 1242 1256 38 A Sharma and K Hawkey 2012 Understanding the Contextual Role that Modalities Play in On Demand Training for Aircraft Mechanic Online Available http ankur sharma weebly com uploads 7 7 3 2 7732735 anderstanding the contextual role that m odalities_play_in on demand training for_aircraft_mechanics pdf 63 APPENDIX A EMAIL RECRUITMENT NOTICE We are recruiting participants to take part in a research study evaluating audio text and audio text as output modalities for viewing training manuals on mobile devices in on demand learning scenario We are looking for Dalhousie University students faculty or staff members Some previous experience with mobile devices like smart phones tablets is required Both expert and novice people in performing Lego robot assembly tasks or other mechanical installation of parts etc jobs are welcome We are looking for people who are not colour blind have good hearing abilities and have good English comprehension skills The study will be conducted in The Mona Campbell Building at Dalhousie University and will take about 60 minutes to complete You will first meet with a researcher to go over an orientation session to get the study det
68. rocess In this work two user studies were conducted to investigate the tradeoffs of using text voice and a combination of both modalities on the learning experience in a just in time mobile learning scenario In such a scenario end users are managing two very visual tasks at the same time i e the primary task of carrying out the assembly disassembly job and the secondary task of learning how to perform the task xii LIST OF ABBREVIATIONS USED ANOVA Analysis of Variance CBT Computer Based Training CLT Cognitive Load Theory CPR Cardiopulmonary Resuscitation GPS Global Positioning System GV Graphics and Visualisation HDD Hard Disk Drive HMD Head Mounted Display JIT Just In Time REB Research Ethics Board xiii ACKNOWLEDGEMENTS I would like to express my gratitude towards my supervisor Dr Kirstie Hawkey for helping me in planning and executing my research work The supervision and support she provided me throughout this entire project enabled me to develop a deep understanding of the project and academic research in general I would also like to thank Dr Derek Reilly for his help and support with this project I would like to thank my parents for their unending love and support Last but not the least I would like to thank The Boeing Company for providing the financial support to carry out this work under the Mobile Graphics research project at the Faculty of Computer Science Dalhousie University xiv CHAPTER1
69. rtable mobile devices for instructional purposes may even be more 56 suitable in precarious industrial settings Our study posed no safety risks on the participants as they were sitting comfortably in a chair and weren t handling any dangerous tools It would be really interesting to study how the learning is affected when they have the thought of their personal safety at the back of their mind 5 2 4 WORKPLACE VS HOME SETTING In a workplace setting the maintenance personnel have professional responsibilities and have to put the interests of company they work for ahead of their own interests They are on deadlines to finish up a job Whereas at home they can be more relaxed and can go at their own pace It would be worth including this factor and the effects on learning and efficiency could be compared in these two different settings in the future studies 5 2 5 ENHANCEMENT OF THE USER INTERFACE In the future we think that interface design of the mobile manual could be further refined For this research we restricted the design just to a basic digitized version of the user guides which was sufficient enough to evaluate the effect of modalities on the learning During the interview some of the participants provided some suggestions when asked about what changes could be made to the interface to better support the learning and implementation scenario For instance N8 mentioned it would be amazing to have an interface where I can
70. rts It also requires the coordination of a number of sensory inputs e g visual and touch For instance if someone wants to perform a simple task of pausing a video running on a tablet device he or she has to first see and identify which area to press then sensing the movement of one s own hand movement towards the screen feeling one s finger to touch the desired area on screen and finally sensing that one has actually tapped the screen Thus motor skills of a person come into play while performing even the easiest mechanical task In a just in time learning and implementation context the learner has to perform two tasks the primary task of performing the mechanical job which requires motor skills and the secondary task of learning how to do the job using the instructional material i e the 16 manual Both the tasks require a fair amount of visual attention The modality of voice may allow the learner to focus on the assigned mechanical task in the real world as they learn about the details of how to perform the job as there is no need to constantly gaze at the screen and try to comprehend from the textual information 5 In a recent field experiment carried out by Jensen et al 33 it was found that the number of eye glances at the GPS visual display decreased with the introduction of voice output as the medium for delivering navigation instructions However it is not clear whether this finding will generalize to just in time
71. s administered exactly as the first study Figure 24 and 25 represent the long and short textual instructions respectively 41 Push out the release latches on both sides of the DIMM socket to release the DIMM module Figure 24 Memory module removal action diagram with long textual instruction Disconnect the antenna cables from the board Figure 25 Wireless card removal action diagram with short textual instruction 42 Figure 26 A participant performing the laptop disassembly task 4 2 RESULTS Similar to the previous study s results section the results in this section are also divided in two sections time data amp feedback from the post task questionnaire and then the next section covers the responses from the interview 4 2 1 TIME DATA amp POST TASK QUESTIONNAIRE FEEDBACK Similar to study one we had 3 independent variables that were varied during the second controlled laboratory study Modality with three levels level 1 text level 2 audio and level 3 audio text i e both Instruction length with two levels level 1 short level 2 long Expertise with two levels level 1 novice level 2 expert We measured the time taken dependent variable by the participants to complete the tasks as we varied the above mentioned variables Table 5 below presents the mean time values 43 Table 5 Time data for task completion study 2 Descriptive Statistics Le Prenz _Mean__ sid Dev
72. s and give consent to do the study You will then complete a set of Lego robots assembling tasks with guidance through user manuals on a mobile device You will be answering questions about your experience after the tasks and at the end of the session Compensation is a 15 for participation in the study Do I have to sign a consent form Yes in order to participate in this study you will have to sign an informed consent form as it is part of a research project Who can I contact for more information You can contact Ankur Sharma at ankur cs dal ca if you have any questions 65 APPENDIX C INFORMED CONSENT Evaluating audio text and audio text as output modalities for viewing maintenance Manuals on mobile devices in an industrial setting on demand learning Principal Investigator Ankur Sharma MCS Thesis Student Faculty of Computer Science ankur cs dal ca Co Investigators Dr Kirstie Hawkey Faculty of Computer Science hawkey cs dal ca Sukhveer Dhillon MCS Thesis Student Faculty of Computer Science sdhillon cs dal ca Contact Person Ankur Sharma MCS Thesis Student Faculty of Computer Science We invite you to take part in a research study being conducted by Ankur Sharma and Dr Kirstie Hawkey at Dalhousie University Your participation in this study is voluntary and you may withdraw from it at any time Your academic or employment performance evaluation will not be affected by whether or not you par
73. s nor extensive extracts from it may be printed or otherwise reproduced without the author s written permission The author attests that permission has been obtained for the use of any copyrighted material appearing in the thesis other than the brief excerpts requiring only proper acknowledgement in scholarly writing and that all such use is clearly acknowledged Signature of Author iii DEDICATION PAGE To my family and friends for their support TABLE OF CONTENTS LIS PGE EE LIST OF FIGURES ege TS RNAEEENER ENNER E REENEN EEN SNE elen EE ee LIST OF ABBREVIATIONS USED E ACKNOWLEDGEMENTS eebe deeg see r nEn E E E EI TEGE EEEN CHAPTER 1 INTRODUCTION eur deeseet ung gege tege e deeN ee R dng Ke ENNER eg H EE ees 1 2 RESEARCH PROCESS Ee EE ege LA RESEARCH QUESTIONS jusscssssstcsuvsesvaxcteenirieessseersdeesaseveeseteeeas 14 RESEARCH CHALLENGES u sstes cwncacseneiesersseveneirawncadtontissaietater 1 5 CONTRIBUTION sisccosssasansssnctestansscansiosanmesnvssie sivsahanssssneiesuaaviansinoaaemavavan sige 16 AES TS EN IN WE CHAPTER 2 RELATED WORK ssscssusccsssssescacsvescssacnssnassavstuss casereressedssensrssnsasevsrenss 2 1 MULTIMEDIA INSTRUCTIONAL DESIGN ssissceasocessassivunsaarcsnseasnavdtacsiunns 2 2 MOBILE LEARNING IT IS DIFFERENT FROM CLASSROOM 2 3 DESKTOP VS MOBILE geogr eegee uge ssenivinsevncuanyasssnnsbeastaosadsavan sive ens 2 4 JUST IN TIME LEARNING E 2 5 COGNITIVE LOAD AND THE MODALITY OF AUDIO 2 6 WORKIN
74. s study is voluntary and you may withdraw from it at any time Your academic or employment performance evaluation will not be affected by whether or not you participate There is no risk involved in participating in this study beyond that inherent in working in an office environment Participating in the study might not benefit you but we might learn things that will benefit others You should discuss any questions you have about this study with Ankur Sharma The study will take about 60 minutes The facilitator will describe how to use the manuals provided over mobile device using touch interaction and then you ll have to perform some assembling tasks using Lego robot kits The researcher will observe and take notes while you ll be performing the tasks also video audio recordings will be made You will be compensated for participating in the study You can withdraw from the study at any time without consequence A researcher is always available 79 over the study period by email or to meet in person to answer any questions you may have or address any problems that you may experience with the tasks All personal and identifying data will be kept confidential Anonymity of textual data will be preserved by using pseudonyms All data collected in the logs video audio recording and questionnaires will use pseudonyms e g an ID number to ensure your confidentiality The informed consent form and all research data will be kept in a secu
75. s to 21 the structural diagrams because they represented the steps required to attach every part in a detailed manner Figure 9 and 10 represent structural and action diagrams respectively Figure 9 Structural diagram of a LEGO mindstorm robot 22 Figure 10 Action diagram assembly instructions for LEGO mindstorm 3 3 3 SEQUENCE OF DIAGRAMS OVER A SINGLE DIAGRAM WITH INFORMATION OVERFLOW According to Novick and Morse 37 people are able to comprehend more when assembly instructions are presented spanning multiple diagrams rather than a single diagram amalgamating all the steps to be carried out They also mentioned that if the assembly is made up of a lot of small but significant parts each diagram should only show how to assemble or disassemble one important part in a single step The Insignificant parts on the other hand can be shown in a combination For instance in case of a LEGO Mindstorm kit numerous connection pegs and bushings could be considered insignificant and parts such as motors and beams are highly significant 23 3 3 4 PROVIDING SPATIAL CONTEXT IN EACH DIAGRAM Agrawala et al 35 also discussed the importance of providing spatial context in each instructional diagram They recommended that the parts installed or disassembled in a step should be visible in the next step to follow so that the learner has a context to refer to and thus can better sense the orientation and geometry of the subassembl
76. t tested to ensure that it was clear enough The associated textual instructions were placed like subtitles in a movie thus avoiding on screen clutter and any occlusion The short textual instructions spanned only a single line refer figure 12 whereas the long textual instructions spanned two or more lines on the screen refer figure 13 Similarly the associated audio narrations which were the exactly the same as textual instructions delivered by voice were short approx 5 7 sec and long gt 7 sec respectively The mobile manual was deployed on a Motorola Xoom Tablet with a screen resolution of 1280 x 720 pixels It was made sure that the video fits the frame of the tablet The manual was played back using the Mobo Video Player for Android There was a delay created before the video proceeds to the next step in the instructional sequence so that the learner can have additional time to recover if lagging behind with the mechanical task The delay time was pilot tested to make sure that it was not excessive or so short that it affected the learner s performance 3 3 6 INTERACTION WITH THE MOBILE MANUAL The mobile manual had a touch user interface A single tap on the screen would make the control panel appear and disappear on the bottom of the screen A double tap on the screen would play or pause the video Also the seek bar on the control panel could be used to go back or forth in the video This could be achieved by simply tapping
77. the learning material explained how an electric motor works The animation was presented in two forms one with on screen instructions and the other without the on screen instructions and had audio instructions instead The group with the animation audio combination performed better on a test given after the animations were played as compared to the group which had animation text combination Moreover these animations were controllable by the students i e they were able to stop and start it at any time as per their convenience Also the animations were segmented in small sections and each section explained a specific function of the electric motor Mayer et al 20 thus also concluded that students were able to learn even better when they had control over the animation These studies 19 20 only concluded that a person s problem solving capacity could be enhanced by the introduction of audio But it has never been evaluated that whether the replacement textual instructions with audio could be helpful for just in time training where the learner while learning from an instructional set also performs a mechanical task Thus makes use of the acquired knowledge in practice 13 2 6 WORKING MEMORY The instructional material may get more difficult to comprehend if the material has very specific instructions For instance the use of technical terms in instructional manuals or the use of mathematical formulas can increase the cognitive
78. thin a maintenance manual can result in clutter and occlusion Furthermore a maintenance technician performing a task on site has to look at the manuals learn from it and then implement the gained knowledge to accomplish the task This division of visual attention between the screen and task at hand can affect their efficiency These issues led us to consider the role of audio in training materials Audio as an output modality has been an integral part of computer based training Clark and Mayer 4 state that using audio rather than on screen text to convey associated instructions with graphics can lead to an increased ability of users to grasp the understanding This is because the cognitive load of the user gets divided into two channels i e visual and auditory This work was started considering the tradeoffs of using the modalities of text and audio for delivering instructions to accompany the instructional diagrams Reading text requires continuous visual attention of the person reading it but with audio instructions one doesn t have to look at screen Thus the modality of audio may allow the mechanics to focus on the assigned task in the real world as they learn about the details of how to perform the job as there is no need to gaze at the screen continuously and try to comprehend 5 That being said processing lengthy audio instructions could be difficult as it requires cognitive effort 4 If the real world task to be accomplished is co
79. ticipate There is no risk involved in participating in this study beyond that inherent in working in an office environment Participating in the study might not benefit you but we might learn things that will benefit others You should discuss any questions you have about this study with Ankur Sharma The study will take about 60 minutes The facilitator will describe how to use the manuals provided over mobile device using touch interaction and then you ll have to perform some assembling tasks using Lego robot kits The researcher will observe and take notes while you ll be performing the tasks also video audio recordings will be made You will be compensated for participating in the study You can withdraw from the study at any time without consequence A researcher is always available over the 66 study period by email or to meet in person to answer any questions you may have or address any problems that you may experience with the tasks All personal and identifying data will be kept confidential Anonymity of textual data will be preserved by using pseudonyms All data collected in the logs video audio recording and questionnaires will use pseudonyms eg an ID number to ensure your confidentiality The informed consent form and all research data will be kept in a secure location under confidentiality in accordance to University policy for 5 years post publication In the event that you have any difficulties with or wish to
80. ual and audio through the auditory refer figure 6 Clark and Mayer 18 suggested this recommendation as they described that learners may overload their visual channel while processing both the diagram and the associated text Also they introduced the concept of memory support via on screen text They specified that if the instruction is complex then the learner might forget it if it is only presented via audio as the audio once narrated is not present 12 anymore Whereas text could stay on screen while the visual is there and hence could be referred to again Previous studies have compared the effects of textual instructions spoken narration and a combination of spoken narration with text in a computer based multimedia learning environment Moreno and Mayer 19 conducted a study where the participants were presented with an animation illustrating the process of lightning formation The pictures were complimented with on screen text in one case and with concurrent narration for the second case Then the participants were asked questions about the process The participants in the concurrent narration group provided twice as many correct solutions as compared to the participants in the on screen text group Thus they concluded that people learn more when instructions explaining pictures are presented as audio rather than text In another study Mayer et al 20 provided the students an interactive experience The animations used in
81. ual interfaces For the diagram with associated text and diagram with associated audio interfaces every participant indicated that diagram was their 1 priority For the multimodal interface diagram was the unanimous 1 priority When it came to prioritize between audio and textual instructions provided as a support 10 out of the 12 novice participants amp 9 of the 12 experts preferred listening to the audio Novice Expert Figure 33 Preference between text and audio multimodal for study 2 4 2 2 RESPONSES COLLECTED THROUGH THE INTERVIEW This time around when we asked the participants in the interview if they would have been able to carry out the tasks by just looking at the diagrams with no help from the audio or textual instructions 4 out of the 12 participants in the expert group and 3 out of the 12 participants in the novice group agreed with this statement Although the action diagrams were their top priority the participants felt that they also needed explicit 49 instructions so that there was no room for confusion especially for the orientation and spatial direction related instructions Below we mention some of the participant views when we asked How useful the audio was when presented with the diagrams when the text was there already Verbal is easier than reading to read the text I had to go switch back and forth said NS E2 had similar views and said I don t have to lock my eyes on
82. uter hardware repair or any other mechanical repair jobs e experts as well as those who didn t Ge novices Thus we modified our recruitment criteria so that we can 40 recruit the target population as per the new tasks see Appendix I and J For this study we recruited 24 participants 15 were male and 9 were females The age range of the participants was 21 to 54 see table 4 Table 4 Participant s Demographics for study 2 Experts Novices Total 12 12 Male 8 7 Female 4 5 Average Age 29 35 The study design remained precisely the same Also the data collection instruments i e the post task questionnaires and the interview questions remained the same Although we did have to change the consent forms to reflect the new inclusion criteria see APPENDIX K The amendment proposal was approved by the REB see APPENDIX H We piloted the computer hardware disassembly tasks with 5 participants 2 males and 3 females to make sure that the textual and auditory instructions were needed in addition to the action diagrams to perform the tasks before running the final user study with 24 participants The pilot study participants found additional instructions in form of text or audio necessary as the action diagrams did not include specific instructions involved such as how many screws to remove in which direction to move the HDD interface connector etc Our research questions remained the same and the study wa
83. voice concern about any aspect of your participation in this study you may contact Catherine Connors Director Office of Research Ethics Administration at Dalhousie University s Office of Human Research Ethics for assistance phone 902 494 1462 email Catherine connors dal ca T have read the explanation about this study I have been given the opportunity to discuss it and my questions have been answered to my satisfaction I hereby consent to take part in the study However I understand that my participation is voluntary and that I am free to withdraw from the study at any time Participant Researcher Name Name Signature Signature Date Date T understand and consent that my participation in the experiments will be video recorded for the purpose of analysis I understand that this is a condition of participation in the study and I understand that this video record may be used in publication or presentation of results 67 Participant Researcher Name Name Signature Signature Date Date Please select one of the options below T agree to let you directly quote any comments or statements made in any written reports without viewing the quotes prior to their use and I understand that the anonymity of textual data will be preserved by using pseudonyms Participant Researcher Name Name Signature Signature Date Date Or I want to
84. would cause a disturbance 11 The visual output is limited when displayed over mobile devices This is attributed to the small screen size 13 Therefore audio could be used as a second modality to convey the information without taking anymore space on the screen 10 Figure 5 Different screen form factors desktop tablet and smartphone 2 4 JUST IN TIME LEARNING Just in Time learning JIT as the name suggests is providing a quick fix to a problem when it is needed 14 Elaborating further you learn something when you are in a condition where you learn how to do a task and then implement it immediately It could be achieved via different sources For instance a mechanic on the factory floor could learn how to perform a particular task by reading an instruction manual while on the job or can learn with the help from his or her colleague which is called on the job training This enables the learner to get trained when they desire and not months before they are actually going to perform It may reduce the need for conducting refresher training sessions as the learner has just acquired the knowledge to accomplish the task and is highly unlikely to forget the perceived information This Just in time learning can be related to the case of an aircraft maintenance routine The maintenance personnel look for defects to be fixed in the cabin logbook and then gather the relevant tools and printed manuals and head to the hangar to perform the j
85. y condition after the task completion The participants rated their preference for different scenarios At the end of the session the participants presented their views and answered general questions about their experience in interacting with the mobile maintenance manual and carrying out the mechanical task in an interview with the principal investigator The results are divided in two sections time data amp participant s feedback from the post task questionnaire Section 3 7 1 and the responses from the interview Section 3 7 2 30 3 7 1 TIME DATA amp POST TASK QUESTIONNAIRE FEEDBACK For the study we had 3 independent variables that were varied during the controlled laboratory study Modality with three levels level 1 text level 2 audio and level 3 audio text i e both Instruction length with two levels level 1 short level 2 long Expertise with two levels level 1 novice level 2 expert We measured the time taken dependent variable by the participants to complete the tasks as we varied the above mentioned variables Table 3 below presents the mean time values Table 3 Time data for task completion study 1 Text Short Text Long Audio Short Audio Long Multimodal Short Multimodal Long Descriptive Statistics epertise Mean Std Deviation Expert Novice Total Expert Novice Total Expert Novice Total 1 00871 12 1 96461 12 2 08274 24 1 57372 12 2 291
86. y to be produced 3 3 5 INTERFACE DESIGN AND IMPLEMENTATION The mobile reference manuals were generated using Adobe Premiere Pro CS 5 5 video editing software as shown in Figure 11 Be Wert the cross anle into the center hole of the orange estate reegen anerian amp then side the bushing aver the egen ale 2 i X ij Figure 11 The Manual developed using Adobe Premiere Pro CS 5 5 To support the experimental design three layouts were implemented 1 Action diagrams with textual instructions 2 Action diagrams with audio instructions and 3 Action diagrams with textual as well as audio instructions For this study the Lego robot assembly tasks were divided into subtasks of part identification and assembling Thus the diagrams in the digital manual first showed the parts to be used and then the assembling instructions in a sequential manner The colored 24 diagrams used in the mobile manual were derived from the paper based user guide included in the LEGO Mindstorm kit A simple and traditional approach of recording and then embedding the audio with the step by step instructional video was followed We ensured that there was no background noise or disturbance in the voice recordings so that the prospective participants won t have any difficulties understanding it Audio was recorded using Sony s Digital Voice Recorder Model number ICDAX412B in the MP3 file format The recorded audio was pilo

MCSc- CSCI- June- 2013 - DalSpace Home

Contents

Download Pdf Manuals

Related Search

Related Contents