Just-in-time information presentation: Improving learning a
Contents
1. ProcD ProcD M SD M SD M SD M SD Practice performance maximum 30 16 00 4 13 18 02 5 54 15 07 5 29 17 28 3 76 Mental effort during practice 1 9 4 44 1 16 4 13 1 46 4 88 1 33 4 80 89 Mental effort for subject matter 1 9 4 29 1 49 4 08 2 25 393 191 5 56 2 06 2 Dec declarative information Proc procedural information B before and D during p lt 05 for timing of declarative information on practice performance and for the interaction of declarative information x procedural information on mental effort during practice ARTICLE IN PRESS L Kester et al Contemporary Educational Psychology xxx 2005 xxx xxx 11 3 1 2 Mental effort Perceived mental effort during practice and for understanding the subject matter were considered separately ANOVA yielded an interaction between the timing of declarative and procedural information on mental effort during practice F 1 85 4 26 MSE 6 36 p lt 05 n 05 see Fig 3 If declarative information was presented before practice participants reported expending less mental effort to solve the tasks when procedural information was presented during practice M 4 4 SD 1 16 than when procedural information was also presented before practice M 4 9 SD 1 33 If declarative information was presented during prac tice the participants reported expending less mental effort to solve the tasks when procedural information was presented before practice M 4 1 SD 1 46 th2. s respectively 2 2 5 Mental effort measurement Mental effort was used as an index of cognitive load It refers to the amount of working memory capacity allocated to problem solving Mental effort was measured during both practice and the test with a 9 point rating scale Paas 1992 Paas van ARTICLE IN PRESS 8 L Kester et al Contemporary Educational Psychology xxx 2005 xxx xxx Merri nboer amp Adam 1994 The mental effort measures ranged from very very low mental effort 1 to very very high mental effort 9 The rating scale was admin istered electronically during both practice and the test directly after each trouble shooting task After each task participants were asked How much mental effort did it require to find a solution for the problem s in the preceding circuit More over after the 10 practice tasks a separate mental effort measurement was adminis tered for the subject matter The participants were asked How much mental effort did it require to understand all subject matter The internal consistency of the men tal effort measures was 83 Cronbach s for the 10 practice tasks 69 for the five equivalent test tasks and 68 for the five transfer test tasks 2 2 6 Learning and performance efficiency The Paas and van Merri nboer procedure 1993 see also Paas et al 2003 was used to calculate efficiency First the performance measures and the mental effort measures for each partici
3. A van Merri nboer J J G amp Baumer A 2001 Just in time information presentation and the acquisition of complex cognitive skills Computers in Human Behavior 17 373 391 Merrill M D 2002 First principles of instruction Educational Technology Research and Development 50 3 43 59 Miller G A 1956 The magical number seven plus or minus two Some limits on our capacity for processing information Psychological Review 63 81 97 Paas F 1992 Training strategies for attaining transfer of problem solving skill in statistics A cognitive load approach Journal of Educational Psychology 84 429 434 Paas F amp van Merri nboer J J G 1993 The efficiency of instructional conditions An approach to combine mental effort and performance measures Human Factors 35 737 743 Paas F Tuovinen J E Tabbers H K amp van Gerven P W M 2003 Cognitive load measurement as a means to advance cognitive load theory Educational Psychologist 38 1 63 71 Paas F van Merri nboer J J G amp Adam J J 1994 Measurement of cognitive load in instructional research Perceptual and Motor Skills 79 419 430 Reigeluth C M Ed 1983 Instructional design theories and models An overview of their current status Hillsdale NJ Lawrence Erlbaum Reigeluth C M Ed 1999 Instructional design theories and models A new paradigm of instructional theory Mahwah NJ Lawrence Erlbaum Spir
4. have been just in time from a theoretical point of view the learners might not have perceived it as being presented just in time for them This might have interfered with the learning processes involved in cognitive skill acquisition In addition the learners might not perceive the declarative and procedural infor mation as being really different The declarative information contained mostly linear prose in which some analogies and examples were added It is possible that the lin earity of the declarative information made it possible for the learners to serially pro cess this information which is associated with low element interactivity and thus low intrinsic load In this way the declarative information resembles the procedural infor mation which contained successive sentences which could also be processed serially In other words the declarative information may have had a lower intrinsic load than expected The unexpected success of the declarative during procedural before for mat could be explained based on this alternative explanation When the declarative and procedural information both have a low intrinsic load both formats could be rephrased as a low intrinsic load information before low intrinsic load information during format The only difference is that the procedural information is temporally and spatially integrated in the electrical circuits used during practice while the declar ative information is only temporally i
5. in the piece by piece formats who were apparently better able to form more adequate specific schemata than the learners in the simultaneous formats Participants in all formats applied what they had learned equally well on familiar test tasks while participants in the piece by piece formats were better able to solve the transfer test tasks In other words participants in all formats were equally able to compile specific schemata but the participants in the piece by piece formats were able to construct more adequate specific schemata Since the practice tasks were ARTICLE IN PRESS L Kester et al Contemporary Educational Psychology xxx 2005 xxx xxx 17 solution oriented 1 e participants had to give a problem description a cause and a solution for the problem all participants were forced to apply the procedural infor mation during practice to come up with an answer The assignment itself thus directed the attention of the participants to the procedural information Additional ly since the practice tasks did not cause cognitive overload 1 e the invested mental effort during practice never exceeded the rating not low not high knowledge com pilation and the forming of specific schemata was possible in all formats This solu tion orientedness of the participants might mean that they did not pay attention to the declarative information unless their attention was specifically directed to it by presenting this information separ
6. reduced when mutually referring information resources are integrated rather than when they are separated in time or space Eliminating split attention in instructional material en ables learners to allocate all available working memory capacity to processes rele vant for learning By presenting the procedural information fully integrated within the practice tasks temporal and spatial split attention is avoided and all relevant information is active in working memory when it is applied to problem solving a necessary precondition for knowledge compilation to occur Kester Kirschner van Merri nboer amp Baumer 2001 To investigate the effectiveness and efficiency of piece by piece and just in time information presentation four information presentation formats are compared namely 1 Presenting declarative information before practice and procedural information during practice 1 e piece by piece and just in time 2 Presenting declarative information during practice and procedural information before practice 1 e piece by piece Presenting both declarative and procedural information before practice 4 Presenting both declarative and procedural information during practice 0S Effectiveness is measured by performance on two types of test tasks Equivalent test tasks are tasks that are analogous to the practice tasks and therefore have a high level of familiarity to the learner These tasks make use of the same circuit elements
7. with the topic of the lesson because they all studied relevant theory in the previous aca demic year As compensation for their participation they received a music compact disc of their own choice 2 2 Materials 2 2 1 Physics lesson Crocodile Physics 1 5 a simulation program for secondary school science classes was used to develop the physics lesson for this experiment The computer based les son contained an introduction declarative information procedural information 10 practice troubleshooting tasks and 10 test troubleshooting tasks During practice the participants had to use the declarative and procedural information given to trou bleshoot a malfunctioning electrical circuit They had to give a description of the problem diagnose the cause of the problem and present a solution to the problem The aim of the lesson was to teach the participants to solve problems related to cur rent e g too high or too low wrongly connected elements e g lamps switches and meters in series or parallel and short circuits This was tested in 10 test tasks where participants had to apply what they had learned during practice to 10 novel malfunctioning circuits No declarative or procedural information was available dur ing the test 2 2 2 Information presentation The troubleshooting practice tasks consisting of malfunctioning electrical cir cuits were accompanied by declarative and procedural information presented either before prac
8. xxx 2005 xxx xxx hypothesis that piece by piece presentation of declarative and procedural informa tion has beneficial effects on learning outcomes was confirmed These findings are particularly important for the design of problem solving instruction because more and more educational approaches stress the importance of meaningful problem solv ing with realistic learning tasks This study showed that the distribution of informa tion presentation in relation to problem solving strongly affects the success of these instructional methods Acknowledgments The authors gratefully acknowledge the assistance of Robert Bouwens Stan Wil den and their colleagues for their help in carrying out this experiment at Bernardi nus College References Anderson J R 1996 ACT A simple theory of complex cognition American Psychologist 51 355 365 Baddeley A D 1992 Working memory Science 255 556 559 Hofer B K Yu S L amp Pintrich P R 1998 Teaching college students to be self regulated learners In D H Schunk amp B J Zimmerman Eds Self regulated learning From teaching to self reflective practice pp 57 85 New York NY Guilford Publications Kester L Kirschner P A amp van Merri nboer J J G 2005 The management of cognitive load during complex cognitive skill acquisition by means of computer simulated problem solving British Journal of Educational Psychology 75 1 16 Kester L Kirschner P
9. ARTICLE IN PRESS Available online at www sciencedirect com science pincer Educational Psychology ELSEVIER Contemporary Educational Psychology xxx 2005 xxx xxx www elsevier com locate cedpsych Just in time information presentation Improving learning a troubleshooting skill Liesbeth Kester Paul A Kirschner Jeroen J G van Merri nboer Educational Technology Expertise Center Open University of the Netherlands P O Box 2960 Heerlen 6401 DL The Netherlands Abstract Troubleshooting in a practice situation requires two types of information namely for rea soning about the problem cause and for finding an adequate solution declarative information and for manipulating the environment procedural information It is hypothesized that pre senting this information piece by piece during practice 1 e presentation of declarative and procedural information separately frees up working memory and facilitates learning More over this effect is augmented when both information types are presented just in time 1 e declarative information before practice and procedural information during practice This should yield highest test performance and instructional efficiency which is defined as higher test performance combined with lower mental effort during practice Eighty five students 49 male 36 female M 15 2 years SD 59 participated in a 2 x 2 factorial experiment with the factors timing of declarative information a
10. N 0 4 0 3 0 2 0N 02 03 04 D 0 1 DecBProcB A E 28 0 2 3 DecDProcD a E 17 04 Low efficiency Mental effort Fig 5 The mean instructional efficiency scores A and the mean performance efficiency scores B drawn in a Cartesian coordinate system present procedural information before practice M 26 SD 1 22 than to also present it during practice M 17 SD 98 see Fig 5B No effects of the infor mation presentation formats were found on either the instructional efficiency or on the performance efficiency for the equivalent test tasks ARTICLE IN PRESS L Kester et al Contemporary Educational Psychology xxx 2005 xxx xxx 15 This interaction effect indicates that piece by piece presentation of declarative and procedural information leads to a higher performance efficiency than simultaneous presentation of these information types This was confirmed with an additional con trast analysis Meaningful differences were found between the declarative before procedural during format and the declarative during procedural before format in which the information was presented piece by piece and the declarative before procedural before format and the declarative during procedural during format in which the information was presented simultaneously F 1 85 4 20 MSE 4 53 p lt 05 n 05 A second contrast analysis carried out to determine whether the declarative before pr
11. an when procedural information was also presented during practice M 4 8 SD 9 Tukey s HSD test showed no significant differences between information presentation formats No differences of the information presentation formats were found on the mental effort reported for understanding the subject matter 3 2 Test tasks See Table 2 for an overview of the results for the test tasks 3 2 1 Test performance For performance on the transfer test tasks ANOVA revealed an interaction be tween timing of declarative and timing of procedural information in the expected direction F 1 85 4 68 MSE 30 5 p lt 05 y 06 see Fig 4 If declarative information was presented before practice the presentation of procedural informa tion during practice led to higher performance on the transfer test tasks M 6 90 SD 2 00 than when procedural information was also presented before practice M 5 57 SD 2 68 Conversely when declarative information was presented during practice the presentation of procedural information before practice led to Mental effort during practice N O O Procedural During OO ee 4 8 DT x 4 1 Procedural Before 4 9 4 4 Mean mental effort N wo A A Q Before _ During Declarative Fig 3 The interaction between the timing of declarative information i e before or during practice and the timing of procedural information 1 e before or during practice for mental effort during pra
12. ately from the procedural information This would explain why participants in the piece by piece formats performed equally well on the equivalent test tasks and better on the transfer test than participants in the other simultaneous formats Participants in these formats were more inclined to not only include the procedural information but also the declarative information in schema acquisition which could explain why their specific schemata were more effective and efficient than the specific schemata of participants in the other formats Future research should help strengthen the theoretical explanations for the results of this study First experiments should be carried out to explore the effects of learn er control over information presentation Although self regulated learning some times leads to superior learning outcomes Zimmerman 2002 studies show that only a few learners exhibit self regulated behaviors of their own accord Hofer Yu amp Pintrich 1998 It seems therefore wise to not only give learners control over the information presentation but also to support them in properly using this infor mation Second eye movement measurements could be used to gain more insight into the learner s attentional processes during information processing In this way just in time information presentation can be refined or optimized by re directing learners attention during problem solving Special attention should be given to for mulat
13. ctice ARTICLE IN PRESS 12 L Kester et al Contemporary Educational Psychology xxx 2005 xxx xxx Table 2 Overview of results for the test tasks Information presentation format Piece by piece Simultaneous DecB DecD DecB ProcB DecD ProcD ProcB ProcD M SD M SD M SD M SD Transfer test tasks Performance on the transfer test tasks 6 90 2 00 6 31 2 83 5 57 2 68 5 24 2 59 maximum 15 Mental effort 1 9 5 61 1 35 5 18 1 87 5 88 1 19 5 46 1 44 Instructional efficiency 32 83 34 1 20 29 1 13 33 98 Performance efficiency 21 87 26 1 22 28 1 05 17 98 Equivalent test tasks Performance on the equivalent test tasks 7 00 2 60 6 98 2 66 7 64 2 83 7 78 2 35 maximum 15 Mental effort 1 9 4 71 127 481 2 01 5 08 1 52 4 88 1 27 Instructional efficiency 03 18 14 1 29 09 1 34 02 90 Performance efficiency 03 98 08 1 35 02 1 27 11 99 Dec declarative information Proc procedural information B before and D during p lt 05 for the interaction of declarative information x procedural information on performance and performance efficiency of the transfer test p lt 01 for the interaction of declarative information x procedural information on instructional efficiency of the transfer test Transfer test task performance 7 5 7 Procedural Before noa O e 6 31 2 5 6 O O lt 5 5 Procedural S During 0 Before During Declarative Fig 4 The interaction between th
14. culated on the basis of perceived mental effort during practice and test per formance The performance efficiency 1s calculated on the basis of perceived mental effort during the test and test performance 2 3 Procedure Participants were randomly assigned to one of the four formats The computers in a computer room were configured beforehand with one of the four formats so that all formats were divided per row over the available computers The monitors were shut off and the participants blindly choose a computer to work on when they walked in Once all participants had chosen a computer and were seated they received an oral instruction which stressed that they had to work independently observe the time limit work seriously carefully study the information that precedes the practice tasks and not ask questions during the experiment It was emphasized that they were not allowed to skip any part of the answer problem description cause or solution even if they did not know the answer In the latter case they were advised to give the re sponse no answer or do not know It was made clear that all of the responses would be checked after the session to determine whether there were omissions regarding the answers on the troubleshooting tasks and the mental effort measures They were told that the aim of the experiment was to determine if it is useful to inte grate this kind of simulation software into regular education and if this is t
15. e timing of declarative information 1 e before or during practice and the timing of procedural information 1 e before or during practice for the transfer test task performance higher performance on the transfer test tasks M 6 31 SD 2 83 than when pro cedural information was also presented during practice M 5 24 SD 2 59 This interaction effect indicates that piece by piece presentation of declarative and procedural information leads to better performance on the transfer test tasks than simultaneous presentation of both information types This was confirmed with an ARTICLE IN PRESS L Kester et al Contemporary Educational Psychology xxx 2005 xxx xxx 13 additional contrast analysis Meaningful differences were found between the declar ative before procedural during format and the declarative during procedural be fore format in which the information was presented piece by piece and the declarative before procedural before format and the declarative during procedural during format in which the information was presented simultaneously F 1 85 4 68 MSE 30 50 p lt 05 n 06 Moreover a second contrast analysis was carried out to see whether the declarative before procedural during format led to better perfor mance on the transfer test tasks than the other three formats This analysis yielded no significant differences F 1 85 4 68 MSE 24 01 p 058 n 04 No sign
16. ed during practice and procedur al information before 1 e piece by piece yield a higher test performance instruc tional efficiency and performance efficiency than those formats that present both information types simultaneously either before or during practice because the piece by piece formats better manage intrinsic load Moreover it is hypothesized that the information presentation format that presents declarative information be fore practice and procedural information during practice 1 e piece by piece and just in time will yield higher test performance instructional efficiency and perfor mance efficiency than the other three formats because in this format the intrinsic load is properly managed and the extraneous load is minimized 2 Method 2 1 Participants Eighty five 10th grade students at Bernardinus College an academic high school in Heerlen The Netherlands 49 male 36 female M 15 2 years SD 59 partic ARTICLE IN PRESS 6 L Kester et al Contemporary Educational Psychology xxx 2005 xxx xxx ipated in this study All participants spoke Dutch as their first language the lan guage in which the instruction was given They voluntarily participated in a physics lesson on electrical circuits using a computer based learning environment No spe cific grade was given for this lesson All participants followed the same physics edu cation curriculum which started in ninth grade They were all equally familiar
17. efficient and indicates that the allocated working memory capacity was used for processes relevant to learning 1 e low extraneous load Low test per formance in combination with high allocation of working memory capacity during practice is less efficient and indicates that a substantial amount of working memory capacity was allocated to processes not relevant to learning 1 e high extraneous load Performance efficiency provides information on the efficiency of the test perfor mance by looking at test performance in relation to the working memory capacity used to reach this performance Performance efficiency indicates the extent to which learners were able to acquire adequate schemata during practice High test perfor mance in combination with low allocation of working memory capacity during the test is highly efficient and indicates that learners were able to form adequate schema ta during practice to help them solve the test problems Low test performance in combination with high allocation of working memory capacity during the test is less efficient and indicates that learners were not able to form adequate schemata during practice and possibly had to resort to more cognitively demanding weak method problem solving strategies The main hypothesis in this study is that the format where the declarative infor mation is presented before practice and procedural during practice along with the format where the declarative information is present
18. ented piece by piece and the declarative before procedural before format and the declarative during procedural during format in which the information was presented simultaneously F 1 85 7 92 MSE 8 7 p lt 01 n 09 A sec ond contrast analysis carried out to determine whether declarative before procedur al during led to a higher instructional efficiency than the other three formats yielded no significant results For the performance efficiency of the transfer test ANOVA also revealed an inter action between timing of declarative information and timing of procedural informa tion showing the same pattern as for instructional efficiency F 1 85 4 20 MSE 4 53 p lt 05 y 05 If declarative information was presented before prac tice it was more efficient to present procedural information during practice M 21 SD 87 than to also present it before practice M 28 SD 1 05 If declarative information was presented during practice it was more efficient to ARTICLE IN PRESS 14 L Kester et al Contemporary Educational Psychology xxx 2005 xxx xxx A Learning efficiency High efficiency a E 0 DecBProcD 0 E 32 DecDProcB 0 2 E 34 Q 0 1 C K 0 4 0 3 X 4 0 1 02 0 3 04 D 0 DecBProcB a E 29 0 2 DecDProcD 0 3 E 33 Low efficiency 0 4 Mental effort B Performance efficiency 0 4 7 High efficiency oo eae 0 3 0 2 oO DecDProcB 0 1 E 26 Z
19. er performance on the transfer test tasks but also reach this performance with a proportionally lower investment of mental effort in test problem solving It appears that learners in the piece by piece formats were better able to form adequate schemata which helped them solve the transfer test tasks With regard to the second hypothesis just in time information presentation did not result in higher instructional efficiency or perfor mance efficiency compared to the other three formats These results raise two questions First why did the transfer test scores of learners in the declarative before procedural during format not significantly differ as pre dicted from the transfer test scores of learners in the declarative during procedural before format The same is true for the mental effort instructional efficiency and performance efficiency scores A possible explanation for this is that a system con trolled approach was taken to test the hypotheses regarding just in time information presentation This means that learners received the necessary information just when it was needed to meet the task requirements based on theoretical assumptions made ARTICLE IN PRESS 16 L Kester et al Contemporary Educational Psychology xxx 2005 xxx xxx by the experimenters The learners themselves had little or no control over informa tion presentation it simply was presented on the computer screen Although the information presented might
20. he case how this should be done It was made clear that the physics lesson contained 10 prac tice tasks and 10 test tasks and that it was their job to give a description cause and solution for the problem in the malfunctioning circuit Before the participants actu ally started with the physics lesson they were walked through an example of the type of troubleshooting task they were about to receive in class with respect to the func tional aspects of the learning environment only 1 e how Crocodile worked During this example they could only ask questions about the learning environment and the procedure This example was not accompanied by declarative or procedural informa tion All efforts were made to ensure that the whole procedure was clear to all par ticipants before the actual experiment started ARTICLE IN PRESS 10 L Kester et al Contemporary Educational Psychology xxx 2005 xxx xxx All participants had 100 min available to work through the introduction and to complete the lesson In the introduction participants received information on 1 what to expect such as the number of tasks available time and how to switch a cir cuit on and off 2 how to navigate within the lesson such as left and right arrows to go back or forth in the lesson and how to jump to a practice task or a test task by clicking on different icons and 3 the rules during the experiment such as that taking notes or changing the computer s configu
21. he focus of this study is on managing intrinsic load and minimizing extraneous load in relation to the information needed to solve practice tasks and acquire trou bleshooting skills In this study two types of information are distinguished namely declarative and procedural Declarative information refers to the conceptual model of how a domain is organized see the textual information in the right half of Fig 1 for an example and enables learners to construct cognitive schemata through elaboration Reigeluth 1983 1999 The declarative information presented is gradually coupled to already existing relevant cognitive schemata in long term memory of the learner Elabora tion of declarative information yields cognitive schemata that contain domain gen eral knowledge which is particularly useful when learners have to deal with unfamiliar problem situations Such situations require interpretation of cognitive schemata that is different use of the same domain general knowledge In the study presented here reasoning about the differences between various connections in elec ARTICLE IN PRESS Go L Kester et al Contemporary Educational Psychology xxx 2005 xxx xxx odile Physics 01 Problem 1 cyp ele tele Eln Problem 1 What s wrong with this circuit Click on next to give your answer In aparallel connection the amperage is divided over the parallel branches The an electrical device is turned on the electr
22. ical circuit is premiered the A volimeter is ro losed and current starts to flow To explain how current flow regardless of the number of Se rough on electrical circuit a model is used A suitable model tral heating system of a house In a central heating system lows through a closed circuit A pump sets the water in motion es it flow Closing a top in a pipe interrupts the flow of wa In an electrical ciruit electrons flow through a closed circuit r source mokes the electrons flow through the circuit e g rators batteries A switch interrupts the circuit temporc or example to switch off a device branches The more elements connected in parallel the higher the amperage of the circuit flow through this meter N ma A power source A resistor voltage variable 90mA always 100 Ohm The difference between the positive pole and the negative pole aov The electrical current the same varioble r source is called voltage A voltmeter is used to measure flows from the Itage across elements Voltage is expressed in volt V or mil positive pole to the mv negative pole of the power source The number of electrons that flow through a wire per second is led amperage An ammeter is used to measure amperage Ampe lis expressed in ampere A or milliampere mA lamp or a resitor are examples of elements that can be found lectrical circuit The lower the resitance the easie
23. ificant differences for the information presentation formats were found on performance of the equivalent test tasks 3 2 2 Mental effort No significant differences of the information presentation formats were found on reported mental effort for either the transfer test tasks or the equivalent test tasks 3 3 Instructional and performance efficiency For the transfer test ANOVA revealed an interaction that partially showed the expected pattern for instructional efficiency between timing of declarative informa tion and timing of procedural information F 1 85 7 92 MSE 8 7 p lt 01 n 09 If declarative information was presented before practice it was more effi cient to present procedural information during practice M 32 SD 83 than to present it before practice M 29 SD 1 13 If declarative information was presented during practice it was more efficient to present procedural informa tion before practice M 34 SD 1 2 than to present it during practice M 33 SD 98 see Fig 5A This interaction effect indicates that piece by piece presentation of declarative and procedural information leads to higher instructional efficiency than simultaneous presentation This was confirmed with an additional contrast analysis Meaningful differences were found between the declarative before procedural during format and the declarative during procedural before format in which the information was pres
24. ing the assignment accompanying the practice tasks because this might be very powerful in directing attention to particular types of information Third to test the nature of the schemata 1 e specific or general formed during practice more direct measures of these schemata are needed such as in depth analysis of verbal protocols to yield more insight into learners reasoning and underlying cognitive schemata Finally it is necessary to replicate the results of this study using more complex declarative information other types of tasks e g design problems or categorization problems and tasks in other learning domains The practical implications of this study are straightforward Relevant information should be distributed in a way that helps manage cognitive load The presentation of all information relevant to solving problems both before learners start to work on the tasks and during the problem solving process is sub optimal It is better to pres ent part of the information before problem solving and part of it during problem solving To conclude this study reveals a positive difference in effectiveness and efficiency for information presentation formats presenting declarative and procedural informa tion piece by piece before or during practice above information presentation formats presenting both information types simultaneously before or during practice The ARTICLE IN PRESS 18 L Kester et al Contemporary Educational Psychology
25. ize a short circuit and be able to fix it Intrinsic load can first be managed by presenting both the declarative and the procedural information piece by piece Instead of allocating working memory capac ARTICLE IN PRESS 4 L Kester et al Contemporary Educational Psychology xxx 2005 xxx xxx ity to processing both declarative and procedural information at the same time learners can allocate the same amount of working memory capacity to the declara tive and the procedural information one piece at a time In addition intrinsic load can be managed by presenting the declarative information just in time that is before learners start solving the practice tasks Since this declarative information typically has a higher degree of element interactivity 1 e more interconnected elements than the procedural information presenting it during practice might require too much working memory capacity When declarative information is presented before prac tice all working memory capacity can be allocated to elaborating it and thus to schema acquisition To minimize extraneous load procedural information should also be presented just in time but here that means during practice task solution Extensive research has been carried out on the split attention effect for an overview see Sweller et al 1998 for a study in this domain see Kester Kirschner amp van Merri nboer 2005 This research indicates that extraneous load is significantly
26. l to facilitating the acquisition of cognitive skills in such environments Cognitive load theory distinguishes three types of cognitive load namely intrin sic extraneous and germane load These three types of load need to be optimally balanced in order to make good use of limited working memory capacity Badde ley 1992 Miller 1956 According to Sweller van Merri nboer and Paas 1998 intrinsic cognitive load is inherent to a learning task and is determined by the de gree of element interactivity in that task Extraneous cognitive load is caused by those processes a learner engages in while interacting with the instructional mate rial that are not beneficial for learning Examples of activities that induce extrane ous cognitive load are mentally integrating different sources of information e g separate information in a figure and a text or searching for relevant information in order to understand the subject matter Finally germane cognitive load is load associated with processes that are beneficial for learning Variability of learning tasks for example may stimulate learners to construct better cognitive schemata Spiro Coulson Feltovich amp Anderson 1988 Sweller et al 1998 and can be con sidered thus to being germane to learning In general a well designed learning environment should properly manage intrinsic load minimize extraneous load and optimize germane load within the boundaries of working memory capacity T
27. nd timing of procedural information both before or during practice Transfer test scores and transfer efficiency scores support the first hypothesis the second hypothesis was not supported 2005 Elsevier Inc All rights reserved Corresponding author Fax 31 45 5762802 E mail address iesbeth kester ou nl L Kester 0361 476X see front matter 2005 Elsevier Inc All rights reserved doi 10 1016 j cedpsych 2005 04 002 ARTICLE IN PRESS 2 L Kester et al Contemporary Educational Psychology xxx 2005 xxx xxx 1 Introduction Modern curricula make use of powerful e learning environments for the acquisi tion of cognitive skills Such environments contain practice problems e g simula tions and related information e g text and animation see Merrill 2002 Reigeluth 1999 Van Merri nboer amp Kirschner 2001 During problem solving in such environments learners 1 master cognitive skills that require integration of the knowledge skills and attitudes necessary for effective performance 2 learn to coordinate cognitive skills and eventually and 3 become able to transfer what is learned to their daily life or future work settings Acquiring cognitive skills while working in complex learning environments however tends to cognitively tax the learner to such an extent that skill acquisition may be hampered or frustrated Imple menting guidelines from cognitive load theory Sweller 1988 could prove beneficia
28. nt wrongly connected elements and short circuits that were designed in Crocodile Phys ics but without the accompanying information Five of the 10 test tasks were equiv alent to the practice tasks that is they contained the same elements as the practice tasks The other five tasks also contained one or two new elements namely a vari able resistor a fuse an LED a buzzer and push button switch or a motor and gears The equivalent test tasks were meant to determine whether the participants had formed specific schemata to help them solve the familiar aspects of the test task The test tasks that contained a new element the transfer test tasks were meant to determine whether the participants were able to construct specific schemata plus those general schemata that help them solve the unfamiliar aspects of the test task The participants could obtain a maximum of 15 points for the five equivalent tasks and 15 points for the five transfer tasks As was the case in practice they received one point for each correct description cause or solution To determine the interrater reli ability the total test performance scores of eight participants were determined by two raters The interrater reliability for the test performance on the equivalent tasks of the two raters was 87 Intraclass Correlation Coefficient SPSS and for the transfer tasks it was 92 The internal consistencies of the equivalent tasks and the transfer tasks were 64 and 63 Cronbach
29. ntegrated This however might have been suf ficient since the procedural information directly refers to the electrical circuit which makes spatial integration necessary while the declarative information only indirectly refers to the electrical circuit which makes temporal integration sufficient The second question is Why is there no difference between formats for the equiv alent test tasks In this respect the un familiarity of the transfer test tasks to the practice tasks has to be discussed On the continuum from near transfer to far trans fer the transfer tasks used in this study were more at the near transfer end of this continuum than the far transfer end Although in general the participants scored higher on the equivalent test tasks than on the transfer test tasks see Table 2 which seems to indicate that the transfer test tasks were more difficult for them the transfer tasks were still very similar to the equivalent test tasks and the practice tasks This has consequences for the underlying assumptions It was assumed that the more unfamiliar the test task was the more learners had to rely on their general schemata of the problem domain acquired through elaboration of the declarative information Since the transfer test tasks were rather similar to the practice tasks the specific schemata might have been sufficient to solve these tasks However the transfer test performance and performance efficiency were higher for the learners
30. o R J Coulson R L Feltovich P J amp Anderson D K 1988 Cognitive flexibility theory Advanced knowledge acquisition in ill structured domains In V Patel Ed Proceedings of the 10th Annual Conference of the Cognitive Science Society Hillsdale NJ Lawrence Erlbaum Sweller J 1988 Cognitive load during problem solving Effects on learning Cognitive Science 12 257 285 ARTICLE IN PRESS L Kester et al Contemporary Educational Psychology xxx 2005 xxx xxx 19 Sweller J van Merri nboer J J G amp Paas F 1998 Cognitive architecture and instructional design Educational Psychology Review 10 251 296 Tuovinen J G amp Paas F 2004 Exploring multidimensional approaches to the efficiency of instructional conditions Instructional Science 32 135 152 Van Merri nboer J J G amp Kirschner P A 2001 Three worlds of instructional design State of the art and future directions Instructional Science 29 429 441 Zimmerman B J 2002 Achieving academic excellence A self regulatory perspective In M Ferrari Ed Pursuit of excellence through education pp 85 110 Mahwah NJ Lawrence Erlbaum
31. ocedural during led to a higher performance effi ciency than the other three formats yielded no significant results 4 Discussion Support is found for the first hypothesis It is clearly better to present the declar ative information and the procedural information piece by piece instead of simulta neously Learners in the declarative before procedural during and the declarative during procedural before information presentation formats piece by piece ob tained higher transfer test scores than learners in the declarative before procedural before and declarative during procedural during formats simultaneous Further more learners in the piece by piece formats reported lower mental effort ratings dur ing practice than learners in the simultaneous formats The combination of low investment of mental effort during practice and high performance on the transfer test tasks is reflected in a high instructional efficiency Thus as hypothesized learners in the formats that presented the necessary information piece by piece had higher instructional efficiency scores than learners in the formats where both information types were presented simultaneously whether before or during practice Presumably learners in the piece by piece formats were better able to allocate their working memory capacity to processes relevant for learning Moreover the performance eff ciency scores indicate that these learners not only show a high
32. oggle switch a lamp a battery a resistor a voltmeter and an ammeter During practice all circuits contained all six elements Every prac tice task consisted of three parts The participants could obtain a maximum of 30 points by giving a correct problem description 1 point a correct problem cause 1 point and a correct problem solution 1 point for the malfunctioning circuits For example the inserted battery in one practice task is too strong for the elements in the circuit In this task the following correct responses could be made 1 problem description the lamp explodes 1 point 2 problem cause the power supply for instance the battery is too strong for the lamp 1 point and 3 problem solution insert a weaker battery 1 point The problem causes that were implemented in the practice tasks were related to current 1 e too high or too low wrongly connected elements or a short circuit Every task contained only one problem To determine interrater reliability practice performance scores of eight partici pants were determined by two raters The interrater reliability for practice perfor mance of the two raters was 87 Intraclass Correlation Coefficient SPSS The internal consistency of the practice items was 82 Cronbach s 2 2 4 Test tasks After the 10 practice tasks participants received 10 test tasks The test tasks also consisted of malfunctioning electrical circuits with problems related to curre
33. pant are transformed into z scores using the grand mean across conditions Then the mean z scores for every condition is represented in a Cartesian coordinate system with Mental effort z scores on the horizontal axis and Performance z scores on the vertical axis see Fig 2 The line P M through the H ig h Performance efficiency 18 E 0 0 5 eim botal effort Low efficiency Fig 2 Efficiency measures in a Cartesian coordinate system ARTICLE IN PRESS L Kester et al Contemporary Educational Psychology xxx 2005 xxx xxx 9 origin indicates a neutral efficiency slope 45 The efficiency E is calculated as the perpendicular distance from a data point in the coordinate system to the line P M Paas amp van Merri nboer 1993 The formula for calculating this distance is Performance Mental effort ae Equal performance P and mental effort M z scores yield an instructional efficien cy of zero a neutral score When P gt M the instructional material is more efficient indicated by a positive value because the performance is higher than might be expect ed on the basis of perceived mental effort 1 e the data point is to the left of the diag onal When P lt M the material is less efficient indicated by a negative value because the performance is lower than might be expected on the basis of perceived mental effort 1 e the data point is to the right of the diagonal Instructional efficien cy 1s cal
34. r In aseries connection the amperage is always the current flows through a circuit same while the voltage is divided over all elements The more elements in a series connection the lower the amperage through the connection Click here for more information Fig 1 An example of a practice task in the declarative during procedural during format The declarative information is presented in the right hand frame and the procedural information is integrated in the circuit diagram trical circuits e g series or parallel and their influence on the circuit and the prop erties of elements in the circuit is just such a situation Procedural information refers to task specific rules along with their related facts principles or concepts which are necessary for schema automation see the textual information in the left half of Fig 1 for an example This information enables learn ers to form productions through knowledge compilation Anderson 1996 the trans lation of procedural information into procedural knowledge 1 e internalized rules Knowledge compilation of procedural information yields productions containing domain specific knowledge particularly useful for dealing with familiar problem sit uations because such situations require the same use of the same domain specific knowledge In the study presented here this allows the learner to recognize a switch and the need to throw it to close a circuit or to recogn
35. ration was not allowed and the work had to be done individually After the introduction the participants could go through the practice phase and the test at their own pace but in a fixed order Once the partic ipants started working on the practice tasks they were not allowed to return to infor mation presented before practice and they could not go back to the practice tasks once they started working on the test tasks Participants could however move back and forth freely within each segment that is in the before information part the practice part and the test part 3 Results 3 1 Practice tasks See Table 1 for an overview of the results for the practice tasks 3 1 1 Practice performance An level of 05 was used for all statistical tests ANOVA revealed a main effect for timing of declarative information presentation on practice performance F 1 85 4 26 MSE 95 04 p lt 05 yn 05 Participants receiving declarative information during practice M 17 64 SD 4 65 performed better than those receiving declarative information before practice M 15 52 SD 4 72 There was no effect of timing of procedural information and no interaction effect on prac tice performance Tukey s HSD test showed no significant differences between infor mation presentation formats Table 1 Overview of results for the practice task Information presentation format Piece by piece Simultaneous DecB DecD ProcB DecB ProcB DecD
36. tice during practice or before and during practice The declar ative information was divided over three screens while the procedural information fit on one screen All conditions contained the exact same amount of information both declarative and procedural Four information presentation formats were dis tinguished in a 2 x 2 factorial design with the factors being timing of declarative information presentation before or during practice and timing of procedural information presentation also before or during practice In the declarative be fore procedural during format the declarative information was presented before practice while the procedural information was presented during practice In the declarative during procedural before format the declarative information was pre sented during practice while the procedural information was presented before practice In the declarative before procedural before format both declarative information and procedural information were presented before practice Finally in the declarative during procedural during format both declarative and proce dural information were presented during practice For an example of a practice task see Fig 1 ARTICLE IN PRESS L Kester et al Contemporary Educational Psychology xxx 2005 xxx xxx 7 2 2 3 Practice tasks Participants received 10 practice tasks The circuits in the practice tasks made use of six elements namely a t
37. which were used during learning Transfer test tasks are tasks that use some of the same ele ments used during learning along with new elements and have thus a lower level of familiarity In such a transfer task a motor would be used instead of a lamp to draw current in a circuit Although learners need both specific schemata and general schema ta to solve both types of test tasks solving equivalent tasks relies more on using specific schemata acquired through compilation of procedural information while solving transfer tasks increasingly relies on general schemata acquired through elaboration of the declarative information as the familiarity of these tasks decreases ARTICLE IN PRESS L Kester et al Contemporary Educational Psychology xxx 2005 xxx xxx 5 While test performance indicates the effectiveness of the information presentation formats the costs at which this performance is obtained indicates their efficiency Instructional efficiency looks at the learning outcomes 1 e test performance in rela tion to the working memory capacity allocated during practice Paas Tuovinen Tabbers amp van Gerven 2003 Paas amp van Merri nboer 1993 Tuovinen amp Paas 2004 Instructional efficiency indicates the extent to which learners during practice were able to allocate their working memory capacity to processes relevant for learn ing High test performance in combination with low allocation of working memory capacity is highly
Download Pdf Manuals
Related Search