Home

SITES 2006 User Guide for the International Database

Contents

1. APPENDIX C SITES 2006 TECHNICAL QUESTIONNAIRE 5 Hardware In your school about how many computers including laptops are Count terminals if they have a keyboard and a screen as computers Count laptops as computers Exclude computers which are not in use Exclude computers which are only used as servers Exclude graphical calculators and Personal Digital Assistants PDAs hand held computers and smartphones phone integrated with PDA Please write a whole number Write 0 zero if none Available in the school altogether BCTOSA1 Available to students in Grades lt grade range gt BCTOSB1 Available only to teachers BCTOSC1 Available only to administrative staff BCTOSD1 Connected to the Internet World Wide Web BCTOSE1 Connected to a local area network LAN BCTOSF1 Multimedia computers equipped with a CD ROM and or DVD BCTOSG1 How many of the computers in your school are laptops Please write a whole number Write 0 zero if none Laptops BCTO6A1 In your school about how many of the following school owned technologies are available A Personal Digital Assistant PDA is a palmtop with roughly the same functionalities a
2. BTG16B1 BTG16B2 C Complete worksheets exercises BTG16C1 BTG16C2 D Give presentations BTG16D1 BTG16D2 E Determine own content goals for s learning e g theme topic for project BTG16E1 BTG 16E2 F Explain and discuss own ideas with s teacher and peers BTG16F1 BTG16F2 G_ Collaborate with peers from other schools within and or outside the country BTG16G1 BTG 16G2 H Answer tests or respond to evaluations BTG16H1 BTG 16H2 I Self and or peer evaluation BTGI6I1 BTG 1612 J Reflect on own learning experience s review e g writing a learning log and adjust own learning strategy BTG16J1 BTG 16 2 K Communicate with outside parties s e g with experts BTG16K1 BTG16K2 L Contribute to the community through 16 their own learning activities e g by conducting an environmental protection project BTG16L1 BTG16L2 APPENDIX D SITES 2006 TEACHER QUESTIONNAIRE Part V Learning Resources and Tools 17 How often do you incorporate the following in your teac
3. 95 96 K pS Resource Materials SITES 2006 IDB USER GUIDE For each of the following technology applications indicate whether it is available and whether you need it in your school for teaching and or learning in Grade lt target grade gt Please mark only one choice in each row Equipment and hands on materials e g laboratory equipment musical instruments art materials overhead projectors slide projectors electronic calculators BCTO4A1 Tutorial exercise software BCT04B1 General office suite e g word processing database spreadsheet presentation software BCT04C1 Multimedia production tools e g media capture and editing equipment drawing programs webpage multimedia production tools BCT04D1 Data logging tools BCTO4E1 Simulations modeling software digital learning games BCTO4F1 Communication software e g e mail chat discussion forum BCT04G1 Digital resources e g portal dictionaries encyclopedia BCT04H1 Mobile devices e g Personal Digital Assistant PDA cell phone BCT0411 Smart board interactive whiteboard BCT04J1 Learning management system e g web based learning environments BCT04K1 Mail accounts for teachers BCT04L1 Mail accounts for students BCT04M1 1 2 3 Available Needed but Not needed not available and not available
4. and evaluating learning outcomes BT G21L1 M_Icanuse ICT to give effective presentations explanations BTG21M1 N Icanuse ICT for collaboration with others BTG21N1 O I caninstall educational software on my computer BTG2101 PI can use the Internet e g select suitable 120 websites user groups discussion forums to support student learning BTG21P1 APPENDIX D SITES 2006 TEACHER QUESTIONNAIRE 22 Looking ahead to the coming two years what priority will you give to the use of ICT in enhancing K your teaching practice in the following areas Please mark only one choice in each row To monitor more effectively the progress of my students BTG22A1 To provide exercises to students in order to practice skills and procedures BTG22B 1 To provide better and more interesting lectures presentations to my students BTG22C1 To engage students in multimedia production projects BTG22D1 To provide more activities that address the individual differences among my students BTG22E1 To involve students in collaborative short projects 2 weeks or shorter BTG22F1 To involve students in extended collaborative projects longer than 2 weeks BTG22G1 To involve my students in scientific investigations involving laboratory work BTG22H1 To
5. an orderly attentive classroom BTG14G1 BTG 14G2 Organize monitor and support team building and collaboration among students BTG14H1 BTG14H2 Organize and or mediate communication between students and experts external mentors BIG1411 BTG1412 Liaise with collaborators within or outside school for student collaborative activities BTG14J1 BTG14 2 Provide counseling to individual students BTG14K1 BTG14K2 Collaborate with parents guardians caretakers in supporting monitoring students learning and or in providing counseling BTG14L1 BTG14L2 APPENDIX D SITES 2006 TEACHER QUESTIONNAIRE 15 In your teaching of the target class in this school year moa ws e Q m a Do you use the following methods of assessing student performance b Do you use ICT to carry out these assessments Please mark only one choice for each of the two parts in each row a Assessment method used b I
6. 111 112 SITES 2006 IDB USER GUIDE Part Ill Teacher Practice In your teaching of the target class in this school year a How often is the scheduled learning time of the class used for the following activities b Has ICT been used when these activities took place Please mark only one choice for each of the two parts in each row a How often is the scheduled learning b ICT used time used for the following activities 1 2 3 4 1 2 Never Sometimes Often Nearly No Yes always Extended projects 2 weeks or longer BTGO9A1 BTGO09A2 Short task projects BTGO09B1 BTG09B2 Product creation e g making a model or a report BTGO9C1 BTG09C2 Self accessed courses and or learning activities BTGO9D1 BTG09D2 Scientific investigations open ended BTGO9E1 BTGO9E2 Field study activities BTGO9F1 BTG 09F2 Teacher s lectures BTG09G1 BTG09G2 Exercises to practice skills and procedures BTGO9H1 BTG09H2 Laboratory experiments with clear instructions and well defined outcomes BTG09I11 BTG0912 Discovering mathematics prin
7. J Demonstrating learning achievement BTG41J1 Dependent K Monitoring progress BTG41K1 Dependent L Providing feedback BTG41L1 Dependent M Choosing learning activities strategies BTG41M1 Dependent This is the end of the questionnaire Thank you very much for your time and effort Return Instructions 128 Appendix E Characteristics of the Australian E Sample E 1 Australia Characteristics of National Sample e School level exclusions consisted of all non mainstream schools e g correctional schools hospital schools environmental schools distance education schools language support centers non English curriculum schools and special schools e Explicit stratification by region Tasmania Continent and school size e Implicit stratification by state for Continent and school type Government Catholic Independent for a total of 100 implicit strata Table E 1 Allocation of School Sample in Australia Total ee Ree Participating Schools Non Explicit Stratum PEENE Seen Participatin Sample Schools Not Met Sampled ee ane Sree Schools Replacement Replacement Schools Tasmania Very Large Schools 7 o o 3 2 o 2 Tasmania Medium Size Schools 8 o o 6 o o 2 Tasmania Very Small Schools 15 o 2 5 1 1 6 Continent Very Large Schools 54 o o 26 7 T 14 Continent Large Scho
8. 119 SITES 2006 IDB USER GUIDE Part VII Information about You and Your School 21 To what extent are you confident in accomplishing the following Please mark only one choice in each row General use of ICT 1 2 3 4 Not at all A little Somewhat Alot A Ican produce a letter using a word processing program BTG21A1 B Ican e maila file e g the notes of a meeting to a colleague BTG21B1 C Ican take photos and show them on the computer BTG21C1 D Icanfile electronic documents in folders and sub folders on the computer BTG21D1 E I canusea spreadsheet program for budgeting or student administration BTG21E1 F Ican share knowledge and experiences with others in a discussion forum user group on the Internet BTG21F1 G Ican produce presentations with simple animation functions BTG21G1 H Icanuse the Internet for online purchases and payments BTG21H1 Pedagogical Use of ICT I Ican prepare lessons that involve the use of ICT by students BTG2111 J Iknow which teaching learning situations are suitable for ICT use BT G21J1 K Ican find useful curriculum resources on the Internet BTG21K1 L_Icanuse ICT for monitoring students progress
9. educational practices critically BCPO9F1 Encourage teachers to assess their educational practices in the context of our school s goals BCPO9GI1 Discuss with parents guardians caretakers what pedagogical changes are taking place in our school BCP09H1 Discuss with students the teaching and learning in our school BCPO9I1 APPENDIX B SITES 2006 PRINCIPAL QUESTIONNAIRE 10 To what extent do you agree or disagree that the school leadership you and or other school leaders encourages the following activities to take place in Grade lt target grade gt Please mark only one choice in each row 1 2 3 4 Strongly Disagree Agree Strongly disagree agree A Teachers co teach with their colleagues BCP10A1 B Teachers collaborate with teachers from other schools BCP10B1 C Teachers discuss the problems that they experience at work with their colleagues BCP10C1 D Teachers collaborate with teachers from other countries BCP10D1 79 80 11 w J a we O H SITES 2006 IDB USER GUIDE To what extent do you agree or disagree that the school leadership you and or other school leaders encourages teachers to use each of the following types of assessment at Grade lt target grade gt Please mark only one choice in each row Writ
10. This is the end of the questionnaire Thank you very much for your cooperation Return Instructions AppendixC SITES 2006 Technical Questionnaire SITES 2006 Second Information Technology in Education Study I Technical Questionnaire International English Version IEA International Association for the Evaluation of Educational Achievement Copyright 2005 2006 APPENDIX C SITES 2006 TECHNICAL QUESTIONNAIRE Introduction The Second Information Technology in Education Study SITES 2006 is an international assessment of teaching and learning practices and of how Information and Communication Technologies ICT support these in secondary schools around the world Approximately 20 countries will provide information from representative samples of teachers on how they organize their teaching and learning the ICT facilities they have available at school how they use ICT for teaching and learning and the obstacles or difficulties they experience in relation to these technologies This information will give better insight into the current state of pedagogical approaches and of how technologies support them It will also allow educational practitioners and policy makers to gain a better understanding of areas needing intervention and additional support Name of country along with about 20 other countries is taking part in this international study of pedagogical practices and the way that ICT supports t
11. computer connectivity and internet access NCQ2d e Funding for ICT has moved from being earmarked provisions to 0 0 00 being an integral part of the total school funding from the government and the school is free to decide the percentage of school funds a ctually spent on ICT NCQ2Se f Schools do not receive any government funding for purchasing u hardware and or software NCQ25f g Other NGOS NNO nA LON UeROLDOy aoa onan R Please be specific NCQ25gt Is language an obstacle for schools in ICT implementation in learning and teaching in your system For example important websites may not be written in a language that students can read Yes language EEEO s eTo C ING NOt amMODStacle eee cdo Lh ele etd Le dl NCQ26 If language is an obstacle please describe to what extent it is an obstacle and what steps have been taken to minimize it Does your system have a system wide program regarding student ICT related skills at the target grade NCQ281 If YES is part of the school curriculum Is it associated with a specific subject If so what subject ee E ean eee E e rer cee NCQ282 If no skip to question 30 APPENDIX A SITES 2006 NATIONAL CONTEXT QUESTIONNAIRE 29 30 31 32 Does the program promote using ICT in Yes No a traditionally important approaches NCQ29a b student c
12. 1 No gt Please proceed to the end of the questionnaire 2 Yes gt Please continue 5 Not applicable 103 SITES 2006 IDB USER GUIDE 19 Doyou use this computer for the following activities Please mark only one choice in each row No Yes A School related activities BCT19A1 Dependent B Connecting to the internet BCT 19B1 Dependent This is the end of the questionnaire Thank you very much for your cooperation Return Instructions 104 Appendix D SITES 2006 Teacher D Questionnaire SITES 2006 Second Information Technology in Education Study Teacher Questionnaire International English Version This questionnaire comprises the following parts Parti Information about the Target Class Part ll Curriculum Goals Part IIl Teacher Practice Part IV Student Practice PartV Learning Resources and Technology Infrastructure Part VI Impact of ICT Use Part VII Information about You and Your School Part VIII Specific Pedagogical Practice that Uses ICT 105 International Association for the Evaluation of Educational Achievement Copyright 2005 2006 APPENDIX D SITES 2006 TEACHER QUESTIONNAIRE Introduction The Second Information Technology in Education Study SITES 2006 is an international assessment of teaching and lear
13. a school level data file the letter T a teacher level data file Note that the letter C for the school level may not be intuitive but was chosen for reasons of consistency with other IEA studies such as TIMSS and PIRLS e The third character indicates the context or scope of the information in a file The letter G is used for general questionnaire data the letter M for mathematics teacher related data and S for science teacher related data e The fourth to sixth characters identify the participating education system using a three character alphanumeric abbreviation based on the ISO 3166 coding scheme Table 1 1 in Chapter 1 lists the codes of all participating education systems The national context questionnaire data file uses the three letter combination SIT e The seventh and eighth characters indicate the study cycle The combination S1 is used for SITES 2006 e The three character file extensions used for the data files are SAV for the SPSS format and XPT for the SAS XPORT format For example the file name BTMZAFS1 SAV indicates a SITES 2006 mathematics teacher file for South Africa in SPSS format All information related to the structure of the data files as well as the source format descriptive labels and response option codes for all variables is additionally contained in codebook files Each data file is accompanied by a codebook file The naming convention for a codebook f
14. ANALYSING THE DATA Figure 4 13 Merge Module for Example School level Analysis B 108 Analyzer I1 Merge Module Q SelectFileTypes SelectVariables Available Variables Selected Variables E School Background Backgound Variables and Scores 362 1D and Samping Variables 0 Background Variables and Scores 1 1D and Sampling Variables 11 H E Background Name Dacia Na ecnin QB BCPOIAI PEDAS ASP OF TL INDEFENDENT LEARNING RESMAT TECHAPP HANDS ON MATERIAL QBBCPOIBI PEDAS ASP OF TL SEA PRO PRE INFORMATION BCPOCI PEDAS ASP OF TL CNTRL OWN LEARN PROG QBBCPOIDI PEDAS ASP OF TL LEARN AT THEIR OWN PAC B BCPOIE1 PEDAS ASP OF TL PROJECT BASED LEARNING B BcPoIF PEDAS ASP OF TLAWHEN TO TAKE A TEST BBCPOIG PEDAS ASP OF TL SEARCH STRAT TO FIND IN B BcPOIHI PEDAS ASP OF TL ASS ORG INTG INFORMATI Becr PEDAS ASP OF TL CRITICALLY EVAL VALIDITY Bcra PEDAS ASP OF TL SEV FRMS OF PRESENTATI QB BCPOIKI _ PEDASVASP OF TL SEV PASNS WORK TOGET Becroi PEDAS ASP OF TL AUTONOMY TO DECIDE TO B acro2at PEDAS GOALS PRESCA CURRICULUM CONTE QB BCP0281 PEDAS GOALSAMPR PERFORMANCE ON ASS BCPO2C1 PEDAS GOALSANDN LEARNING EXPERIENCE B BcPo201 PEDAS GOALS NCREASE LEARNING MOTIVA B BCPO2E1 PEDAS GOALS FOSTER ABILITY AND READIN SortVerisblesBy _ BCPO2F1 PEDAS GOALS COLLAB AND ORGA SKILLS BCPO2G1 PEDAS GOALS REALWORLD EXAMPLES SET Name QB BTPO2H1 PEDAS GOALS OPP TO LEARN FROM EXPERTS
15. I am able to implement innovations in my classroom according to my own judgment and insights BTG26C1 To what extent do the following statements about professional collaboration among teachers apply to you Please mark only one choice in each row 1 2 3 4 Not at all A little Somewhat Alot I co teach with my colleagues BTG27A1 I discuss the problems that I experience at work with my colleagues BTG27B1 I work with teachers in other schools on collaborative activities BTG27C1 I work with teachers in other countries on collaborative activities BTG27D1 123 124 28 29 30 31 32 SITES 2006 IDB USER GUIDE To what extent do the following statements about support to teachers apply to you Please mark only one choice in each row 1 2 Not at all A little When necessary I receive sufficient technical support from my school region state e g by having a technician in my classes to support my teaching BTG28A1 My students can access computers easily outside scheduled class time without my help BTG28B 1 The administrative work arising from the use of ICT in my teaching e g booking computer laboratories changing class schedules is easy to do in my school BTG28C1 Do you
16. Using computers for monitoring student progress BCP12E1 Organizing forms of team teaching BCP12F1 Collaborating with other teachers via ICT BCP12G1 Communicating with parents via ICT BCP12H1 Tra 7moa Being knowledgeable about the pedagogical issues of integrating ICT into teaching and learning BCP 1211 J Using subject specific learning software e g tutorials simulation BCP 12J1 81 82 J Developing a strategic plan for integrating ICT SITES 2006 IDB USER GUIDE 13 How much ofa priority is it for your school leadership you and or other school leaders to acquire competencies in the following areas Please mark only one choice in each row 1 2 3 4 Not Low Medium High considered priority priority priority A Developing a common pedagogical vision among teaching staff in the school BCP13A1 B Managing the innovation of pedagogical practices in the school BCP13B1 C Explaining to teachers the relevance of encouraging students to be responsible for their own learning process and outcomes BCP13C1 D Identifying best practices that exist outside the school regarding the integration of ICT in learning BCP13D1 E Promoting collaboration between teachers of diff
17. B ecroai PEDAS GOALS FOSTER COMMUNICATION SKI ROPNAN PFNAS GNAI S AFSPONS INTFRNF T RFHAVINLS gt Location In File O OutpuiFiles pecan TTT lt lt Back Stat SPSS Exit SITES 2006 Popuiation2 lane School Background For this analysis there is no need to recode the data The analysis is conducted with the IEA IDB Analyzer analysis module Figure 4 14 shows the set up for this analysis and Figure 4 15 shows the results In the set up screen for the analysis module the analysis type selected is Percentages only Variables IDCNTRY and BCT04A1 have been selected as Grouping Variables As Weight Variable the IEA IDB Analyzer automatically selected the variable SCHWGT Figure 4 14 Analysis Module for Example School level Analysis 108 Analyzer II Analyze Module O Arsvsirle CCGA haf OQ SelectAnalysis Type Percentages and Means Percentages only C Regession Conelations Benchmarks Clear all Selections I With Achieverert Scores FF Exclude Missing From Analysie I Cumuteive I Wah Analysis Variable Select Variables Desciption Description IDSCHOOL SCHOOL PRINCIPAL ID S IDCNTRY COUNTRY ID S IDTECH TECHNICAL COORDINATOR ID 4 p amp BCTo1 RESMAT TECHAPP HANDS ON MATERIAL S DPOP POPULATION ID Sf INSITOS IN SITES 2006 POPULATION Sf IDSTRATE EXPLICIT STRATUM Analysis Vericbles Name Description Sf IDSTRATI IMPLICIT STRATUM S
18. Less than 2 4 years S 9 years 10 19 years 20 years 2 years or more 125 126 37 38 39 SS et oy Se ey SS SITES 2006 IDB USER GUIDE Part VIII Specific Pedagogical Practice that Uses ICT Which of the following description is applicable to you BTG37AI Filter Please mark only one choice Iuse ICT once a week or more in the target class gt Please continue Iuse ICT extensively in the target class during a limited period during the year e g in a project or a theme gt Please continue None of the above gt Please go to the end of the questionnaire Please describe the one most satisfying pedagogical practice that you applied in the target class in this school year in which you and or your students used ICT extensively with specific content related to mathematics science BTG38AT Dependent Please describe the pedagogical practice e g a research project or a multimedia production the ICT used e g data logging tools spreadsheets or web search and its content e g curricular goals topic in a maximum of 20 words Has the use of ICT in this pedagogical practice contributed to changes in the following students outcomes in the target class Please mark only one choice in each row 1 2 3 Decreased Made no Increased difference Subject matter knowledge mastery BTG39A1 Dependent ICT skills B
19. WJ Eds 2009 IEA SITES 2006 Technical Report Amsterdam International Association for the Evaluation of Educational Achievement International Organization for Standardization 2008 ISO 3166 Maintenance agency ISO 3166 MA ISO focal point for country codes Retrieved from http www iso org iso country_codes htm on July 8 2008 Law N Pelgrum WJ amp Plomp T Eds 2008 Pedagogical practices and ICT use around the world findings from an international comparative study CERC Studies in Comparative Education Hong Kong Comparative Education Research Centre The University of Hong Kong and Dordrecht Springer Wikipedia 2008 ISO 3166 1 Retrieved from http en wikipedia org wiki ISO_3166 1 on July 8 2008 The SITES 2006 Data Files Falk Brese Ralph Carstens This SITES 2006 User Guide is produced as part of a larger package containing the SITES 2006 international database The package is available at www sites2006 net for download as a zipped archive consisting of the following parts e CODEBOOKS Codebook files with descriptions of all variables in the SITES 2006 data e DATA Data files containing data from school principals ICT coordinators and teachers in SPSS and SAS XPORT formats The National Context Questionnaire data file is available in SPSS format only e PROGRAMS IEA IDB Analyzer setup file e REPORTS The SITES 2006 IDB User Guide and the SITES 2006 Technical Report The file names w
20. curriculum textbook lists and teaching eon methods are prescribed e None of above prescriptions are prescribed o mmncnnnnnnnninnnnnniinnnnnse Explanations as needed Please explain if your situation does not fit above categories 8 Which government levels take the responsibility for determining the curriculum taught in the schools Please Tick all that apply a Central government NCQ08 1 a mnnnnnnnintnniniinninitnniniinnininnisnninnnt b State and or provincial government NCQO8 1b c District and or Local government NCQ08 lc d Non statutory and or professional body NCQO8 1 id AE E e Schools are free to decide NCQO08 1 funcscnmnmnncnnnenennennnennneinnanannanninsnti f Other Please specify N CQOS 1f i NCQ08 1ft If you selected more than one of the above answers please answer 8 1 and 8 2 8 1 Which level has the primary most responsibility for these policies Enter atO firussen NCQ082 8 2 Which level has the aime second most responsibility for these policies Enter a to f or none iussisse NCQ083 Explanations as needed 55 NCQ084 SITES 2006 IDB USER GUIDE 9 What is the approximate percentage of schools in the sector containing the target grade that are classified as special education schools and do not follow the mainstream syllabi or curriculum This does not include schools for gifted students only NCQO91
21. learning can take place any time anywhere BCPO6E1 To improve the technical skills of teachers BCPO6F1 To improve the ability of teachers to make good pedagogical use of ICT BCP06G1 To broaden teachers pedagogical repertoire and to widen their pedagogical competence to engage in new methods of teaching and learning BCPO6H1 To improve students ICT skills BCPO6I1 To provide teachers with incentives including salary adjustment promotion etc to integrate ICT use in their teaching BCP06J1 To increase the number of teachers using ICT for teaching learning purposes BCPO6K1 75 76 K Has the school leadership you and or other school leaders taken any of the following actions during the past few years Please mark only one choice in each row Re allocating workload to allow for collaborative planning for innovations in the classrooms BCP07A1 Re allocating workload to allow for the provision of technical support for innovations BCP07B1 Organizing workshops to demonstrate the use of ICT supported teaching and learning BCP07C1 Meeting teachers to review their pedagogical approach BCP07D1 Monitoring and evaluating the implementation of pedagogical changes BCPO7E1 Establishing new teacher teams to
22. 32 G1 Russian Federation 47 3 9 61 3 2 7363 5 34 2 7 10 1 8 27 3 2 36 2 7 49 3 9 15 2 1 02 0 5 05 1 2 18 2 3 13 2 1 Singapore BAA 85M 100 00 BEY 958 664 2 BUY 9222 34 G5 28G4 9508 10000 589 Slovak Republic 75 2 3 48 28 97 09 68 2 6 25 2 3 40 2 9 97 1 0 83 20 21 20 17 23 25 26 81 2 0 72 2 7 Slovenia NAH HO 10003 HCY BUP 5526 BOD BEY 21 23 O4 10 485 IUG 91 1 5 South Africa 17 1 4 10 14 35 R2 07 1 0 11 1 6 04 0 8 14 1 4 20 2 0 13 18 09 1 0 07 1 3 13 1 4 08 1 4 Denmark 94 2 0 931 9 99 1 0 89 24 4 3 7 53 3 9 97 1 2 93 2 0 11 2 3 25 3 2 51 G 96 14 89 2 3 Estonia 66 38 MBA BUDY IED 35GA AGH BO GH 2BY 2128 AGH France 86 2 5 803 5 99 1 0 68 3 3 76 3 5 50 3 6 71 3 3 83 2 6 18 2 7 14 2 6 26 3 5 78 2 8 48 3 7 Norway EA BS 1000 0 BEY 28 G3 349 9503 HBB BAD 07 18 Notes School participation rote after including replocement schools 15 below 70 Less than 70 of the school level questionnaires 1m the participating schools were reamed ops eee nen ee nce eee eee 4 Nanonally defined population covers less than 90 of the nationally desired population Figure 4 13 belowshows the set up screen for the IEA IDB Analyzer Merge module with the variables selected for the analysis Notice that the school file BCG is selected The identification variables are automatically selected by the IEA IDB Analyzer
23. Dependent Dependent Available Available provider is an Filter provider is external Not available school based organization A Introductory course for Internet use and general BCT12A1 BCTIZA2 BCT12A3 applications basic word processing spreadsheet databases etc B Technical course for operating and maintaining BCT12B1 BCT12B2 BCT 12B3 computer systems C Advanced course for applications standard tools BCT12Cl1 BCT 12C2 BCT12C3 e g advanced word processing complex relational databases D Advanced course for Internet use e g creating websites _ BCT12D1 BCT12D2 BCTI2D3 developing a home page advanced use of Internet video conferencing E Course on pedagogical issues related to integrating ICT BCTI2EL BCTI2E2 BCTIZE3 into teaching and learning F Subject specific training with learning software for s BCT12F1 BCT12F2 BCT12F3 pecific content goals e g tutorials simulation etc G Course on multimedia use e g digital video and or BCT12G1 BCT12G2 BCT12G3 audio equipment 100 APPENDIX C SITES 2006 TECHNICAL QUESTIONNAIRE 13 mit ons gt mit GaAs D gt Support Facilities for ICT Do you hold any of the following positions at your school Please mark only one choice in each row Principal BCT13A1 Deputy principal BCT 13B1 Head of department BCT 13C1 Teacher BCT13D1 Librarian BCT13E1 Other than a
24. IEA IDB Analyzer analysis module is the next step Figure 4 18 shows how the set up screen of the analysis module looks when performing this analysis Notice that for analysis type Percentages only is selected Under Grouping Variables IDCNTRY BTG32A1 and BTG18A1 are selected MTOTWGT is selected as the Weight Variable since this analysis used mathematics teacher data This is unlike the analysis at the school level where the IDB Analyzer predefines the appropriate weight variable When conducting teacher level analysis with the IEA IDB Analyzer the user has to define the weight The output of this set up is shown in Figure 4 19 ANALYSING THE DATA Figure 4 18 Analysis Module for Example Teacher level Analysis iT IS YOUR GENDER 90 YOU USE ICT IN TEACH BIRTH YEAR INFORMATION ON TRACKING FORM FINAL PARTICIPATION INDICATOR EXCLUSION INFORMATION ADMINISTRATION LANGUAGE SCHOOL ID _ SUBJECT IDENTIFICATION z SECONDARY ID FOR SCIENCE TEACHERS _ SECONDARY ID FOR MATH TEACHER POPULATION ID IN SITES 2006 POPULATION EXPLICIT STRATUM SIMPLICT STRATUM IMPLICIT STRATUM UNIQUE IDB Identifier Output5 Document5 SPSS Viewer loj x SHAR B E oo OLS 00 amp Swe tes OB 2 Percentages by IDCNTRY BTG32A1 BTG18A1 1 IMPACTICT DO YOU USE ICT Percent IN TEACHING Percent Chinese Taipei Hong Kong SAR In this example each education
25. International Report for the Example Teacher level Analysis Law Pelgrum amp Plomp 2008 pp 187 Table 6 2 Number of male teachers and female teachers and the percentage of teachers in each gender group who used ICT with their target classes Mathematics teachers Science teachers Male Female Sis Male Female Sie use ICT N use ICT z N use ICT N use ICT z 37 0 03 344 40 0 03 318 57 0 03 DCE SH E Es Ee Se EE 0 Ge Rs ee 39 0 02 429 32 0 02 493 51 0 02 301 45 0 03 isi iin EE GE GEO TE RES GES BE Ee i Se 68 0 03 240 73 0 03 263 78 0 03 184 89 0 02 ba eg a as Ea 59 0 05 533 58 0 02 124 64 0 05 545 57 0 03 ia CT E L0G LD LED Li Ee LES LS Gs 78 0 03 228 74 0 03 ue 128 76 0 04 17 76 0 03 BOB 169 52 0 05 452 51 0 02 265 56 0 04 796 56 0 02 62 0 04 171 62 0 04 174 0 0 03 154 78 0 04 m oo no 760008 280 6700 03 el 45 0 10 209 39 0 04 70 56 0 06 302 54 0 03 Fee OOH OO OO SOO 56 0 10 350 62 0 03 n 64 0 07 396 66 0 03 iOS GS se S370 Ls 1 0 03 114 80 0 04 175 76 0 03 101 72 0 04 T Russian Federation 53 301 n sow m sow m sowy 19 0 03 259 17 0 02 kal 183 15 0 03 242 17 0 03 K T use percentages are weighted statistics However N Listed in the tables is the unweighted Differences between male teachers and female teachers is significant at p lt 0 05 umber of teachers Gendar was not part of the sampling criteri
26. NCQ152 2 Teacher Preparation APPENDIX A SITES 2006 NATIONAL CONTEXT QUESTIONNAIRE Reminder Questions in this section refer to your overall educational system that is the target grade is not explicitly specified However if school types in your system differ greatly please answer for the school type within which your target grade of the SITES 2006 study belongs and then explain how the school types differ in the area marked Explanations as needed 16 What is the normal requirement for being certified as a teacher Tick only one choice NCQ161 a b c Postsecondary diploma and or certificate in education only Postsecondary degree in a major discipline OnLy nencnnnmnninnnnininnnne Postsecondary degree in a major discipline and diploma and or uuu certificate in education Other requirements please specify NCQ1 61 t s Requirements are defined only at local or school level esmnnnninnnnnnnin Explanations as needed NCQ162 17 Are there ICT specific requirements for being certified as a teacher Tick all that apply oT x No requirements NCQ171a pease Requirement to meet specified Doaa in Fenai ee er ee competence NCQ171b Requirement to meet specified benchmarks in using ICT in onsen subject based teaching NCQ171c Requirement to meet specified benchmarks in general ssus pedagogical ICT competence NCQ171d ie competence in integrating ICT into students learning activities Othe
27. OF TLAVHEN TO TAKE A TEST PEDAS ASP OF TL SEARCH STRAT TO FIND INF PEDAS ASP OF TL ASS ORG INTG INFORMATION PEDAS ASP OF TL CRITICALLY EVAL VALIDITY PEDAS ASP OF TL SEV FRMS OF PRESENTATION _PEDAS ASP OF TL SEV PRSNS WORK TOGETHER PEDAS ASP OF TL AUTONOMY TO DECIDE TOPIC _ PEDAS GOALS PRESCR CURRICULUM CONTENT PEDAS GOALS IMPR PERFORMANCE ON ASS EXAM _ PEDAS GOALS INDIV LEARNING EXPERIENCE 201 __ PEDAS GOALS INCREASE LEARNING MOTIVATION PEDAS GOALS FOSTER ABILITY AND READINESS PEDAS GOALS COLLAB AND ORGA SKILLS PEDAS GOALS REALWORLD EXAMPLES SET APPL _ PEDAS GOALS OPP TO LEARN FROM EXPERTS _PEDAS GOALS FOSTER COMMUNICATION SKILLS PEDAS GOALS RESPONS INTERNET BEHAVIOR _ PEDICTAS IMPICT PREPARE FOR FUTURE JOBS A printout of the results is presented in Figure 4 6 It shows the number of school principals answering in the area where their school is located and the average number of computers with internet access in each of the categories of school location for the selected five education systems Figure 4 6 SPSS Output for Percentages and Means Average for BCTOSE1 by IDCHTRY BCP21Al SHCHA PEOPLE WHERE SCHOOL N oof Sum of Percent BCTOSEL BCTOSEL Std Dev Percent COUNTRY ID IS LOCATED Cases SCHWGT Percent a 6 Mean s Std Dev s Missing Chile 3 000 PEOPLE OR FEWER 53 689 16 40 1 55 5 47 62 5 62 1 33 6 99 3 001 TO 15 000 PEOPLE 63 594 14 12 1 56 14 68 1 50 11 52 1 0 5 99 15 001 TO 5
28. STOTWGT must be used 23 24 SITES 2006 IDB USER GUIDE If researchers plan to analyze data from more than one level they must carefully choose the correct weight e Analyzing combined teacher data and school data is straightforward with the IDB Analyzer The software can be used to easily add school level data to the teacher data This way school information becomes an attribute of the teacher and the user can analyze information from both files For example the research question could be of this type What percentage of teachers who use the internet for teaching work at schools that have computers in most classrooms MTOTWGT or STOTWGT respectively should be used for this type of data analysis e Descriptive statistics obtained from a combined file can deviate from results obtained from the two original files If a school variable and a teacher variable are analyzed jointly a missing value in the school variable can set the teacher variable to missing and vice versa e If teacher information is regarded as an attribute of the school information for example the research questions could state what percentage of schools that have computers in most classrooms employ any teachers who use the Internet for teaching this cannot be handled easily with the IDB Analyzer The researcher must use other software e g SPSS SAS to aggregate the teacher data and to merge the teacher information to the school file When statements
29. and collaboration among students BCP04A1 B Organize and or mediate communication between students and experts external mentors BCPO4B1 C Facilitate collaboration within or outside of school on student activities BCP04C1 D Collaborate with parents guardians caretakers in supporting monitoring students learning a nd or in providing counseling BCP04D 1 E Provide students with experiences that show them how certain activities are done in real life or by experts BCP04E1 73 14 T QAM K Are the following actions with regard to ICT at Grade lt target grade gt taken in your school Please mark only one choice in each row Setting up security measures to prevent unauthorized system access or entry BCPOSA1 Restricting the number of hours students are allowed to use the computer BCPOSB1 Allowing students to access school computers outside school hours BCPOSC1 Allowing students to access computers outside class hours but during school hours BCPOSD1 Honouring of intellectual property rights e g software copyrights BCPOSE1 Prohibiting access to adult only material e g pornography violence BCPOSF1 Restricting the playing of games on school computers BCPOSG1 Specifying the compulsory computer related knowledge and skills that students need BCPOSH1 Giving the local community parents an
30. area with more than 50 000 people the estimated difference in the student computer ratio is 7 57 REC21A1 estimate This means that the student computer ratio is 31 48 students per computer in schools that are in an area with more than 50 000 people and therefore this ratio is higher than in schools which are located in an area with 50 000 people or fewer With an estimated standard error of 3 17 REC21A1 s e this difference is statistically significant at a 95 confidence level because REC21A1 t test is bigger than the absolute value of 1 96 In contrast the computed t test for Finland and Hong Kong suggests that the null hypothesis no difference in student computer ratio due to community size can not be rejected 4 4 4Computing Correlations and their Standard Errors The analysis type Correlations is used to calculate correlation coefficients between selected analysis variables This example will compute the Pearson correlation between the priority of resource allocation that a school principal gives to improve the ability of teachers to 39 make good pedagogical use of ICT BCP06G1 and the priority of resource allocation to broaden teachers pedagogical repertoire and to widen their pedagogical competence to engage in new methods of teaching and learning BCP06H1 The steps in the IEA IDB Analyzer are as follows 1 Open the analysis module of the IEA IDB Analyzer SITES 2006 IDB USER GUIDE 2 Select the da
31. c bcgall sav Compute new variable RECCSRAT as student computer ratio if BCTOSA1 gt 0 RECCSRAT BCP19A1 BCP19B1 BCTO5A1 variable labels RECCSRAT Computed student computer ratio Compute new variable REC21A1 recode BCP21A1 1 0 2 0 3 0 4 1 5 1 6 1 else sysmis into REC21A1 variable labels REC21A1 Recoded school location value labels REC21A1 o 50 000 people or fewer 1 more than 50 000 people save outfile c bcgall sav 37 38 SITES 2006 IDB USER GUIDE For the purpose of this example analysts can simply copy the above SPSS code into the SPSS syntax window and run it This will add the two recoded variables to the data file which then can be selected for the regression analysis in the IEA IDB Analyzer Poo bor The steps in the IEA IDB Analyzer are as follows Open the Analysis module of the IEA IDB Analyzer Select the data file called BCGALL SAV that was recoded earlier As type of the analysis select Regression In the next steps all variables need to be defined a As Grouping Variable the software always selects variable IDCNTRY by default No other variable needs to be added for this example b Next the analysis variable needs to be defined To activate this section press the radio button Analysis Variables This time select variable REC21A1 from the list of variables and move it to the analysis variables field by pressing the
32. have access to a computer at home BTG29A1 Filter No gt Please go to question 31 Yes gt Please continue Do you use this computer for the following activities Please mark only one choice in each row Teaching related activities BTG30A1 Dependent Connecting to the internet BTG30B1 Dependent To what age group do you belong BTG31A1 i 2 3 4 S Below 25 25 29 30 39 40 49 50 59 What is your gender BTG32A1 1 2 Male Female LI LI 3 4 Somewhat Alot 1 Zz No Yes 6 60 or above APPENDIX D SITES 2006 TEACHER QUESTIONNAIRE 33 34 35 36 What is your highest level of education BTG33A1 Please mark only one choice 1 2 3 4 Secondary or Post secondary Bachelor s Master s high school education degree degree or above e g teachers college Do you have a Bachelor s degree in Science or Mathematics BTG34A1 Please mark only one choice 1 2 3 4 No Degree in Mathematics Degree in Science Degree in both only Science only Mathematics and Science Do you have a teaching license or certificate BTG35A1 1 2 No Yes How many years of experience do you have in teaching Mathematics or Science BTG36A1 1 2 3 4
33. is 0 0527 In addition to the correlations the output also displays the mean and standard deviation for each variable in the analysis together with the corresponding standard errors Due to limitations in the current implementation of the correlation analysis the grouping variables are not displayed with labels The user will need to keep a record of the labels for each of the values of the grouping variables in order to interpret the results These are available in the technical documentation for the study and can also be obtained by using the Percentages Only procedure within the IEA IDB Analyzer or looking them up in the codebooks for the files 4 5 Performing Analyses from the SITES 2006 Report 4 5 1 Analysis with School level Variables The example of a school level analysis will investigate the percentages and standard errors of schools in which common types of technology applications and facilities were available The results of this analysis are presented in Table 4 5 of the SITES 2006 international report see Figure 4 12 below Parts of the results are also reproduced here in Figure 4 15 This example uses the analysis type Percentages only since the purpose is to compute the percentage of schools having equipment and hands on materials available The first step in the analysis is to identify the variables of interest in the appropriate data files and to review the documentation for any specific cultural and national adaptations to the
34. on this Not applicable 31 Altogether how often do you personally use a computer BCP31A1 Filter Please mark only one choice Never gt Please proceed to the end of the questionnaire A few times per year Almost monthly Weekly Daily nan BW N e 32 Do you use your computer for any of the following Please mark only one choice in each row Writing documents and letters BCP32A1 Dependent Budgeting monitoring or controlling expenses BCP32B1 Dependent Planning purposes BCP32C1 Dependent Communicating with teachers BCP32D1 Dependent Communicating with parents BCP32E1 Dependent Teaching instruction BCP32F1 Dependent Time tabling BCP32G1 Dependent Searching for information BCP32H1 Dependent Developing and making presentations BCP32I1 Dependent r ie OS me me gt Own professional development BCP32J1 Dependent 33 Do you have access to a computer at home BCP33A1 Filter 1 No gt Please proceed to the end of the questionnaire 2 Yes gt Please continue 89 SITES 2006 IDB USER GUIDE 34 Do you use this computer for the following activities Please mark only one choice in each row No Yes A School related activities BCP34A1 Dependent B Connecting to the internet BCP34B1 Dependent
35. shows the setup screen with the selections made for computing the correlations between the two selected variables Figure 4 11 shows the results from the analysis Figure 4 10 Analysis Module Computing Correlations IDB Analyzer II Analyze Module gt PEDAS ASP OF TL INDEPENDENT LEARNING PEDAS ASP OF TL SEA PRO PRE INFORMATION _PEDAS ASP OF TL CNTRL OWN LEARN PROGRESS 1 PEDAS ASP OF TL LEARN AT THEIR OWN PACE _PEDAS ASP OF TL PROJECT BASED LEARNING ciptio PEDAS ASP OF TLAVHEN TO TAKE A TEST _ PEDICTAS RESALLOCAMP ABILITY OF ICT U _ PEDAS ASPOF TL SEARCH STRATTOFINDINF PEDICTAS RESALLOC BROADEN REPERTO _PEDAS ASPOF TL ASS ORG INTG INFORMATION he ies een __PEDAS ASP OF TL CRITICALLY EVAL VALIDITY _ _PEDAS ASP OF TL SEV FRMS OF PRESENTATION _PEDAS ASP OF TL SEV PRSNS WORK TOGETHER PEDAS ASP OF TL AUTONOMY TO DECIDE TOPIC _ _PEDAS GOALS PRESCR CURRICULUM CONTENT _ PEDAS GOALS IMPR PERFORMANCE ON ASS EXAM _ PEDAS GOALS INDIV LEARNING EXPERIENCE PEDAS GOALS FOSTER ABILITY AND READINESS PEDAS GOALS COLLAB AND ORGA SKILLS PEDAS GOALS REALWORLD EXAMPLES SE T APPL ____ PEDAS GOALS OPP TO LEARN FROM EXPERTS ___ PEDAS GOALS FOSTER COMMUNICATION SKILLS DAS GOALS RESPONS INTERNET BEHAVIOR BCPO3A1 PEDICTAS IMPICT PREPARE FOR FUTURE JOBS The output in Figure 4 11 displays for each group defined by the grouping variables the correlation coefficients for each possible pair of variables The diagonal and elements
36. staff at the IEA DPC proportional to the school size taking into account national stratification needs After agreeing to participate in SITES 2006 schools provided national centers with a list of all eligible mathematics and science teachers teaching in the target grade National centers drew two random samples of mathematics and science teachers using software provided by the IEA DPC Once schools and teachers had been sampled administration of the survey began National centers could opt to administer the survey questionnaires online or on paper National centers were free to choose which modes would be the default at the national school or individual level Usually national centers asked individual respondents to select their preferred mode The SITES 2006 survey was administered from March to June 2006 in the northern hemisphere and from September to October 2006 in the southern hemisphere Due to low initial response rates several national centers were granted an extension of the administration period For more information about the sample design survey operations and online data collection procedures please refer to Chapters 6 7 and 8 of the SITES 2006 Technical Report Carstens amp Pelgrum 2009 OVERVIEW OF THE INTERNATIONAL DATABASE After data collection and capture several quality control and assurance procedures were implemented to ensure high quality and international comparability of the data from participation educatio
37. the survey in the first half of calendar year 2006 Four education systems administered the survey towards the end of the calendar year 2006 following the southern hemisphere timeline SH In the Russian Federation regions were selected as a first sampling stage A sample of schools from within each region was selected In the case of the Moscow region the sample was extended to arrive at optimized estimates for the region Therefore data from schools sampled for the Moscow region contributed to both the estimates for the Moscow region and for the Russian Federation All records from the Moscow region sample were consequently also added to the Russian Federation data 1 2 2 Management SITES 2006 was managed by a consortium involving the following partners and key staff e University of Twente Tjeerd Plomp study director Hans Pelgrum international coordinator and responsible for the development and reporting of the school questionnaires Joke Voogt 2 See Appendix E for characteristics of the Australian sample 3 Australia also administered the survey towards the end of the school year following a southern hemisphere timeline but this occurred one year later towards the end of 2007 OVERVIEW OF THE INTERNATIONAL DATABASE Table 1 1 Participating Education Systems in SITES 2006 Education System ISO 3166 1 ISO 3166 1 Hemisphere SITES Module 1 Alpha 3 code Numeric code Participation IDCNTRY Alberta Province Canada C
38. these practices as well as the factors associated with them SITES 2006 is the fourth project in this series It consists of a survey of schools and teachers of mathematics and science Its aim is to understand what pedagogical practices teachers and schools in different education systems apply and how ICT is used in these practices The study s initial findings are presented in Law Pelgrum amp Plomp 2008 1 2 1 Participating Education Systems Table 1 1 shows information about the 22 education systems that participated in SITES 2006 In addition to these 22 Australia participated as a benchmarking country and collected data during the following school year Altogether SITES 2006 covered education systems from five different continents Fifteen of these systems had already participated in SITES Module 1 The ISO 3166 1 definition for country codes was used to identify education systems For education systems not listed in the ISO 3166 1 the IEA Data Processing and Research Center DPC derived new alphanumeric and associated numeric codes The three letter alphanumeric code is used in filenames and identifies the education systems associated with that file The numeric code is used within the data files to identify the education system for each record The survey was administered towards the end of the school year In order to conform to this timeline 18 education systems followed the northern hemisphere NH timeline and administered
39. you use ICT in the teaching and learning activities of the target class 19 BTG18A1 Filter No gt Please go to question 21 Yes gt Please continue To what extent do you agree that the use of ICT has had the following impacts on you Please mark only one choice in each row My ICT skills have improved BTG19A1 Dependent I incorporate new teaching methods BTG19B1 Dependent I provide more individualized feedback to students BTG19C1 Dependent I incorporate new ways of organizing student learning BTG19D1 Dependent I monitor more easily students learning progress BTG19E1 Dependent I access more diverse higher quality learning resources BTG19F1 Dependent I collaborate more with colleagues within my school BTG19G1 Dependent I collaborate more with peers and experts outside my school BTG19H1 Dependent I complete my administrative tasks more easily BTG19I1 Dependent My workload has increased BTG19J1 Dependent There is increased work pressure BTG19K1 Dependent Ihave become less effective as a teacher BTG19L1 Dependent 1 2 3 Not at all A little Somewhat Alot APPENDIX D SITE
40. 0 000 PEOPLE 66 623 14 62 1 56 17 44 1 87 14 45 1 65 5 15 50 001 TO 100 000 PEOPLE 56 492 11 70 1 43 20 21 2 62 19 14 2 76 6 10 100 001 TO 500 000 PEOPLE 127 1080 25 70 2 03 21 35 1 50 19 10 2 44 5 25 NORE THAN 500 000 PEOPLE a0 725 17 25 1 59 27 39 2 82 25 89 3 07 6 30 Chinese Taipei 3 000 PEOPLE OR FEVER 6 17 2 12 30 69 07 13 24 45 44 15 95 00 3 001 TO 15 000 PEOPLE 4l 120 14 99 1 82 73 21 4 96 33 84 6 02 ao 15 001 TO 50 000 PEOPLE 75 188 23 53 2 03 123 14 8 73 72 37 9 23 2 01 50 001 TO 100 000 PEOFLE 62 115 14 35 1 73 142 15 9 85 55 09 5 33 ag 100 001 TO 500 000 PEOPLE 120 216 26 93 2 00 208 27 15 42 143 48 16 65 67 NORE THAN 500 000 PEOFLE 76 145 18 08 1 56 212 01 13 39 106 23 14 43 2 30 Finland 3 000 PEOPLE OR FEWER 19 a7 11 19 2 38 34 15 3 17 13 90 2 7L 2 55 3 001 TO 15 000 PEOPLE 83 193 37 59 3 35 51 79 2 75 24 31 1 87 10 70 15 agi TA cn aan papi ER 112 ne ul pia an aea z uo 36 In the first line of the results shown in Figure 4 6 in Chile there are 53 schools located in villages with 3 000 people or fewer representing an estimated 689 schools in the whole population being located in such an area On average schools located in an area with 3 000 people or fewer in Chile have only 5 47 computers with internet access available The standard error of the mean estimate is 0 62 The standard deviation is 5 62 with a standard error of 1 33 The output s last column also reports the percentage of responses missing for the analysi
41. 006 Technical Report Carstens amp Pelgrum 2009 for more details on national stratification variables and their meaning e IDSTRATE Explicit stratum the school was allocated to Stratification codes are comparable only within but not across education systems e IDSTRATI Implicit stratum based on the sampling forms and plans and as assigned during the selection 18 SITES 2006 IDB USER GUIDE e IDSTRATU Unique information about the implicit stratification that takes the same number for each level of IDSTRATE Secondary analysts are therefore encouraged to use this variable for the purpose of analysis and for grouping schools and teachers sharing implicit stratification Note however that the school and teacher samples selected for SITES 2006 may not have been optimized to yield precise estimates for domains or sub populations that were only used in implicit stratification 2 4 Questionnaire Variables All variables in the SITES 2006 questionnaires can be identified by their variable names The variable names follow a systematic naming convention e The first three digits indicate the source of the data Variable names beginning with BCP indicate data from the Principal Questionnaire variable names beginning with BCT indicate data from the Technical Questionnaire and variable names beginning with BTG indicate data from the Teacher Questionnaire e The fourth and fifth digit represents the question number within the questionnaire
42. 1 For each of the following activities to what extent is pedagogical support available for teachers in Grade lt target grade gt Note Pedagogical support may consist of advice and guidance via persons manuals etc with regard to the activities mentioned below Please do not consider support that is only technical Please mark only one choice in each row 1 2 3 4 5 Notatall A little Somewhat Alot Not applicable Having students produce outcomes of media production projects e g development of websites BCP15A1 Having students work on short projects 2 weeks or shorter BCP15B1 Having students work on extended projects longer than 2 weeks BCP15C1 Having students collaborate with others by online means such as online discussion forums BCP1S5D1 Having students conduct open ended scientific investigations BCP1SE1 Having students engage in field study activities BCP15F1 83 84 SITES 2006 IDB USER GUIDE Obstacles 16 To what extent is your school s capacity to realize its pedagogic
43. 2006 international database regardless of country is hardly ever desirable Analysts will need to add BCPO6A1 for this example To do this select the variable from the variable list on the left hand side of the window and press the right arrow button belonging to the section of the grouping variable This will move the variable BCPO6A1 from the variable list on the left hand side into the field for the grouping variables on the right hand side 5 The weight variable is automatically defined by the software As this is an example for analysis at the school level the weight SCHWGT is selected by default Additionally the jackknifing variables JKCZONE and JKCREP are defined 6 Specify the name and folder of the output files in the Output Files field The IEA IDB Analyzer will use this name and folder to create three output files i an SPSS syntax file that contains the code for performing the analysis ii an SPSS data file with the results and iii an Excel file with these same results 7 Press the Start SPSS button to create the SPSS syntax file and open it in an SPSS syntax window The syntax file will be executed by opening the Run menu of SPSS and selecting the All menu option If necessary the IEA IDB Analyzer will produce a prompt to confirm the overwriting of already existing files 6 For information on the use of replicate weights please refer to the corresponding chapter in the SITES 2006 Technical Report and Chapter 3 in this User Gui
44. AB 9134 NH Catalonia Spain EGH 7241 NH Chile CHL 152 SH Chinese Taipei TWN 158 NH Denmark DNK 208 NH Estonia ESTI 233 NH Finland FIN 246 NH France FRA 250 NH Hong Kong SAR HKG 344 NH Israel ISR 376 NH Italy ITA 380 NH Japan JPN 392 SH Lithuania LTU 440 NH Moscow Russian Federation RUM 6431 NH Norway NOR 578 NH Ontario Province Canada COI 9132 NH Russian Federation RUS 643 NH Singapore SGP 702 SH Slovak Republic SVK 703 NH Slovenia SVN 705 NH South Africa ZAF no SH Thailand THA 764 NH Benchmarking Participant Australia AUS 36 SH Education systems marked with an asterisk are not part of the ISO 3166 1 definition Codes have been defined by the IEA DPC e Hong Kong University Nancy Law co international coordinator and Angela Chow research officer e IEA Data Processing and Research Center Dirk Hastedt Ralph Carstens Falk Brese procedures manuals data collection and processing analysis Olaf Zuehlke sampling e IEA Secretariat Barbara Malak Minkiewicz member relations translation verification Jur Hartenberg financial management e University of Liege Belgium Christian Monseur sampling referee and analysis A Steering Committee provided overall guidance It consisted of Ron Anderson from the University of Minnesota and Alfons ten Brummelhuis from the Foundation for ICT at school in the Netherlands Each participating education system appointed a National Research Coordinator N
45. B Analyzer is used to analyze the data whether or not these have been pre processed with the merge module It can also create code for several analysis procedures Table 4 1 lists the analysis available with the SITES 2006 data using the IEA IDB Analyzer Table 4 1 Types of Analysis for SITES 2006 Data Percentages only Percentages and Means Regression Correlations Computes percentages by subgroups defined by grouping variable s Computes percentages means and standard deviations for selected variables by subgroups defined by the grouping variable s Computes regression coefficients for selected variables predicting a dependent variable by subgroups defined by grouping variable s Computes means standard deviations and correlation for selected variables by subgroups defined by the grouping variable s Depending on the type of analysis chosen the analysis module of the IEA IDB Analyzer requires variables to be selected for the following purposes Grouping Variable s This is the list of variables that are to be used to define the subgroups The list can consist of one or more variables The IEA IDB Analyzer always includes IDCNTRY as the first grouping variable and there should always be at least one grouping variable If the option Exclude Missing from Analysis is checked only cases that have non missing values in the grouping variables will be used in the analysis Note that this box needs to remain checked w
46. BCPO6A1 1 PEDICTAS RESALLOC DECR N of COUNTRY ID STUDENT PER COMP Cases Percent NOT A PRIORITY LOW PRIORITY MEDIUM PRIORITY HIGH PRIORITY Chinese Taipei NOT A PRIORITY LOW PRIORITY MEDIUM PRIORITY HIGH PRIORITY NOT 4 PRIORITY LOW PRIORITY MEDIUM PRIORITY HIGH PRIORITY NOT A PRIORITY LOW PRIORITY MEDIUM PRIORITY HIGH PRIORITY NOT 4 PRIORITY LOW PRIORITY MEDIUM PRIORITY HIGH PRIORITY ANALYSING THE DATA From the first four lines of the results shown in Figure 4 4 it can be seen that in Chile there are 34 schools in the sample where the school principal answered that decreasing the student computer ratio is not a priority The 34 schools in this category represent 319 schools in the whole population in Chile Schools were the decrease of the student computer ratio is not a priority made up 7 11 of Chilean school population The standard error of this percentage is 1 31 4 4 2 Computing Percentages and Means and their Standard Errors To compute percentages and means of continuous variables with their corresponding standard errors researchers will need to select Percentages and Means as type of analysis An example will help to demonstrate this This analysis type computes the percentages of schools within specified subgroups and their mean and standard deviation on the continuous variable selected This analysis type also computes the appropriate standard errors for those percentages means and standard deviatio
47. CP26F 1 Parents community groups initiated the change and I supported its realization BCP26G1 Students initiated the change and I supported its realization BCP26H1 Including this school year how many years have you been Please mark only one choice in each row 1 2 3 4 gt Lessthan 3 5 years 6 10 years 11 20 years 21 years 3 years or more Principal of any school including years as principal in this school BCP27A1 Principal of this school BCP27B1 Working in any professional capacity at this school including years as teacher vice principal and principal BCP27C1 What is your age BCP28A1 30 years or less 31 35 years 36 45 years 46 55 years More than 55 years APPENDIX B SITES 2006 PRINCIPAL QUESTIONNAIRE 29 Please indicate whether you are BCP29A1 1 Female 2 Male 30 Are you involved in fundraising for ICT related matters in your school BCP30A1 Please mark only one choice Yes I personally spend quite some time doing this I am involved in this but another person other people in the school do the major part of the job No we outsource fundraising matters A WwW N me No I and those of my colleagues involved in the school s leadership spend no or very little time
48. CT used 1 2 1 2 No Yes No Yes Written test examination BTG1SA1 BTG15A2 Written task exercise BTG15B1 BTG15B2 Individual oral presentation BTG1SC1 BTG15C2 Group presentation oral written BTG1SD1 BTG15D2 Project report and or multimedia product BTG1SE1 BTG1SE2 Students peer evaluations BTGISF1 BTGISF2 Portfolio learning log BTG15G1 BTG15G2 Assessment of group performance on collaborative tasks BTG15H1 BTG1SH2 115 SITES 2006 IDB USER GUIDE Part IV Student Practice 16 In your teaching of the target class in this school year a How often do your students engage in the following activities b Do your students use ICT for these activities Please mark only one choice for each of the two parts in each row a How often do your students engage b ICT used in the following 1 2 3 4 1 2 Never Sometimes Often Nearly No Yes always A Students working on the same learnings materials at the same pace and or sequence BTG10A1 BTG16A2 B Students learning and or working during lessons at their own pace
49. DSTRATU IMPLICIT STRATUM UNIQUE F icbid 108 Identifier fx fr Achievement Scores uea Dependent Variable TT Achievement Benchmarks Name Description Weight Variable Name Description jira SCHWGT SCHOOL WEIGHT Jackknifing Variables Hame Description a Search SB IKCZONE _ JACKKNIFE ZONE gt 4 gt In Name Description Number OfDecimals 2 x OuputFies EBCG pot only_O4AT 5 Start SPSS e 43 44 SITES 2006 IDB USER GUIDE Figure 4 15 SPSS Output for Example School level Analysis ital Output4 Document4 SPSS Viewer Ele Edt View Data Transform insert Format Analyze Graphs Lilies Add ons Window Help SHAR BE of OLB 0 A Gus tjes OB o Percentages by IDCNTRY BCT0441 1 RESMAT TECHAPP HANDS ON Percent COUNTRY ID MATERIAL Percent s e AVAILABLE NEEDED BUT NOT AVAILABLE NOT NEEDED AND NOT AVAILABLE Chinese Taipei AVAILABLE NEEDED BUT NOT AVAILABLE NOT NEEDED AND NOT AVAILABLE Finland AVAILABLE NEEDED BUT NOT AVAILABLE NOT NEEDED AND NOT AVAILABLE Hong Kong SAR AVAILABLE NEEDED BUT NOT AVAILABLE NOT NEEDED AND NOT AVAILABLE Israel AVAILABLE NEEDED BUT NOT AVAILABLE NOT NEEDED AND NOT AVAILABLE In this example each education system s results are presented in three lines one for each value of the BCT04A1 variable The education systems are identified in the firs
50. GO6A1 Please answer this questions with reference to the subject domain that is specified on the cover page 1 2 3 4 gt Less than 2 4 hrs S5 6 hrs 7 8 hrs More than 8 hrs two hours 109 110 Z Ta JAWS SITES 2006 IDB USER GUIDE What proportion of students in your class has competence in the following Please mark only one choice in each row 1 2 3 4 5 Operation Nearly Some Majority Don t skills none students of students know Word processing BTG07A1 Database software BTG07B1 Spreadsheet BTGO7C1 Presentation software BTGO7D1 Application of multimedia BTGO7E1 E mail BTGO7F1 Internet BTG07G1 Graphic calculator BTGO7H1 Data logging tools BTGO7I1 APPENDIX D SITES 2006 TEACHER QUESTIONNAIRE Part Il Curriculum Goals K In your teaching of the target class in this school year how important is it for you to achieve the following goals Please mark only one choice in each row To prepare students for the world of work BTGO8A1 To prepare students for upper secondary education and beyond BTG08B1 To provide opportunities for students to learn from experts and peers from other schools countries BTG08C1 To pro
51. NDEPENDENT LEARNING PEDAS ASP OF TL SEA PRO PRE INFORMATION PEDAS ASP OF TL CNTRL OWN LEARN PROGRESS PEDAS ASP OF TL LEARNAT THEIR OWN PACE PEDAS ASP OF TL PROJECT BASED LEARNING 2 PEDAS ASP OF TLAVHEN TO TAKE A TEST _ PEDAS ASP OF TL SEARCH STRAT TO FIND INF PEDAS ASP OF TL ASS ORG INTG INFORMATION PEDAS ASP OF TL CRITICALLY EVAL VALIDITY CA g S IDCNTRY COUNTRY ID ja ita l n b p RECAI l Recoded school location Ef EE l PEDAS ASP OF TL SEV FRMS OF PRESENTATION eea PEDAS ASP OF TL SEV PRSNS WORK TOGETHER _Descriptio PEDAS ASP OF TL AUTONOMY TO DECIDE TOPIC PEDAS GOALS PRESCR CURRICULUM CONTENT PEDAS GOALS IMPR PERFORMANCE ON ASS EXAM PEDAS GOALS INDIV LEARNING EXPERIENCE on PEDAS GOALS INCREASE LEARNING MOTIVATION ir PEDAS GOALS FOSTER ABILITY AND READINESS PEDAS GOALS COLLAB AND ORGA SKILLS Achieve i PEDAS GOALS REALWORLD EXAMPLES SET AFPL PEDAS GOALS OPP TO LEARN FROM EXPERTS p PEDAS GOALS FOSTER COMMUNICATION SKILLS PEDAS GOALS RESPONS INTERNET BEHAVIOR D ptio PEDICTAS IMPICT PREPARE FOR FUTURE JOBS x ga PSCHWwGT SCHOOL WEIGHT cae gt vaks ANALYSING THE DATA A printout of the results is presented in Figure 4 9 It performs a linear regression in each education system with the community size variable REC21A1 as a predictor of the student computer ratio RECCSRAT it regresses RECCSRAT on REC21A1 and displays the results for the five selected education system
52. NDIX C SITES 2006 TECHNICAL QUESTIONNAIRE Staff Development 11 Have teachers in your school acquired knowledge and skills in using ICT for teaching and learning in mi oases gt Qa any of the following ways Please mark only one choice in each row Via informal contacts communication BCT 11A1 Via the ICT coordinator or technical assistant BCT11B1 Via in school courses BCT11C1 Via training from a teacher who has attended a course BCT11D1 Via the school s working group or committee for ICT in education BCT11E1 During meetings of the teaching staff where the use of ICT computers in education is a regular item for discussion BCT 11F 1 Via a regular newsletter printed or electronic BCT11G1 Via courses conducted by an external agency or expert in the school or on distance BCT11H1 Via observation of and discussion with colleagues BCT 1111 Via reading professional journals and similar publications BCT11J1 99 SITES 2006 IDB USER GUIDE 12 For each of the following ICT related courses please indicate whether it is available to teachers in your school and who provides the course inside or outside the school Please mark all that apply in each row 1 checked 2 not checked For all variables
53. O FIND IN PEDAS ASP OF TL ASS ORG INTG INFORMATI PEDAS ASP OF TL ORITICALLY EVAL VALIDITY PEDAS ASP OF TL SEV FAMS OF FRESENTATI PEDAS ASP OF TL SEV PRSNS WORK TOGET PEDAS ASP OF TL AUTONOMY TO DECIDE TO TT PEDAS GOALS PRESCA CURRICULUM CONTE BCPO2B1 PEDAS GOALSAMPR PERFORMANCE ON ASS BCPO2CI PEDAS GOALSANDIV LEARNING EXPERIENCE B EAD QB BCPO2FI PEDAS GOALS COLLAB AND ORGA SKILLS BCPI2G1 PEDAS GOALS REALWORLD EXAMPLES SET QP BCPO2HI PEDAS GOALS OPP TO LEARN FROM EXPERTS 31 32 SITES 2006 IDB USER GUIDE Researchers should check the resulting SPSS output file for possible warnings If warnings appear this may indicate that the merge process was not performed properly and the resulting merged data file might not be the one expected 4 4 Performing Analyses with the IEA IDB Analyzer This section describes a few analysis types available within the IEA IDB Analyzer to compute specific statistics with their correct standard errors Making use of these analysis types for analyzing the SITES 2006 data is the best way to ensure that the analyses will be done properly By using the IDB Analyzer sampling weights are always used to analyze the SITES 2006 data and standard errors are correctly computed using the required jackknife repeated replication JRR method The analysis module of the IEA ID
54. RC who together with staff members at their national center was responsible for implementing the study within their education system in accordance with the SITES 2006 guidelines and procedures The quality of the SITES 2006 collection and data was dependent on the work of the NRCs and their colleagues in carrying out the complex sampling translation and data collection tasks with dedication competence and energy 10 SITES 2006 IDB USER GUIDE A complete list of involved persons and their contact details is included in the SITES 2006 international report Law Pelgrum amp Plomp 2008 as well as the SITES 2006 Technical Report Carstens amp Pelgrum 2009 1 2 3 Target Populations and Questionnaires SITES 2006 surveyed schools that enrolled students in the target grade that represents the 8th year of schooling counting from the first year of ISCED Level 1 The target population included school principals ICT coordinators and mathematics and science teachers from those schools For a detailed description of the target populations see Chapter 6 in the SITES 2006 Technical Report Carstens amp Pelgrum 2009 On average 400 schools enrolling students in the target grade and four teachers per school two mathematics and two science teachers were surveyed in each participating education system The study administered three questionnaires to respondents in schools a school questionnaire to the principals a technical questionnaire t
55. Remarks AA AE A essence tsa tend bad enacted tee quence NCQO092 10 What is the method for evaluating student progress in order to decide on promoting students from the target grade to the next grade level Tick all that apply a National examination NCQU1O 1a wicnnnnmecmncnnemnemenememueeieinienieutnientettt b School internal examination NCQU01D wu eesmnmsnnnennennannennenennuannanaaanstt c Oral and or written examinations throughout the school year NCQ101c d Portfolio of student work during the school year NCQUOI1d oem e Other NCOI IG en rere ee Explanations as needed Deenen 0 11 Would all students in the target grade be required to attain specific standards in none all or only some of the school subjects in order to be promoted to the next grade level Tick all that apply Noisubj cts NGOI aeiiicmiiaisiaininnena a All school subjects NCQ111b z Only some subjects NGCOQIIIE ninsis innn APPENDIX A SITES 2006 NATIONAL CONTEXT QUESTIONNAIRE 12 13 11 1 Ifyou have answered only some subjects please indicate if these subjects are any of the following by ticking the boxes as appropriate Mother Tongue NCO LIZA isens Mathematics NCO LI 2 Diinan ORR Science NEO LIZE eisime on ee eee ee eae ee Remarks if any Seetee ANCOL Does your system have a system wide curriculum that includes mathematics at the ta
56. S 2006 TEACHER QUESTIONNAIRE 20 To what extent has the use of ICT impacted your students in the target class in the following areas fa 4 mo O Z a hr Please mark only one choice in each row Subject matter knowledge BTG20A1 Dependent Learning motivation BTG20B1 Dependent Information handling skills BTG20C1 Dependent Problem solving skills BTG20D1 Dependent Self directed learning skills BTG20E1 Dependent Collaborative skills BTG20F1 Dependent Communication skills BTG20G1 Dependent ICT skills BTG20H1 Dependent Ability to learn at their own pace BTG2011 Dependent Self esteem BTG20J1 Dependent Achievement gap among students BTG20K1 Dependent Time spent on learning BTG20L1 Dependent School attendance BTG20M1 Dependent Assessment results BTG20N1 Dependent Digital divide i e inequity between students from different socioeconomic backgrounds BTG2001 Dependent 1 Decreased Decreased alot 2 a little 3 No impact 4 Increased Increased a little bs alot
57. Second Information Technology in Education Study SITES 2006 User Guide for the International Database ye gt Second Information Technology in Education Study SITES 2006 User Guide for the International Database Edited by Falk Brese Ralph Carstens Contributors Eugenio J Gonzalez Juliane Hencke Olaf Zuehlke TEA International Association for the Evaluation of Educational Achievement 2009 International Association for the Evaluation of Educational Achievement IEA SITES 2006 User Guide for the International Database edited by Falk Brese and Ralph Carstens Publisher IEA Secretariat Amsterdam the Netherlands ISBN 978 90 79549 03 0 For more information about the IEA SITES 2006 International Database contact IEA Data Processing and Research Center Mexikoring 37 22297 Hamburg Germany email sites iea dpc de Contents Overview of the SITES 2006 International Database 1 1 Introduction 1 2 An Overview of SITES 2006 1 2 1 Participating Education Systems 1 2 2 Management 1 2 3 Target Populations and Questionnaires 1 2 4 Sampling and Survey Operations 1 2 5 Quality of Samples and International Reporting The SITES 2006 Data Files 2 1 Data Files Codebooks and Formats 2 2 Cases Included 2 3 Survey Variables 2 3 1 Identification Variables 2 3 2 Tracking and Administrative Variables 2 3 3 Sampling and Weighting Variables 2 4 Questionnaire Variables 2 5 Coding Missing Data 2 5 1 Not Admin
58. TES 2006 IDB USER GUIDE Introduction The Second Information Technology in Education Study SITES 2006 is an international assessment of teaching and learning practices and of how Information and Communication Technologies ICT support these in secondary schools around the world Approximately 20 countries will provide information from representative samples of teachers on how they organize their teaching and learning the ICT facilities they have available at school how they use ICT for teaching and learning and the obstacles or difficulties they experience in relation to these technologies This information will give better insight into the current state of pedagogical approaches and of how technologies support them It will also allow educational practitioners and policy makers to gain a better understanding of areas needing intervention and additional support Name of country along with about 20 other countries is taking part in this international study of pedagogical practices and the way that ICT supports these The study is being conducted under the auspices of the International Association for the Evaluation of Educational Achievement IEA We are asking you for your help in order to determine the current state of pedagogical approaches to and the use of ICT Please try to answer each question as accurately as you can Confidentiality All information that is collected in this study will be treated confidentially At no time will
59. TG39B1 Dependent Learning motivation BTG39C1 Dependent Ability to learn at own pace BTG39D 1 Dependent Communication skills BTG39E1 Dependent Information handling skills BTG39F 1 Dependent Collaborative skills BTG39G1 Dependent Self directed learning skills BTG39H1 Dependent Problem solving skills BTG391I1 Dependent Achievement gap among students BTG39J1 Dependent Self esteem BTG39K1 Dependent APPENDIX D SITES 2006 TEACHER QUESTIONNAIRE 40 Has the use of ICT in this pedagogical practice contributed to changes in the following aspects of K your teaching of the target class Please mark only one choice in each row Quality of coaching students BTG40A1 Dependent Time available to help individual students BTG40B1 Dependent Time needed to solve technical problems BTG40C1 Dependent Time needed for preparation BTG40D1 Dependent Quality of instructions given to students BTG40E1 Dependent Time needed for classroom management BTG40F1 Dependent Quality of classroom discussion BTG40G1 Dependent Collaboration between students BTG40H1 Dependent Communication with the outside world BTG4011 Dependent Availability of new learning content BTG40J1 Dependent Variety of learning resources materials BTG40K1 Dependent Varie
60. Table 4 5 Percentages standard errors of schools in which common types of technology applications and facilities were available s Communi A Mail Mail r Tutorial General Multimedia Digital Mobile Smart Equipment i Simulation cation LMS accounts accounts software software production resources devices board e er ecko software teachers students 23 Alberta Province Canada 8 3 0 68 4 0 100 0 0 81 8 1 44 3 6 43 4 0 76 3 5 87 3 1 22 34 47 4 1 48 G1 95 1 5 53 4 3 Catalonia Spain BBL 5E 9907 UEI BEH GY VUY 8707 AA O A 4EY BUY HQ l Chile 47 1 8 45 Q2 90 1 3 54 R4 63 2 1 48 2 2 79 1 8 72 19 13 1 5 06 1 3 39 Q4 68 2 0 32 2 2 T E E S O E E E S E CAG CAS RA ER A Finland 96 1 2 66 2 8 99 0 7 77 2 6 64 3 2 20 2 4 92 1 7 78 26 11 21 10 18 4627 97 1 1 59 2 8 mogKogsat TL RGY 00H NAY TAH IEY WOD BA WED WED AAD AH WEY Israel 70 2 9 46 3 3 96 1 3 44 81 55 3 6 16 2 1 84 2 5 53 3 1 13 1 7 08 1 5 46 3 0 54 3 1 40 3 1 E COIS UN O EA EN OA A A A E S A OER Japan 94 1 3 58 5 97 0 7 76 2 4 2 2 1 39 2 2 62 2 5 51 24 03 0 8 20 22 35 R4 36 2 2 22 2 1 Litwana GI UEY VED OMY mas Moscow Russian Federation 65 2 6 65 24 81 2 1 47 2 9 24 24 24 2 4 81 2 0 55 2 6 26 2 3 21 1 9 09 1 4 53 2 5 38 2 3 Ontario Province Canada 81 2 5 78 27 99 05 BH IBO SIGA GGA W 09 20 21 23 54 GO 100002
61. a and N is uncluded to indicate the Differences barween male teachers and female teachers is significant at p lt 0 01 size of the actual sample within each gender School participation rate before including replacement schools is below 85 Value labels for the response categories 1 never 2 sometimes 3 often 4 uearly always School participation rate after including replacement schools is below 85 School participation rate after including replacement schools is below 709 Tescher participarion data were collected after survey administration International procedures for target class selection was not followed in all schools Nationally defined population covers Jess than 90 of the nationally desired population As with the previous analysis the first step is to identify the variables relevant to the analysis in the appropriate files and review the documentation for any specific cultural and national adaptations to the questions of interest Appendix D of the SITES 2006 Technical Report Since the example uses a teacher level variable the relevant variables can be found in the mathematics teacher file From this file the variable that contains the information on the teachers gender BTG32A1 and the variable that contains the information on teachers ICT usage in the target class BTG18A1 are extracted For combining the data from the desired education systems this example will use the merge module of IEA IDB Analyzer The mathe
62. above the diagonal of the correlation matrix are displayed with their corresponding standard errors ANALYSING THE DATA Figure 4 11 SPSS Output for Correlations itz BCG_corr_06G1 06H1 spo Document6 SPSS Viewer Fie Edit View Data Transform Insert Format Analyze Graphs Utilities Add ons Window Help Stella c cl Delti Ole S l 4l l 216 21213 B E Output Correlation matrix for IDCNTRY 152 H bog Variable Sum of Wgts e e Correlations and s e BCPO6G1 4510 87 1 0000 0000 5919 BCPO6H1 4510 87 Correlation matrix for IDCNTRY 156 Variable Sum of Wgts ee A E BCPO6G1 623 50 1 0000 0000 5544 BCPO6H1 823 50 SS a a fe Correlation matrix for IDCNTRY 246 Sum of Wgts e e Correlations and s e BCPO6G1 596 98 1 0000 0000 8273 BCPO6H1 596 98 Correlation matrix for IDCNTRY 344 Sum of Wgts BCPO6G1 Fj 1 0000 0000 5803 BCPO6H1 Correlation matrix for IDCNTRY 376 Variable Sum of Wgts BCPO6G1 632 49 1 0000 0000 6069 BCPO6H1 632 49 The results displayed in Figure 4 11 indicate that the correlation between the two variables in the analysis for the first group IDCNTRY 152 Chile is 0 5919 whereas the standard error is 0 0621 For the second group IDCNTRY 158 Chinese Taipei the correlation between these variables is 0 5544 and the standard error for the correlation
63. al goals hindered by each of the following obstacles Please mark only one choice in each row 1 2 3 4 5 ICT related obstacles Notatall A little Somewhat Alot Not applicable A Insufficient qualified technical personnel to support the use of ICT BCP16A1 B Insufficient number of computers connected to the Internet BCP16B1 C Insufficient Internet bandwidth or speed BCP16C1 D Lack of special ICT equipment for disabled students BCP16D1 E Insufficient ICT equipment for instruction BCP16E1 F Computers are out of date BCP16F1 C C G Not enough digital educational resources for instruction BCP16G1 H Lack of ICT tools for science laboratory work BCP16H1 I Teachers lack of ICT skills BCP 1611 J Insufficient time for teachers to use ICT BCP 16J1 Other obstacles K Pressure to score highly on standardized tests BCP16K1 L Prescribed curricula are too strict BCP16L1 M Insufficient or inappropriate space to accommodate the school s pedagogical approaches BCP16M1 N Insufficient budget for non ICT supplies e g paper pencils BCP16N1 O Us
64. ally Not Applicable if e The previous filter question was answered in a way that made a response to dependent questions logically impossible and the dependent questions were validly skipped This code was assigned at the IEA DPC after data collection Logically not applicable responses are coded in the SPSS files as 6 96 996 and so on depending on the field length of the variable The code for logically not applicable responses in SAS files is B Attention Point Depending on the analysis attempted it is recommended that analysts review the questionnaire to determine whether any of their included questions are dependent on previous responses as in such a case coverage may be reduced It might then be advisable to filter the dataset and only select respondents to which the question applied 2 5 4 Not Reached A special missing code was assigned to questions that were deemed Not Reached to distinguish them from Omitted responses Omitted questions are those that a respondent most certainly read but either consciously decided not to answer or accidentally skipped Not reached codes are assigned for questions the respondent did not reach that is the respondent started answering the questionnaire but stopped answering before he or she reached the end of the questionnaire most likely due to a lack of time Not reached variables are exclusively located towards the end of questionnaire Before
65. and Communication Technologies are used interchangeably in this questionnaire e Please note that some questions refer to the entire school other questions refer to Grades lt grade range gt while some questions pertain to Grade lt target grade gt only For countries in which the definition of school is not obvious to respondents add appropriate description depending on how sampling units were defined in the national sampling plan When questions refer to your school we mean by school lt national school definition gt e Guidelines for answering the questions are typed in italics Most questions can be answered by marking the one most appropriate answer When a question states Please mark all that apply you may give more than one answer e Ifyou are completing the paper version of this questionnaire please use a writing pen or ballpoint to write your answers e When you have completed this questionnaire please National Return Procedures and Date 93 94 SITES 2006 IDB USER GUIDE Further information e When in doubt about any aspect of the questionnaire or if you would like more information about it or the study you can reach us by phone at the following numbers National Center Contact Information Thank you very much for your cooperation APPENDIX C SITES 2006 TECHNICAL QUESTIONNAIRE An A UNE mi yopaw gt ICT in Your School How many years has your school been using ICT for teach
66. apply a Central government NC OQOT I a aiasicsicdionnsssauinad sotsunaeitonisietitnanssnnnnatinetas b State and or provincial government NCQO11D ou eseenmnnnnnnnnnnnnnnnneiinn c District and or Local government NCQOI Ic 0 0 d Non statutory and or professional body NCQO1 1d e Schools are free to decide NCQO11e weenie f Other Please specify WOOGIE niii NCQO011ft If you selected more than one of the above answers please answer 1 1 and 1 2 1 1 Which level has the primary most responsibility for these policies Enter a to fi s NCQO12 1 2 Which level has the secondary second most responsibility for these policies Enter a to f or none us NCQ013 Explanations as needed E E E E E A E AE A NCQ014 Which government levels take the responsibility for monitoring and implementing the examinations for compulsory education Please Tick all that apply a Central government NCQ021 a revscoierocanene n b State and or provincial government NCQO21b c District and or Local government NCQO021c nsi d Non statutory and or professional body NCQ021d ssis e Schools are free to decide NCQO2 Le wesnnncnicnnenneneiemienieinenieinietienienienenienie f Other Please specify NCQO21f ii NCQO21ft If you selected more than one of the above answers please answer 2 1 and 2 2 2 1 Which level has the primary most responsibility for these p
67. are made about schools that have aggregated teacher information attached SCHWGT should be used e It is neither possible nor meaningful to combine the files of mathematics teachers and science teachers and to make statements about teachers who teach mathematics or science or both for two key reasons First teachers of mathematics and science constitute two separate target populations and the questionnaires were administered within the subject areas Secondly some teachers taught both subjects and consequently could and actually were selected twice This is not reflected by the sampling weights e When analyzing weighted school or teacher data for groups of education systems the researchers must be aware of a pitfall If the education system identification variable IDCNTRY is used as a grouping variable all weighted results are correct within each education system However if any international average gets directly computed using SCHWGT MTOTWGT or STOTWGT larger education systems contribute more to this average than smaller education systems Usually this is not the intention of the researcher though it might just well be Instead of performing weighted analyses across groups of education systems it would be necessary to do weighted analyses separately for each education system and to calculate an average of these results afterwards see also the section below about the senate weight 3 2 3 Example for Analyzing Weighted Data No
68. base contains variables that allow for the use of a variance estimation method called Jackknife Repeated Replication JRR These variables are referred to as jackknife zones and as jackknife replicates With the IDB Analyzer software it is possible to implement the JRR method easily for details about the JRR technique please refer to Chapter 10 of the SITES 2006 Technical Report 3 4 2 Selecting the Appropriate Variance Estimation Variables For school level analysis the variables to use are JECZONE and JKCREP For teacher level analysis the variables to use are JKZONE and JKREP In most education systems the teachers in a school are assigned to the same jackknife zone as the school itself In some of the smaller participating education systems schools and teachers had to be assigned to different zones Just as with weights researchers should choose the correct jackknife variables when working with combined data sets When the calculations are performed with the IDB Analyzer the correct variables will be selected automatically If other software is used zone variables usually have to be specified as stratum or strata variables while the rep variables commonly are referred to as cluster variables Attention Point To limit the size and complexity of the IDB the data files do not include the 100 replicate weight variables used for variance estimation The IEA IDB Analyzer computes the replica
69. bers National Center Contact Information Thank you very much for your cooperation 107 108 SITES 2006 IDB USER GUIDE Subject and Target Class References When a question refers to the target class please think only about the class course you are teaching in this school year that is specified on the cover page You will answer all questions with reference to the teaching of the subject domain that is specified on the cover page in this class APPENDIX D SITES 2006 TEACHER QUESTIONNAIRE Part I Information about the Target Class 1 How many students are there in the target class BTGOIA1 2 What is the gender mix of this class BTG02A1 1 2 3 All boys All girls Both boys and girls 3 Which curriculum track is the target class in BTGO3A1 1 2 3 Academic Vocational No tracking 4 Approximately what percentage of students are absent in the target class on a typical school day BTG04A1 1 2 3 4 Less than 5 S 10 11 20 More than 20 5 Approximately what percentage of students in the target class are native speakers of the language of instruction BTGOSA1 1 2 3 4 More than 76 90 50 75 Less than 90 50 6 Howmany hours of scheduled class time do you spend with the target class on Mathematics Science lessons per week BT
70. bove BCT 13F1 Which of the following duties do you have Please mark only one choice in each row I teach ICT courses to students BCT 14A1 I teach ICT courses to teachers and other school staff BCT14B1 I teach Mathematics and or Science BCT 14C1 I teach other subjects BCT14D1 I formally serve as ICT coordinator BCT 14E1 I informally serve as ICT coordinator BCT 14F1 1 2 No Yes 1 2 No Yes Approximately how many 60 minute periods on average per week do the following persons spend on providing ICT support to teachers and students at your school Note Support includes any services formal or informal technical or pedagogical that help teachers and students use ICT Please write a whole number Write O zero if none Yourself BCT15A1 ICT staff not including yourself BCT15B1 Teachers BCT15D1 Personnel from external companies BCT15G1 Others BCT15H1 Other administrators and staff e g media specialist BCT15C1 Students from own school who are assigned to provide this service BCT15E1 Volunteers from outside the school e g parents BCT15F1 101 102 SITES 2006 IDB USER GUIDE 16 To what extent is technical support available in your school if teachers want to use ICT for th
71. c investigations BCPO8E1 Undertaking field study activities BCPO8F1 Using virtual laboratories simulations BCPO8G1 Applying exercises to practice skills and procedures BCPO8H1 Involving students in laboratory experiments with clear instructions and well defined outcomes BCPO8I1 Involving students in studying natural phenomena through simulations BCP08J1 Involving students in processing and analyzing data BCPO8K1 77 SITES 2006 IDB USER GUIDE During this school year how often did the school leadership you and or other school leaders undertake each of the following Please mark only one choice in each row 1 2 3 4 Notat All Afewtimes Monthly Weekly Organize activities to develop a common vision of what is meant by quality education BCPO9A1 Inform teachers about pedagogical changes taking place in the school BCP09B1 Inform teachers about educational developments outside the school BCPO9C1 Consult teachers about desired pedagogical changes BCPO9D1 Discuss with teachers what they want to achieve through their lessons BCPO9E1 Motivate teachers to critically assess their own
72. ciples and concepts BTGO09J1 BTG 09J2 Studying natural phenomena through simulations BTGO9K1 BTGO9K2 Looking up ideas and information BTGO9L1 BTG 09L2 Processing and analyzing data BTG09M1 BTG 09M2 APPENDIX D SITES 2006 TEACHER QUESTIONNAIRE 10 11 12 When I am instructing students in the target class excluding field trips they are BTG10A1 Please mark only one choice 1 2 3 4 Always in the same Sometimes in Often in locations Always in locations location with me locations away from me way from me away from me When students in the target class participate in planned learning activities they BTG11A1 Please mark only one choice 1 2 3 4 Always work in Sometimes work in Often work in Always work in the same location ifferent locations different locations different locations The learning activities for students in the target class are planned so that these take place BTG12A1 Please mark only one choice l 2 3 4 Always during Sometimes outside Often outside At any time scheduled school hours scheduled school hours scheduled school hours no scheduled school hours 13 I provide feedback to students
73. coordinate the implementation of innovations in teachers teaching and learning BCP07F1 Changing class schedules to facilitate the implementation of innovations BCP07G1 Implementing incentive schemes to encourage teachers to integrate ICT in their lessons BCPO7H1 Encouraging teachers collaborate with external experts to improve their teaching and learning practices BCPO7I1 Featuring new instructional methods in the school newspaper and or other me dia e g the school website BCP07J1 Involving parents in ICT related activities BCPO7K1 SITES 2006 IDB USER GUIDE APPENDIX B SITES 2006 PRINCIPAL QUESTIONNAIRE K To what extent do you agree or disagree that the school leadership you and or other school leaders encourages teachers in Grade lt target grade gt to undertake the following activities Please mark only one choice in each row 1 2 3 4 Strongly Disagree Agree Strongly disagree agree Assigning extended projects 2 weeks or longer BCPO8A1 Assigning short task projects BCP08B1 Assigning production projects e g making models or reports BCPO8C1 Involving students in self accessed courses and or learning activities BCP08D1 Involving students in open ended scientifi
74. coordinators within schools within an education system is simply the combination of the school ID IDSCHOOL and the sequential number 01 IDTEACH Five to eight digit numeric code to identify teachers within schools It is composed of the school ID IDSCHOOL the teacher population identification 01 for mathematics teachers and 02 for science teachers and a sequential two digit number within each school IDPOP Identifies the population For SITES 2006 the value is always 2 indicating secondary education This is similar to the population coding in previous IEA studies 2 3 2 Tracking and Administrative Variables The SITES 2006 international database includes several variables that provide information about survey administration participation and some basic characteristics of respondents The following variables are used for this purpose in the school and teacher level files PSYSTEM CSYSTEM TSYSTEM Numeric variable uniquely identifying each record in the original data files provided by participating education systems PSYSTEM was used for records from the School Questionnaire data files CSYSTEM for records from the Technical Questionnaire data files and TSYSTEM for records from the Teacher Questionnaire data files An offset of 5 000 was added to the sequential numbers if the questionnaire was answered on line An offset of 10 000 was added if the information about the assigned questionnaire mode was omitte
75. d PITPART CITPART ITPART These variables provide the final participation information for school principals PITPART ICT coordinators CITPART and teachers ITPART PITODC CITODC ITODC Indication of the questionnaire administration mode for school principals PITODC ICT coordinators CITODC and teachers ITODC The value was set to 0 for questionnaire administered on paper and 1 for questionnaires administered online INSIT06 Indication if the record belongs to the SITES 2006 international sample In the final international database this variable takes the values 1 for all records VERSION Release version number of the SITES 2006 international database DPCDATE File creation date The teacher data files BTM BTS consist of additional variables providing information about teachers and further details about the administration ITSEX Teacher gender information as stated in the Teacher Tracking Form If the information was not available on the Tracking Form but the teacher provided the gender information in the questionnaire question BTG32A1 the questionnaire information was copied over ITBIRTHY Teacher birth year information as stated in the Teacher Tracking Form ITEXCLUD Variable indicating if a teacher was excluded from the survey In the final IDB this variable takes the value 9 not excluded for all records ITLANG Language used for the Teacher Questionnaire Ifan education system only used o
76. d or others access to school computers and or the Internet BCPOSI1 Complementing printed lesson materials with digital resources for teaching and learning BCPOSJ1 Providing teachers with laptop computers and or other mobile learning devices BCPOSK1 Providing students with laptop computers and or other mobile learning devices BCPOSL1 SITES 2006 IDB USER GUIDE APPENDIX B SITES 2006 PRINCIPAL QUESTIONNAIRE K What priority level do you give to resource allocation in your school in order to enhance the use of ICT in teaching and learning for the Grade lt target grade gt students in your school Please mark only one choice in each row 1 2 3 4 Nota Low Medium High priority priority priority priority To decrease the number of students per computer BCPO6A1 To increase the number of computers connected to the Internet BCPO6B1 To increase the bandwidth for Internet access of the computers connected to the Internet BCPO6C1 To increase the range of digital learning resources related to the school curriculum BCP06D1 To establish enhance an online learning support platform and its management so that teaching and
77. ddress different learning needs BCP02C1 D To increase learning motivation and make learning more interesting BCP02D1 E To foster students ability and readiness to set own learning goals and to plan monitor and evaluate own progress BCPO2E1 F To foster collaborative and organizational skills when working in teams BCP02F1 G To provide activities which incorporate real world examples settings applications for student learning BCP02G1 H To provide opportunities for students to learn from experts and peers from other schools organizations countries BCP02H1 I To foster communication skills in face to face and or on line situations BCP02I11 J To prepare students for responsible Internet behavior e g not to commit mail bombing such as spam etc and or to cope with cybercrime e g Internet fraud illegal access to secure information etc BCP02J1 vA 72 SITES 2006 IDB USER GUIDE Pedagogy and ICT in your school This section asks you to answer questions about pedagogy and ICT in your school For each of the following how important is the use of ICT at Grade lt target grade gt in your school Please mark only one choice in each row To prepare students for t
78. de 33 SITES 2006 IDB USER GUIDE Figure 4 3 Analysis Module Computing Percentages only TSCHOOL PRINCIPAL ID S DONTRY COUNTRY ID TECHNICAL COORDINATOR ID B BCPOSA1 PEDICTAS RESALLOC DECR STUDENT PER POPULATION ID IN SITES 2006 POPULATION _ EXPLICIT STRATUM L AMPUCIT STRATUM IMPLICIT STRATUM UNIQUES IDB Identifier PEDAS ASP OF TLANDEPENDENT LEARNING PEDAS ASP OF TL SEA PRO PRE INFORMATION PEDAS ASP OF TL CNTRL OWN LEARN PROGRESS PEDAS ASP OF TL LEARN AT THEIR OWN PACE PEDAS ASP OF TL PROJECT BASED LEARNING PEDAS ASP OF TL SEARCH STRAT TO FIND INF PEDAS ASP OF TL ASS ORG INTG INFORMATION PEDAS ASP OF TL CRITICALLY EVAL VALIDITY _PEDAS ASP OF TL SEV FRMS OF PRESENTATION PEDAS ASP OF TL SEV PRSNS WORK TOGETHER PEDAS ASP OF TL AUTONOMY TO DECIDE TOPIC 1 PEDAS GOALS PRESCA CURRICULUM CONTENT PEDAS GOALS IMPR PERFORMANCE ON ASS EXAM PEDAS GOALS INDIV LEARNING EXPE A printout of the results is presented in Figure 4 4 It produces the percentages of school principals answering a decrease of the student computer ratio has no low medium or high priority for the selected five education systems Figure 4 4 SPSS Output for Percentages only Output 1 Document1 SPSS Viewer loj x Ble Edt View Daa Transform isei Formet Analyze Graphs Lies Add cns Window Help OO 8 Gwe feo MM eet Percentages by IDCNTRY
79. dentified While results will be made available by country and by type of school within a country you are guaranteed that neither your school nor any of its personnel will be identified in any report of the results of the study For countries which have ethical survey guidelines which emphasize voluntary participation Participation in this survey is voluntary and any individual may withdraw at any time About this Questionnaire e This questionnaire asks for information from teachers about education and policy matters related to pedagogical practices and computers The questionnaire will take you approximately 30 minutes to complete e The words computers and ICT Information and Communication Technologies are used interchangeably in this questionnaire e Guidelines for answering the questions are typed in italics e Most questions can be answered by marking the one most appropriate answer A few questions 9 14 15 and 16 require responses to two parts a and b Mark one most appropriate answer for each of the two parts in each row e Ifyou are completing a paper version of this questionnaire please use a writing pen or ballpoint to write your answers e When you have completed this questionnaire please National Return Procedures and Return Date Further information e When in doubt about any aspect of the questionnaire or if you would like more information about it or the study you can reach us by phone at the following num
80. describes the use of the IEA International Database IDB Analyzer software IEA 2009 for analyzing the SITES 2006 international data files Example analyses will illustrate the capabilities of the IEA IDB Analyzer to compute a variety of statistics including percentages means regression coefficients and correlations and their corresponding standard errors The examples use school and mathematics teacher data to replicate some of the SITES 2006 results included in the main international publication for SITES 2006 Pedagogical practices and ICT use around the world findings from an international comparative study Law Pelgrum amp Plomp 2008 as well as other useful analyses for investigating policy relevant research questions 4 2 The IEA IDB Analyzer Developed by the IEA Data Processing and Research Center IEA DPC the IEA IDB Analyzer is a stand alone software that operates in conjunction with the Statistical Package for the Social Sciences SPSS 2008 The IEA IDB Analyzer enables users to combine SPSS data files from IEA s large scale assessments and conduct analyses using SPSS without actually writing syntax code The IEA IDB Analyzer generates SPSS syntax that takes into account information from the sampling design in the computation of statistics and their standard errors In addition the generated SPSS syntax correctly handles plausible values multiple imputation for calculating estimates of achievement scores and their cor
81. e K following activities Please mark only one choice in each row 1 3 4 No support Some Extensive Not support support applicable Assigning extended projects 2 weeks or longer BCT16A1 Assigning short task projects BCT16B1 Assigning production projects e g making models or reports BCT16C1 Involving students in self accessed courses and or learning activities BCT16D1 Involving students in scientific investigations open ended BCT16E1 Undertaking field study activities BCT16F1 Using virtual laboratories simulations BCT16G1 Applying exercises to practice skills and procedures BCT16H1 Involving students in laboratory experiments with clear instructions and well defined outcomes BCT16I1 Involving students in studying natural phenomena through simulations BCT16J1 Involving students in processing and analyzing data BCT16K1 APPENDIX C SITES 2006 TECHNICAL QUESTIONNAIRE Obstacles 17 To what extent is your school s capacity to realize its pedagogical goals hindered by each of the following obstacles Please mark only one choice in each row A Insufficient qualified technical personnel to support t
82. e The sixth digit indicates the dimension of the question For single item questions the letter A is used If a question is further divided into several items letters indicate the items in alphabetic order e If questions are even further divided e g in complex matrix questions like Question 12 in the Technical Questionnaire the seventh digit sequentially identifies the sub items If there is no such further division this digit is set to 1 2 5 Coding Missing Data A response to a question can be missing for one or more reasons The question could have been excluded from the questionnaire Not Administered the respondent may have chosen not to respond to the question Omitted response or simply did not have time to reach the question in the instrument Not Reached response SITES 2006 did not use a special code to indicate invalid responses These kinds of missing data are handled differently during data processing and therefore are coded differently in the data files Depending on the file format different codes are used to represent the missing data The following missing codes are used in the data files 2 5 1 Not Administered Questions A response to a question is coded as Not Administered in the following cases e Ifareturned questionnaire is empty ora lost all variables referring to that questionnaire are coded as not administered For online questionnaires this applies to respondents who never logged i
83. e School level Analysis SPSS Output for Example School level Analysis Table 6 2 from the SITES 2006 International Report for the Example Teacher level Analysis Law Pelgrum amp Plomp 2008 pp 187 Merge Module for Example Teacher level Analysis Analysis Module for Example Teacher level Analysis SPSS Output for Example Teacher level Analysis Allocation of School Sample in Australia Participation Rates in Australia 20 24 25 27 27 31 31 32 34 34 36 36 37 38 39 40 41 42 43 43 44 45 46 47 47 129 130 Overview of the SITES 2006 International Database Falk Brese Ralph Carstens 1 1 Introduction To support and promote secondary analyses the International Association for the Evaluation of Educational Achievement IEA is making the SITES 2006 international database and accompanying User Guide available to researchers analysts and public users The database comprises national contexts and school and teacher level data from 23 education systems all of which gave IEA permission to release their national data It includes information from roughly 8 000 schools including more than 13 000 mathematics and over 16 000 science teachers The SITES 2006 User Guide describes the content format and usage of the SITES 2006 international database Chapter 2 describes the structure of the database in detail including information about the response sampling and derived variables and their coding Chapter 3 deta
84. e it will be stored in the Output Files field The IEA IDB Analyzer also will create an SPSS syntax file SPS of the same name and in the same folder with the code necessary to perform the merge In the example shown in Figure 4 2 the merged file BCGALL SAV and the syntax file BCGALL SPS will both be stored in the root directory of your computer The merged data file will contain school data with the variables shown in the Selected Variables panel to the right ANALYSING THE DATA 8 Click on the Start SPSS button to create the SPSS syntax file and open it in an SPSS syntax window ready for execution The syntax file must be executed by opening the Run menu of SPSS and selecting the All menu option The IEA IDB Analyzer will give a warning if it is about to overwrite an existing file in the specified folder Figure 4 1 Merge Module Selecting Participants cs gf Chile Chinese Taipei TU Lithuania Pussian Federation Moscow Singepore SVK Slovak Republic _ Slovenia Figure 4 2 Merge Module Selecting File Types and Variables PIDB Analyzer I1 Merge Module PEDAS ASP OF TL NDEPENDENT LEARNING PEDAS ASP OF TL SEA PRO PRE INFORMATIO _PEDAS ASP OF TL CNTRL OWN LEARN PROG PEDAS ASP OF TL LEARN AT THEIR OWN PAC PEDAS ASP OF TL PROJECT BASED LEARNING Fl PEDAS ASP OF TLAVHEN TO TAKE A TEST 1 PEDAS ASP OF TL SEARCH STRAT T
85. en INFIHOW MANY STUDENTS IN 11971 31 81 lt D TARGET CLASS Valid N listwise 11971 Using the JRR technique with the IDB Analyzer it is apparent that the correct estimate for the actual standard error is almost three times as large Figure 3 4 Example of Correct Variance Estimation using the IDB Analyzer Average for BTGO141 by IDCNTRY N of Sum of BTGOLAL BTGO1AL COUNTRY ID Cases MNTOTUGT Mean s e Thailand 664 11971 31 61 This difference could be explained by the fact that classrooms within a school usually have a similar size Teachers from the same school are likely to give similar answers to this question This effect is not taken into account by SPSS directly but by the IDB Analyzer References Carstens R amp Pelgrum WJ Eds 2009 IEA SITES 2006 Technical Report Amsterdam International Association for the Evaluation of Educational Achievement SAS Institute Inc 2008 SAS 9 2 Computer software Cary NC SAS Institute Inc Information retrieved from http www sas com software sas9 on February 23 2009 Westat Inc 2008 Wesvar Version 5 1 Replication Based Variance Estimation for Analysis of Complex Survey Data Computer software Rockville MD Westat Inc Retrieved from http www westat com westat statistical_ software wesvar on February 23 2009 27 Analyzing the SITES 2006 Data Using the IEA IDB Analyzer Juliane Hencke Eugenio J Gonzales 4 1 Overview This chapter
86. entered pedagogies NCQ29b c online learning or distance education NCQ29c d connecting with other schools and cultures NCQ29d e collaborative team learning NCQ29e f communication and presentation NCQ29f Do any of your educational system s policy documents promote approaches that mention 21st Century skills The SITES2006 Conceptual Framework document defines 21st Centuryskillsas having two components collaborative inquiry and connectedness Yes No 21st Century skills NCQ301 If yes please describe the policy and how it relates to ICT Also describe how 21st Century skills are defined And include a website address for any relevant documents Please list other useful website addresses with key information for understanding either the general education policies or the ICT in education policies and practices in your country and or system Are there any other things that we should know about your educational system for purposes of SITES2006 analysis Thank you very much for your time and effort in completing this questionnaire for SITES2006 It will contribute greatly to the study 65 Appendix B SITES 2006 Principal B Questionnaire SITES 2006 Second Information Technology in Education Study Principal Questionnaire International English Version IEA International Association for the Evaluation of Educational Achievement Copyright 2005 2006 68 SI
87. erent subjects BCP13E1 F Managing the adoption of ICT supported methods for assessing student progress BCP 13F 1 G Organizing cooperation with other schools regarding the development of teaching and learning materials BCP 13G1 H Organizing cooperation with other schools regarding the development of ICT based teaching and learning BCP13H1 I Promoting the integration of ICT in the teaching and learning of traditional subjects BCP 1311 use in teaching and learning BCP13J1 APPENDIX B SITES 2006 PRINCIPAL QUESTIONNAIRE Pedagogical Support for Persons Using ICT 14 How frequently does each of the following persons provide pedagogical support to those teachers in w o a we Grade lt target grade gt who want to use ICT for their teaching and learning activities Note Pedagogical support may consist of giving advice and guidance on issues related to teaching and learning Please do not consider support that is only technical Please mark only one choice in each row 1 2 3 4 5 Never Few times Monthly Weekly Not a year applicable Experienced colleagues BCP14A1 The school principal BCP14B1 The technology coordinator BCP14C1 Other staff from the school BCP14D1 Experts from outside the school BCP14E
88. for the international database To simplify logistics a single teacher data entry file was used The file contained responses from both the mathematics and science teachers Since teachers belonged to only one of two independently sampled populations analysis of combined mathematics and science teacher data is neither conceptually meaningful nor possible from a sampling and weighting perspective To reflect this the international database includes one file for the mathematics teacher data and one for the science teacher data Thus the SITES 2006 international database consists of the following file types file type acronyms in brackets e a school level data file with information collected from principals and ICT coordinators BCG e a mathematics teacher file with information collected from mathematics teachers BTM and e a science teacher file with information collected from science teachers BTS Additionally there is a single national context questionnaire data file with all systems data included see Section 2 7 The filenames consist of an eight character string followed by a three character file extension and use the following naming conventions e The first character of a file name is always B denoting populations associated with ISECD level 2 teaching and learning e The second character indicates the level of the information included in a file The letter N indicates a national level data file the letter C
89. he database is for analysts who wish to apply a different data editing or cleaning approach to these data 2 7 National Context Data National or country aspects sometimes account for elements of the education system and educational processes In SITES 2006 a National Context Questionnaire NCQ was designed to explore these possibilities This section describes the format and content of the questionnaire The NCQ was administered to all National Research Coordinators NRC in English Responses are qualitative in nature and were collected via the internet by means of an online questionnaire The responses were verified as far as possible discussed with NRCs and reconciled if needed The final NCQ data set comprises data from all 22 core education systems plus Australia Variables for data taken from the United Nations Development Program UNDP Human Development Report 2006 are set to system missing for Australia which entered the survey as a benchmarking participant at a stage where the NCQ file was almost finalized Full documentation of the NCQ design contents administration analysis outcomes and reporting are provided in Chapter 3 of the international report Law Pelgrum amp Plomp 2008 and Chapter 2 of the technical report Carstens amp Pelgrum 2009 The questionnaire itself including variable names can be found in Appendix A of this User Guide The NCQ data is provided as a single file called BNGSITS 1 sav in fu
90. he use of ICT BCT17A1 B Insufficient number of computers connected to the Internet BCT17B1 C Insufficient Internet bandwidth or speed BCTI7GI D Lack of special ICT equipment for disabled students BCT 17D 1 E Insufficient ICT equipment for instruction BCTI7E1 F Computers are out of date BCT17F1 Not enough digital educational resources for instruction BCT17G1 H Lack of ICT tools for science laboratory work BCT17H1 I Teachers lack of ICT skills BCT1711 J Insufficient time for teachers to use ICT BCT17J1 Other obstacles K Pressure to score highly on standardized tests BCT17K1 L Prescribed curricula are too strict BCT17L1 M Insufficient or inappropriate space to accommodate the school s pedagogical approaches BCT17M1 N Insufficient budget for non ICT supplies e g paper pencils BCT17N1 O Using ICT for teaching and learning is not a goal of our school BCT 1701 1 2 3 4 Notatall Very little Somewhat Toa great extent 18 Do you have access to a computer at home BCT18A1 Filter
91. he world of work BCPO3A1 To improve students performance on assessments examinations BCP03B1 To promote active learning strategies BCP03C1 To individualize student learning experiences in order to address different learning needs BCPO3D1 To foster collaborative and organizational skills when working in teams BCP03E1 To develop students independence and responsibility for their own learning BCPO3F1 To do exercises to practice skills and procedures BCPO3G1 To increase learning motivation and make learning more interesting BCP03H1 To satisfy parents and the community s expectations BCPO3I1 To act as a catalyst in changing the pedagogical approaches of teachers BCP03J1 1 2 3 4 Not at all A little Somewhat Alot disagree agree APPENDIX B SITES 2006 PRINCIPAL QUESTIONNAIRE 4 Towhat extent do you agree or disagree that the school leadership you and or other school leaders encourages teachers at Grade lt target grade gt to use ICT in each of the following activities Please mark only one choice in each row 1 2 3 4 Strongly Disagree Agree Strongly disagree agree A Organize monitor and support team building
92. hen reproducing results from the international report Analysis Variable s This is the list of variables for which the statistics are to be computed Researchers may select more than one analysis variable Each of these variables will be analyzed separately Dependent Variable This is the dependent variable to be predicted by the list of analysis or independent variables in regression analysis Only one dependent variable can be listed for regression analysis Weight Variable The sampling weight that will be used in the analysis Generally SCHWGT should be used when analyzing school data files MTOTWGT should be used when analyzing mathematics teacher data files and STOTWGT should be used when analyzing science teacher data The selection of a weight variable is mandatory ANALYSING THE DATA e Jackknifing Variables This window lists the variables that capture the assignment of cases to sampling zones JKCZONE JKZONE and whether the case is to be dropped or have its weight doubled when computing the set of replicate weights JKCREP JKREP The IEA IDB Analyzer automatically uses these variables to compute the replicate weights that are used in all analysis types The examples presented in this section all use the SPSS data file called BCGALL SAV as input which contains the school data files for the five participating education systems Chile Chinese Taipei Finland Hong Kong SAR and Israel used in the merge example before In
93. hers and the programs they participate in These questions attempt to characterize using fixed choice and open ended questions the critically important activities and policies at educational levels higher than schools You may want to involve other experts in your country or education system to validate your answers Please note that e While most questions refer to your overall system some refer only to schools with the target grade e When questions refer to your overall system but school types in your system differ greatly please answer for the school type within which your target grade of the SITES 2006 study belongs and then explain how the school types differ in the area marked Explanations as needed This questionnaire has been developed by Ron Anderson ISC member in collaboration with Nancy Law Tjeerd Plomp Hans Pelgrum and Alfons ten Brumelhuis ISC member If you have any questions please don t hesitate to contact Ron Anderson and or Tjeerd Plomp We are looking forward to your responses Ron Anderson rea umn edu amp Tjeerd Plomp Study Director Your name Country System Date APPENDIX A SITES 2006 NATIONAL CONTEXT QUESTIONNAIRE 1 Educational System Structure and Responsibility Which government levels take the responsibility for the formulation of educational policies regarding structure overall organization of the school system for compulsory education Please Tick all that
94. hese The study is being conducted under the auspices of the International Association for the Evaluation of Educational Achievement IEA We are asking you for your help in order to determine the current state of pedagogical approaches to and the use of ICT Please try to answer each question as accurately as you can Confidentiality All information that is collected in this study will be treated confidentially At no time will the name of any school or individual be identified While results will be made available by country and by type of school within a country you are guaranteed that neither your school nor any of its personnel will be identified in any report of the results of the study For countries which have ethical survey guidelines which emphasize voluntary participation Participation in this survey is voluntary and any individual may withdraw at any time About this Questionnaire e This questionnaire asks for information from schools about education and policy matters related to pedagogical practices and ICT If you are the person answering this questionnaire it is important that you are someone who knows about the ICT facilities in your school and about practices regarding their use in your school If you do not have the information to answer particular questions then please consult other persons in your school The questionnaire will take you approximately 30 minutes to complete e The words computers and ICT Information
95. hing of the target class in this school year Please mark only one choice in each row Equipment and hands on materials e g laboratory equipment musical instruments art materials overhead projectors slide projectors electronic calculators BTG17A1 Tutorial exercise software BTG17B1 General office suite e g word processing database spreadsheet presentation software BTG17C1 Multimedia production tools e g media capture and editing equipment drawing programs webpage multimedia production tools BTG17D 1 Data logging tools BTG17E1 Simulations modeling software digital learning games BT G17F1 Communication software e g e mail chat discussion forum BTG17G1 Digital resources e g portal dictionaries encyclopedia BTG17H1 Mobile devices e g Personal Digital Assistant PDA cell phone BTG17I1 Smart board interactive whiteboard BTG17J1 Learning management system e g web based learning environments BTG17K1 1 Never 2 Sometimes 3 Often 4 Nearly always 117 118 Part VI Impact of ICT Use SITES 2006 IDB USER GUIDE 18 Do
96. ile is as follows e The first three characters of the filename are identical to the file naming convention presented above e The next three characters identify the files as a SITES 2006 codebook and are always S e The seventh and eighth characters are always S1 to indicate the SITES 2006 study cycle e The three character file extension is either SDB for standard dBase format TXT for plain ASCII format and PDF for Adobe Acrobat format DATA FILES The standard dBase SDB codebook files can be read using Microsoft Excel or most standard database and spreadsheet programs Important codebook fields include FIELD_LABL which contains extended textual information for all variables QUEST_LOC which provides the location of questions within their respective survey instruments and FIELD _CODE which lists all acceptable responses allowed in the database The SPSS dictionary information and the separate codebook files should be sufficient to inform any necessary conversion of data files into other formats for use outside of SPSS or SAS such as raw ASCII tab delimited Microsoft Excel Stata or SPSS Portable The standard format for the SITES 2006 public use international database is SPSS which is accessible to virtually all secondary analysts The SPSS files include full dictionary information i e variable name format type width and decimals label value labels missing values and appropriately set measurement level nomi
97. ils the weighting and variance estimation methods used in SITES 2 006 and provides information on the use of weights and guidelines for estimating variance when conducting secondary analysis Chapter 4 provides instructions on analyzing the SITES 2006 international database using IEA s International Database IDB Analyzer software to replicate the results presented in the SITES 2006 international report as well as to conduct secondary analysis The appendices contain the international version of each of the SITES 2006 questionnaires Since SITES 2006 was an ambitious and demanding study involving complex procedures for drawing samples collecting data and analyzing and reporting findings it is necessary to have an understanding of the characteristics of the study in order to work effectively with the data Although the SITES 2006 design operations and data gathering resembled to a large extend the procedures used in IEA s past and current student achievement studies such as TIMSS and PIRLS SITES adult target populations for example implied certain consequences and necessities for data collection and analysis The SITES 2006 User Guide describes the organization content and usage of the international database while the SITES 2006 Technical Report Carstens amp Pelgrum 2009 provides a comprehensive account of the methodological and analytical implementation of the study It is imperative therefore that this user guide be used in conj
98. in secondary analysis Chapter 11 of the SITES 2006 Technical Report Carstens amp Pelgrum 2009 includes comprehensive information about the scale and indicator construction in the study For the above example information about the included variables as well as country by country scale reliability coefficients is included in Section 11 4 9 of the technical report Given that all scales and indicators reported in SITES 2006 followed the cautious and straightforward calculation described above it should be easy for secondary analysts to re compute a case s derived variable using the questionnaire data included in the international database and by following the documentation in Chapter 11 of the SITES 2006 Technical Report Derived scale and indicator variables are therefore not included in the international database and should be computed as needed and desired by database users The authors and editors of the international as well as the technical report are available for further support regarding the computation and any conceptual questions DATA FILES Attention Point It is important to note that the SITES 2006 international report flagged education systems in relation to the achieved participation rate and whether the internationally defined procedures for the teacher s target class selection were followed Details about these rules and reporting standards are included in Chapters 10 and 12 of the SITES 2006 Technical Report Additiona
99. in the target class BTG13A1 Please mark only one choice 1 2 3 4 Always during Sometimes outside Often outside At any time school hours scheduled school hours scheduled school hours no scheduled school hours 113 14 SITES 2006 IDB USER GUIDE 14 In your teaching of the target class in this school year a How often do you conduct the following b Do you use ICT for these activities Please mark only one choice for each of the two parts in each row a How often do you conduct the following b ICT used 1 2 3 4 1 2 Never Sometimes Often Nearly No Yes always Present information demonstrations and or give class instructions BTG14A1 BTG14A2 Provide remedial or enrichment instruction to individual students and or small groups of students BTG14B1 BTG14B2 Help advise students in exploratory and inquiry activities BTG14C1 BTG14C2 Organize observe or monitor student led whole class discussions demonstrations presentations BTG14D1 BTG14D2 Assess students learning through tests quizzes BTG14E1 BTG14E2 Provide feedback to individuals and or small groups of students BTGI14F1 BTG14F2 Use classroom management to ensure
100. ing and teaching NCQ201h Pedagogy i Emphasis on students completing set tasks to demonstrate mastery NCQ201i j Emphasis on providing individualized student learning experiences to address different learning needs NCQ201j k Emphasis on open ended inquiry based learning learning tasks in classrooms NCQ201k 1 Emphasis on student collaboration for project based learning NCQ2011 m Emphasis on inter classroom collaboration within schools NCQ201m n Emphasis on inter school collaboration among teachers NCQ201n Assessment o Encouragements and support to set up international collaborative learning projects NCQ201o p Importance of students performance in standardized tests for tracking student progress NCQ201p q Importance of students performance in standardized tests for assessing school effectiveness NCQ201q 61 62 SITES 2006 IDB USER GUIDE r Importance of alternative authentic modes of assessment NCQ20Ir s Encouragement of collaborative tasks and peers assessment NCQ201s Explanations asriced ede iis iisctacaisentineiieacsi annsoinintadcansnsscteaenatiptenastactnann a a e ai LOD 21 Please review yo
101. ing ICT for teaching and or learning is not a goal of our school BCP 1601 APPENDIX B SITES 2006 PRINCIPAL QUESTIONNAIRE Organization of Learning The questions below are about grouping of students and time schedules 17 18 How often would visitors who walk into a lesson in your school ona typical day observe the following in Grade lt target grade gt Please mark only one choice in each row 1 2 3 4 Never Sometimes Often Nearly always Whole classes of students in their classroom with one teacher BCP17A1 In large classrooms students working under the supervision of a team of teachers BCP 17B1 Individuals or small groups of students being coached by teachers BCP17C1 Individuals or small groups of students working on their own at places they choose themselves BCP17D1 How often could students at your school expect the following to occur at Grade lt target grade gt Please mark only one choice in each row 1 2 3 4 Never Sometimes Often Nearly always Students working in different groups according to the projects they are engaged in or the subjects they are taking BCP18A1 Students all working in the same group class BCP18B1 Students spending their time in school fo
102. ing and or learning purposes for students in Grades lt grade range gt BCTOIA1 Please mark only one choice 0 2 years 3 5 years 6 10 years 11 15 years More than 15 years Don t know To what extent do you agree with each of the following statements about the use of ICT in your school Please mark only one choice in each row 1 2 3 4 Strongly Disagree Agree Strongly disagree agree ICT is considered relevant in our school BCTO2A1 Our school has integrated ICT in most of our teaching and learning practices BCT02B1 We have started to use ICT in the teaching and learning of school subjects BCT02C1 We still do not know which ICT applications are useful for our school BCTO2D1 Constraints rule out the use of ICT in our school BCTO2E1 Approximately how often during this school year will students in Grade lt target grade gt be using ICT for learning in the following subject domains Please mark only one choice in each row 1 2 3 4 Never Sometimes Often Nearly always Mathematics BCTO3A1 Natural Sciences BCT03B1 Social Sciences BCT03C1 Language of instruction mother tongue BCT03D 1 Foreign languages BCT03E1 ICT as separate subject BCTO3F1
103. ired ICT tools outside of the school premises BTG23G1 I do not have the time necessary to develop and implement the activities BTG23H1 I do not know how to identify which ICT tools will be useful BTG23I1 My school lacks digital learning resources BTG23J1 I do not have the flexibility to make my own decisions when planning lessons with ICT BTG23K1 I do not have access to ICT outside of the school BTG23L1 Have you participated in any of the following professional development activities If no would you wish to attend Please mark only one choice in each row Introductory course for Internet use and general applications e g basic word processing spreadsheets databases etc BTG24 A1 Technical course for operating and maintaining computer systems BT G24B1 Advanced course for applications standard tools e g advanced word processing complex relational databases BT G24C1 Advanced course for Internet use e g creating websites developing a home page advanced use of the Internet video conferencing BTG24D 1 Course on pedagogical issues related to integrating ICT into teaching and learning BTG24E1 Subject specific training with learning software for specific content goals e g tutorials simulation etc BTG24F1 Course on multimedia operatio
104. istered Questions 2 5 2 Omitted or Invalid Responses 2 5 3 Logically Not Applicable 2 5 4 Not Reached 2 6 Variables and Scales Derived from the Questionnaire Data 2 7 National Context Data Weights and Variance Estimation 3 1 Overview 3 2 Sampling Weights 3 2 1 Why Weights Are Needed 3 2 2 Selecting the Appropriate Weight Variable 3 2 3 Example for Analyzing Weighted Data 3 3 Normalizing Weights 3 4 Variance Estimation 3 4 1 Why Variance Estimation Is Needed 3 4 2 Selecting the Appropriate Variance Estimation Variables 3 4 3 Example for Variance Estimation Analyzing the SITES 2006 Data Using the IEA IDB Analyzer 4 1 Overview 4 2 The IEA IDB Analyzer 4 3 Merging Files with the IEA IDB Analyzer 4 4 Performing Analyses with the IEA IDB Analyzer 4 4 1 Computing Percentages only and their Standard Errors 4 4 2 Computing Percentages and Means and their Standard Errors 4 4 3 Computing Regression Coefficients and their Standard Errors 4 4 4 Computing Correlations and their Standard Errors 4 5 Performing Analyses from the SITES 2006 Report 4 5 1 Analysis with School level Variables 4 5 2 Analysis with Teacher level Variables 4 6 Other Analyses using the IEA IDB Analyzer o NN SITES 2006 IDB USER GUIDE Appendices Appendix A SITES 2006 National Context Questionnaire Appendix B SITES 2006 Principal Questionnaire Appendix C SITES 2006 Technical Questionnaire Appendix D SITES 2006 Teacher Questionnaire Appendi
105. ithin the package generally follow the DOS file naming convention file names with up to eight characters followed by a three character extension as in FILENAME EXT Files with the same names are complementary to each other and the extension identifies their function or type The extensions used for the files contained in the package are the following e PDF Codebooks and reports in PDF format e SDB Codebooks in standard Dbase format readable in Excel e TXT Codebooks in ASCII text format e XPT SAS Export data files e SAV SPSS data files e EXE IEA IDB Analyzer setup executable file Attention Point At a later stage the IEA SITES 2006 IDB will also be made available through IEA s Study Data Repository system at http rms iea dpc org 2 1 Data Files Codebooks and Formats This section describes the file types contents and formats of the SITES 2006 data For each file type a separate file is available for each participating education systems Files of the same type are structurally isomorphic i e each file includes the same uniformly defined set of variables During data capture one file type for the school principal data one for the ICT coordinator data and one for teacher data were used Since both the principal and the ICT coordinator responses related to the school level and matched each other 1 1 they 14 SITES 2006 IDB USER GUIDE were consequently merged to form a combined school level data file
106. learning and teaching NCQ191d Explanations as needed 3 Change in past five years 60 APPENDIX A SITES 2006 NATIONAL CONTEXT QUESTIONNAIRE Note The next question block applies to your entire compulsory educational system in the past five years Keep in mind that the country chapters were written about five years ago in 2001 20 In the past five years to what extent have each of these items decreased not changed or increased Tick all that apply Decreased N o Increased Increased Dont Change Slightly ALot Know Spending for ICT in education a Total spending on ICT in Education NCQ201a b Spending on Internet connections and networking NCQ201b c Spending on classroom ICT hardware and software NCQ201c D Spending on instructional technology support staffing NCQ201d E Spending on professional development related to ICT NCQ201le F Funding for professional development related to using ICT in teaching NCQ20If g Funding for professional development related to using ICT in teaching for such methods as inquiry collaboration and authentic assessment NCQ201g h Funding for school level leadership development for strategic implementation of ICT in learn
107. ling 1 2 5 Quality of Samples and International Reporting The quality of the achieved sample within each education system participating in SITES 2006 was assessed by categorizing the participation rates of each of the three surveyed population schools mathematics teachers and science teachers The categorization resulted in either full reporting or annotated reporting due to the potentially reduced quality of the sample In addition education systems in which the achieved sample did not cover at least 90 of the desired population were flagged in the SITES 2006 international report For teacher data a flag also indicated one education system that collected data on teacher participation only after survey administration Finally annotated reporting was used for those education systems that did not completely follow the internationally defined procedure to randomly select and provide a target grade reference to teachers for answering Parts I VI and Part VIII of the SITES 2006 Teacher Questionnaire Attention Point To identify possible limitations users of the SITES 2006 international database are encouraged to consult Chapter 10 of the SITES 2006 Technical Report Carstens amp Pelgrum 2009 for more details about the quality rating of the SITES 2006 samples and Chapter 12 for details about the flagging and reporting policy in the SITES 2006 international report 12 SITES 2006 IDB USER GUIDE References Carstens R amp Pelgrum
108. llowing lessons according to a fixed schedule BCP18C1 Changes to the usual time schedule if students need time to complete their projects BCP18D1 Students having a lot of freedom to plan their own learning time BCP18E1 85 86 SITES 2006 IDB USER GUIDE School Characteristics The intention of this set of questions is to describe the general characteristics of your school 19 What is the total number of boys and girls in the entire school Please write a whole number Write O zero if none Total number of girls Total number of boys BCP19A1 BCP19B1 20 What are the lowest and highest grade levels in your school Please mark only one choice in each row 1 2 3 4 5 6 yi 8 9 10 il 12 13 Kindergarten 1 2 3 4 5 6 7 8 9 10 11 12 13 A Lowest BCP20A1 B Highest BCP20B1 21 How many people live in the city town or village where your school is located BCP21A1 Please mark only one choice 3 000 people or fewer 3 001 to 15 000 people 15 001 to 50 000 people 50 001 to 100 000 people 100 001 to 500 000 people More than 500 000 people DnB WY Fe 22 Approximately what percentage of students are absent from your school on a ty
109. lly Chapter 11 of the SITES Technical Report may indicate limitations arising from low or only marginally acceptable scale reliabilities that may indicate reduced measurement invariance across education systems Users of the SITES 2006 international database are strongly advised to consult these pieces of documentation carefully should the intended analysis involve the comparison of one or more derived composite variables across countries The school level file BCG includes a set of derived variables for the Questions 5 and 15 in the SITES 2006 Technical Questionnaire e The variable BCTOSA1O holds the original information for the total number of computers in the school as reported by ICT coordinators Because implausible combinations were detected the main questionnaire variable BCTOSA1 was recoded if either the sum of the items B D computers available only to students plus those only available to teachers plus those only available to administrative staff or any of the values in items E F or G was higher than the value in A e The variables BCT15A10 to BCT15H10 hold original information for the amount of time spent on ICT related support on average per week Ifa value larger than 50 was detected in the first item A time spent by the ICT coordinator this and all values in items B H were set to omitted in the main questionnaire variables BCT 15A1 to BCTISH1 The main value for including the original variables for Questions 5 and 15 in t
110. lly labeled SPSS format Given its qualitative nature there is neither a corresponding SAS file nor a separate codebook file 21 22 SITES 2006 IDB USER GUIDE References Carstens R amp Pelgrum WJ Eds 2009 IEA SITES 2006 Technical Report Amsterdam International Association for the Evaluation of Educational Achievement Law N Pelgrum W J amp tPlomp T Eds 2008 Pedagogical practices and ICT use around the world Findings from an international comparative study CERC Studies in Comparative Education Hong Kong Comparative Education Research Centre The University of Hong Kong and Dordrecht Springer Weights and Variance Estimation Olaf Zuehlke 3 1 Overview This chapter gives a brief introduction to the use of weighting and variance estimation variables in SITES 2006 The names and locations of these variables in the international database are described and their specific roles in school and teacher analysis are explained Examples for the importance of using the appropriate weighting and variance estimation techniques are given 3 2 Sampling Weights 3 2 1 Why Weights Are Needed All data in the SITES 2006 international database are derived from randomly drawn samples of schools and teachers Of course the results of the study should not only hold for the sampled schools and teachers but for the entire education system that is participating in the SITES 2006 study To make correct inferences abo
111. matics teacher data for five education systems Chile Chinese Taipei Finland Hong Kong SAR and Israel are merged by clicking on the Merge button In general merging teacher data files works the same way as merging school data files Figure 4 17 shows the set up screen of the IEA IDB Analyzer merge module where BTM has been selected as the file type In this example the merged file will be saved to a file called BTMALL SAV The identification variables are automatically selected by the IEA IDB Analyzer 46 SITES 2006 IDB USER GUIDE Figure 4 17 Merge Module for Example Teacher level Analysis IDB Analyzer IT Merge Module J IMPACTICT DO YOU USE ICT IN TEACHING __INFOYOUAWHAT IS YOUR GENDER EEIS AS INF NATIVE SPEAKERS OF LANGUAGE OF INST INF HOW MANY HAS OF MATH SCIENCE LESS _INF COMPETENCE WORD PROCESSING __ INF COMPE TENCE DATABASE SOFTWARE _INF COMPE TENCE SPREADSHEET EO INF COMPETENCE PRESENTATION SOFTWARI __INF COMPETENCE GRAPHIC CALCULATOR Ii INF COMPETENCE DATA LOGGING TOOLS _ CURRGOALS GOAL IMPORTANCE WORLD OF CURRGOALS GOAL IMPORTANCE JUPPER EDU _ CURRGOALS GOAL IMPORTANCE LEARN FRM _ CURRGOALS GOAL IMPORTANCE REAL WORL _ CURRGOALS GOAL IMPORTANCE PERFORMA _ CURRGOALS GOAL IMPORTANCE NC MOTIVA _CUIRRGNALS GNAL IMPNRTANCE ANDIV ran al For this analysis there is no need to recode the data and conducting the analysis with the
112. menu item or WEIGHT BY syntax 3 3 Normalizing Weights In other IEA studies a senate weight SENWGT and a house weight HOUWGT were calculated These weights are not included in the SITES 2006 international database but researchers can easily calculate them since they are the result of a linear transformation of the SITES 2006 weighting variables For cross education system analyses in which education systems should be treated equally a senate weight can be computed that transforms the weights in such a way that the sum of weights across all cases is equal to a fixed constant 500 in the example below For each mathematics teacher i the senate weight variable can be computed as 500 SENWGT MTOTWGT gt __ XMTOTWGT 1 Similarly researchers who are interested inscience teachers shoulduse STOTWGT instead of MTOTWGT and those who are interested in schools SCHWGT Instead of 00 any other constant value can be used The senate weight will give inaccurate results if a lot of cases in a variable of interest are missing and if the missing data varies substantially between countries In student achievement studies where every student is assigned an achievement score this is not a large problem In SITES 2006 however almost every variable has some entries set to missing therefore the senate weight should be used cautiously Some researchers pursuing advanced analyses may use software
113. n e For socio cultural reasons a country may have chosen not to administer a certain question in its national questionnaire as documented in the national adaptation forms The variables corresponding to the question that was removed are coded as not administered The code for not administered questions in SPSS files is SYSMIS The code for not administered questions in the SAS files is A This code was assigned at the IEA DPC after data collection 2 5 2 Omitted or Invalid Responses The response to a question was coded as omitted when the question was administered but not answered or when an invalid response was given The following cases are coded as an omitted or invalid response e The question was administered but no response was provided DATA FILES e The respondent selected more than the expected number of checkboxes or gave a response to a question that was not interpretable e Ifa particular question or a whole page was misprinted or left out ofa questionnaire or for other reasons was not available to the respondent The code for these responses in SPSS files are 9 99 999 and so on depending on the field length of the variable The code for omitted or invalid responses in SAS files is 7 This code was assigned by the national center during data capture or by the IEA DPC during data cleaning and editing 2 5 3 Logically Not Applicable The response to a variable is coded as Logic
114. n systems National centers ran standardized checks on their data in order to detect inconsistencies duplicate records or problematic reliability of data entry During data processing the IEA DPC investigated the quality of the data using about 135 different structural validity and consistency checks The DPC and national centers worked closely to resolve any issues that were detected The cleaned data from participating education systems were then weighted and transformed to the international database structure such that every dataset consisted of the same variables with the same coding scheme Adaptations done by national centers were recoded to recover the international data structure according to rules devised by the DPC and the national centers In addition school level data from responses to the school and the technical questionnaires were merged into one file per education system at the school level Furthermore data from the teacher questionnaire were split into one file per education system containing data from mathematics teachers and one file per education system containing data from science teachers Please refer to Chapter 9 of the SITES 2006 Technical Report Carstens amp Pelgrum 2009 for details on data processing and the creation of the SITES 2006 international database Chapter 10 for details of the weighting and variance estimation procedures and Appendix D for details about education system specific adaptations and their hand
115. nal ordinal or scale The dictionary information can be accessed in list form through the SPSS View gt Variables menu or in output form from through the File gt Display Data File Information menu Given this SPSS users will most likely not need to consult the separate codebooks files as all relevant meta information has become part of the SPSS files The international database is additionally included in SAS XPORT transport format The national context questionnaire data is only available in SPSS format and there is no separate codebook file describing its structure Analytical support is provided for SPSS only through the IEA IDB Analyzer see Chapter 4 of this User Guide 2 2 Cases Included The international database includes all cases that satisfied the SITES 2006 sampling standards Data from those respondents who either did not participate or did not pass adjudication for example because within school participation was not sufficient were removed from the final data Detailed information about the data collection capturing processing editing weighting and adjudication of the international database are included in the SITES 2006 Technical Report Carstens amp Pelgrum 2009 Details regarding the file types and the variables they contain are described in the following sections For the international database the data cleaning process at the IEA DPC ensured that information coded in each variable was in fact interna
116. nal report were mistakenly reported 100 times lower than the correct values 4 6 Other Analyses using the IEA IDB Analyzer The examples above are just illustrations of the possible analyses that can be conducted using the IEA IDB Analyzer Questions about the IEA IDB Analyzer should be directed to sites iea dpc de References Carstens R amp Pelgrum WJ Eds 2009 IEA SITES 2006 Technical Report Amsterdam International Association for the Evaluation of Educational Achievement IEA 2009 International database analyzer version 2 0 0 0 Computer software Hamburg Germany IEA Data Processing and Research Center Law N Pelgrum W J amp Plomp T Eds 2008 Pedagogical practices and ICT use around the world Findings from an international comparative study CERC Studies in Comparative Education Hong Kong Comparative Education Research Centre The University of Hong Kong and Dordrecht Springer SPSS Inc 2008 SPSS for Windows version 16 0 Computer software Chicago IL SPSS Inc Appendices Appendix A SITES 2006 National Context Questionnaire Copyright 2006 International Association for the Evaluation of Educational Achievement IEA I IEA SITES 2006 National Context Questionnaire NCQ Dear NRC As discussed at the NRC meeting last December in Phuket Thailand we are asking each NRC to complete this questionnaire so that we have up to date characteristics of each system for pu
117. ne language the value was set to 1 for all teachers DATA FILES e ITSUBJ Teacher population identification with value 1 for mathematics teachers and value 2 for science teachers e ITMATH Secondary ID for science teachers Teachers teaching both mathematics and science or one or more science subjects in education systems where science is taught in separate subjects were listed twice once in the tracking form for mathematics teachers and once in the tracking form for science teachers On each mathematics teacher tracking form the corresponding ID from the science teacher tracking form was copied for teachers teaching both subjects e ITSCI Secondary ID for math teachers Teachers teaching both mathematics and science or one or more science subjects in education systems where science is taught in separate subjects were listed twice once in the tracking form for mathematics teachers and once in the tracking form for science teachers On each science teacher tracking form the corresponding ID from the mathematics teacher tracking form was copied for teachers teaching both subjects 2 3 3 Sampling and Weighting Variables To calculate population estimates and correct jackknife variance estimates sampling and weighting variables are provided in the data files Further details about weighting and variance estimation are provided in Chapter 3 of this User Guide The following weight variables are included in the SITES 2006 inte
118. ning practices and of how Information and Communication Technologies ICT support these in secondary schools around the world Approximately 20 countries will provide information from representative samples of teachers on how they organize their teaching and learning the ICT facilities they have available at school how they use ICT for teaching and learning and the obstacles or difficulties they experience in relation to these technologies This information will give better insight into the current state of pedagogical approaches and of how technologies support them It will also allow educational practitioners and policy makers to gain a better understanding of areas needing intervention and additional support Name of country along with about 20 other countries is taking part in this international study of pedagogical practices and the way that ICT supports these This questionnaire is being administered to representative samples of teachers in these countries The study is being conducted under the auspices of the International Association for the Evaluation of Educational Achievement IEA We are asking you for your help in order to determine the current state of pedagogical approaches to and the use of ICT in Name of country Please try to answer each question as accurately as you can Confidentiality Allinformation that is collected in this study will be treated confidentially At no time will the name of any school or individual be i
119. ninnuninnnnniinnnniiinnniininnnininiist b State and or provincial government NCQ041 eennnninnnnnmnnnnnnininnnss c District and or Local government NCQ04 1 msnnninnnininnnnniiiinnnniinte d Other Please specify NCQ041 nmmmnnnnmnnnmnnnnnnininnnse NCQO041t e There are no publicly funded schools in our system NCQO4 1 wmemmnnenmnene Explanations as needed one NCQO42 What percentile of schools at the SITES 2006 target grade is funded by the government at the central and or state provincial levels Tick only one choice NCQOS None lt 10 10 25 26 50 51 75 76 90 gt 90 APPENDIX A SITES 2006 NATIONAL CONTEXT QUESTIONNAIRE 6 What percentile of the schools at the SITES 2006 target grade is funded at the local or district government levels Tick only one choice NCQ06 None lt 10 10 25 26 50 51 75 76 90 gt 90 7 For schools that have the target grade and also are funded by the government which alternative below best describes your system Tick only one choice NCQO71 a Only attainment targets are prescribed enenennnnnnninnnnniiinnnniiniinnneste Only attainment targets and curriculum are prescribed ceeeennnnnne b c Attainment targets curriculum and textbook lists are sreactibed sau d Attainment targets
120. nnnnninninnnnnnnn f Goal to reduce the digital divide NCQ24 siicscimcpiinnncnnrsmgacsnmnamiaaiens g Steps to be taken to ensure ICT access outside of school i emnnnninnnnnnnn NCQ24g h Specifications on teachers professional development requirement in ICT NCQ24h i Policy to stimulate greater teachers professional development in ICT NCQ 24i j Specifications on evaluation of implementations of the policy on u ICT in education NCQ24j k Specifications on funding level and or arrangement associated 0 0 with the policy NCQ24k l Other specify WCQ QA veprimin T NCQ 24lt 63 64 25 26 27 28 SITES 2006 IDB USER GUIDE How is the provision of hardware and software to schools managed Tick all that apply a Schools hardware and or software are specified and provided come through a centralized system run by the government NCQ2Sa b Schools are allocated funding for the purchase of hardware and or software as specified in pre approved applications submitted by schools NCQ25Sb c Schools are only given partial funding for the purchase of hardware and or software as specified in pre approved applications and have to supplement the difference from the schools own funds NCQ25c d Schools are provided with government funding total or partial for
121. ns This example will compute the average number of computers with internet access BCTOSE1 for each category of school location BCP21A1 and their associated standard errors for each education system IDCNTRY using the weighting variable SCHWGT The data will be read from the data file BCGALL SAV and the standard errors will be computed based on 100 replicate weights The steps in the IEA IDB Analyzer are as follows 1 Open the analysis module of the IEA IDB Analyzer 2 Select the data file called BCGALL SAV that was merged in the previous step 3 As type of the analysis select Percentages and Means Please note that the program will by default exclude from the analysis cases with missing grouping variables This can be deactivated by removing the checkmark from the option Exclude Missing from Analysis 4 Inthe next steps the variables need to be defined a As Grouping Variables the software always selects variable IDCNTRY by default BCP21A1 will need to be added for this example To do this select the variable from the variable list on the left hand side of the window and press the right arrow button belonging to the section of the grouping variable This will move the variable BCP21A1 from the variable list on the left hand side into the field for the grouping variables on the right hand side Note that subgroups are created and reported using the order of the variables as they appear in this list b Next the analysis va
122. ns e g using digital video and or audio equipment BTG24G1 1 2 3 No Ido No I would Yes Ihave not wish like to attend to attend if available APPENDIX D SITES 2006 TEACHER QUESTIONNAIRE 25 26 27 To what extent do the following statements about school vision apply to the staff in your school Please mark only one choice in each row 1 2 3 4 Not at all A little Somewhat Alot We discuss what we want to achieve through our lessons BTG25A1 Teachers are constantly motivated to critically assess their own educational practices BTG25B1 Teachers are expected to think about the school s vision and strategies with regard to educational practices BTG25C1 To what extent do the following statements about teachers participation in decision making apply to you Please mark only one choice in each row 1 2 3 4 Not at all A little Somewhat Alot I can influence the development of the school s innovation implementation plans BTG26A1 When implementing innovations our school considers teachers opinions and adjusts its action plan as needed BTG26B1
123. o ICT coordinators and a teacher questionnaire to mathematics and science teachers Furthermore NRCs were asked to complete the National Context Questionnaire This questionnaire was administered online and gathered data about system level information related to structure funding curriculum and educational processes Appendices A to D contain all four questionnaires including the variable names as used in the international database To conduct a valid analysis of the SITES 2006 data and to correctly interpret the results it is important to be aware of adaptations to the questionnaires that might have been made by national centers In some instances the international version of a question or item had to be adapted by all participating education systems e g the target grade In other cases national centers adapted questions or items for socio cultural reasons or because the international version was partly or completely not applicable in their education system Users of the international database are encouraged to refer to Appendix D of the SITES 2006 Technical Report Carstens amp Pelgrum 2009 for more details about national adaptations in SITES 2006 and potential consequences for comparability across education systems 1 2 4 Sampling and Survey Operations In SITES 2006 a two stage stratified cluster sampling procedure was used National centers provided a list of all target grade schools to the IEA DPC The school samples were drawn by
124. olicies Enter a to fin NCQ022 2 2 Which level has the secondary second most responsibility for these policies Enter a to f or none uc NCQ023 Explanations as needed NCQ024 E E E EAE TE A E E ETE E E A E 55 SITES 2006 IDB USER GUIDE Which government levelstake the responsibility for determining requirements for teacher certification Please Tick all that apply Central government NCQ 3 else itaceen aia ete g taarele State and or provincial government NCQO03 1D onnnninnnninnnninnnninnnnnss District and or Local government NCQO03 16 m m nnnnunnnnnnnniininnnnniiinninss Non statutory and or professional body NCQ031d weesmmnnmnnmnninniniennnetese Schools are free to decide NCQO3 Le weeiscmncnncnnememenieinemnenieiieinemieieuenuties Other Please specify NCQO31f s NCQO31ft mo aaora If you selected more than one of the above answers please answer 3 1 and 3 2 3 1 Which level has the primary most responsibility for these policies Enter to fs visccessisscsousecsses NCQ032 3 2 Which level has the secondary second most responsibility for these policies Enter a to f or none iss NCQ033 Explanations as needed eg EE E E EET NCQ034 What is the_main funding source for public constitutionally mandated schooling Please Tick only one source the primary source a Central government NCQ041 oi mnnnn
125. ols 60 o o 33 12 5 10 Continent Medium Size Schools 65 o 1 32 19 3 10 Continent Small Schools 72 o 1 24 7 10 20 Continent Very Small Schools 135 o 4 44 34 10 43 Total 416 o 8 173 92 36 107 E 2 Australia Sample Implementation Australian data did not undergo a formal sampling adjudication process However some participation statistics can be given here E 2 1 Exclusion Rates e School exclusion rate 0 4 e Mathematics teacher exclusion rate 0 3 e Science teacher exclusion rate 0 3 129 130 E 2 2 Participation Rates Table E 2 Participation Rates in Australia SITES 2006 IDB USER GUIDE School participation rate before replacement School participation rate after replacement Mathematics teacher participation rate Science teacher participation rate School principal participation rate ICT coordinator participation rate 41 6 72 4 83 9 85 9 84 4 87 4 38 9 70 7 84 0 86 2 85 5 86 8
126. pical school day BCP22A1 Please mark only one choice Less than 5 S 10 11 20 More than 20 AUNE 23 Approximately what percentage of students in your school are native speakers of lt national language language of instruction gt BCP23A1 Please mark only one choice 1 Less than 50 2 50 75 a 76 90 More than 90 APPENDIX B SITES 2006 PRINCIPAL QUESTIONNAIRE 24 Has your school been involved in any of the following activities during the past few years Please mark only one choice in each row Making changes to pedagogical practices BCP24A1 Adopting new assessment practices BCP24B1 Connecting to the Internet BCP24C1 Adapting buildings to suit the school s pedagogical approaches BCP24D 1 Setting up computers in classrooms BCP24E1 MmmoawPD gt Installing computer laboratories BCP24F1 25 Who at your school has the primary responsibility for making decisions about each of the following Please mark only one choice in each row 1 2 3 4 5 External School Subject Teachers Not agency leadership department applicable Purchasing ICT equipment BCP25A1 B Selecting subject content to be learned BCP2S5B1 C Determining which pedagogical approaches will be used BCP25C1 D Choosing whether ICT is used BCP25D1 E Assessing learning
127. progress in the classroom BCP25E1 F Using mobiles and or handheld devices for instructional purposes BCP25F1 87 88 26 27 28 1 2 3 4 5 SITES 2006 IDB USER GUIDE Personal Background Information Below are a few questions about your personal background Think about a new development change that you consider highly satisfying related to the learning experiences of students that occurred in your school and under your principalship during the current academic year Did you play any of the following roles in this new development Please mark only one choice in each row I initiated the change and teachers in our school further developed and implemented it BCP26A1 I initiated the change and I contributed substantially to its development and implementation BCP26B1 Teachers initiated the change The change was basically a bottom up initiative that did not require my support BCP26C1 Teachers initiated the change My role was mainly in the form of moral support BCP26D1 Teachers initiated the change and I allocated resources and necessary staffing to support it BCP26E1 The school management board initiated the change and I led its development and implementation B
128. provide more opportunities for my students to collaborate with or learn from people outside of their classroom including peers and external experts BTG2211 To collaborate more with fellow teachers and others within and outside my school BT G22J1 To provide more opportunities for my students to collaborate with their classmates BTG22K1 To arrange self accessed activities for my students BTG22L1 1 Not at all 2 Low priority 3 Medium priority 4 High priority 121 122 SITES 2006 IDB USER GUIDE 23 Do you experience the following obstacles in using ICT in your teaching QTA K 24 Please mark only one choice in each row ICT is not considered to be useful in my school BTG23A1 My school does not have the required ICT infrastructure BTG23B1 I do not have the required ICT related skills BTG23C1 I do not have the necessary ICT related pedagogical skills BTG23D 1 I do not have sufficient confidence to try new approaches alone BTG23E1 My students do not possess the required ICT skills BTG23F1 My students do not have access to the requ
129. questions of interest Appendix D of the SITES 2006 Technical Report The variable BCT04A1 in the school files contains information on the availability of equipment and hands on materials The next step is to review the documentation of national adaptations to the questionnaires to ensure that there were no deviations listed 41 42 SITES 2006 IDB USER GUIDE for this variable see Appendix D of the SITES 2006 Technical Report Ifno adaptations were made the analysis can continue without additional modifications The next step is to identify the variables relevant to the analysis in the appropriate files Since this example uses a school level variable it should be identified in the school file To support this the appendices of this User Guide provide all questionnaires and include the respective variables names as used in the international database This data file will provide the variable that contains the information about the availability of equipment and hands on materials BCT04A1 The merge module of the IEA IDB Analyzer will combine data from the school files of the desired education systems The school data for five education systems Chile Chinese Taipei Finland Hong Kong SAR and Israel are merged by clicking on the Merge button The merged data file will be called BCGALL SAV Figure 4 12 Table 4 5 from the SITES 2006 International Report for Example School level Analysis Law Pelgrum amp Plomp 2008 pp 81
130. re typed in italics Most questions can be answered by marking the one most appropriate answer e Ifyou are completing the paper version of this questionnaire please use a writing pen or ballpoint to write your answers e When you have completed this questionnaire please National Return Procedures and Date APPENDIX B SITES 2006 PRINCIPAL QUESTIONNAIRE Further information e When in doubt about any aspect of the questionnaire or if you would like more information aboutit or the study you can reach us by phone at the following numbers National Center Contact Information Thank you very much for your cooperation 69 70 Pedagogy at Your School SITES 2006 IDB USER GUIDE The following questions address the characteristics of teaching and learning in your school To what extent is each of the following aspects of teaching and learning currently present in your school Please mark only one choice in each row Students develop abilities to undertake independent learning BCPO1A1 Students learn to search for process and present information BCP01B1 Students are largely responsible for controlling their own learning progress BCPO1C1 Students learn and or work during lessons at their own pace BCPO1D1 Students are involved in cooperative and or project based learning BCPO1E1 Students determine for themselves when to take a test BCPOIFI1 Students learn search strategies to find diverse types of relevant info
131. responding standard errors by combining both sampling and imputation variance The IEA IDB Analyzer consists of two modules the merge module and the analysis module which are executed as independent applications The merge module is used to create analysis datasets by combining data files of different types and from different education systems and selecting subsets of variables for analysis The analysis module provides procedures for computing various statistics and their standard errors for variables of interest These procedures can be applied for an education system and for specific subgroups within an education system Both modules can be accessed by using the Windows Start menu Start gt All Programs gt IEA gt IDB Analyzer gt Merge Module Analysis Module The IEA IDB Analyzer requires PCs running Microsoft Windows 2000 XP or any later version The IEA IDB Analyzer further requires a working installation of the Microsoft NET 2 0 framework and includes this if required as part of the setup routine Local administrator rights are required for a successful installation of the IEA IDB S In SITES plausible values are not used so this functionality will not be discussed in this chapter 29 30 SITES 2006 IDB USER GUIDE Analyzer The software will produce scripts to be run with SPSS Therefore SPSS needs to be installed for merging files and running analysis The IEA IDB Analyzer currently is available free of charge from
132. rget grade If only yes for some types of students or schools please describe these types NCQ122 Remarks if any If no skip to question 14 How much emphasis does the mathematics curriculum at the target grade place on each of the following None Very Little Some Alot a Mastering basic skills NCQ131a b Applying math in real life contexts NCQ131b c Communicating about math NCQ131c d Integrating math with ICT NCQ131d Comments NCQ132 57 58 SITES 2006 IDB USER GUIDE 14 Does your system have a system wide curriculum that includes science at the target grade INO EE ETEA PETT PE E Me atta cette AA AA A cate Ne dete LB hh If only yes for some types of students or schools please describe these types NCQ142 Remarks if any If no skip to question 16 15 How much emphasis does the science curriculum at the target grade place on each of the following None Very Little Some Alot a Knowing basic science facts NCQ151a b Applying science to real life problems NCQ151b c Communicating about science NCQI1S1c d Integrating science with ICT NCQ151d Remarks For example if there are several science subjects at the target level with differing emphasis across them please describe the situation aeaee
133. riables need to be defined To activate this section press the radio button Analysis Variables This time select variable BCTOSE1 from the list of variables and move it to the analysis variables field by pressing the right arrow button in this section Note that more than one analysis variable can be selected 5 The weight variable is automatically defined by the software As this is an example for analysis on school level the weight SCHWGT is selected by default Additionally the jackknifing variables JKCZONE and JKCREP are defined 6 Specify the name and folder ofthe output files in the Output Files field The IEA IDB Analyzer will use this name and folder to create three output files an SPSS syntax file that contains i the code for performing the analysis ii an SPSS data file with the results and iii an Excel file with these same results 7 Press the Start SPSS button to create the SPSS syntax file and open it in an SPSS syntax window The syntax file will be executed by opening the Run menu of SPSS and selecting the All menu option If necessary the IEA IDB Analyzer will produce a prompt to confirm the overwriting of already existing files 35 SITES 2006 IDB USER GUIDE Figure 4 5 Analysis Module Computing Percentages and Means IDB Analyzer II Analyze Module T E __ Change _PEDAS ASP OF TLANDEPENDENT LEARNING COUNTRY ID PEDAS ASP OF TL SEA PRO PRE INFORMATION SHCHA4 PEOPLE WHERE SCHOOL IS LOCAT _ PEDAS ASP
134. right arrow button in this section Please note that if this variable does not exist in the variable list the recoding step before was not conducted successfully c Select the radio button for the Dependent Variable Select variable RECCSRAT from the variable list and move it to the dependent variable field by pressing the right arrow button in this section Please note that if this variable does not exist in the variable list the recoding step before was not conducted successfully The weight variable is automatically selected by the software As this is an example for analysis on school level the weight SCHWGT is selected by default Additionally the jackknifing variables JKCZONE and JKCREP are defined Specify the name and folder of the output files in the Output Files field The IEA IDB Analyzer will use this name and folder to create three output files i an SPSS syntax file that contains the code for performing the analysis ii an SPSS data file with the results and iii an Excel file with these same results Press the Start SPSS button to create the SPSS syntax file and open it in an SPSS syntax window The syntax file will be executed by opening the Run menu of SPSS and selecting the All menu option If necessary the IEA IDB Analyzer will produce a prompt to confirm the overwriting of already existing files Figure 4 8 Analysis Module Computing Regression IDB Analyzer II Analyze Module c C Gro PEDAS ASP OF TL I
135. rmation BCP01G1 Students learn to assemble organize and integrate information BCPO1H1 Students learn to critically evaluate the validity and value of information obtained from their searches on the Internet BCPOII1 Students present work using several forms of presentation e g text visual verbal electronic BCPO1J1 Students are assigned projects that require several persons working together for an extended period of time BCP01K1 Students have autonomy to decide what topics to study BCPOIL1 1 2 3 Notatall Tosomeextent Alot APPENDIX B SITES 2006 PRINCIPAL QUESTIONNAIRE 2 To what extent do you agree or disagree that the school leadership you and or other school leaders encourages Mathematics and Science teachers at Grade lt target grade gt to achieve the following goals Please mark only one choice in each row 1 2 3 4 Strongly Disagree Agree Strongly disagree agree A To cover the prescribed curriculum content BCP02A1 B To improve students performance on assessments examinations BCP02B1 C To individualize student learning experiences in order to a
136. rnational database e SCHWGT School weight included in the school level files e MTOTWGT Total mathematics teacher weight included in the mathematics teacher level files e STOTWGT Total science teacher weight included in the science teacher level files In addition the following weighting factors are included e WGTFACI School weighting factor included in the school and teacher level files e WGTADJI School non response adjustment included in the school and teacher level files e WGTFAC2 Teacher weighting factor included in the teacher level files e WGTADJ2 Teacher non response adjustment included in the teacher level files The following variance estimation variables or jackknife variables are included in the SITES 2006 international database The actual replicate weights are computed on the fly and are not part of the data files e JKCZONE Jackknife zone to which the school is assigned included in the school level files e JKCREP Jackknife replicate to which the school is assigned included in the school level files e JKZONE Jackknife zone to which the teacher s school is assigned included in the teacher level files e JKREP Jackknife replication unit to which the teacher s school is assigned included in the teacher level files Furthermore the school level and teacher level files contain several stratification variables which are listed below See Chapter 6 and Appendix C of the SITES 2
137. rposes of our analysis As you may recall this information will be used in writing one of the chapters of the final report chapter 2 In addition it will be used in analyzing and interpreting the school and teacher results from the study It is very important for the study that you provide this information For some open ended questions you may want to write a long answer or revise a section of the chapter written about your country system in 2001 and published in 2003 in the Cross national ICT Policy book by Plomp Anderson Law amp Quale If so you may attach such answers to this questionnaire Among the SITES2006 countries only Canada and Estonia did not have such a chapter You will be interested to know that the publisher has invited us to publish a 2nd edition of that volume Our plan is to finalize the revised chapters by next sammer You will be invited to help us with that project in the near future 51 52 SITES 2006 IDB USER GUIDE Instructions This questionnaire consists of four brief subtopics A Educational system structure and responsibility B Teacher preparation C Change in past S years and D System wide ICT policies Answers to the questions in all of these sections are needed to contextualize the findings from the school and teacher survey data School specific policies are not sufficient for understanding the sources and root causes of pedagogical practices especially with respect to ICT of teac
138. rs please specify NCOIT ies opel ia epee a haute Requirements are defined only at local or school level NCQU7 Lf ween Explanations as needed 18 Are qualified teachers in the target grade required to undertake regularly any inservice and or professional development activities on any of the following NCQ172 59 SITES 2006 IDB USER GUIDE aspects Tick all that apply Major subject area of teaching NCQU81 a ss Pedagogical practice NCQ181b PE nen Dee no Cn ese E Enhancing students motivation to fon NCQI8Ic enero Child development NCQU81 evn ICT Skills NOOTE TE irinin aea R Use of ICT in subject based or cross curricular teaching NCQ181f Use of ICT in project based learning NCQ181g ccsncnnnnnnnnnnnnnnnnnnnnn Others please specify NCQ18 1h Requirements are defined only at local or school level NCQ18 Linn moa Moan mis Explanations as needed e e a E heien ima aaa nanencee NCQ182 19 Do any government agencies subsidize inservice training or professional development courses for teachers in any of the following areas Tick all that apply a ICT skills NCQ191a oe b Use of ICT in subject Tasio or cross Ca teaching NCQI91b tae Use of ICT in school and or classroom administration and or management work NCQ191c d Use of ICT for new approaches in
139. s Figure 4 9 SPSS Output for Regression is Oulpul3 Ducument3 SPSS Yiewer File Edit View Data Transform Insert Format Analyze Graphs Utilities Add ons Window Help SHAR B amp 6o BLS QO E Gus Fem OB 233 o Predictors RECZ1A1 Predicted RECCSRAT 1 N of Constant Constant RECZ1A1 REC2141 REC21A1 gt COUNTRY ID Cases Mult_RSQ estimate 3 e estimate s e t test Chile Chinese Taipei Finland Hong Kong SAR Israel SPSS Processor is ready The regression performed by our example uses the recoded variables REC21A1 and RECCSRAT By using these variables the intercept or constant will be the estimated student computer ratio for schools located in an area with 50 000 people or fewer whereas the regression coefficient REC21A1 estimate will be the estimated difference in the student computer ratio in schools located in an area with more than 50 000 people This will also allow us to perform a t test to determine if the student computer ratio is significantly different between schools located in areas with 50 000 people or fewer and schools located in areas with more than 50 000 people The results displayed in Figure 4 9 show that in Chile the estimated mean student computer ratio in schools located in an area with 50 000 people or fewer is 23 91 students per computer Constant estimate with a standard error of 2 25 Compared to this if a school is located in an
140. s a PC Please write a whole number Write O zero if none PDAs and smartphones phone integrated with PDA BCT07A1 Graphic calculators BCT07B1 Smartboards interactive whiteboard system BCT07C1 Projectors for presentation of digital materials BCTO7D1 In your school about what percentage of students bring any of the following to school Please mark only one choice in each row 1 2 3 4 S Lessthan 10 24 25 49 50 75 Morethan 10 75 PDAs smartphones BCTO8A1 Graphic calculators BCT08B1 Laptops BCT08C1 97 98 w J a we J awe SITES 2006 IDB USER GUIDE Where are the computers for teaching and learning in Grade lt target grade gt located Please mark only one choice in each row Most classrooms BCTO9A1 Some classrooms BCTO9B1 Computer laboratories BCTO9C1 Library BCTO9D1 Other places BCTO9E1 Who is involved in the maintenance of computers in your school Please mark only one choice in each row The school s own staff BCT10A1 Staff from other schools BCT 10B1 An external company hired by the school BCT 10C1 An external unit arranged by the ministry local regional authorities BCT 10D 1 1 2 No Yes 1 2 No Yes APPE
141. s variable BCTOSE1 within each level of the grouping variables IDCNTRY and BCP21A1 For about 7 of the schools located in areas with 3 000 people or fewer the school principals did not respond to the question about the number of computers with internet access ANALYSING THE DATA Conversely Chile has 80 schools representing an estimated 725 schools in the whole population being located in a city with more than 500 000 people Here the estimated mean number of computers with Internet access is 27 39 with a standard error of 2 82 4 4 3 Computing Regression Coefficients and their Standard Errors The analysis type Regression is used to perform single or multiple linear regression between a dependent explained variable and one or more independent to be explained variables This example demonstrates the use of this type of analysis which computes the regression coefficients and their corresponding standard errors Note that the IDB Analyzer provides convenient tools to estimate coefficients and sampling errors reflecting the sample design only and that analysts would need to verify any assumptions about the data s characteristics distributions etc beforehand e g normality outliers multicolinearity or heteroscedascity This example will show a linear regression with the location of the school recoded variable REC21A1 as a predictor of the student computer ratio computed variable RECCSRAT using the weight variable SCHWGT It
142. system s results are presented in four lines two lines for each value of teachers gender variable BTG32A1 and one line for each value of usage of ICT variable BTG18A1 within each teachers gender group The results are presented in much the same manner as in the previous example where the education systems are identified in the first column and the second and third columns describe the 47 48 SITES 2006 IDB USER GUIDE categories of BTG32A1 gender and BTG18A1 ICT use being reported The first two lines in Figure 4 19 show that in Chile there are 49 02 male mathematics teachers using no ICT in the teaching and learning activities of the target class while 50 98 of the male mathematics teachers use ICT In comparison 40 81 of the female mathematics teachers in Chile do not use ICT whereas 59 19 of the female mathematics teachers use ICT in their teaching and learning activities in the target class For all four percentages the appropriate standard errors are displayed in the last column of the output Please note that the second and the fourth line for each education system match the results presented in Columns 3 and S in Table 6 2 of the SITES 2006 international report see Figure 4 16 The standard errors in Figure 4 19 are correct Attention Point Note that the standard errors are different from those displayed in the SITES 2006 international report because the standard errors in Chapter 6 of the SITES 2006 internatio
143. t column The second column describes the categories of BCT04A1 being reported The third column reports the number of valid cases and the fourth the sum of weights of the sampled schools The last two columns display the percentage of schools in each category and its standard error The first three lines in Figure 4 15 show that in Chile 46 90 of schools have equipment and hands on materials available 51 29 of schools need equipment and hands on materials but do not have it available and 1 81 of schools reported that they do not have and do not need this type of technology applications and facilities available They also show the standard errors for the percentages in each category Note that the first line for each education system in the output in Figure 4 15 matches the percentages and standard errors reported in Table 4 5 Figure 4 12 in the SITES 2006 international report 4 5 2 Analysis with Teacher level Variables The example of a teacher level analysis will investigate the percentage of mathematics teachers using ICT by gender The results of such an analysis are presented in Table 6 2 of the SITES 2006 international report Figure 4 16 Parts of it are reproduced here in Figure 4 19 After merging the mathematics teacher data for five education systems Chile Chinese Taipei Finland Hong Kong SAR and Israel the example will use the analysis type Percentages only ANALYSING THE DATA Figure 4 16 Table 6 2 from the SITES 2006
144. t using weights in data analysis can lead to severely biased results The following example illustrates the importance of using weights in research with SITES 2006 data A researcher may be interested in the average number of girls in schools in Thailand variable BCP19A1 in file BCG Using unweighted data e g in SPSS it seems to him that the average number of girls in each school is approximately 47 Figure 3 1 Example of Un weighted Analysis in SPSS Descriptive Statistics N Mean SHCHA TOTAL NUMBER OF GIRLS IN SCHOOL 417 546 99 Valid N Ilistwise 417 WEIGHTS AND VARIANCE ESTIMATION Using weighted data with the IDB Analyzer it shows that in Thailand the estimate for the average number of girls per school taking differential selection probabilities into consideration is actually only 301 Figure 3 2 Example of Weighted Analysis Using the IDB Analyzer Average for BCP1941 by IDCNTRY N of Sum of COUNTRY ID Cases SCHUGT Thailand 417 9003 The large difference between the un weighted and the weighted result can easily be explained by the SITES 2006 sampling design in which the proportion of large schools in the sample is higher than in the population The sampling weights compensate for that disproportional school sample allocation and not using weights leads to an incorrect and biased population estimate Results equivalent to the above can be obtained by using the SPSS Weight Cases
145. ta file called BCGALL SAV that was merged in the previous step As type of the analysis select Correlations 4 In the next steps all variables need to be defined a As Grouping Variable the software always selects variable IDCNTRY by default No other variable needs to be added for this example b Next select the analysis variables To activate this section press the radio button Analysis Variables This time select variables BCPO06G1 and BCPO6H1 from the list of variables and move them to the analysis variables window by pressing the right arrow button in this section S The weight variable is automatically selected by the software As this is an example for analysis at the school level the weight SCHWGT is selected by default Additionally the jackknifing variables JKCZONE and JKCREP are defined 6 Specify the name and folder of the output files in the Output Files field The IEA IDB Analyzer will use this name and folder to create three output files i an SPSS syntax file that contains the code for performing the analysis ii an SPSS data file with the results and iii an Excel file with these same results 7 Press the Start SPSS button to create the SPSS syntax file and open it in an SPSS syntax window The syntax file will be executed by opening the Run menu of SPSS and selecting the All menu option If necessary the IEA IDB Analyzer will produce a prompt to confirm the overwriting of already existing files Figure 4 10
146. te weights on the fly when an analysis is conducted 3 4 3 Example for Variance Estimation Not using the jackknife variables in data analysis will lead to incorrect estimates of sampling precision The following example illustrates the importance of using the JRR technique in research and analysis with SITES 2006 data In this example problems occur when the statistical analysis package SPSS is used for data analysis The IEA IDB Analyzer was specifically designed to overcome these problems Analysts should note that in addition to the IDB Analyzer a growing number of other software packages become available to handle the paired jackknife replication method implemented in SITES 2006 The software package Wesvar Westat 2008 for complex sample analysis is made available free of charge by Westat Inc and may be downloaded from Westat s webpage SAS Version 9 2 SAS Institute 2008 also includes support for the replication method used in SITES 2006 and is commercially available from SAS Institute Inc WEIGHTS AND VARIANCE ESTIMATION A researcher may be interested in the average size of the SITES 2006 target class in Thailand variable BTGO1A1 Using SPSS directly and thereby ignoring the complex sample design and clustering effects he finds that the weighted average class size is about 31 8 and the standard error seems to be close to 0 1 Figure 3 3 Example of Incorrect Variance Estimation in SPSS Descriptive Statistics oN m
147. ten test examination BCP11A1 Written task exercise BCP11B1 Individual oral presentation BCP11C1 Group presentation oral written BCP11D1 Project report and or multimedia product BCPI1E1 Students peer evaluations BCP11F1 Portfolio learning log BCP11G1 Group assessment scores for collaborative tasks BCP11H1 1 2 3 4 Strongly Disagree Agree Strongly disagree agree APPENDIX B SITES 2006 PRINCIPAL QUESTIONNAIRE Staff Development for Teachers and the School Leadership The following contains a number of questions about staff development for Mathematics and or Science teachers teaching Grade lt target grade gt and for the school leadership 12 Are teachers of Mathematics and or Science at Grade lt target grade gt required or encouraged to acquire knowledge and skills in each of the following Please mark only one choice in each row 1 2 3 No Yes Yes Encouraged Required A Integrating Web based learning in their instructional practice BCP12A1 Using new ways of assessment portfolios peer reviews etc BCP12B1 Developing real life assignments for students BCP12C1 ios Using real life assignments developed by others BCP 12D 1
148. that does not automatically normalize weights but demands the sum of the weights to be equal to the achieved sample size i e the mean of all weights is equal to 1 A house weight with this property can easily be computed With n being the number of responding mathematics teachers in an education system for each mathematics teacher i the variable can be computed as n HOUWGT MTOTWGT gt XMTOTWGT 1 BEBCP1941 Mean 25 26 SITES 2006 IDB USER GUIDE Similarly for science teachers STOTWGT instead of MTOTWGT should be used and for schools SCHWGT should be used 3 4 Variance Estimation 3 4 1 Why Variance Estimation Is Needed Since all estimates statements and inferences in SITES 2006 are based on sample data they can only be stated with a degree of uncertainty With all research that is performed using SITES 2006 data the precision of the population estimates should be reported Because of the complex sampling design used in SITES 2006 it is not possible to calculate standard errors or to easily perform significance tests with standard software packages While these programs implicitly assume that the data is derived from a simple random sample the SITES 2006 teacher data come from a two stage stratified cluster sample each school being regarded as a cluster of teachers Any method for estimating sampling variance must take this design into account The SITES 2006 international data
149. the IEA website at http www iea nl 4 3 Merging Files with the IEA IDB Analyzer Data from IEA studies such as SITES 2006 are generally distributed separately by education system and file type The merge module of the IEA IDB Analyzer allows the user to combine data from different education systems into a single dataset for analysis It also facilitates the combination of data from different sources e g school and mathematics teacher files into one SPSS dataset Combining files from different education system will permit conducting cross education system analyses When running the merge module the IEA IDB Analyzer creates SPSS code that can be used later without need for the IEA IDB Analyzer itself The data files created using the merge module can be processed either with the analysis module of the IEA IDB Analyzer or by any other analysis software that accepts SPSS input files To create an SPSS data file with more than one education system s data or to combine more than one file type analysts should do the following 1 Open the merge module of the IEA IDB Analyzer from the Windows Start menu 2 In the Select Data Directory field browse to the folder where the SPSS IDB data files are located For example in Figure 4 1 all SPSS data files are located in the C IntData SITES Data folder The program will automatically recognize and complete the study Type Year and Grade fields and list all education systems available in
150. the name of any school or individual be identified While results will be made available by country and by type of school within a country you are guaranteed that neither your school nor any of its personnel will be identified in any report of the results of the study For countries which have ethical survey guidelines which emphasize voluntary participation Participation in this survey is voluntary and any individual may withdraw at any time About this Questionnaire e This questionnaire asks for information from schools about education and policy matters related to pedagogical practices and computers We would like the person who completes this questionnaire to be the principal of the school Ifyou do not have the information to answer particular questions please consult other persons in the school This questionnaire will take approximately 30 minutes to complete e The words computers and ICT Information and Communication Technologies are used interchangeably in this questionnaire e Please note that some questions refer to the entire school while other questions refer to Grade lt target grade gt only For countries in which the definition of school is not obvious to respondents add appropriate description depending on how sampling units were defined in the national sampling plan When questions refer to your school we mean by school lt national school definition gt e Guidelines for answering the questions a
151. this folder as possible candidates for merging If the folder contains data from more than one IEA study or from more than one grade the IEA IDB Analyzer will prompt users to select files from the desired study and grade for analyses 3 Select the education systems of interest from the Available Participants list and use the arrow buttons to include them in the list of Selected Participants To select multiple education systems hold the CTRL key of the keyboard when selecting the education systems In Figure 4 1 Chile Chinese Taipei Finland Hong Kong SAR and Israel have been selected see page 31 Notice that the education systems are displayed in alphabetical order sorted by the education system s name and not by the system s three letter code 4 Press the Next gt gt button to proceed The software will open the second window of the merge module as shown in Figure 4 2 to select the file types and the variables to be included in the merged data file 5 Select the file types for merging by checking the appropriate boxes to the left of the window For example in Figure 4 2 the school data file is selected 6 Select the variables required from the list of variables available in left panel Please note that identification and sampling variables are selected automatically by the IEA IDB Analyzer In the example shown in Figure 4 2 all school variables are selected 7 Specify the desired name of the merged data file and the folder wher
152. this example all school variables were selected The merged file was created using the merge module of the IEA IDB Analyzer 4 4 1 Computing Percentages only and their Standard Errors To compute percentages of variables with their JRR standard errors analysts will need to select Percentages only as the type of analysis This type can compute the percentages within specified subgroups and will also compute the appropriate standard errors for those This example will compute the percentages for school principals giving high priority to a decrease of the student computer ratio BCP06A1 and its standard error within each education system IDCNTRY using the weighting variable SCHWGT The data will be read from the data file BCGALL SAV and the standard errors will be computed based on 100 replicate weights The steps in the IEA IDB Analyzer are as follows 1 Open the analysis module of the IEA IDB Analyzer 2 Select the data file called BCGALL SAV that was merged in the previous step 3 As type of the analysis select Percentages only Note that the program by default will exclude missing values on the grouping variable from the analysis This can be deactivated by removing the checkmark from the option Exclude Missing from Analysis 4 In the next step analysts will need to define the grouping variables As Grouping Variable the software always selects variable IDCNTRY by default since computing estimates for the entire SITES
153. tionally comparable that national adaptations were reflected appropriately in all concerned variables that questions not internationally comparable were removed from the database and eventually that all entries could be successfully linked between levels 2 3 Survey Variables In addition to the data from the responses to the questions in the questionnaires Section 2 4 the data files contain some more variables with identification Section 2 3 1 tracking and administrative information Section 2 3 2 Furthermore sampling and weighting variables Section 2 3 3 are included in all school level and teacher level data files 2 3 1 Identification Variables The following identification variables are used to uniquely identify each record in the data files e IDCNTRY Two to four digit numeric code identifying the education system If possible the ISO 3166 1 codes were used For some education systems that are not 16 SITES 2006 IDB USER GUIDE represented in the ISO 3166 1 code list a numeric code was created and assigned by the IEA DPC see Table 1 1 IDSCHOOL In all school and teacher level files this one to four digit numeric code identifies the school within an education system The codes are only unique within an education system To uniquely identify schools across education systems IDSCHOOL has to be used in combination with IDCNTRY IDTECH This three to six digit numeric code in the school level file identifying ICT
154. truct of interest than the individual variables could on their own The key methods for extracting and verifying multi item constructs were exploratory and confirmatory factor analysis Scale variables in SITES 2006 were computed as simple sum scores of the manifest variables in contrast to other methods employed in similar research such as polytomous IRT scoring or regression based factor scores based on score coefficients For example teachers self reported technical competence as reported on Pages 191 194 in the international report was based on eight manifest variables from the teacher questionnaire Question 21 Items A H The corresponding syntax in SPSS would be as follows Figure 2 1 Example SPSS Syntax to Create a Sum score for the scale GEN_IT GEN_IT MEAN 8 BTG21A1 BTG21B1 BTG21C1 BTG21D1 BTG21E1 BTG21F1 B1G21G1 BTG21H1 As can be seen the mean GEN_IT of variables BTG21A1 to BTG21H1 is computed for any record that has data available for all eight variables In the above SPSS function MEAN the 8 portion specifies the minimum number of valid arguments In general records were included in the derived variable calculation only if there were data available for all of the variables involved This approach is usually referred to as listwise deletion Attention Point To consistently reproduce the composites and their analysis in the international report it is necessary to follow the listwise deletion approach
155. ty of learning activities BTG40L1 Dependent Adaptation to individual needs of students BTG40M1 Dependent Amount of effort needed to motivate students BTG40N1 Dependent Insight into the progress of student performance BTG4001 Dependent Self confidence BTG40P1 Dependent 1 Decreased 2 Made no difference 3 Increased 127 SITES 2006 IDB USER GUIDE 41 In this pedagogical practice who was the main actor in initiating the following aspects of teaching and learning Please mark only one choice in each row NA Not applicable for this specific pedagogical practice 1 2 3 Teacher Students NA Determining content BTG41A1 Dependent Determining learning goals BTG41B1 Dependent Getting started BTG41C1 Dependent Organizing grouping BT G41D1 Dependent wt o awe Choosing learning resources materials BTG41E1 Dependent Deciding on the location of learning BTG41F1 Dependent G Planning of time BTG41G1 Dependent H Deciding on the time needed for learning BTG41H1 Dependent I Deciding on when to take a test BTG4111 Dependent Ti
156. unction with the technical report The international report Law Pelgrum amp Plomp 2008 is another key resource for the analysis itself Using all these publications will allow analysts to confidently replicate the procedures used and to accurately undertake new analyses in areas of special interest 1 Further information about the international report its supplementary online appendices as well as order details and links are provided on the study s website at http www sites2006 net SITES 2006 IDB USER GUIDE 1 2 An Overview of SITES 2006 For more than 50 years IEA has been conducting comparative studies in education IEA s Second Information Technology in Education Study SITES produced internationally comparable indicators on information and communication technology ICT usage and practices as well as an in depth analysis of the way in which ICT impacts teaching and learning processes SITES 2006 is the fourth in IEA s series of ICT related surveys which began in 1987 with the IEA Computers in Education Study COMPED followed by SITES Module 1 1997 2000 and SITES Module 2 2000 2004 COMPED focused on the use of computers in schools and the effects of this usage within schools SITES Module 1 looked at the readiness of schools to integrate ICT into teaching and learning SITES Module 2 concerned 174 case studies of innovative pedagogical practices in different countries examining and comparing the characteristics of
157. ur answers to the previous question question 20 regarding recent 5 year trends in ICT spending pedagogy and assessment Consider if there were any trends that were different for the schools with the target grade Please describe any such trends and how these differed from compulsory schools as a whole within your system NCQ21 22 In the past 5 years were there any national or system wide curriculum priorities in Math and or Science that changed or policies in these areas that were initiated Please summarize these changes and how they might help to explain findings from the principal and teacher surveys NCQ22 APPENDIX A SITES 2006 NATIONAL CONTEXT QUESTIONNAIRE 4 23 Does your system have a national or system wide ICT policy in education National or system wide policies and practices on the use of ICT Such a policy may consist of several separate policies Yes Please continue with next question No Go to question 25 NCQ23 24 Which of the following components are included in this ICT policy Tick all that apply a Clear vision and goals for ICT NCQ24 a eennnnnnnnnnnnnnnnne b An explicit goal that the use of ICT is to support curriculum uuu innovation NCQ24b c Descriptions of specific desired modes for integrating ICT in on teaching and learning NCQ24c d Desired minimum level of access e g student computer ratio one NCQ24d e Desired level of internet connectivity NCQ24 u mmcnnnnnnn
158. using the Not Reached code the last valid answer given ina questionnaire was identified The first omitted response after this last answer was coded as Omitted but all following responses were then coded as Not Reached For example and assuming the SPSS data file format the response pattern 1 9 4 2999999 where 9 represents Omitted is recoded to 1942977777 where 7 represents Not Reached When recoding Omitted values to Not Reached all Not administered values were ignored For example the pattern 3 1 5 2 9 9 9 8 9 9 where 8 represents Not Administered would be recoded to 3152977877 This code was assigned by the IEA DPC after data collection In SPSS files Not Reached variables are coded to 7 97 997 and so on depending on the field length of the variable In SAS data files the code for Not Reached is R 20 SITES 2006 IDB USER GUIDE 2 6 Variables and Scales Derived from the Questionnaire Data The SITES 2006 international report Law Pelgrum amp Plomp 2008 was based on analyses of individual variables as well as composites created from multiple variables In the SITES 2006 questionnaires there were several questions about various aspects of a single construct In these cases responses to the individual items were combined to create a derived variable that provided a more comprehensive picture of the cons
159. ut the education system it is necessary to take into account the complex nature of the sampling design implemented in SITES 2006 Details about the sampling design are reported in Chapter 6 of the SITES 2006 Technical Report Carstens amp Pelgrum 2009 The SITES 2006 sampling design called for different selection probabilities at the school and at the teacher sampling level Sampling weights reflect and compensate the disproportional selection probabilities of the schools as well as of the teachers If any unit of response had a small selection probability this was compensated with a large weight and vice versa Given that some sampled schools and teachers refused to participate in SITES 2006 it was necessary to adjust the sampling weights for the sample size loss Thus the sampling weights were multiplied by non response adjustments The final total weights are the product of weight factors and adjustment factors that reflect the selection probabilities and the non response patterns at all levels of analysis Details about weighting and adjustment are reported in Chapter 10 of the SITES 2006 Technical Report Carstens amp Pelgrum 2009 3 2 2 Selecting the Appropriate Weight Variable For analysis concerning a single level different weights must be applied e For school level analyses principals technical coordinators SCHWGT must be used e For mathematics teacher analyses MTOTWGT must be used e For science teacher analyses
160. vide activities which incorporate real world examples settings applications for student learning BTGO8D1 To improve students performance in assessments examinations BTG08E1 To increase learning motivation and make learning more interesting BT GO8F1 To individualize student learning experiences in order to address different learning needs BTGO8G1 To foster students ability and readiness to set their own learning goals and to plan monitor and evaluate their own progress BTGO8H1 To foster students collaborative and organizational skills for working in teams BTGO8I1 To foster students communication skills in face to face and or online situations BTG08J1 To satisfy parents and the community s expectations BTGO8K1 To prepare students for competent ICT use BTGO8L1 To prepare students for responsible Internet behavior e g not to commit mail bombing etc and or to cope with cybercrime e g Internet fraud illegal access to secure information etc BTGO8M1 1 Not at all 2 A little 3 Somewhat 4 Very much
161. will compute the regression coefficients and their standard errors The data will be read from the data file BCGALL SAV and the standard errors will be computed based on 100 replicate weights Please note that this example uses the values of the variables BCP19A1 and BCP19B1 as well as the values of variable BCTOSA1 to compute the student computer ratio in variable RECCSRAT This is done by dividing the sum of the enrollment of girls BCP19A1 and boys BCP19B1 in the school by the number of computers in the school BCTOSA1 The values of the variable BCP21A1 indicating the location of the school are recoded into variable REC21A1 The categories were combined into two groups often referred to as dummy coding The first group of schools is located in an area with 50 000 people or fewer The other group of schools is located in an area with more than 0 000 people Please refer to the output section for an explanation of why this was done These recodes should be done using SPSS directly No recodes can be performed using the IEA IDB Analyzer The IEA IDB Analyzer does not support direct recoding of data The SPSS code for the recodes that reads the merged data file BCGALL SAV and saves it under the same name including the new variables RECCSRAT and REC21A1 is presented in Figure 4 7 Note that this code would need to be run directly in SPSS Figure 4 7 Example SPSS Syntax to Dummy Recode Variables for Analysis get file
162. x E Characteristics of the Australian Sample Sl 67 91 105 129 List of Tables and Figures Table 1 1 Figure 2 1 Figure 3 1 Figure 3 2 Figure 3 3 Figure 3 4 Figure 4 1 Figure 4 2 Table 4 1 Figure 4 3 Figure 4 4 Figure 4 5 Figure 4 6 Figure 4 7 Figure 4 8 Figure 4 9 Figure 4 10 Figure 4 11 Figure 4 12 Figure 4 13 Figure 4 14 Figure 4 15 Figure 4 16 Figure 4 17 Figure 4 18 Figure 4 19 Table E 1 Table E 2 Participating Education Systems in SITES 2006 Example SPSS Syntax to Create a Sum score for the scale GEN_IT Example of Un weighted Analysis in SPSS Example of Weighted Analysis Using the IDB Analyzer Example of Incorrect Variance Estimation in SPSS Example of Correct Variance Estimation using the IDB Analyzer Merge Module Selecting Participants Merge Module Selecting File Types and Variables Types of Analysis for SITES 2006 Data Analysis Module Computing Percentages only SPSS Output for Percentages only Analysis Module Computing Percentages and Means SPSS Output for Percentages and Means Example SPSS Syntax to Dummy Recode Variables for Analysis Analysis Module Computing Regression SPSS Output for Regression Analysis Module Computing Correlations SPSS Output for Correlations Table 4 5 from the SITES 2006 International Report for Example School level Analysis Law Pelgrum amp Plomp 2008 pp 81 Merge Module for Example School level Analysis Analysis Module for Exampl

Download Pdf Manuals

Related Search

Related Contents

  サポート＆サービスのご案内  Canada - Buyandsell.gc.ca  Rowenta Sew 'n' Press Steam Iron User's Manual  Adobe Fireworks CS3 User's Manual  Descarga aquí  exquis mots 4  Title ものづくりから気づくこと : 誰が製作するのか  EURO 22 - 29 BAC Plus 2/A  ML 5.0 Smoothed Aggregation User's Guide  

Copyright © All rights reserved. Failed to retrieve file