Home

View/Open - San Diego State University

1. 33 B DIAGNOSTIC ASSESSMENT EXTENSION OF CAST RENEWAL APPROVAL FOR PROTOCOL 02 09 278 2 2 04042 00000 0 35 Es USER MANUALE aba 39 viii LIST OF TABLES Table 2 1 Comparing Language Proficiency 05 6 Tale estne e ce ioo EE A acs ne pe et RA 29 1X LIST OF FIGURES PAGE Figure 3 1 2 J2EE atebitecturec scot cat ae eds Se rach eeu ead eee 9 Figure 4 1 MIL class dia gran uo S o 12 Figure 4 2 Activity diagram for the 13 Figure 4 3 Part A of activity diagram in Figure 4 2 exporting data in Excel sheets 14 Figure 4 4 Part B of activity diagram in Figure 4 2 pop up 15 Figure 4 5 Part C of activity diagram in Figure 4 2 show hide columns 16 Figure 4 6 Database diagram for facilitating research with learner s data 17 Figure 4 7 Main screen of the application for facilitating research at LARC 17 Figure 4 8 Data columns show hide and Excel sheet report functionality 18 Figure 4 9 Excel report of the pre assessment survey data with global ratings 19 Figure 4 10 Post survey data with global ratings exported in 240242422221 19 Figure 4 11 Uncheck a categ
2. 8 3 2 System Regue MENiS ais O tM d 8 223 TEM Lee se S 9 3 3 1 TayazServlets itae iouis Cete us ved oe cedit 9 LM C ia 9 AN o di 10 JEX APE e o en aage Sen con 10 4 SOFTWARE AND SYSTEM DESIGN AND DEVELOPMENT 11 4 1 Class Dia Brain sini EP 12 4 2 DIA STAINS A ties 12 4 3 Database Design oe He taa bee 14 4 4 Application Ee Ree 16 4 4 1 Excel Sheet Functionality and Show Hide Functionality 16 4 4 2 Pre Survey Excel Sheet ee n edet Reto ei 18 4 43 Post uryey Excel oso Sca uto d gie p RUND e CU V nde tes 18 4 4 4 Column Show Hide Functionality eere 18 2 4 5 View User s Informati es aereo sees ee ope A aedis 21 AAG Th POPUP Men TT NE 21 4 4 7 Multiple Column Filters treten e 24 5 gt TESTIN Geiri ii 28 6 CONCLUSION AND FUTURE WORK ten PEE neti 31 eee 31 6 2 F t re Work 31 lt lla Os E 32 APPENDICES A DIAGNOSTIC ASSESSMENT USING CAST CONSENT
3. Add Language to DA Databases y Add Language Demographics Add Context User Logged in v_shah Add Function Edit Items Approve Create Excel Sheet For CAST Pre Survey Data CAST Post Survey Data CAST Post Survey Data amp CAST Pre Survey Data Email CAST Users Select Columns To Be Displayed In The Table View Algorithm Tests Demographics CAST Pre Survey Language often spoken in your home CAST Post Survey Give responses that reflect Recorded voice properly guage poken in y A ee Test ID Your native language Current TESTING course number ability to speak The test delivery software Hindi Administrator Name B live in area language was predominant Years of formal language study Felt comfortable giving my could be improved by Please review translations and edits Language language of instruction primary Residence length Test Directions regularly and determine when an it Test Date Lenguas of insiniction secondary Understood what I was expected to do Enjoy dni Timed Pauses is ready to be used in the CAST test Registration Code Taken a Study Abroad course Understood questions being asked Items could be improved by Nervousness During the Test Review Hindi Items Age Instructions for giving responses clear Y TESTING programran Ideas for Test Improvement ly Gender Global Ratings Had enough time to prepar
4. ADN Figure 4 12 Uncheck TestID and name categories to hide the corresponding columns in the learner s data table In Figure 4 13 the Test ID checkbox is checked again and this display the corresponding Test ID column in the Learner s Data Table keeping the Name column is still hidden Therefore the user can randomly select the columns he she wants to see by checking their corresponding checkboxes 21 Add Language to DA Databases a Select Columns To Be Displayed In The Table Add Language Add Context CAST Pre Survey Language often spoken in your home CAST Post Survey Give responses that reflect Recorded voice properly Add Function Your native language Current TESTING course number ability to speak The test delivery software Edit tams Name liveinara language was predominant Years of formal language study Felt comfortable giving my be improved by Language a ON amp Residence length responses 8 Test Directions Approve Items language of instruction primary 4 Enjoyed taking this test Test Date language of instruction secondary Understood what I was expected to do 3 Timed Pauses Email CAST Users 84 Registration Code 4 Taken a Study Abroad course Understood questions being asked Items could be improved Nervousness During the Test x ct jvi 4 TESTING program ran Visi Aigorkdun Tesis E Age f highest level of educat Instruct
5. This is an attempt to provide test takers feedback There are several levels defined to indicate the proficiency of the speaker Advance level At level Not at Level ACFTL works together with CAST and NCATE for training teacher to understand about the proficiency levels For a foreign language researcher just looking at a single score of pass or no pass at an audio test is not useful enough in terms of understanding to contribute to achieving advance level so it is essential to find out what factors or individual profiles like the background education etc combine together that contribute in achieving Advance level Ray T Clifford LARC board member and Associate Dean Director Center for Language Studies rightly said Life begins at the Advance Level It is a part of the language researcher s job to determine how a person who has attained an advance level handles a situation In addition what are the factors that facilitate achieving an Advance level I interviewed Dr Mary Ann Lyman Hager Director of LARC Nadezda Novakovic Researcher at LARC Dr Trevor Shanklin Director of Lab Operations LARC and Mike Pasamonik Software Engineer as a part of requirements gathering Dr Mary Ann Lyman Hager had in mind a project idea for creating an application for language proficiency data that can be used by the researchers at LARC for various different research purposes She introduced me to Dr Trevor Shanklin and Nadezda to col
6. I m not very Strongly Strongly Strongly Strongly Strongly Strongly liked iL Easy to unc I had all the No I would 21 na Agree Disagree Agree Agree Agree Agree Strongly D Disagree Disagree foelthatiiAgree Strongly Strongly A Agree Strongly Strongly A it worked piciear were about A LOT otv 22 Spanish205 5 i Strongly A Strongly A Strongly A Strongly A Strongly A Agree Strongly A Strongly Strongly My test incl Agree Strongly A Strongly Strongly Strongly A Strongly A Sometimes They were It was just I think wot 23 322 355 451 b n Strongly A Strongly Agree Strongly Strongly A Agree Agree Agree Agree the real Agree Agree Agree Disagree Agree Agree didn t reali Make sure 1 think it affi 24 SPAN 451 SPAI S Strongly A Strongly A Strongly A Strongly A Strongly Strongly A Strongly null Strongly A More intera Strongly A Strongly A Strongly A Strongly A null Strongly would pre They were The pause was not ni 25 iyear Strongly Agree Agree Agree Agree Strongly AAgreo Agree Agree Well my S Strongly Agree Agree Agree Agree Agree lthoughtit Mention ho Fine Tm always 26 iyear Strongly Agree Agree Agree Agree Strongy Agree Agree Agree Weil my SjStrongly Agree Agree Agree Agree Agree thought it Mention ho Fine Im always 27 iyear Strongly A Agree Agree Agree Agree Strongly A Agree Agree Agree Well my Sj Strongly A Agre
7. Accurate idea of reponses Ul Had enough time to say what I wanted SERA heaudio _ what extent Hindi Administrator to say Listening to test instructions Was asked to talk about a variety of Procedure for recording Please review translations and edits regularly and determine when an i is ready to be used in the CAST Review Hindi Items Arabic Item Review Dates As a member of the CAST Consortium we ask that please review the prompts linked below for contextual appropriateness for the test takers that you deal with For example Questions about careers may not be contextually relevant Military Personnel just startir v 21 2007 n RETA om Mala Na 74 4 Figure 4 11 Uncheck a category to hide it in learner s data table In Figure 4 12 the Name checkbox is unchecked and this hides the corresponding Name column in the Learner s Data Table Add Language to Databases Select Columns To Be Displayed In The Table Add Language Add Context CAST Pre Si i CAST Post Surve i i jarvey Language often spoken in your home y Give responses that reflect Recorded voice properly Add Function Test ID Your native language Current TESTING course number ability to speak The test delivery software ams O Name B live in area language was predominant Years of formal language study Felt comfortable givi
8. LARC Language Acquisition Resource Center LRC Language Resource Center LTI Language Teaching Institute xi ACKNOWLEDGEMENTS This thesis would have been a difficult undertaking without the direction of my thesis committee chair Professor Carl Eckberg I am also grateful to my other committee members Professor Mary Ann Lyman Hager and Professor Joseph Lewis for their constant support and guidance towards the development of this thesis I would also like to thank Mike Pasamonik Nada Novakovic and Trevor Shanklin for their help and support 1 INTRODUCTION A language learning resource center promotes learning and teaching of foreign languages It offers professional development workshops generates language learning materials and conducts research on foreign language learning Language Acquisition Resource Center LARC at San Diego State University SDSU is also a similar LRC that conducts research on foreign language learning The language researchers at LARC have to study a language corpus which is a large collection of learner data to understand with precision how a learner performs at different difficulty levels and many other factors that help improve the quality of the CAST Computer Assisted Screening Tool test at LARC Consequently there was a need to create an application that will facilitate this research at LARC 1 1 ABOUT LARC In 1990 the first Language Resource Centers LRCs were establishe
9. DIAGNOSTIC ASSESSMENT EXTENSION OF CAST RENEWAL APPROVAL FOR PROTOCOL 02 09 278 35 1 SAR DIEGO STATE Phase ATARA Liv ERSITY TAPSE IN APPROVAL 8 11 2011 through amp 22 2011 NEW IRE REQUIREMENTS August 22 2011 IAB Number 3333 Erentz Contracts 52543 ahe federat other Rigulatory Determination Risk Levert Minimal Conbrustion approved per 45 CFR 25 110 Catesores 6 27 Deer Dr Lyrran Hager The S061 institubonal Review Board approved the project referenced for continustion on August 22 204 in accomiance with SDGLIs Assurance and federal requirements pertaining tn human subjects protections within the Code of Federal Regulations 45 CFR H CFR 50 This approval only applies to thee analysis of data collected prior to 6 11 11 and the recruitment al human subjects and data collection alter B 22 2011 and analysis of that data based on procedures described your protocol amd tha information you provided within pour report of progresa Approval carries with it the understanding that you will contact the IR bo obtain authorization to implement any proposed change to the protmcnis to document a change in your alison with SDSU student fsculby or staff and far to report the completion of data Since a review of your request to continue ta recruit human subject wes not able to be conducted until deba m Report of Progress must be submitted prior tn the expiration de
10. Felt comfortable giving my could be improved by Language language of instruction primary Residence length responses Test Directions Hindi Administrator amp Tes Date language c instruction E Understood what I was expected to do Enjoyed uiking thistest Y Timed Pauses Please review translations and edit 8 Registration Coleg Taten a Study Abroad course 8 Understood questions being asked ems could be improved by 4 Nervousness During the Test regulary and determine when Age highest level of education 8 Instrcsions for giving responses clear 9 TESTING program ran for Test Improvement ls ready to be used inthe CAST te 6 Genter Global Ratings enough time to prepare my p MM Other Comments Review Hindi Items responses get Accurate ides of reponses 8 Had enough ime to say what wand Volume level ofthe audio Listening 0 test instructions Was asked to tlk abouta variety of 8 Procedure for reconding topics responses Arabic Item Review linked below for contextual appropriateness for the takers that you deal with For example 1 J 1 1 1 Questions about careers 1 not be contextually relevant Military Personnel just out 1 1 T 1 1 1 Approve Test items View My Approval Record View CAST Ratings M Jonathan Williams es 2007 03 23 00 00 00 0 BETA OPI n Male No English No
11. Acquisition Research Center LARC at San Diego State University The Institutional Review Board at SDSU has approved this study The purpose of this study is to obtain information about what foreign language learners speech sounds like at various levels of language proficiency Another purpose of our research is to develop a framework to give language learners feedback on their language performance Anyone 18 years and older may participate Your participation including the pre test questionnaire and the on line speaking test is very important to us and should last less than one hour Depending on where you participate you may or may not receive financial compensation If you are participating at an institution where you do receive an incentive you may receive up to In a pre test questionnaire we will ask information about your language background and your experiences using the language In this questionnaire there are no right and wrong answers You will next be given an on line speaking test and will record your responses to specific questions Questions will consist of two parts The first part describing a situation is both written and recorded The second part is an audio prompt Your recorded answers will be sent to an evaluator for analysis You may receive some basic feedback on your language skills Data from the speaking test will also be valuable to linguistic researchers who examine speech patterns or structures used by language l
12. CAST Post Survey Data Il CAST Post Survey Data amp CAST Pre Survey Data Select Columns To Be Displayed In The Table View Algorithm Tests Demographics CAST Pre Survey Language often spoken in your home CAST Post Survey Give responses that reflect Recorded voice properly ee Test ID Your native language Current TESTING course number ability to speak The test delivery software Hindi Administrator Name liven area language was predominant M Years of formal language study Felt comfortable giving my could be improved by Please review translations and edits Language language of instruction primary Residence length RR Test Directions regularly and determine when an it Test Date language of instruction secondary Understood what I was expected to do Enjoyed taking this test Timed Pauses s ready to be used in the CAST testi Registration Code Taken a Study Abroad course A Understood questions being asked Items could be improved by Nervousness During the Test Review Hindi Items Age highest level of education Instructions for giving responses clear m program ran Haces for Test Improvement Gender Global Ratings Had enough time to prepare my Fidelity of the audio E Any Other Comments Arabic Item Review Tesponses get Accurate idea of reponses Had enough time to say what I wanted Volume level of the audio hat extent As a mem
13. a reliable floor or baseline rating 2 Provide feedback to the examinees on how their proficiency can be improved 3 Estimate examinees likely score on the official ACTFL Oral Proficiency Interview OPI 4 Provide positive feedback for proficiency based teaching With the newly added reviewer interface instructors can effectively use the CAST as a tool to rate their students oral production and to increase their own understanding of the criteria by which ACTFL ratings are given 4 With regard to CAST it is the intersection of three fields computational linguistics applied linguistics corpora and language pedagogy A researcher would have his her hands full in any one of these fields The researcher may not end up helping the test taker but the field as a whole or the instructor Perhaps where CAST best shows the intersection of research and benefits for the test taker are the way the feedback cycle is set up A language corpus is created of spoken learner data from the CAST which is helpful for the following Anevaluation of the reliability and validity of the test itself e Facilitating the feedback loop so that the test taker gets meaningful assistance e Inthe analysis of the elements that constitute successful communication To study rater s strategies 1 3 NEED FOR FACILITATING RESEARCH WITH LEARNER DATAIN CAST A person may be not yet ready to take the full blown OPI There is no direct feedback from
14. main application 2 2 Generate Excel Reports The section Generate Excel Report highlighted in red in the figure below allows you to create Pre Survey Post Survey and Pre Survey Post Survey data reports which enables you to work with the survey data in Excel format These reports also include the Global Ratings of the language learner which enables you to relate their survey results with their performance on the test 43 5 httpi cast sdsu edu e 8B f Please login using your assigned administrator user name and password Computer User name v shah gt Assisted Password eee eee To Login to CAST Admin System Tool Note Your Administrative session will time out after 15 minutes of inactivity CAST Tech Test Start Here Returning User Login FAQ Email Us Tech Support Administrator Rater Login Secured Figure 3 Login Screen 2 2 1 Create Pre Survey Report Click on the link Pre Survey Report as shown in Figure 4 It will open up an Open Save Dialog box Click on open to open up the Excel report or click on save to save it at a selected location The Excel report will be generated in the format as shown in Figure 4 15 CAST Administration s Add Language to DA Databases gt Demographics 0 Add Language Add Context User Logged in v shah Add Function Edit Items Create Excel Sheet For CAST Pre Survey Data
15. search criterias Check the columns you want to display in the Data Grid Data Grid shows the checked search criterias Figure 4 5 Part C of activity diagram in Figure 4 2 show hide columns identify the instance with respect to other instances Figure 4 6 represents the PK in bold at the top of each entity table Another entity may manipulate a given instance by accessing the instance s PK In such case the PK referred to as the FK of the manipulating entity identifies the instance For example USER ID in BETAREG represents a suitable PK since no two identical User IDs exist in the database This key appears as the FK in CASTPRESURVEY ANSWERS CASTPOST ANSWERS RATEDTESTS and DEMOGRAPHICS_GROUP Any database query fired on an entity would reflect the changes in that entity as well as any other dependant entity Therefore any change to the PK of BETAREG will also change the attribute values of the related entities 4 4 APPLICATION SCREENS This section gives the details of the GUI used to give a web view to the learner s data along with its important components 4 4 1 Excel Sheet Functionality and Show Hide Functionality Figure 4 7 depicts the main screen of the application which shows all the functionalities of the application in brief Figure 4 8 screen shot zooms in to show the Excel sheet functionality and the show hide column functionality There are three links as follows 1 CAST Pre Survey Data This Excel sheet contains al
16. the exam and reporting progress or retreat There are many different ways practiced by different organizations to protect the confidentiality of the participants or informants The options that can be implemented at LARC were as follows 1 Determine in advance whether the test taker or participant wants to remain anonymous or receive recognition 2 Restrict access to such data to bona fide users Make data at the data entry level anonymous to make sure it does not cause loss of data In a meeting with Dr Mary Ann Lyman Hager it was decided that the second option would be selected as the other two options would require changes to the current flow of the CAST system 2 2 LEARNING FROM RESEARCH The technique adopted in this application is to restrict access to such data to authorized users and make it anonymous to all the users except administrators A robust system has been set up in the application to make the personal data anonymous when required LARC and the users need to follow a code of ethics while working on test takers data LARC has to assure that users respect the rights of participants The IRB at the university reviews all research conducted at the institution SDSU also has an IRB LARC has a document from IRB see Appendix A and B 3 REQUIREMENTS AND FUNCTIONAL SPECIFICATIONS This section discusses the necessary software resources and hardware components required for the Learner s da
17. to DA Databases Gender enough time to say what I wanted Volume level ofthe audio extent Add La 1o say Listening to test instructions id Language Was asked to talk about a variety of 84 Procedure for recording Add Context Ade Function Learner s Data items Approve items Email CAST Users View Algorithm Tests Demographics Spanish Administrator Please review transiations end edi regularly and determine when an i is reat used in the CAST Review Spanish Items Arabic Item Review As a member of the CAST Consortium we ask that please review the prompts linked below for contextual aXoSjl 1007 es User Info 2007 03 23 00 00 00 0 BETA OPI Complete Rated 24 Female appropriateness for the test SIUE RS takers that you deal with ut Users By For example Questions about careers mar not be contextually relevant Military Personnel just startit y out Test Items View My Approval Record b ti e es 2007 03 23 00 00 00 0 BETA OPI Complete Rated 24 Mae Research Links T X Figure 4 19 Menu to view learner s audio recordings and global ratings by selected rater for administrators fest ISSi highest level of education Review Reported Concert WI Instructions for giving Tesp T locas for Test I Approve Administrative User Rated amp Global Ratings Had enough time to prepare el la adig Any Other C
18. will tale place through an outside agency or organization confirm wath thet institution thet you have permission to conduct the study prior to mbabon of any study activities Ki human subjects tutorial found at a sce aein phn by ail members of the research teem This ER a HERD ER d any changes to your study planned you must submit s rocdificstion request and receive IRE approval prior to the implementation of study changes To submit s mocdificstion request 800855 the protocol the WebPortal on the protocol Main Page you will need to dick om Modifications under Protocol Maintenance and enter a report Ono you have filled in your responses on the report form NEW FECGLIIREMENTS Tncdocument your modiScstion in detail sones your currenthy apor protocol in the Full Document Viewer Copy and paste the document into Word amd use track changes to document revisions tn your protocol Seve the file Marne Modification Dete and upload it to your potou file Wien approved by the IRE this document will be the current version of your ADE poino to the approved conient fonts or other uploaded document 1 being requested changes must be documented using the track changes festure in Word Upload the reise form tn your viE B protocol file This form wall be reviewer by the IRE you do not have B af your approwed consent form in a Wo
19. 0 2 Please review translations and edits regularly and determine when an ite is ready to be used in the CAST test CjgDNn es 2007 03 23 00 00 00 0 BETA OPI Complete Rated 23 Review Hindi Items Arabic Item Review RL 57 zm Figure 4 14 User s personal information visible to administrators Approve Test Takers Add A New Administrator Arm Disarm Questions User Admin Control System Review Test Status Review Ratings Review Reported Concerns Test 19 pCCvAR m es 2007 03 23 00 00 00 0 BETA OPI Complete Rated 22 Approve Administrative User Add Language to DA Databases Add Language Add Context Function es B Test ID 000000 o 2 Edit Items Rated 23 a 74 View User s Exam View Users Exam Audio recordings and rater s ratings Name XYZ ABC Approve Items Email CAST Users CjgDNn EIER es View Algorithm Tests Complete Address 1111 Evergreen 09990 Demographics Org Code 0OOXXXX Hindi Administrator Please review translations and edits regularly and determine when an itc is ready to be used in the CAST test Telephone 000 222 999 Review Hindi Items Arabic Item Review Figure 4 15 User s personal information access restrictions 2 Sort Users By When you click on this option the User can sort the data by the selected Category 3 View User s Info This Inf
20. Eng View Recording Scripts M ont PA y KIG Figure 6 Check box to hide the appropriate column Test ID column is visible the Test ID column is visible If the Test ID box is unchecked the Test ID column in the table will be hidden as shown in the Figure 7 Add Language to DA Databases 7 0 P Select Columns To Be Displayed In The Table Maxon sr i CAST Post i LAS Language often spoken in your home Survey Give responses that reflect V Recorded voice properly Add Function Your native language Current TESTING course number ability to speak h The test delivery software po ame E Hive in area language was predominant Years of formal language study Felt comfortable giving my could be improved by Language g of instruction primary Residence length posed Test Directions ape linta TestDate E language of instruction secondary Understood what was expected Enjoyed aking Timed Pauses Email CAST Users Registration Code amp Taken a Study Abroad course Understood questions being asked Heras could be improved by Nervousness During the Test ivi TESTING program ran View Algorithm Tests Age highest level of education Greini hr smoothly eas for Test Improvement Demographics s E Global Ratings cf pac co idea of ARA es e Volume eve ofthe audio 9 89 Accurate idea of reponses
21. FACILITATING RESEARCH WITH LEARNER DATA IN ONLINE SPEAKING TEST A Thesis Presented to the Faculty of San Diego State University In Partial Fulfillment of the Requirements for the Degree Master of Science in Computer Science by Rachana Santosh Bedekar Fall 2011 SAN DIEGO STATE UNIVERSITY The Undersigned Faculty Committee Approves the Thesis of Rachana Santosh Bedekar Facilitating Research with Learner Data in Online Speaking Test C Le Carl Eckberg CHair Department of Computer Science fe Department of Computer Science t UO Le A lt La AAA gt Ve ges Mary Ann Lytnan Hager Latrguage esource Center j a 227 J DOT Approval Date Copyright O 2011 by Rachana Santosh Bedekar Rights Reserved iii DEDICATION I would like to dedicate this thesis to my father Santosh Bedekar and my mother Manali Bedekar for inculcating in me the importance of hard work and for supporting me always to my sister Tanaya Bedekar for her encouragement and love iv ABSTRACT OF THE THESIS Facilitating Research with Learner Data in Online Speaking Test by Rachana Santosh Bedekar Master of Science in Computer Science San Diego State University 2011 Language Acquisition Resource Center LARC is a Language Resource Center at San Diego State University LARC has an online speaking test called CAST Computer Assisted Screening Tool CA
22. Had enough time to say what I wanted what extent Hindi Administrator to say Listening to test instructions Was asked to talk about a variety of Procedure for recording Please review translations and edits 2 regularly and determine when an is ready to be used in the CAST Review Hindi items WML o 1 ILearner s Data Arabic Item Review 1 1 1 Consortium we ask that y 1 1 1 1 please review the prompts linked below for contextual appropriateness for the test takers that you deal with L For example Questions about careers may not be contextually relevant Military Personnel just startir 4 l 00 00 00 1 RETA MDI Mala No 74 Je Figure 7 Test ID Column is hidden 46 2 4 Filter Learner s Data Columns The textbox above each column as shown with a dotted line in Figure 8 acts as search box As the user starts typing in the box the column is filtered accordingly The user can filter multiple columns at the same time For example when the user enters two in the Age column and c in language column filter the rows in the column Age are narrowed to the ones with value 2 Volume level of the audio to say Listening to test instructions Was asked to talk about a variety of Procedure for recording topics responses Had enough time to say what I wanted 2 what extent
23. ST tests the speaking proficiency of the test takers in English Spanish Japanese Hindi French Modern Standard Arabic Persian Mandarin Chinese Iraqi Arabic and Egyptian Arabic For research purposes a language corpus is created from the data obtained from the CAST test the pre survey and post survey taken before and after the test A language researcher at LARC uses this language corpus to study various different factors that help language speakers achieve the Advance level proficiency He also uses the language corpus to evaluate the reliability and validity of the CAST test to facilitate the feedback loop that gives the test takers meaningful assistance to figure out the elements that constitute successful communication to study the rater s rating strategies and to study a test taker s attitude towards the computer based test The main objective of this thesis is to make language researchers lives easy at LARC by providing them with an interface that allows them to efficiently search and navigate through the language learners data This data can be viewed as a web page and in the form of an Excel sheet The web interface is an easy to use and consolidated outlook to the test data pre survey data and post survey data stored in the back end The research data can be sorted and filtered to develop data patterns and it can be modified and downloaded into a workable format as required The researcher has access to the test ratings and recording
24. State University 42 2 Getting Started This section describes the guidelines to use this application by listing the steps to use each feature of the application 2 1 User Login Process This section will guide you through the User login process to access the demographics application First go to the page http cast sdsu edu Click on the Administrator Rater login as shown in the Figure 2 http cast sdsu edu Computer Assisted Introduction Screening The CAST test has been designed to gauge oral Tool proficiency for students of English Spanish Japanese Hindi French Modern Standard Arabic Persian Comp uter Mandarin Chinese Iraqi Arabic and Egyptian Arabic Assisted The 30 minute Online test can be taken by anyone with a Screening CAST Tech Test Start Here Windows computer and an Internet connection at Tool Returning User Login cast sdsu edu FAQ CAST advisory ratings are based on the ACTFL Proficiency Email Us Tech Support Rating Scale Possible outcomes based on those descriptors are A dministrator Rater Login 1 Does not meet level 2 Approaching level 3 Meets level Test takers will receive diagnostic assessment data with their score Figure 2 CAST Screen Enter User ID and password as shown in the Figure 3 Contact the administrator if you do not have required login credentials Scroll down and click on the Demographics Survey results link to go to the
25. Strongly Strongly Strongly A Strongly A Strongly A Agree Strongly In the begir Agree Strongly A Strongly Strongly Strongly A When I pre just didn t about right wayyy less Strongy Disagree Strongly A Strongly Strongly Agree Disagree Disagree Agree I didn t und Strongly Strongly Strongly Strongly Strongly Strongly would hav Very clear Good paus It made Strong Disagree Strongly Strongly A Strongly A Agree Disagree Disagree Agree I didn t und Strongly Strongly A Strongly A Strongly A Strongly A Strongly A would hav Very clear Good paus It made Strong Disagree Strongly A Strongly Strongly Agree Disagree Disagree Agree 1 didn t undi Strongly Strongly Strongly Strongly Strongly A Strongly would hav Very clear Good paus It made me Strongly A Strongly A Strongly A Strongly A Strongly A Strongly A Most of the Strongly Strongly Strongly A Strongly A Strongly A Strongly A Maybe a ci Strongly Strongly St gly A Strongly Strongly Most of the Strongly A Strongly A Strongly Strongly Strongly Strongly Maybe a em Cast Post Survey Data gt Normal View SCRL CAPS NUM Figure 4 10 Post survey data with global ratings exported in Excel 4 10 the checkboxes are connected to the corresponding columns and when one checks or un che
26. View Algorithm Tests Demographics Hindi Administrator Please review translations and edits Learner s Data regularly and determine when an it is ready to be used in the CAST tes Review Hindi Items Arabic Item Review As a member of the CAST Consortium we ask that you please review the prompts linked below for contextual appropriateness for the test takers that you deal with et 2007 11 06 14 42 05 0 F07 Complete Rated 20 Male Yes Mandarin For example Questions about careers ma not be contextually relevant Military Personnel just starti E ct 2007 11 06 14 42 05 0 BCA_F07 Complete Rated 20 Male Yes Mandarin Approve Test Items View My Approval Record Research Links CAST Ratings ct 2007 11 06 14 41 44 0 BCA_F07 Complete Rated 20 Female Yes chinese View CAST SurveyResults View Recording Scripts View CAST Function Context Appro Grid Y 2007 11 06 14 41 440 F07 Complete Rated 20 Female Yes chinese Reviewei 1 dibres 4 4 Figure 8 Column Filter functionality 2 5 Using Popup Menu When the user hits the right click on the Learner s data table a context menu pops up as shown in the Figure 9 This context menu has three options as shown in the figure below e View User Info When the user clicks on View User Info he can see the user s contact information e Sort Users By User can use this option to implement the functionality to s
27. ber of the CAST to say Listening to test instructions Consortium we ask that you Was asked to talk about a variety of V Procedure for recording please review the prompts topics responses linked below for contextual appropriateness for the test takers that you deal with Learner s Data For example b 7 Questions about careers may 4 not be contextually relevant AS jx n ssIinm A Figure 4 Generate Excel Sheet Report 44 2 2 2 Create Post Survey Report Click on the link Post Survey Report as shown in Figure 4 It will open up an Open Save Dialog box Click on to open up the Excel report or click on save to save it at a selected location This link will create an excel sheet report similar to the one shown in the Figure 5 demographicsPreSurveyData 1 xls S we E EJ d New Open Save Print Import Paste Format Undo Redo AutoSum Sort A Z SortZ A Gallery Toolbox Zoom Sheets Charts SmartArt Graphics Wordart D E F 6 H J E n spoke Is it your native lar Have an arein what language w In what Have you taken a Stu What is the highe iti English English English No High School full English English English No High School full English English English No High School full English English English No High School full English English E
28. cel Report 2 2 3 Create Pre Post Survey Report Click on link Pre Post Survey Report as shown in Figure 4 It will open up an Open Save Dialog box Click on to open up the Excel report or click on save to save it at a selected location This link will create an Excel sheet report similar to the one shown in the Figure 5 2 3 Show Hide Column s of Learner s data The checkboxes under the Select columns to be displayed in the table section are mapped to the columns in the learner s data table If you uncheck a checkbox the column mapped to it in the table will also be hidden If you want to column again uncheck the checkbox mapped to it For example as shown in the Figure 6 the Test ID box is selected so 45 Approve Administrative User Demographics Add Language ta DA Databases 0 Mid Language User Logged in _shah Add Context Add Function Create Excel Sheet For CAST Pre Survey Data CAST Post Survey Data CAST Post Survey Data de CAST Pre Survey Data Edk tems Approve Select Columns To Be Displayed In The Table Email CAST Users View Algorithm Tests PASTES vey 8 Language often spoken in your home Post Survey 8 Give responses that ree Recorded voice properly 1 1 8 Your native 8 Coren TESTING course number ability to speak 8 The test delivery software live in area gt language was predominant 80 Years of formal language study
29. cks a particular check box its corresponding column is shown or hidden respectively In Figure 4 11 the Test ID checkbox is unchecked and this hides the corresponding Test ID column in the Learner s Data Table 20 Add Language to DA Databases Select Columns Be Displayed In The Table Add Language Add Context CAST Pre Surv CAST Post Surve i ji ey Language often spoken in your home Survey Give responses that reflect 64 Recorded voice properly Add Function Your native language Current TESTING course number ability to speak B 8 The test delivery sof Bit home Name live in area language was predominant Y Years of formal language study Felt comfortable giving my could be improved by Language tanga gt Residence length responses Test Directions age of instruction primary Approve Items Test Date Understood what I was expected todo 94 Enjoyed taking this test Timed Pauses Email CAST Users Wf Registration Code 4 Taken a Study Abroad course Understood questions being asked Hems could be improved by Nervousness During the Test i ivi TESTING program ran View Algorithm Tests Age B highest level of education Instructions giving responses clear y Ideas for Test Improvement Demographics Wi Global Ewing tim Ib paene y amp Fidelity of the audio Mj Any Other Comments Volume level of the audio
30. criteria Includes spoken responses from French German Japanese Spanish and other language learners in single web interface The application is searchable by a number of parameters including school level class level proficiency level and immersion or heritage status A user manual is created Appendix C to make sure users can utilize this application with less difficulty 6 2 FUTURE WORK As a part of the future work the following improvements can be made to the application 1 Implement a method in application to export research data after applying different sorting categories Implement functionality in the application to generate graphs using the results from the existing report results 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 32 BIBLIOGRAPHY Foreign Language Resource Center About LRCs 2010 http nflrc msu edu accessed Sept 27 2011 Language Acquisition Resource Center About LARC 2011 http larc sdsu edu accessed Sept 27 2011 The University of Texas at Austin Spanish Corpus Proficiency Level Training n d http www laits utexas edu spt intro accessed Sept 27 2011 Language Acquisition Resource Center Computer Assisted Screening Tool n d http cast sdsu edu accessed Sept 27 2011 University of Oregon About CASLS 2010 http casls uoregon edu pages about index php accessed Sept 27 2011 U
31. d at U S universities by the Department of Education to address the increasing national need for expertise and proficiency in foreign languages Now there 15 LRCs to promote the teaching and learning of foreign languages LRCs provide language learning and teaching materials to promote professional development opportunities for teachers and instructors and carry out research on foreign language learning Some LRCs concentrate on specific language areas and some on foreign languages in general all have the common goal to develop to improve foreign language education in the United States 1 LARC at SDSU is one of LRCs LARC s mission is to develop and support the teaching and learning of foreign languages in the United States through research technology and publications Particular attention is paid to less commonly taught languages cross cultural issues language skills assessment and teacher training 2 1 2 ABOUT CAST CAST is one of the many programs hosted by LARC CAST is used to test the oral proficiency of a test taker in a number of foreign languages Examples of other projects would be the SPT at UT Austin They have created a Spanish corpus and proficiency level the training It will help in the assessment of a learner s level of proficiency and create a model for training of teachers in the application of language proficiency 3 The intention of the CAST is to 1 Elicit an adequate speech sample from examinees to assure
32. d written productions The data of the database is composed of learners in grades 6 16 by using an online proficiency assessment test called Standards Based Measurement of Proficiency STAMP 6 The SLAWeb tool has research productions from more learners of Chinese French German Hebrew Japanese Spanish and Turkish The CASLS databank can be searched using different parameters like school level class level immersion and heritage status and proficiency 6 The research on SLAWeb was useful to see what others are doing in the field of language proficiency research what are the pros and cons of the software and how can this analysis be used to improve the LARC language proficiency research In case of LARC data is collected from language learners who take the CAST test which is an online proficiency assessment The data is from learners of Persian ESL Spanish Testing Chinese Egyptian Iraqi French Pashto German Hindi Tagalog Urdu Bengali Japanese Therefore there was a need for this thesis project to assist a researcher in understanding with precision how a learner performs at different test levels and come up with ways to improve feedback to the test takers Table 2 1 4 5 represents the comparison between LARC s learners database application and CALSYS s learners database application Table 2 1 Comparing Language Proficiency Tools LARC CALSYS SLAWeb Data collected from Type of test data Sho
33. e Agree Agree Agree Agree Ithoughtit Mention ho Fine Im always 28 205 311 5 Strongly A Agree yA Strongly Strongly A Strongly A Agree y_A Agree Agree Agree Strongly A Strongly A Strongly They were 1was just a 29 Agree Agree Agree Agree Agree Agree Agree Agree Agree Agree Agree Agree Agree Agree Taking the Agree Agree Agree Agree Agree Agree Agree Agree Agree Agree Agree Agree Agree Agree Taking the 5 Agree Agree Agree Agree Agee Agree Agree Agee Agree Agree Agree Agree Agree Agree Agree Taking the 1366 447 2 1 year 10 n Strongly_A Strongy Strongly Strongly Strongly Agree Agree Agree Agree Ithinxihat Agree Strongy Strongly A Agree Strongly Strongly don t think They were had no pr took the 355 447 2 1 year 10 n Strongly_A Strongly Strongly Strongly A Strongly A Agree Agree Agree Agree Ithinkihat Agree Strongly A Strongly Agree Strongly Strongly don t think They were had pri took the 1355447 2 1 year 10 Strongly A Strongly A Strongly A Strongly A Strongly A Agree Agree Agree Agree Agree Strongly Strongly A Agree Strongly A Strongly I don t think They were had no pre took the 5 Strongly_A Agree Strongly A Strongly A Strongly A Agree Agree Agree Agree not si Strongly_A Agree Strongly Strongly Strongly A Strongly A everything The directi about right
34. e my en A Any Other Comments Arabic Item Review responses i get Accurate idea of reponses Had enough time to say what I wanted Volume level of the audio _ ist extent As a member of the CAST to say Listening to test instructions Consortium we ask that you Was asked to talk about a variety of V Procedure for recording please review the prompts topics responses linked below for contextual appropriateness for the test takers that you deal with Learner s Data For example Questions about careers may a not be contextually relevant iY jar 4 AA Figure 4 8 Data columns show hide and Excel sheet report functionality 2 CAST Post Survey Data This Excel sheet contains all the CAST s Post Survey with global ratings It is used by the researchers to compare the test takers mind set by after the test and test results by viewing their global results and evaluation their post survey answers at the same time 3 CAST Post Survey and CAST Pre Survey This Excel sheet contains all the CAST S Post Survey as well as Pre Survey data with the global ratings Each link is responsible for creating an Excel sheet This gives the researcher an easy way to download the data in a workable format 4 4 2 Pre Survey Excel Sheet Figure 4 9 shows and Excel sheet of the Pre Survey Data and the Global Rating of different test takers This Excel sheet gives researcher a workable format for the data 4 4 3 Post Surve
35. e privacy of the learners and cause no harm to them A method was to be selected to implement these criteria keeping in mind the consistency with the existing system While creating the corpus from the test takers data the confidentially issue of the test taker s data is given utmost importance and it is done under a lot of protection According to Talk Bank The British Association for Applied Linguistics LSA Ethics Discussion and Confidentiality and Informed consent it is a responsibility of linguistics individually and collectively to anticipate ethical dilemmas and avoid harm to those with whom they work According to the code of ethics no personal data about individuals is to be made available to the users apart from audio records and basic facts 8 9 10 and 11 According to Talk Bank and British Association group of individuals involved in this research is classified into three groups 8 9 Participants people who have been recorded 2 Contributors researchers who have collected recordings 3 Users researchers who receive data from TalkBank The LARC has a similar classification for the users for learners data application and additionally a fourth category called administrators This classification is used when giving administrators or users access to learner s data The administrators can see a learner s contact details in case they want to contact the learner for reasons like retaking
36. earners at different levels Your data may be used in linguistic research following this study Your identity will be coded so that the researchers will be unable to associate your voice with your identity There will be several pages of instruction to the oral test You may complete the oral test at your leisure taking all the time you need and responding as completely as you can Please answer each question verbally by speaking clearly into the microphone Your oral response will be sent to the server All responses to these questions are TOTALLY CONFIDENTIAL In summary the study involves completing a number of questions that ask you about your language experiences the answers you provide will be kept confidential You are free to withdraw your consent at any time The speaking data you provide will be evaluated These data may be used for future research on language learning but your identity will be hidden from researchers Your completion of the questionnaire survey and the oral test are completely voluntary We encourage you to contact us at 619 594 5445 if you feel you would like to discuss the survey your score or the speaking test If you have concerns about your treatment as a subject please feel free to contact the SDSU Institutional Review board at 619 594 6622 Thank you for your participation consent I do not consent Approval Expires 8 22 2012 SAN DIEGO STATE UNIVERSITY Study Number 3555 APPENDIX
37. etsbesd to this correspondence please contact the IRE office 515 554 5622 or P rA involvement of human subjects in IEE n web zite at Sadete Students This notification may be used as documentation bo register in Thesis 79394 Attach a hard copy af this notice to your Appointment of Thess Project Committe form prior to submitting Ge completed form to Graduate and Research Affairs Student Services Division Sincerely jp A a APPENDIX USER MANUAL 39 40 User Manual Index Te GBNERAD INFORMA TON 42 dl 1 te 42 1 2 Point of Contat oe edente to 42 2 GETTING STARTED este est siad e ORITUR Eb sania eae qve das 43 2 T User Log PU indi 43 212 Download Excel Sheet Reports e te dat vi p 43 2 3 Filter Leamer s Data 22 nne il 45 241 Shown Hide Learner s Dala oido v eed dut ttai imd aes 47 NS 2 ne me 47 41 1 General Information This section describes the general information about the application It also describes the application overview and the point of contact in case of any questions about the application 1 1 Application Overview The Application has the following features Create Excel Sheet The Create Excel Sheet F
38. expected to do Enjoyed taking this test 8 Rejstu onCole Taken a Study Abroad course Undersood questions being asked couki be improved by 4 Nervousness During the Test 9 Ag 8 highest level of education 4 lnstroctions for giving responses clear Y pen programran dens for Test Improvement smoothly Cender i 8 Had time to prepare my MN Any Other Commens SS responses gnum SA get Accurate ides of reponses 8 Hod enough time to say what wanted 4 Volume level of the audio us exent isay 8 Listening to test instructions Was asked to talk about a variety of Procedure for recording topics responses Arabic Item Review As a member of the CAST Consortium we ask that yo please review the prompts linked below for contextual lini low for contextual 22 PCCvAR Jonathan Williams es 2007 03 23 00 00 00 0 OPI 2 Male No English No Eng appropriateness for the test takers that you deal with For example Questions about careers not be contextually relevant Military Personnel just star out Approve Test Items View My Approval Record Research Links View CAST Ratings CAST Surveyesalts View Recording Scots I Context Aore 14 14 Figure 1 Main Application Screen 1 2 Point of Contact This application is created for the linguistic research work that is carried out at the Language Acquisition Resource Center In case of any questions contact LARC at San Diego
39. ge string ode string unrateUser void getFormValues void seetUserinto void getUserinfo string 4 2 ACTIVITY DIAGRAMS The activity diagram in Figure 4 2 demonstrates the logical flow of actions in this application when accessing the data contained in objects of its classes If the user wants to access the proficiency database he has to login using his user ID and password It he is an administrator he will have a different view of the application as compared to the other users He can then access the application by clicking on the Demographics link The user will then have four different choices like 1 export data in Excel sheet 2 show hide data columns to see only required columns 3 use the Popup menu to access test takers information 4 filter the columns to narrow the number of rows visible As not all the activities can be depicted in one diagram the diagram is split into three different sections A B and C Figure 4 3 depicts the Excel functionality of the application and section A of Figure 4 2 in detail The user can generate Pre Assessment Survey report with global ratings 13 CAST Login Screen Enter UseriD Password Admin login successful User login successful cid CAST navigation screen Click on Demographics link Application Main Screen Export data in excel sheet Popup Quick Menu Show Hide Columns Filter Data Columns Start typing Filter Cr
40. imed believe it t 114 321 326 339 35 Strongly Strongly A Strongly Strongly A Agree Strongly Disagree Disagree Agree vague follo Strongly Strongly Strongly Strongly Strongly Strongly the softwar This made 15 7205 Strongly A Strongly A Agree Strongly A Strongly A Agree Disagree Disagree Agree I didn t feel Strongly A Strongly A Strongly A Strongly A Strongly A Strongly They were Some of thi think wat 16 329 425 444 4 25 months Agree Agree Agree Stongy A Strongly A Agree Disagree Agree Strongly it felt like Ix Strongly A Agree Agree Agree Strongly Agree there was c yes gota 17 Agree Agree Strongly_A Disagree Strongly A Agree Agree Disagree Agree Making itr Agree Strongly A Strongly Strongly A Strongly Strongly A Allowing or they were f they were f It would be 8 Pa Agree Strongly Agree Strongly Strongly_A Disagree Agree Agree Strongly_A The test we Strongly Strongly Strongly Strongly Strongly Strongly No improve the test dirt the pauses To be hone 19 5 Strongly Strongly Strongly Strongly Strongly Strongly A Strongly Strongly Strongly A There were Strongly A Strongly A Strongly Strongly Strongly A Strongly was gooc Perhaps to about right It was the s 20 425and451 two years Strongly Strongly Strongly Strongly Strongly Strongly A Strongly Strongly Strongly
41. included as a module of the CAST application as a whole and as a result the same programming language standards were used for this application as well This application is built using the J2EE architecture as shown in the Figure 3 1 Servlets M HTML Database Presentation Layer C Application Data Server Business Logic Layer Figure 3 1 J2EE architecture 3 3 2 jQuery jQuery is a fast and concise JavaScript Library that simplifies HTML document traversing event handling animating and Ajax interactions for rapid web development jQuery is designed to change the way that you write JavaScript 13 jQuery code can be easily embedded in HTML as a result jQuery was a choice for implementing show hide data functionality as HTML is used for front end coding for this application jQuery was used to map the checkboxes with the data table This functionality can also be implemented using JavaScript but jQuery reduces the amount of code and also is easier to maintain compared to JavaScript Moreover some people disabled JavaScript but this is not done for jQuery 10 3 3 3 JavaScript JavaScript is a lightweight programming language JavaScript is generally embedded directly into HTML pages or imported as a 18 file JavaScript is an interpreted language and as a result it is fast JavaScript is useful to respond to user events like click mouse in mouse etc on the interface This application has a
42. ions for giving responses clear E Ideas for Test Improvement SOUND 84 Gender Global Ratings Had enough time to prepare my Fidelity of the audio Any Other Comments responses 8 Volume level ofthe audio 9 Accurate idea of reponses S Had enough time to say what I wanted lume level of the audio _ what extent Hindi Administrator to say Listening to test instructions Was asked to talk about a variety of amp Procedure for recording Please review translations and edits topics resi ponses regularly and determine when an it is ready to be used in the CAST tes Review Hindi Items Learner s Data Arabic Item Review As a member of the CAST Consortium we ask that please review the prompts linked below for contextual appropriateness for the test takers that you deal with For example Questions about careers may not be contextually relevant Military Personnel just startir 2 1 2007 03 nn n nn n RETA Mala Na X Figure 4 13 Checked TestID and TestID column 4 4 5 View User s Information For users other than the administrators test takers contact details are not visible as it is considered a breach of confidentiality When users other than administrators try to access the data they will see it as anonymous However when the administrators right clicks on the learner s data table as shown in Figures 4 14 and 4 15 the
43. iteria in Filter Box Narrows the number of rows in the table that match the search O Figure 4 2 Activity diagram for the application 14 Export data in excel sheet Generate Excel Reports on Generate Pre Survey on Generate Pre amp Post Survey eron on Generate Post Survey Opens up a confirmation window Opens up a confirmation window Opens up a confirmation window Click OK to save the report Click OK to save the report Open Saved Report Open Saved Report rese data represented in workable ema and Post Survey data represented in workable ema c data represented in workable Figure 4 3 Part A of activity diagram in Figure 4 2 exporting data in Excel sheets Click OK to save the report Open Saved Report Post Assessment Survey report with global ratings Pre and Post Assessment Survey report with global ratings Figure 4 4 depicts the activity diagram for the pop up menu functionality and section B of Figure 4 2 The pop up menu gives the user three different options like 1 Sort the records by Name ID and Language 2 View User s Information and 3 View the Audio recordings and ratings of the user Figure 4 5 is the activity diagram is an extension of section C of Figure 4 2 The application has the capability to limit the data the user wants to see on the screen as there is a large am
44. l the CAST s Pre Survey data 17 CASTPOSTSURVEY ANSWRS Q1 Q2 Q3 Q26 RESPDATE CREDITDATE USERID USERID DEMOGRAPHICS GROUP CASTPRESURVEY ANSWRS NAME USER NAME Q1 Q2 Q3 Q38 RESPDATE CREDITDATE Approve Admnistratwe User Demographics 0 User Logged in shah Create Excel Sheet For CAST Pre Survey Data 1 CAST Post Survey Data I CAST Post Survey Data de CAST Pre Survey Data Approve items Select Columns To Be Displayed In The Table Give responses that reflec Recorded voice properly 4 Caren TESTING course number abliym The test delivery sofware 8 Years of formal language study Feli comfortable giving my be improved Test amp Residence length response Understood what I was expected to do Enjoyed taking thistest y Timed Pauses 4 Undersiood questions being asked hems could be improved by Nervousness During the Test 8 Instructions for giving responses clear Y a 8 Ideas for Tes Improvement smoot Had eno y ciun aa 8 Fidelity of the reponses get Accurate idea of reponses 8 Had enough time to say whatlwameg 80 Volume level ofthe sodio to say 8 Listening ten instructions Was asked to talk about a variety of 9 Procedure for recording topics responses 2007 03 23 00 00 00 0 BETA OPI 22 Male No English No Eng Figure 4 7 Main screen of the application for facilitating research at LARC 18
45. lect their requirements and Michael Pasamonik to know more about the CAST software system When interviewed Dr Trevor Shanklin Director of Lab Operations said that a researcher at LARC is responsible for studying a language corpus created using the CAST test data and involves at least one of the following 1 of the reliability and validity of the test itself 2 Facilitating the feedback loop so that the test taker gets meaningful assistance 3 It will help in the analysis of the elements that constitute successful communication He wanted to see a test takers demographic data like name age gender country of residence etc along with other test data This is useful in evaluating an individual by comparing him with heritage speakers When interviewed Nadezda Novakovic thought it would be great to have research data presented in a coordinated formatted She also thought it would be a good idea to link the post assessment survey data to the candidates final CAST ratings She came up with the following additional requirements for this project 1 separate grid of responses to the pre assessment survey with a column containing candidates CAST ratings 2 Aseparate grid of responses to post assessment survey again with a column containing candidates CAST ratings 3 big grid containing the responses to both the pre assessment and post assessment surveys with a column containing CAST rating
46. lish Yes English English No High School full full full approach approach fails fails fails fails approach approach gt gt Cei 9157 H k L Mae recie m Quar xod ors mm een uo E pce vcf mm Vd 1 2 clearly ur clearly ur The instru had enou I had enou was aske was able felt comf enjoyed t The items TESTI The fidelity The volum The procei The procei The TESTI Timed 3 Strongly A Strongly A Strongly A Strongly A Strongly A Strongly A Agree Strongly A Agree headpi Strongly Strongly A Strongly Strongly A Strongly A Strongly Everything don t reme Less nervo 4 Strongly Strongly Strongly Strongly A Strongly Strongly A Agree Strongy Agree headp Strongly Strongly A Strongly A Strongly Strongly Strongly A Everything 1 don t reme Less nervo 5 5 Strongly Strongly Strongly Strongly A Strongly A Strongly Agree Strongly A Agree headpi Strongly Strongly Strongly Strongly Strongly Strongly Everything 1 don t reme Less nervo 6 5 Strongly Strongly Strongly Strongly Strongly_A Strongly A Agree Strongly A Agree Strongly Strongly Strongly A Strongly A Strongly Strongly loss nervou 7 421451328 5 3months Strongly A Strongly A Strongly Strongly A Strongly A Agree Strongly Str
47. more than one category For instance as shown in the Figures 4 23 4 24 and 4 25 29 gt Meets Task Requirements Comments PER This Respontet Meets Task Requirements No Comments Download This Response 2 Meets Task Requirements No Comments Download This Response Le JL b de o Meets Task Requirements Comments Download This Response JL T JE JJ s Meets Task Requirements Comments Download This Response 5 3 3 Figure 4 21 Review CAST ratings screen ms Email CAST Users View Algorithm Tests Demographics Hindi Administrator Please review translations and edite regularly and determine when an it is ready to be used in the CAST tes Review Hindi Items Arabic Item Review As a member of the CAST Consortium we ask that you please review the prompts linked below for contextual appropriateness for the test takers that you deal with For example Questions about careers not be contextually relevant Military Personnel just starti out Approve Test Items View My Approval Record Research Links View CAST Ratings View CAST SurveyResults View Recording Scripts bon CAST Function Context Appro Learner s Data Was asked t
48. ng my could be improved by amp Language d amp Residence length responses Test Directions Approve Items Test Date Ls del E Understood what I was e Y Enjoyed taking this test Timed language of instruction secondary co gag igit insit Email CAST Users Wf Registration Code Taken a Study Abroad course Understood questions being asked improved Nervousness During the Test ie TESTING program ran View Algorithm Tests amp Age E highest level of educati Instructions for giving responses clear Ideas for Test Improvement amp Gender Global Ratings Had enough time to prepare my es of the audio Any Other Comments a responses a edm get Accurate idea of reponses AI Had enough time to say what I wanted Y Volume level of the audio Hindi Administrator to say Listening to test instructions s Was asked to talk about a variety of Procedure for recording Please review translations and edits topics responses regularly and determine when an i is ready to be used in the CAST Review Hindi Items Learner s Data Arabic Item Review As a member of the CAST Consortium we ask that yoi please review the prompts linked below for contextual appropriateness for the test takers that you deal with For example Questions about careers may not be contextually relevant Military Personnel just startir 2 AAN AAA Mes o
49. nglish Yes Associate Degree full English English English No Associate Degree full English English English No High School full English English English No Associate Degree full English English English No High School full English english English No High School unrate English English English No Associate Degree approach English English English No High School full English English English No High School fails English English English Yes High School full English English English Yes High School null English English English No High School null English English English No High School null English English English Yes Associate Degree null English english english No High School approach English English English No High School full English English English No High School full English Englsih English Yes Associate Degree full English English English No High School approach English English English No High School approach English English English No High School approach English English English No High School fails English English English No High School fails English English English No High School fails English English English No High School fails English English English No High School approach 33 22 Male No English English English No High School approach 22 No English English English No High School approach E lan Ds 5 5 i CPV Sil gt A AS el 5 sem Figure 5 Sample Ex
50. niversity of Oregon Center for Applied Second Language Studies n d http casls uoregon edu accessed Sept 27 2011 Harvard University Department of Linguistics Research 2007 http www fas harvard edu herpro research htm accessed Sept 27 2011 TalkBank org TalkBank n d http talkbank org accessed Oct 6 2010 British Association for Applied Linguistics BAAL News 2010 http www baal org uk accessed Sept 27 2011 LSA Ethics Conference on Language Ethics 201 1 http Isaethics wordpress com accessed Sept 27 2011 UCLA Center Confidentiality and Informed Consent 2008 smhp psych ucla edu pdfdocs confid confid pdf accessed Sept 27 2011 Oracle Java Servlet Technology n d http www oracle com technetwork java javaee servlet index html accessed Sept 27 2011 jQuery jQuery Project 2010 http Query com accessed Oct 5 2010 w3schools com JavaScript Tutorial n d http www w3schools com js default asp accessed Sept 27 2011 JExcelApi Java Excel API n d http JExcelapi sourceforge net accessed Sept 27 2011 Wikipedia Software Testing n d http en wikipedia org wiki Software testing accessed Sept 27 2011 APPENDIX A DIAGNOSTIC ASSESSMENT USING CAST CONSENT FORM 33 34 Diagnostic Assessment using the Computer Assisted Screening Tool CAST Consent Form We are asking you to participate in a study being conducted by Dr Mary Ann Lyman Hager of the Language
51. o get Accurate idea of reponses Ade Language to DA Databases amp Gender 4 Had enough time to say what I wanted Volume level of the audio Listening to test instructions Ade Language d G Was asked to talk about a variety of 84 Procedure for recording Ade Context topics responses Add Function Learner s Data Edit items Approve items Email CAST Users View Algorithm Tests Demographics Spanish Administrator an ularly en an i be used in the Arabic Item Review As a member of the CAST Consortium we ask that yo please review the prompts linked below for contextual aXoSjl Coo 2007 03 23 00 00 00 0 OPI Complete Rated 24 Female appropriateness for the test takers that you deal with Sort Users By View Users Exam For example Questions about careers not be contextually relevant Military Personnel just startit out Approve Test ems My Approval Record A es 2007 03 23 00 00 00 0 BETA Complete Rated 24 Male Research Links E Figure 4 20 Selecting rater to view his ratings that shows the test taker s recordings and ratings by that particular rater as shown in Figure 4 21 4 4 7 Multiple Column Filters Figure 4 22 shows the method to start filtering data columns by typing filter criteria in the filter box above each data column The multiple column filter feature enables the researcher to filter the data using
52. o talk about a variety of topics Procedure for recording responses 2007 03 23 00 00 00 0 OPI 2007 03 23 00 00 00 0 BETA OPI 2007 03 23 00 00 00 0 BETA Complete Male No No Figure 4 22 Filtering data columns to narrow number of rows 20 Was asked to talk about a variety of Procedure for recording topics responses Please review translations and edits regularly and determine when an it 15 ready to be used in the CAST tes Review Hindi Items Arabic Item Review As a member of the CAST Consortium we ask that yo please review the prompts linked below for contextual appropriateness for the test takers that you deal with For example Questions about careers ma not be contextually relevant Military Personnel just starti out Approve Test Items My Approval Record Research Links View CAST Ratings CAST SurveyResults View Recording Scripts eater Volume level ofthe Listening to test instructions ae Was asked to talk about a variety of Procedure for recording regularly and determine t is ready to be used in the CAST tes Review Hindi Items please review the prompts linked below for contextual appropriateness for the test takers that you deal with 2007 11 06 14 42 05 0 BCA_F07 2007 11 06 14 41 44 0 BCA_F07 Figure 4 24 Filters column by two let
53. ongly A Strongly A Really the Strongly A Strongly Strongly Strongly Strongly A Strongly A thought it Very under liked that When first 377 421 Strongly Strongly Strongly Strongly A Strongly Strongly A Strongly Strongly Strongly It was very Strongly Strongly A Strongly A Agree Strongly Strongly They am not su The only th 9 Pat 2 Agree Agree Disagree Agree Strongly Strongly Disagree Strongly A Agree It was diffic Strongly Strongly Strongly Strongly Strongly Strongly There is a It was a don t know would ima 10 2 24 Months Agree Strongly A Strongly Strongly A Strongly A Agree Agree Strongly Agree Ifeltthatth Agree Strongly Strongly Strongly Strongly Strongly Seems to b Seemed gc Also good think it ma 11 21 Agree Agree Strongly Strongly Strongly A Strongy A Agree Agree Agree Strongly Strongly Strongly A Strongly A Strongly A Strongly They were It wasn t so 112 21 12 16 months Strongly Strongly A Strongly Agree _ Agree Agree Agree Agree Agree _I think that Strongly Strongly A Strongly A Strongly Strongly Strongly 1 think that 13 Spanisn 425 n Strongly A Strongly A Strongly Strongly A Strongly A Strongly A Agree Strongly A Strongly Strongly Strongly Strongly A Strongly A Strongly Strongly AI wouid be They were The t
54. or section on the screen as show in the Figure 1 is used to generate MS Excel reports for Pre survey Post survey and both pre and post survey data e Learner s Data Table The Learner s Data section on the screen as shown in the Figure 1 is used to display all the learner data together in a table format Column Filter The text box above each column in the Learners data table as shown in the Figure 1 is used as a search box to filter through the column and narrow down the number of rows to match the filter categories entered in this search box Approve Demographics 0 Add Lang Add Language User Logged in v shah Add Context Fre ig Create Excel Sheet For CAST Pre Survey Data CAST Post Survey Data 11 CAST Post Survey Data de CAST Pre Survey Data Edit tens Approve Items Select Columns To Be Displayed In The Table CAST Users View Algorithm Teststll PreSurvey 0 Language often spoken in your home CASTPostSurey 8 Give responses hat reeet Recorded voice properly Your naive language 8 Curent TESTING course number ability to speak 8 Toe test delivery software Demographics Name live in area language was predominant 46 Years of formal language study 8 Felt comfortable giving my be improved by Language amp language of instrucion primary Residence responses s Test Date language of instruction secondary Understood what I was
55. ormation is visible only to Authorized users As shown in Figure 4 19 when the user uses the right click event on the Learner s data grid a quick menu pops up This menu has a View User s Exam option it gives another Rated By list Researcher can then select a rated exam by a particular rater as shown in Figure 4 20 Once the user clicks on the Rater s name it gives you another screen User Admin Control System Review Test Status Review Ratings Review Reported Concerns Test Issu Approve Administrative User Learner s Data Add Language to DA Databases Add Language Add Context Add Function Edit Items Approve Items to say Listening to test instructions 4 Was asked to talk about a variety of topics responses Procedure for recording 23 Email CAST Users View Algorithm Tests Demographics pCCvAR Anonymous Spanish Administrator Please review translations and edits regularly and determine when an ite is ready to be used the CAST test Review Spanish Items pCCvAR Anonymous Arabic Item Review As a member of the CAST Consortium we ask that you please review the prompts linked below for contextual appropriateness for the test 7 e lt 22 CjgDNn Anonymous es 2007 03 23 00 00 00 0 BETA OPI 2007 03 23 00 00 00 0 BETA OPI Sort Users By View User s Exam es 2007 03 23 00 00 00 0 BETA OPI Complete Complete Rated Rated 23 Figure 4 16 Pop up menu fo
56. ort the data by Test ID Name or Language e View User s Exam User can use this option to view the audio recordings of the language learners global ratings and rater comments to say Listening to test instructions User Admin Control System vum Was asked to talk about a variety of Procedure for recording Revieve Test Status topics responses 0 Review Ratings Review Reported Concerns Test Isst Approve Administrative User Learner s Data Add Language to DA Databases Add Language Add Context Add Function Edit Items Approve Items Email CAST Users View Algorithm Tests Demographics 2007 03 23 00 00 00 0 BETA OPI Complete Rated 22 Spanish Administrator Please review translations and edits regularly and determine when an ite is ready to be used in the CAST test Review Spanish Items PCCYAR 2007 03 23 00 00 00 0 Complete Rated 2 Arabic Item Review ort Users By View User s Exam As a member of the CAST Consortium we ask that you please review the prompts CJEDNn 2007 03 23 00 00 00 0 BETA OPI Complete Rated 2L n linked below for contextual appropriateness for the test Y BE qu Figure 9 Quick Menu
57. ory to hide it in the learner s data 20 Figure 4 12 Uncheck TestID and name categories to hide the corresponding columns in the leamer s data table scu iple betont ab ttai ud 20 Figure 4 13 Checked TestID and TestID column 21 Figure 4 14 User s personal information visible to administrators esses 22 Figure 4 15 User s personal information access restrictions 22 Figure 4 16 Pop up menu for Users cite oi br 23 Figure 4 17 Test takers information is anonymous to the users 23 Figure 4 15 Quick pop ss tosta p iet ias 23 Figure 4 19 Menu to view learner s audio recordings and global ratings by selected r ter Tot ASAS aes iade A roe aed tie 24 Figure 4 20 Selecting rater to view his 24 Figure 4 21 Review CAS ratings 25 Figure 4 22 Filtering data columns to narrow number of rows 25 Figure 4 23 Filters column by one letter of the filter 26 Figure 4 24 Filters column by two letters of filter 26 Figure 4 25 Filters data columns by multiple filter criteria 27 LIST OFACRONYMS ACTFL American Council of the Teaching of Foreign Languages CAST Computer Assisted Screening Tool GUI Graphical User Interface IRB Institutional Review Board
58. ount of research at one place 4 3 DATABASE DESIGN DIAGRAM SQL serves as an interface to a Relational Database Management System RDBMS which manages data organized in the form of related tables This feature preserves security and authentication and provides powerful database driven functionality 15 Popup Menu Quick Menu Quick Menu Popup Menu won click on the Data Click on View Ratings Click on User Information Click on Sort By si Select The Sort By Cat User Login se is irs eson User Information Not Visible Data Column sorted by the selected category Select a Rater User Information Visible Recordings amp Ratings Visible anywhere outside the popup to Figure 4 4 Part B of activity diagram in Figure 4 2 up menu Each table represents a class entity i e CASTPRESURVEY ANSWERS CASTPOST ANSWERS RATEDTESTS DEMOGRAPHICS GROUP and BETAREG along with its attributes and its links representing its relationships to other entities Two entities relate to each other in the following ways Primary Key PK and Foreign Key FK For each entity the PK defines a field unique to each instance of the Activity Diagram Part B pop up menu and activity Diagram part C show hide data columns entity helping to 10 Show Results By Uncheck the columns you do not want to display in the Data Grid Data Grid hides the unchecked
59. popup menu or context menu to link to audio recordings and view learners information The context menu is a div element with display property set to none and position property set to absolute The context menu appears on the mouse down event on the learners data table This context menu works with all majors browsers like IE Firefox Chrome and Opera 14 3 3 4 JXL JExcel API Java Excel JExcel API is an open source Java API that facilitates developers to read write and modify Excel spreadsheets dynamically and write the changes to any output stream 15 There was a requirement that the code of this application should be compatible with the code structure of the CAST test which is written in Java using Java Servlets The JExcel API can be invoked from within a servlet thus giving access to Excel spreadsheets over the internet and intranet web applications As a result JExcel API was used as it can be used with Java Servlets to implement the export data component of this application 11 4 SOFTWARE AND SYSTEM DESIGN AND DEVELOPMENT In designing such an application there exist two possible approaches procedural and object oriented The procedural approach involves creating algorithmic procedures then breaking the necessary steps into modules through which data is input logically Whereas the object oriented approach focuses on terms called objects Objects encapsulate both procedures and data in a single entit
60. r users User Admin Control System Review Test Status Review Ratings Review Reported Concerns Test Issu Approve Administrative User Add Language to DA Databases XvAR Anonymous es 2007 03 23 00 00 00 0 BETA OPI Complete Rated 22 Add Context Add Function Edit Items Approve Items Email CAST Users AR Anonymous Complete Rated 22 View Algorithm Tests Demographics Anonymous Spanish Administrator Nn Anonymous us M Complete Rated 23 jonymous reia translations adisi Anonymous Anonymous Anonymous Anonymous regularly and determine when an ite is ready to be used in the CAST test Anonymous Review Spanish Items noB ONE Anonymous Arabic Item Review Anonymous 4 88 Anonymous Complete Complete Rated 24 As a member of the CAST Rated Consortium we ask that you please review the prompts linked below for contextual riateness for the test 7 Y jr Done m Figure 4 17 Test takers information is anonymous to the users View User Info Sort Users By View User s Exam Figure 4 18 Quick pop up menu 24 Review Reported Concerns Test Issi 19 Test St highest level of education Instructions for pv responses clear OT PE II Wess Tor Test Irmproverm Approve Administrative User Global Ratings 6 Had enough tine to prepare my a AS s s rine 0 f responses courate idea of reponses Add Language
61. rd format request a copy from the IRE office 2 The S080 requires report any problems that arie during thi course of an approved research vtae Serios sdyerze events or unanticipeted probleme that are life tmresterine or eve resulted in serious injury desth must be reported to the IRE immediately whenever posible or within st least 48 hours from the onset of the incident All other problems must be reported to the SDSL ot n 38 Diego ace Un Ree cry ON arpani re SAR DIEGO STATE n ra JEVERSITY Fasc 615804 within 1 days complete and submit an adversa event report te the Protocol lain Mena ee information and consultation contact the IRE office directly va Email st IR sd edu or f To sabe request lo extend approval log in tn your WebPortal account and access the protocol On the protocol Main Fage click on Progress Report under Protocol Maintenance and enter a report Once you have filled in your responses on the report form click sunmit You should recejve an sutometed email verifying IRS receipt of your Report of Progress indicate which if any consent formis youare requesting tn renew Refer to the Consent Form Development section of the protocol and provide the IRE with the spedi file names and ste s of upload of the consent document s you requesting tn For questions n
62. responses ciear TESTING program mn Ideas for Test Improvement Had enough time to prepare my smoothly Fidelity of the audio lid Had enough time to say what what extent to say Listening to test instructions Was asked to talk about a variety of 8 Procedure for recording topics responses Please review translations and edite regularly and determine when an is ready to be used in the CAST tes 4 2007 11 06 14 41 44 0 F07 Rated 20 Female Complete Figure 4 25 Filters data columns by multiple filter criteria 28 5 TESTING Software testing determines whether a system as implemented satisfies its formal stated system requirements A test case is a set of conditions or variables under which it is determined whether an application or software system is working correctly or not Software testing can be stated as the process of validating and verifying to make sure that a software program application product meets the requirements that directed its design and development works as expected and can be implemented with the same characteristics 16 Unit testing validates the correct behaviour of every minimal executable segment of the application code Sub system system testing validates the correct interaction between all sub systems of the system Integration testing validates that the sys
63. ries shown or hidden and the column Name respectively will be hidden 10 Verifying Database The title of the data Make sure the data in Pass Results grid columns should the data grid is match the data in the consistent with the data respective column column titles 11 Browser Independence Checkboxes should be Open the Application Pass Checkboxes overflowing out of the bounding box bounded with different browser types and the application should appear same in all the browsers 3l CHAPTER 6 CONCLUSION AND FUTURE WORK 6 1 CONCLUSION This section lists the evaluation of the application by the LARC department and the project s accomplishments Application is successfully to the main server and is well tested by Mike Pasamonik the Software Engineer at LARC Created an easy to use web interface to the learner s database Preserve confidentiality of learner s information by limiting user s access to data It was achieved by creating different levels of users such as administrators and users The web interface helps researcher create reports in form of workable Excel sheets for the Pre survey Post survey and both together The learner s data should be easy to sort and filter The filter can be applied to many categories at the same time and thus help the researcher create various different patterns of data Limit the data to be displayed on the screen by giving user the option to check and uncheck the search
64. s I interviewed Mike Pasamonik to understand the CAST system from a software engineer s perspective This project is incorporated as a CAST module so it had to be compatible with the current CAST structure He explained to me the different technologies used to create the CAST system database structure server requirements hardware requirements and software requirements Keeping all these things in mind it was required to develop a system that will have interfaces and a database to organize store and display the different types of research data in a comprehensible web based view To make sure the confidentiality issue of the test taker is not ignored the personal data is visible only to the Administrators and not to other users researchers LARC has obtained permission from the IRB to use the test taker s data the permission document is included in Appendix A and B 2 COMPARISON WITH OTHER LANGUAGE LEARNING TOOLS This section discusses a few language proficiency research programs how they carry out language research and how the research was useful to make the LARC language proficiency research application helpful for researchers 2 1 OTHER LANGUAGE RESEARCH TOOLS AVAILABLE CALSYS at the University of Oregon is also a Language Resource Center with a goal to improve the teaching and learning of foreign languages 5 CALSYS has a searchable database software called SLAWeb SLAWeb is a searchable database of spoken an
65. s as a part of a quick menu The research data is stored using Oracle databases Java Servlets technology is used to congregate together the data and present it in a grid format jQuery and JavaScript technologies are used to implement the sorting and filtering functionalities on the data table A popup context menu is created for the data table using JavaScript technology The quick menu is used for sorting data viewing the audio samples of the language learner and language learner information jQuery is used to show hide data columns in the grid as required generating data patterns and making it easy to browse vi TABLE OF CONTENTS PAGE ABSTRACT eu mau UE qf tete LIST OF FIGURES at tte MSIE Ra ix LISTPOP ACRONYMS de ias tala X ACKNOWLEDGMENT SS sett d es ee erae eee xi CHAPTER 1 INTRODUCTI N dees RE RU cire Ui phat cds es 1 FLADT EAR o i dl gad d sius 1 O 2 1 3 Need for Facilitating Research with Learner Data in CAST 2 2 COMPARISON WITH OTHER LANGUAGE LEARNING TOOLS 5 2 1 Other Language Research Tools 5 2 2 Learning from Kesearc AAA O ee 7 3 REQUIREMENTS AND FUNCTIONAL 8 3 1 Functional Requirements i e
66. tabase system to run effectively Further it covers the behaviors and properties required by its users and other components of this application 3 1 FUNCTIONAL REQUIREMENTS This section describes the functional requirements of this application Create a user friendly interface for the learner s data Preserve confidentiality of test taker or learner by limiting access to data Create Excel sheets for the pre survey post survey and both The learner s data should be easy to sort and filter e The filter can be applied to one to many categories at the same time Limit the data to be displayed on the screen by giving user and option to select the required data 3 2 SYSTEM REQUIREMENTS The details of the development environment for the CAST are given below The hardware environment Xserve PowerPC G5 Type 64Bit OS and Speed 2 0Ghz e 2GBRAM 153 GB Hard Disk In addition the software environment Mac OS X Serve v 10 4 11 Java Standard Edition v 1 5 SDK Orion Oracle 10g Database e Any browser with JavaScript enabled 3 3 TECHNOLOGIES This section describes the programming technologies used to build the learner s data application 3 3 1 Java Servlets CAST test is written using Java 2 Platform J2EE programming standard A servlet is a Java programming language class used to expand the capacity of servers that host applications accessed via a request response programming model 12 This application is
67. te af your Study pee in approval requires trat all research with human subjects pe suspended until approval is obtained and may result s temporary hold on funds if your study is funded The investigator will be cut of compliance with federal regulation and university policy if human subjects continue to be imeotved in this project without a valid spproval As pour study requires the use of consent docamert ness note that the re stumped forms been uploaded to your protpc file within the system within the Supporting Documents section This document bears the IRE s stamp of approval Print a copy of this TE ade Dir CASALS ECC EI Re gt OLG PALA gt Le ACA LOB A min UT RH e DD ET HORT aD Ea del Race ACA Lacs A CHIC E CC DRUCK CE SAM DIEGO STATE ee UNIVERSITY 5615 554 AL may nit be made to the consent documents without prior review and approwal of the IRE You has been completed or Eerrminated The follwing is the file neme s of the RE epproved forms associabed with thi rontinumstion request verify the of the bransisbed document IRE approval ar this document fior use in subject recroitment is based on your sssuranos that the transisbed document reflects te conbent of the b if recruitment
68. tem meets stated requirements when individual software modules are combined and tested as a group Different students were working on different modules of CAST at the same time so integration testing was carried out to make sure that once all the students deploy their modules to the server the individual applications still work fine For this application the login functionality was caching the user and was not detecting the different types of users logging in As a result through testing of this application was carried and some more code was added to admin class to fix the issue Using Java Random Number Generator functionality a random number is generated per each user login to avoid user login caching The complete set of test descriptions for this project is as follows in Table 5 1 Table 5 1 Testing Results 29 ID Test Case Expected Result Action Result 1 Cast Pre Survey Data Creates Excel sheet for Click on Cast Pre Pass Link Cast Pre Survey Data Survey Data link 2 CastPost Survey Data Creates Excel sheet for Click on Cast Pre Pass Link Cast Post Survey Data Survey Data link 3 Cast Post Pre Survey Creates Excel sheet for Click on Cast Post Pre gt Pass Data Link Cast Post and Pre Survey Data link Survey Data 4 Demographics Link Open up the main 1 Go to the Login Pass screen of the Screen Application 2 Sign up as user administrator 3 Click on the Demographics link in the left column 5 Data Grid Col
69. ters of filter criteria i View DA Materials Situation Script View CAST Resources Development Wiki Super User Functions Internationalize CAST Create New Org Code Approve Test Takers Add New Administrator Arm Disarm Questions User Admin Control System Review Test Status Review Ratings Review Reported Concerns Test Is Approve Administrative User Add Language to DA Databases Add Language Add Context Add Function Edit Items Approve Items Email CAST Users View Algorithm Tests Demographics Hindi Administrator CAST Pre Survey Test ID C Name Test Date amp Registration Code Test Status 9 Rated Age 8 Gender Language often spoken in your home Your native language live in area language was predominant language of instruction primary language of instruction secondary Taken Study Abroad course highest level of education Global Ratings 27 CAST Post Survey Give responses that reflect Recorded voice properly Current TESTING course number ability to speak The test delivery software Years of formal language study Felt comfortable giving my could be improved by Residence length Test Directions Understood what I was expected to do 4 Enjoyed taking this test Timed Pauses Understood questions being asked Items could be improved by gf Nervousness During the Test instructions for giving
70. the sequence of actions that occur in a system including the invocation of object methods modification of object properties and input output between the user interface and database 12 4 1 CLASS DIAGRAM The class diagram for this application Figure 4 1 shows the structure and interaction of the classes Each entity class shown boxed in Figure 4 1 models the properties and behaviors of the data For example the class for Demographics Data will contain properties such as resultSet and statement and the behaviors of demographics getFormValues etc The lines represent the relationship between the classes generator int testid int printFrames void processRequest void AdminLinks printSULinks void demographicsData resultSet ResultSet statement Statement castUtil CASTUtil demographics void getFormValues string pre postSurvey Workbook JExcel Workbook preSurveyWorkSheet JExcel Worksheet postSurveyWorkSheet JExcel Worksheet pre postSurveyWorkSheet JExcel Worksheet createE xcelFilePre void createExcelFilePost void createExcelFilePostPre void demographicsChangePref name string state string city string DemographicsFindRaters ratedTestData string query count int findRaters void gelFormValues getFormValues Figure 4 1 UML class diagram userlD int name string langua
71. umn When we start typing Type in Gender Pass Filter Female in Gender Column Filter column it gets filtered and shows results for gender Female 6 Position of Pop up When user users a right Right click on the data Pass Menu click on data grid a grid quick menu pops up The quick menu s location should be limited to the grid context 7 Expected Columns in When user clicks on Click on the Post Pre Pass Excel Sheet the Post Pre Survey link the data from both post and pre survey is correlated and presented in the same workable sheet Survey Data link compare the columns with the original data table continues Table 5 1 continued 30 ID Test Case Expected Result Action Result 8 personal data User Information is Sign is a Pass anonymous for visible only to the researcher user Click researchers other than Administrators and for the Demographics the administrators other researchers the link The name column personal information of will read the test takers should Anonymous Also be anonymous right click on the Learner s data grid and select View User Info the information will appear as Anonymous 9 Linking checkboxes to When the user checks Uncheck box Name in Pass the research data or un checks a column the categories The columns name only it s column name will be corresponding data grid hidden Check the box column should be Name in the catego
72. was much 5 Strongy A Agree Strongly A Strongly Strongly A Agree Agree Agree Agree iwas not s Strongly_A Agree Strongly A Strongly Strongly A Strongly A everything The direct about right was much 5 Strongly_A Agree Strongly A Strongly A Strongly_A Agree Agree Agree Agree not si Strongly A Agree Strongly_A Strongly Strongly A Strongly everything The directic about right was much 5 Strong Strongly Strong A Agree Strong Agree Agree Agree Agree Strongly Strongly Strongly A Strongly Strongly Strongly Mt was a 5 Strongly Strongly_A Strongly A Agree Strongly Agree Agree Agree Agree quest Strongly A Strongly Strongly Strongly Strongly Strongly twas a lit 5 Strongy Strongly_A Strong A Agree Agree Agree Agree Agree Strongly Strongly Strongly Strongly Strongly Strongly a litt 5 Strongly A Strongly Strongly A Strongly A Strongly A Strongly_A Strongly A Agree Strongly_A In the begir Agree Strongly Strongly Strongly Strongly A Strongly When I pre I just didn t about right wayyy less 5 Strongly Strongly Strongly Strongly Strongly Strongly Strongly Agree Strongly_A In the begir Agree Strongly Strongly Strongly Strongly When I pre just didn t about right wayyy less 5 Strongly A Strongly
73. ws test results Information Security Searchable by CAST is LARC s online speaking test and the post and pre survey carried out during the test Includes speaking results and assessment survey data By demographic data such as age gender country of residence and years of study primary education Login by user name and password A number of parameters including school level class level proficiency level and heritage status An online proficiency assessment the Standards Based Measurement of Proficiency STAMP Includes reading writing and speaking results By language of study native language and grade and demographic data such as age and years of study Login using username and password Parameters including school level class level proficiency level and immersion or heritage status Sources Language Acquisition Resource Center Computer Assisted Screening Tool n d http cast sdsu edu accessed Sept 27 2011 University of Oregon Searchable Proficiency Database n d http casls uoregon edu pages research proficiencydatabase php accessed Sept 27 2011 The Harvard University Department of Linguistics uses Data coding primarily done with Excel OpenOffice or makes use of software called PXLab They can conduct simple experiments but no official support is available when they encounter any problems 7 The intent of the research was to find out an ideal way to preserve th
74. y There also exists an object interface which defines how the object interacts with the rest of the code It was a part of the requirements that this application should be compatible with the current CAST application The CAST application is created using Java Servlets thus an object oriented design methodology was selected In addition structured query language SQL provided a database query language JDBC communicates SQL to Oracle The Unified Modeling Language UML based approach was used design the interface and database components of this project UML design uses four types of structures to facilitate language independent organization visualization and creation of an object oriented program e Class Diagram It depicts system organization by showing the system s classes relationships between the classes and class attributes and behaviors procedures Database Diagram It is an organization of data to represent an outline of how the database will be constructed and depicts the relationship between the different data tables Activity Diagrams A state transition diagram which depicts the consecutive transitions of an object from various states during execution of a given procedure A state represents the conditions of the object as the program executes The transitions begin at an initial state and progress either in parallel or conditionally to a final state e Sequence Diagrams A chronological diagram which depicts
75. y Excel Sheet Figure 4 10 shows an Excel sheet of the Pre Survey Data and the Global Rating of different test takers This Excel sheet gives researcher a workable format for the data Similarly an Excel sheet can be generated for Post Survey and Pre Survey both 4 4 4 Column Show Hide Functionality As there is a large amount of learner data it is required to have a method to facilitate the researcher with functionality so that he she can select only the required data columns they want instead of scrolling through the entire page horizontally As shown in Figures 4 9 and 1 2 3 4 5 6 23 7 24 24 21 10 21 21 12 23 2 25 14 15 18 16 1 18 21 19 19 20 23 english 22 21 Male No English Yes English English No High School 23 Female English No English English No High School 24 24 Male No English Yes Englsih English Yes Associate Degree 25 23 Female English Yes English English No High School 26 23 Female English Yes English English No High School 27 23 Female English Yes English English No High School 28 19 Female Yes English No English English No High School 29 19 Female No English No English English No High School 30 19 Female English No English English No High School 31 19 Female English No English English No High School 32 Male No English Yes English English No High School 33 Male No English Yes English English No High School 34 22 Eng
76. y will be able to see the Test Taker s contact details Different similar applications have used a variety of practices as follows 1 Make the data anonymous while entering it into the system 2 Use login functionality to limit access to data However the anonymity of data seems to be a very fine practice At LARC we have used the second method as listed above to protect data access The language learners data in the Figure 4 14 and Figure 4 15 are removed for confidentiality reasons When users click on the View User Info as shown in Figure 4 16 they will not be able to see the test takers personal information and all the information will appear as anonymous as shown in Figure 4 17 4 4 6 The Popup Menu Figure 4 18 depicts the pop up menu created using JavaScript It facilitates access to the test takers oral recordings When a user clicks on any record a menu pops up with sub menus like 22 ra topics responses m Approve Test Takers Add A New Administrator Arm Disarm Questions User Admin Control System ad Review Test Status Review Ratings Review Reported Concerns Test Issi Approve Administrative User Add Language to DA Databases Add Language Add Context Add Function adas pCCvAR o es 2007 03 23 00 00 00 0 BETA OPI Complete Rated 22 Approve Items Email CAST Users View Algorithm Tests Sort Users pCCv 2007 00 00 0 BETA lecum Rated Hindi Administrator AR bo View User s Exam 0325

View/Open - San Diego State University

Contents

Download Pdf Manuals

Related Search

Related Contents