CTTITEM: SAS macro and SPSS syntax for classical item analysis
Contents
1. and Cronbach s alpha when the item in question is deleted For dichotomous items index of discrimination is computed based on Kelley 1939 s 27 rule in classify ing the high and low scoring groups When there are ties at the cut scores they are included in their corresponding groups This grouping is also used to tabulate conditional relative frequency distributions of option responses by high and low groups expressed in percentages for option analysis For polytomous items marginal distributions of item categories are provided Data sets for scored item responses as well as item statistics are created and can be output for additional analysis The data to be analyzed should have the following format Ist row ID in the 1st column followed by item vari able names this row is optional and must be bypassed with the firstobs 2 option in the data step when the data set is read in if it is present 2nd row uppercase NCAT in the Ist column fol lowed by the number of options categories for each item 3rd row uppercase KEY in the Ist column followed by the keys for objective items to be scored dichoto mously or for polytomous items enter 1 for positively scored or 0 for reversely scored items e g all 1 s when no reverse scoring is necessary 4th to last row subjects id in the 1st column fol lowed by their item responses note use one row for each subject one column for each2. 54 0 77 0 41 0 48 0 70 WRMC7 0 83 0 51 0 75 0 38 0 41 0 71 WRMC8 0 92 0 44 0 80 0 34 0 21 0 71 WRMC9 0 71 0 52 0 69 0 36 0 59 0 71 WRMC10 0 73 0 42 0 56 0 25 0 48 0 72 WRMCI1 0 65 0 51 0 65 0 34 0 72 0 71 WRMCI12 0 88 0 53 0 87 0 43 0 41 0 70 WRMC13 0 78 0 52 0 73 0 38 0 45 0 71 WRMCI14 0 83 0 61 0 90 0 49 0 55 0 69 WRMCI15 0 80 0 48 0 69 0 34 0 48 0 71 Relative frequency distributions by high and low groups for each item indicates the item key IRMC1 N 1 2 3 LOW 27 29 3 7 0 90 HI 27 29 0 0 0 100 IRMC2 N 1 2 4 LOW 27 29 3 0 97 0 HI 27 29 0 0 100 0 Similar outputs are omitted Manuscript received January 24 2006 revision accepted for publication May 23 2006
3. Behavior Research Methods 2007 39 3 527 530 CTTITEM SAS macro and SPSS syntax for classical item analysis Put WA LEI AND QIONG WU Pennsylvania State University University Park Pennsylvania This article describes the functions of a SAS macro and an SPSS syntax that produce common statistics for conventional item analysis including Cronbach s alpha item difficulty index p value or item mean and item discrimination indices D index point biserial and biserial correlations for dichotomous items and item total correlation for polytomous items These programs represent an improvement over the existing SAS and SPSS item analysis routines in terms of completeness and user friendliness To promote routine evaluations of item qualities in instrument development of any scale the programs are available at no charge for interested users The program codes along with a brief user s manual that contains instructions and examples are downloadable from suen ed psu edu pwlei plei htm In any test or instrument development newly written items are necessarily tried out or pilot tested before they can be used to collect information Qualities of individual items such as whether they are functioning the way as in tended have to be evaluated Items deemed problematic are often revised or eliminated from the final form Item statistics based on tryout data are informative for such decisions Conventional item analysis typically includes such t
4. HOICE TESTS 1992 June 26 Retrieved May 6 2006 from support sas com ctx samples index jsp sid 478 amp tab details SPSS WHITE PAPER USING SPSS FOR ITEM ANALYSIS 1998 Re trieved May 6 2006 from www spsstools net Syntax ItemAnalysis UsingSPSSforltemAnalysis pdf Continued on next page 530 LEI AND WU APPENDIX SAS Example for a Hypothetical Test With 15 Multiple Choice Items Taken by 100 Examinees The sample data set c mcitem dat contains ID 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 NCAT 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 KEY 4 3 1 1 2 4 2 1 3 ht 2 3 2 1 4 3 1 2 2 2 4 1 3 1 3 2 3 2 1 2 4 3 1 1 2 4 2 1 3 1 1 2 3 2 l 3 4 3 1 1 2 4 2 1 2 1 1 2 3 2 1I 4 4 3 1 1 2 1 2 1 3 lL 3o 2 2 2 1 5 4 3 2 32 22 2 2 2 2 12 2 3B 2 1 The SAS commands used to read in the external data file and invoke the CTTITEM macro DATA ONE INFILE C MCITEM DAT FIRSTOBS 2 INPUT ID MCI1 MC15 RUN CTTITEM ONE MC 100 15 1 RUN The output generated by SAS TEST STATISTICS FOR MC TOTAL_MC N 100 00 MEAN 12 59 STD 2 43 MIN 5 00 Ql 11 00 MEDIAN 13 00 Q3 15 00 MAX 15 00 SKEWNESS 1 05 KURTOSIS 0 57 ALPHA 0 73 SEM 1 27 item statistics ISTAT_MC P_VALUE POINT_BISERIAL BISERIAL CORRECTED _PTBIS D_INDEX ALPHA_WITHOUT_ITEM WRMCI 0 96 0 41 0 93 0 34 0 10 0 72 WRMC2 0 99 0 27 1 02 0 23 0 03 0 73 WRMC3 0 97 0 38 0 96 0 32 0 10 0 72 WRMC4 0 90 0 25 0 42 0 13 0 21 0 73 WRMC5 0 85 0 48 0 74 0 36 0 34 0 71 WRMC6 0 79 0
5. asks as gauging difficulty prevalence of correct responses and discrimination ability to differentiate respondents on the trait being measured for each item as well as es timating score reliability and distribution for the set of items to be used as a whole Results of an item analysis can help determine the minimum number of items needed for a desired level of score reliability or measurement ac curacy The common statistics used by the classical item and test analysts for dichotomously scored e g multiple choice or true false items and polytomously scored e g Likert type scale or essay questions items are produced by the customized SAS macro and SPSS syntax These programs are customized to generate user friendly outputs that are similar to those produced by a specialized item analysis software program such as ITEMAN Assessment Systems Corporation 1989 but without the accompanied cost of commercial programs They represent an update or improvement over the existing routines within SAS and SPSS as well as supporting materials available from the Web Currently there is not a SAS built in item analysis rou tine though an item macro that performs item analy sis for multiple choice items is available from the Web Sample 478 1992 However the SAS item macro does not provide the index of discrimination D index and it is not clear how ties are treated when forming the high and low scoring groups for distractor ana
6. choice or true false format as well as poly tomously scored items like short answer or Likert type questions Being a SAS macro CTTITEM can be run on any platform with which SAS is compatible and the job P W Lei puiwa psu edu 527 Copyright 2007 Psychonomic Society Inc 528 LEI AND WU size is limited only by your computer s memory allocation to SAS CTTITEM scores option or category responses based on the desired type of scoring dichotomous or polyto mous For dichotomous items the macro reads in option responses and scores the items as right or wrong with the answer keys provided For polytomous items on the other hand it reads in category responses and reverses score categories for negatively stated questions as specified by the user Descriptive statistics for total score such as mean standard deviation quartiles skewness and kurtosis as well as Cronbach s alpha Cronbach 1951 and standard error of measurement SEM for the test are reported Moreover CTTITEM calculates item difficulty p value for dichotomous items and item mean for polytomous items various item discrimination indices corrected and uncorrected point biserial correlation biserial correla tion and index of discrimination for dichotomous items corrected and uncorrected item total correlation for poly tomous items see Crocker amp Algina 1986 and Ebel amp Frisbie 1991 for detailed discussions of these item sta tistics
7. gh scoring groups The scoring groups are formed based on the same scheme as that used in the SAS macro Two sepa rate SPSS syntax files are available one for dichotomous items CTTITEM_D sps and the other for polytomous items CTTITEM_P sps The following instructions on how to use the syntax apply to both cases Input data Input data should be in text format The first line should contain the correct options for dichoto mous items or the number of score categories for polyto mous items the program assumes that your responses scores for the polytomous items start from 1 with an inter val of 1 For dichotomous items item response data start from the second line with each line representing one case For polytomous items the second line instructs the pro gram whether to reversely score the items 0 or not 1 Examinee scores start from the third line for polytomous items Use a space for a missing value Missing value will be treated as incorrect for the calculation of item and test statistics for dichotomous items Listwise deletion is used for missing values of polytomous items The program will score dichotomous items as right coded as 1 or wrong coded as 0 based on the answer keys provided by the user For polytomous items the program will reversely score the items instructed by the user 1 e 0 s on the sec ond line of the data file For example if the user would like to have high scores to represent hi
8. gh levels of the trait being measured then a 1 should be entered on the second line of the data file for all positively stated items and a 0 for all negatively stated items Syntax A few changes need to be made to the syntax to fit the current data structure They are file locations and variable names and have been highlighted with instruc tions in the syntax files see the brief manual included in the package for additional details Output The first part of the output includes the statis tics of the total score distribution The reliability analysis results that follow include Cronbach s alpha summary statistics for interitem correlations and item total statis CLASSICAL ITEM ANALYSIS 529 tics The corrected item total correlation is commonly called the corrected point biserial correlation P value and D index are then listed for every item in its origi nal order For polytomous items p value is replaced by item mean and D index does not apply If the data are dichotomously scored frequencies of options en dorsed by the low and high scoring groups distractor analysis will be displayed If the data are polytomously scored score point frequencies for every item will be provided Scored item responses total scores as well as item p value and D index are saved in separate data sets for later use Program Availability CTTITEM was developed using SAS IML version 9 1 and is compatible with earlier vers
9. ions of SAS IML e g SAS Version 8 1 The SPSS version was developed with the SPSS Version 13 0 matrix language and is also compatible with older versions e g SPSS Version 11 5 Syntax files of the SAS and SPSS program codes brief manuals example data and output files are available for download free of charge from the first author s Web site suen ed psu edu pwlei plei htm AUTHOR NOTE Correspondence concerning this article should be addressed to P W Lei 230 Cedar Building Pennsylvania State University University Park PA 16802 e mail puiwa psu edu REFERENCES ASSESSMENT SYSTEMS CORPORATION 1989 User s manual for the ITE MAN conventional item analysis program version 3 5 for Windows 3 x St Paul MN Author CROCKER L amp ALGINA J 1986 Introduction to classical and modern test theory New York Holt Rinehart amp Winston CRONBACH L J 1951 Coefficient alpha and the internal structure of tests Psychometrika 16 297 334 EBEL R L amp FRISBIE D A 1991 Essentials of educational measure ment 5th ed Englewood Cliffs NJ Prentice Hall KELLEY T L 1939 Selection of upper and lower groups for the valida tion of test items Journal of Educational Psychology 30 17 24 LEVESQUE R 2003 January 27 Syntax for item analysis SPS Re trieved May 6 2006 from www spsstools net Syntax ItemAnalysis SyntaxForltemAnalysis txt SAMPLE 478 PERFORM ITEM ANALYSIS FOR MULTIPLE C
10. item and a for missing response To call the CTTITEM macro simply copy the entire macro text from MACRO to MEND onto with the dashes replaced by the proper parameter values and then submit Inside the parentheses are parameter val ues in the following order data set name to be analyzed the nonnumerical prefix of the item variables sample size number of items and the type of scoring desired enter 1 for dichotomous or 2 for polytomous items An example including SAS data step commands used to read in an external data file is illustrated in the Appendix SPSS Syntax A similar version of CTTITEM was also created in SPSS For dichotomous items it scores response options as right or wrong based on the answer keys provided by the user For polytomous items it reverse scores negatively stated items so that all items have a consistent scoring scheme It produces score distribution statistics mean standard deviation maximum minimum skewness and kurtosis Cronbach s alpha item difficulty index pro portion correct for dichotomous items and item mean for polytomous items item discrimination indices D index and corrected point biserial correlation for dichotomous items and corrected item total correlation for polytomous items score frequency tables for polytomous items and distractor analysis results for multiple choice items fre quency distribution of options endorsed by low and hi
11. lyses Al though SPSS has a reliability routine for item analysis its use is limited Without additional customization scor ing is tedious in current versions of SPSS Moreover the current reliability routine of SPSS does not produce the D index and conditional option distributions for distractor analysis Levesque 2003 provided an SPSS syntax that is supposed to perform item analysis of multiple choice items as described in the SPSS white paper 1998 How ever the syntax produces some error messages and does not generate the D index when it is run in SPSS 13 due perhaps to some incompatibility between different SPSS versions Moreover neither program performs item analy sis for polytomously scored items Instructions on how to use these programs as well as documentations about how some of the statistics are computed are also not very detailed We created the CTTITEM SAS macro and SPSS syntax to overcome these shortcomings Specifically item analy ses are performed for not only dichotomous items but also polytomous items In addition the D index is calculated for dichotomous items The programs are also made more user friendly by reducing the number of required input modifications and providing brief instructional manuals Details of the program functions are described below CTTITEM A SAS Macro CTTITEM is a SAS macro that performs classical item analysis of dichotomously scored items such as those of the multiple
Download Pdf Manuals
Related Search