Home

Bigsteps user Manual - Winsteps and Facets Rasch Analysis Software

Contents

1. 4 3 2 1 0 1 2 3 4 5 PERSON MEASURE ITEMS 1 1 1 11 111 21 21221 1 2111 Q S M S Q These tables are plots of the standardized fit statistics INFIT or OUTFIT against the parameter estimates INFIT is a standardized information weighted mean square statistic which is more sensitive to unexpected behavior affecting responses to items near the person s ability level OUTFIT is a standardized outlier sensitive mean square fit statistic more sensitive to unexpected behavior by persons on items far from the person s ability level The standardization is approximate Its success depends on the distribution of persons and items Consequently the vertical axis is only a guide and should not be interpreted too rigidly The NORMAL variable controls the standardization method used Letters on the plot indicate the misfitting person or items Numbers indicate non extreme fit or multiple references The letters appear on Tables 6 17 18 19 for persons and Tables 10 13 14 15 for items 8 6 Tables 5 2 and 9 2 Fit plots If both infit and outfit plots are requested then a plot of outfit against infit is also produced to assist with the Page 96 A User s Guide to BIGSTEPS identification of the different patterns of observations they diagnose For interpretation of fit statistics see Appendix 2 4 3 2 1 0 1 2 3 4 5 6 7 8 9 4 4 4 4 4 4 4 4
2. l l F A 8 C C B T 7 0 D R 6 K l J 1 5 H a HSpSSas Soo ET L 3 h O 2 fE A 1l b D 0 i g I 1 F G c a N 2 d eL G 3 4 4 k I j M 5 4 Page 102 A User s Guide to BIGSTEPS ACT MEASURE 8 12 Tables 7 and 11 Misfitting responses controlled by FITI FITP These tables show the persons or items for which the standardized outfit or infit if OUTFIT N statistic is greater than the misfit criterion FITP or FITI Persons or items are listed in descending order of misfit The response codes are listed in their sequence order in your data file The residuals are standardized response score residuals which have a modelled expectation of 0 and a variance of 1 Negative residuals indicate that the observed response was less correct or for rating scales lower down the scale than expected The printed standardized residual is truncated not rounded so that its actual value is at least as extreme as that shown Standardized residuals between 1 and 1 are not printed X indicates that the item or person obtained an extreme score M indicates a missing response TABLE OF POORLY FITTING ITEMS PERSONS IN ENTRY ORDER NUMBER NAME POSITION MEASURE INFIT Z
3. 4 ITEM STATISTICS 4 4 ENTRY RAW INFIT OUTFIT PTBIS NUMBR SCORE COUNT MEASURE ERRORIMNSQ ZSTD MNSQ ZSTD CORR ACTS 4 4 20 49 75 1 74 2011 34 2 0 1 85 3 8 21 WATCH BUGS 4 50 75 1 70 20 88 8 89 6 50 WATCH GRASS CHANGE 8 52 75 1 63 19 1 09 6 1 19 1 0 35 LOOK IN SIDEWALK CRACKS A User s Guide to BIGSTEPS MEASURE ORDER Page 97 4 4 4 4 4 NUM is the sequence number of the person or item in your data and is the reference number used for deletion or anchoring NAME is the item name or person id SCORE is the raw score corresponding to the parameter COUNT is the number of valid data points MEASURE is the estimate for the parameter If the score is extreme a value is estimated but as MAXIMUM perfect score or MINIMUM zero score No measure is reported if the element is DROPPED no valid observations remaining or DELETED you deleted the person or item If unexpected results are reported check whether TARGET or CUTLO or CUTHI are specified ERROR is the standard error of the estimate For anchored values an A is shown on the listing and the error reported is that w
4. 6 P B E 5 A R S 4 o N 3 C T D T 2 U Q M 2 N H E F 1 T J I 24 2 T 0 453w K 31 3 PO L S 1 2721 N 30V v 2 _ 2gk w b d 3 a c t4 4 4 4 4 4 4 4 t 4 3 2 1 0 1 2 3 4 5 6 7 8 9 PERSON OUTFIT ZSTD 8 7 Tables 6 1 10 1 13 1 14 1 15 1 17 1 18 1 19 1 Person and item statistics PERSON STATISTICS controlled by USCALE UMEAN UDECIM LOCAL ISORT PSORT OUTFIT ORDER 4 4 4 ENTRY RAW INFIT OUTFIT PTBIS NUM SCORE COUNT MEASURE ERROR MNSQ ZSTD MNSQ ZSTD CORR PERSON fo 4 73 28 25 38 3412 87 5 0 4 83 6 8 A 01 SANDBERG RYNE 71 33 25 97 391 3 15 5 314 93 5 8 B 06 STOLLER DAVE 72 14 25 1 32 var l2 OL 2 815 13 5 7 C 10 JACKSON SOLOMON BETTER FITTING OMITTED 37 27 25 26 34 44 2 7 42 2 3 b 87 AIREHEAD JOHN 21 28 25 38 34 29 3 9 31 3 0 a 84 EISEN NORM L 4 4 4 MEAN 31 25 89 39 98 2 1 08 S D 8 0 1 21 10 50 1 6 1 03 1 9 4
5. 0 5 34E 1 44 4 2 40 77 8 64 92 12 3 94 93 1 4 57 1 07 5 1 77 80 9 1 49 89 13 4 98 1 12 2 3 66 84 6 1 06 86 10 2 30 88 14 5 80E 1 47 3 3 00 me I 7 23 92 11 3 10 89 4 4 4 4 A User s Guide to BIGSTEPS Page 103 RAW SCORE MEASURE OGIVE FOR COMPLETE TEST 4 14 E E 13 a x 124 X P 11 i E 10 Cc 9 T 8 7 E 7 D 6 3 5 S 4 x Cc 3 o 2 a R 1 G E 0 E 6 4 2 0 2 4 6 MEASURE TABLE OF SAMPLE NORMS 500 100 AND FREQUENCIES CORRESPONDING TO COMPLETE TEST 4 4 SCORE MEASURE S E NORMED S E FREQUENCY CUM FREQ PERCENTILE 4 4 0 6 04E 1 45 241 66 aL 2 9 1 2 9 1 7 27 1 11 504 51 12 34 3 23 65 7 49 14 6 51E 1 47 813 67 0 0 35 100 0 100 4 4 The columns in the Table of Sample Norms and Frequencies are Measures on the Complete Test SCORE raw score on a complete test containing all calibrated items MEASURE measure corresponding to sc
6. wk ok KO Il TITLE LIKING FOR SCIENCE wright amp Masters p 18 CONTROL FILE data example hold sf dat OUTPUT FI DATE LE sf out Nov 20 13 45 1996 4 4 ape Sa nh OVERVIEW TABLES EENE EOE E EOE A re ffs ITEM CALIBRATIONS eee A A Lk ee Je Ss at gee ak apo ie 1 PERSON AND ITEM DISTRIBUTION MAP 12 ITEM MAP BY NAME 2 MOST PROBABLE RESPONSES SCORES 13 ITEM MEASURES IN DIFFICULTY ORDER 3 PERSON ITEM AND STEP SUMMARY 14 ITEM MEASURES IN ENTRY ORDER eE EEEO 15 ITEM MEASURES IN ALPHA ORDER PERSON FIT PERSON MEASURES 4 PERSON PLOT OF INFIT vs ABILITY 5 PERSON PLOT OF OUTFIT vs ABILITY 16 PERSON MAP BY NAME 6 PERSON MEASURES IN FIT ORDER 17 PERSON MEASURES IN ABILITY ORDER 7 DIAGNOSIS OF MISFITTING PERSONS 18 PERSON MEASURES IN ENTRY ORDER 19 PERSON MEASURES IN ALPHA ORDER ITEM FIT o o REFERENCE TABLES 8 ITEM PLOT OF INFIT vs DIFFICULTY 9 ITEM PLOT OF OUTFIT vs DIFFICULTY 20 SCORE TABLE 10 ITEM MEASURES IN FIT ORDER 21 CATEGORY PROBABILITY CURVES 11 DIAGNOSIS OF MISFITTING ITEMS 22 SORTED RESPONS
7. nOne cee eeeeeseeeseeesecseesceessecseeseeaesesseeaeeaseessesseeaeeeseneaes 33 4 2 8 KEYSCR reassign scoring keys default 123 oo ee eessssessesseeeeeeseseeeseeseeessecseeaeeesseeaes 35 4 29 Disjomt Strings Of TeSspOnSes 25 a r a a a a Hashes at a a aias 37 4 2 10 FORMAT reformat data default none seseessessessessessessessessessessessessessessessessessessesses 37 4 2 11 RESFRM location of RESCORE default N before amp END s sssssesssseessessessresrsseesessesses 41 4 2 12 KEYFRM location of KEYn default 0 before KEND ss ssesssesssssesssessssessessrssessresessesses 41 4 2 13 CUTHI cut off responses with high probability of success default 0 NO eee 42 4 2 14 CUTLO cut off responses with low probability of success default 0 nO eee 43 4 3 Specifying the structure of rating scales oie ese eseeeeeeeneeseeeeeeeseeseeseesseeseeseasseeseeaeeaseessesseeaeeaseeens 43 4 3 1 MODELS assigns model types to items default R rating scale oo eee eeeeeeereeeeeeeeeees 43 4 3 2 STKEEP keep non observed steps categories default N no csc eeeeeseseeseeeeeeeneeseeeseeeees 44 433 Weighting Wtems ss siestssciylegsitslebecttevacislacdechstie a Ae A aes d edhe yieisevnatesaiiaaees 44 4 3 4 GROUPS assigns items to groups default all in one group 0 eee eseeeeeeeeeteeseeeeeeeees 46 4 3 5 MODFRM location of MODELS default N before amp END ccccecccsceseeteeeteeeteeeeees 47
8. seeeeeeeeeeeeeeeeeeeerere 73 OUTFIT sort misfits on infit or outfit default Y Outfit cccscccesesceeessseeeeessneeeeessneeeeessseeeeeees 72 PAFILE name of person anchor file default no file 0 eee eeescceeeeeeeeneeesseeeeeeecesaecesaeecesneeeeaeessaes 54 PAIRED correction for paired comparison data default N ce eeeceesecsseceseceneeeseeeseeceseeeseseseeneeeaees 64 PANCHQ anchor persons interactively default N nO cceeseccsseccesseeceseeeeeeeeeseecesaeeceeecesaeeeseeeeaees 54 PDELQU delete persons interactively default N NO cecceescccssncceeseeceseeceeaeeceeeeesaeecsaeecesaeeeeeeeesaees 53 PDFILE name of person deletion file default no file ee eee eseseceenceeesneceeeeceeeceeaeeeesneeeeaeeeaes 52 PERSONS title for person names default PERSON ccscccssscccesceceeeeeeeecesneceeaeeceaeeesaeecesneeeeaeesaes 69 PFILE person output file default no file eee ee ceeeeseecesecsseeeseeeeecseeceseceaeeeseeeeesseeseeseaeseaeeeaeees 80 PRCOMP principal components analysis of item residuals in Table 10 default N ee eeeeeeeeeee 78 PSELECT person selection criterion default all persons 0 0 0 eeeeseeesseesseeeneeeneeeseeceseeeseeeseenseeeaees 53 PSORT column within person name for alphabetical sort in Table 19 default 1 ee eeeeeeee 77 PTBIS compute point biserial correlation coefficients default Y yes ces ceseeseeceeeceseceseeeseeeseeesees 73
9. 4 6 4 SANCHQ anchor steps interactively default N no If your system is interactive steps to be anchored can be entered interactively by setting SANCHQ Y between the amp INST and amp END lines If you specify this you will be asked if you want to anchor any steps If you respond yes it will ask if you want to read these anchored items from a file if you answer yes it will ask for the file name and process that file in the same manner as if SAFILE had been specified If you answer no you will be asked to enter the step measures found in Table 3 If there is only one rating scale enter the category numbers for which the steps are to be anchored one at a time along with their logit or rescaled by USCALE step measure calibrations Bypass categories without measures Enter 0 where there is a measure of NONE When you are finished enter 1 in place of the category number If there are several rating scales enter one of the item numbers for each rating scale then the step measures corresponding to its categories Repeat this for each category of an item for each rating scale Enter 0 where there is a step measure for a category of NONE Entering 0 as the item number completes anchoring Example 1 You are doing a number of analyses anchoring the common rating scale to different values each time You want to enter the numbers at your PC SANCHQ Y You want to anchor items 4 and 8 BIGSTEPS asks you
10. 6 3 4 LINLEN length of printed lines in Tables 7 10 16 22 default 80 The misfitting responses name maps scalogram and option frequency tables can be output in any convenient width Specify LINLEN 0 for the maximum page width 132 characters Example You want to print the map of item names with up to 100 characters per line LINLEN 100 set line length to 100 characters 6 3 5 FRANGE half range of fit statistics on plots default 0 auto size Specifies the standardized fit Y axis half range i e range away from the origin for the standardized fit plots FRANGEZ is in units of standardized fit i e expected mean 0 standard deviation 1 Example You want the fit plots to display from 3 to 3 units of standardized fit FRANGE 3 6 3 6 MRANGE half range of measures on plots default 0 auto size Specifies the measure X axis on most plots half range i e range away from the origin or UMEAN of the maps plots and graphs This is in logits unless USCALE is specified in which case it must be specified in the new units defined by USCALE See TFILE for instructions on customizing particular tables Example 1 You want to see the category probability curves in the range 3 to 3 logits MRANGE 3 Example 2 With UMEAN 500 and USCALE 100 you want the category probability curves to cover a range from 250 to 750 UMEAN 500 new item mean calibration USCALE 100 value of 1 logit MRANGE 250 to be plotted ea
11. A User s Guide to BIGSTEPS Rasch Model Computer Program John M Linacre Benjamin D Wright Winsteps com A User s Guide to BIGSTEPS Rasch Model Computer Programs John M Linacre Benjamin D Wright Winsteps com 773 702 1596 FAX 773 834 0326 www winsteps com Copyright 1991 1998 John M Linacre This software and manual are now freeware Permission to copy is granted ISBN 0 941938 03 4 January 2 2006 1 2 3 4 TABLE OF CONTENTS INTRODUC TION Ae en e A AS A EAA EA AAAA EE A IAE A AENA 1 1 1 Rasch analysis and BIGS TERS nccrensnsnnunnone ne e a a a paa 1 1 2 About the User s Guideshop ene aneis ae a eerie N ESE wane dared ASAS N ERNS 2 ka Pypestyle a eta eit hain al aa tien deen an aes 2 GENERALE DESCRIPTION aiio ea u seat la cttat ts srt atteete a te chiagte SU baele otitis Sra a atid 3 2 0 Installing BIGSTEPS s 2 2f 2s 35st aaar a a a aN E aae Aaaa aa at ete toed a aa enean ieie 3 2 0 2 Installation instructions for BIGSTEPS eeeseseseeseeeesesesesesrsseseseresesesestststsrsrstererereeeesrsrsrsesesese 4 2 1 How touse BIGS TERS aihe un eee eaa et ee dad ee aa 4 2A Starting BIGSTEPS 1 DOS itet e e Seaat ena retara eliro aeie taa eteten eiiiai rtiai 4 2 4 1 Reading extra control variables from the DOS prompt line sseeeeeeeeeeesesesesesesesssrsesesrsrsrsrsesee 4 25 Stoppin BIGSTEPS saiae a a a eea aa a aa ea aa a stitched Aa a Ee aita 5 2 6 The BIGSTEPS
12. Across the bottom is the logit or rescaled scale of the variable with the distribution of the person abilities shown beneath it An M marker represents the location of the mean person ability S markers are placed one standard deviation away from the mean Q markers are placed two standard deviations away The Most Probable Response Table selected with CURVES 001 answers the question which category is a person of a particular measure most likely to choose This is the most likely category with which the persons of logit or rescaled ability shown below would respond to the item shown on the left The area to the extreme left is all 0 the area to the extreme right is at the top category Each category number is shown to the left of its modal area If a category is not shown it is never a most likely response An item with an extreme perfect or zero score is not strictly estimable and is omitted here Blank lines are used to indicate large gaps between items along the variable This table presents in one picture the results of this analysis in a form suitable for inference We can predict for people of any particular ability measure what responses they would probably make M depicts an average person The left Q a low performer The right Q a high performer Look straight up from those letters to read off the expected response profiles Page 90 A User s Guide to BIGSTEPS MOST PROBABL
13. ROSSNER JACK 21 ja EISEN NORM L Peano st es Sson teas SoS SSS See Se SSS Sa Seas ae Sea Sa See ore ase esses A User s Guide to BIGSTEPS Page 99 The fit information is shown in graphical format to aid the eye in identifying patterns and outliers The fit bars are positioned by default at 0 7 and 1 3 They may be repositioned using TFILE 8 9 Tables 10 3 13 3 14 3 15 3 Item option frequencies controlled by DISTRT Y ITEM OPTION FREQUENCIES are output if DISTRT Y These show occurrences of each of the valid CODESz and also of MISSING in the input data file Counts of responses forming part of extreme scores are included Only items included in the corresponding main table are listed ITEMS OPTION DISTRACTOR FREQUENCIES OUTFIT ORDER 4 4 4 NUM NONMISS MISSING R SCR 00 SCR 01 SCR 02 SCR 23A 75 de SL SSO 44 58 0 20 26 1 11 14 2 5B 75 LO ye 47 62 0 19 25 1 9 12 2 18F 75 1 1 1 1 0 3 4 1 71 94 2 4 4 NUM is the item sequence number The letter next to the sequence number is used on the fit plots NONMISS is the number of non missing responses to this item MISSING is the number of missing not in CODES responses R is the percent
14. Responses are revalued according to the matching codes in IREFER Example Items identified by Y and Z in IREFER are to be recoded Y type items are 1 3 7 8 Z type items are 4 6 9 10 CODES ABCD NI 10 IREFER YY YZZZYYZZ IVALUEY 1234 IVALUEZ 4321 4 2 7 KEYn scoring key default none Usually only KEY1 is needed for an MCQ scoring key Up to 99 keys can be provided for scoring the response choices with control variables KEY1 through KEY99 Usually KEY1 is a character string of correct response choices The default is one column per correct response or two columns if XWIDE 2 By default responses matching the characters in KEY1 are scored 1 Other valid responses are scored 0 KEY2 through KEY99 are character strings of successively more correct response choices to be used when more than one level of correct response choice is possible for one or more items The default score value for KEY2 is 2 and so on up to the default score value for KEY99 which is 99 The values assigned to these keys can be changed by means of KEYSCR If XWIDE 1 only the values assigned to KEY1 through KEY9 can be changed KEY10 through KEY99 retain their default values of 10 through 99 If XWIDE 2 the all KEYn values can be changed Page 34 A User s Guide to BIGSTEPS Example 1 A key for a 20 item multiple choice exam in which the choices are coded 1 2 3 and 4 with one correct choice p
15. With FORMAT Columns of reformatted record 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 90123456789012345678 CDBACBCAADDDDCDDCACDCBACCBA CZarathrustra xXerxes XWIDE 2 response width FORMAT 4X 28A2 20A1 skip unused characters ITEM1 1 start of item responses in formatted record NI 28 lt number of items NAME1 29 start of name in columns NAMLEN 20 length of name CODES ABCD valid response codes Example 3 Each person s data record is 80 characters long and takes one line in your data file Person id is in columns 61 80 30 1 character item responses A B C or D are in columns 5 34 13 2 character item responses 01 02 or 99 are in 35 60 XXXXDCBDABCADCDBACDADABDADCDADDCCA0N1990201019902010199020201Zarathrustra xXerxes becomes on reformatting Columns 1234567890123456789012345678901 2 3 4 5 6 7 8 9 0 1 2 3 45678901234567890123 DCBDABCADCDBACDADABDADCDADDCCA01990201019902010199020201Zarathrustra xerxes XWIDE 2 analyzed response width FORMAT 4X 30A1 13A2 20A1 skip unused ITEM 1 1 start of item responses in formatted record NI 43 lt number of items NAME1 44 start of name NAMLEN 20 length of name CODES A B C D 010299 valid responses t character code followed by blank Example 4 The person id is 10 columns wide in columns 15 24 and the 50 1 column item responses A B C D are in colum
16. 223 keyed for selected items KEY2 33X the X will be ignored read these columns vertically A User s Guide to BIGSTEPS Page 33 4 2 4 NEWSCORE recoding values with RESCORE default none Says which values must replace the original codes when RESCORE is used If XWIDE 1 the default use one column per code If XWIDE 2 use two columns per code The length of the NEWSCORES string must match the length of the CODES string For examples see RESCORE NEWSCORE is ignored when KEYn is specified 4 2 5 IREFER identifying items for recoding with IVALUE default none Responses are revalued according to the matching codes in IVALUE Example 1 There are are 3 item types Items are to rescored according to Type A and Type B Other items to keep original scoring CODES 1234 IREFER AAAAAAAABBBBBBBB lt amp 3 item types IVALUEA 1223 Recode Type A items IVALUEB 1123 Recode Type B items IVALUE 1234 amp Recode Type item Can be omitted Example 2 There are are 3 item types Items are to rescored according to Type A and Type B Other items to keep original scoring XWIDE 2 XWIDE 2 CODES 1 23 4 IREFER AAAAAAAABBBBBBBB a 3 item types IVALUEA 1223 Recode Type A items IVALUEB 1123 Recode Type B items IVALUE 1234 Recode Type item Can be omitted 4 2 6 IVALUEx recoding for x type items with TIREFER default none
17. 4 3 6 GRPFRM location of GROUPS default N before amp END ccc ecceseescesseeseeeteeeteeeeens 47 4 4 Deleting or anchoring Hemsire niire Ee R A AA Mis EN A R an ES aaka 48 4 4 1 IDFILE name of item deletion file default no file oc cescessceseceseceeceeeeseeeeeeeeeseees 48 4 4 2 IDELQU delete items interactively default N 10 cccsceceesceseeseceeeeeeeseeeceeeeeeeseeeeeeeeenees 50 4 4 3 IAFILE name of item anchor file default no file eee eecceseeseceteeeteeeeeeseenseeeseesseens 51 4 4 4 IANCHQ anchor items interactively default N NO cececeeseeseeseceeeeeeeeceeceeeeeeeaeceeeeeeenees 51 4 5 Deleting or anchoring Persoms cieeeeesesseecseecseeseeseeeseesesseesesesseeseeseeesseeseeseeassesseeaeeaseessesaeeaseaseeens 52 4 5 1 PDFILE name of person deletion file default no file eee eeeeneeseeteeeeeeeeeaeeeeeereenees 52 4 5 2 PDELQU delete persons interactively default N nO cceceecesseseceeeeeseceeceeeeeeeseeeeeeeeenees 53 4 5 3 PSELECT person selection criterion default all persons 0 cc eeeeeeeseeseeeeeeeseeseeeeeeeees 53 4 5 4 PAFILE name of person anchor file default no file 0 eee eeeeseeeeeteeeeeeeeeeeeaeeeeeeeeeeees 54 4 5 5 PANCHQ anchor persons interactively default N NO cceeceseseceeeceeteceeceeeeeeeaeeeeeeeeaees 54 4 6 Categories and steps labeling deleting and anchoring eseseeeeeeseeseeeeeeeeeseeeeeesseeseeaseeseeens 5
18. ITEMI 11 NI 66 66 matches columns in total PAIRED YES INUMBER YES number the matches in the Output amp END Browne ipo 1 1 1 1 D D 1 1 Mariotti 0 1D 0 1 1 1 D 1 D 1 Tatai DO O 1 D D 1 1 1 1 D Hort 1D1 D D 1 D D D 1 0 Kavalek 010D D D 1 D 1 1 D Damjanovic OODDD D D D 1 D 1 Gligoric OODODD D 1 1 1 0 Radulov OOODODD D 1 D 1 Bobotsov DDODDDOD 0 0 1 Cosulich D00D00001 1 1 westerinen 0D000D0D10 1 zichichi 00D1D010000 Part of the output is PLAYER STATISTICS MEASURE ORDER Sa aaa aa al a ENTRY RAW INFIT OUTFIT NUMBR SCORE COUNT MEASURE ERROR MNSQ ZSTD MNSQ ZSTD PLAYER Sea a a ere ate te 1 17 11 1 09 35 1 10 2 1 02 1 Browne l 2 15 11 68 32 1 02 O 96 1 Mariotti 12 6 11 88 33 1 86 1 8 1 90 1 7 Zichichi Se See ar ee So ee eae ee MEAN 11 11 00 32 99 2 e99 2 l S D 3 0 59 01 37 1 1 39 1 1 l aa a ne a a Page 26 A User s Guide to BIGSTEPS 4 READING YOUR DATA 4 1 Specifying the layout of your data Where to find your data its format and the location of person and item identifying names 4 1 1 DATA name of data file default data at end of control file Your data can be the last thing in the control file which is convenient if you only have a small amount of data but if you have a large amount of data you can place it in a separate file and then use DATA to say where it is Exa
19. In Table 1 each person or item is indicated by an X or when there are too many X s to display on one line several persons or items are represented by a The left hand column locates the person ability measures along the variable For dichotomous items the right hand column locates the item difficulty calibrations along the variable Look for an even spread of items along the variable the Y axis with no gaps indicating poorly defined or tested regions of the variable The persons often show a normal distribution Good tests usually have the items targeted lined up with the persons For rating scales each item is shown three times In the center item column each item is placed at its mean calibration i e this is the location of the center of the rating scale the location at which being ratings in the top and bottom category are equally probable In the left hand item column the item is shown at the ability level corresponding to a probability of 5 of exceeding or being rated in the bottom rating scale category In the right hand item column the item is shown at the ability level corresponding to a probability of 5 of being rated in or falling below the top rating scale category Dichotomous items D have only one location MAP OF PUPILS AND ACTS MEASURE P 50 P 50 MEASURE lt more gt PUPILS ACTS BOTTOM ACTS CENTER ACTS TOP lt rare gt 4 0 X 4 0 x x 3 0
20. Like Example 2 Items 1 10 Group 1 are Strong Disagree Disagree Agree Strongly Agree Items 11 20 Group 2 are Never Sometimes Often Always NI 20 CODES 1234 GROUPS 11111111112222222222 CFILE 7 1 Strongly Disagree 7 is any item in Group 1 7 2 Disagree A User s Guide to BIGSTEPS Page 55 7 3 Agree 7 4 Strong Agree 13 1 Never 13 is any item in Group 2 13 2 Sometimes 13 3 Often 13 4 Always x Example 3 To enter CFILE information on the DOS Prompt or Extra Specifications lines using commas instead of blanks as separators C gt BIGSTEPS SF DAT SF OUT CFILE 1 Dislike 2 Don t know 3 Like Example 4 One group of items has a unique response format other groups all have the same format Here each group has only one item i e GROUPS 0 NI 20 CODES 1234 GROUPS 0 CFILE 1 Strongly Disagree This scale is used by most items 2 Disagree 3 Agree 4 Strong Agree 16 1 Never 16 is the one item using this scale 16 2 Sometimes 16 3 Often 16 4 Always x Example 5 Several categories are collapsed into one category The original codes are A H After rescoring there is only a dichotomy 0 1 NI 30 CODES ABCDEFGH NEWSCORE 00011110 CFILE 0 Fail Specify the categories as reported 1 Pass x 4 6 2 SDFILE name of item step category deletion file default no file Deletion of categories from a test analysis i e conversion of responses in these categories to mis
21. PROX ACTIVE COUNT EXTREME 5 RANGE MAX LOGIT CHANGE ITERATION PERSONS ITEMS CATS PERSONS ITEMS MEASURES STEPS 1 76 25 3 3 78 3 20 3 8918 0740 2 74 25 3 4 53 3 67 7628 6167 DROPPING OUT OF RANGE OBSERVATIONS This is reported for CUTLO and CUTHI 3 74 25 3 4 73 3 85 2143 0991 4 74 25 3 4 82 3 90 0846 0326 Control sf dat Output sf out UCON MAX SCORE MAX LOGIT LEAST CONVERGED CATEGORY STEP ITERATION RESIDUAL CHANGE PERSON ITEM CAT RESIDUAL CHANGE 1 3 01 4155 60 24 2 27 64 0184 2 50 0258 53 24 1 6 88 0198 3 37 0292 53 5 1 3 10 0091 4 26 0206 53 21 1 2 74 0079 5 20 0154 53 21 0 1 90 0056 6 15 0113 53 21 0 1 42 0042 7 11 0083 53 21 0 1 05 0030 A User s Guide to BIGSTEPS Page 85 an extra pass to calculate fit statistics Calculating Fit Statistics Processing Misfitting PERSONS for Table 7 Calculating Correlations for Table 10 Calculating Principal Components for Table 10 G one per iteration one per factor for Table 11 amp misfitting items x Processing Misfitting ITEMS Sorting ITEMS for Table 12 Sorting ITEMS for Table 15 Sorting PERSONS for Table 16 Sorting PERSONS for Table 19 Calculating Scores for Table 20 writing Sorted Responses in Table 22 Guttman Scalogram Analysis completed of SF DAT amp name of
22. RCONVE score residual at convergence default 0 5 ceecceescccesnccesseeceeaeeeeeecesnecesaeecesaeceeaeeceeeeeeeaeeeaes 61 REALSE inflate S E for misfit default N no misfit alloWaNCe cc eeessccceesssceeeseteeeesssnseeeeseneeeesees 62 RESCORE response recoding with NEWSCORE or KEYn default 00 eeeceesneeeneeeeeees 32 RESFRM location of RESCORE default N before amp END cccccccccesscccessneeesesneeeeeseneeeeeseneeeeeees 41 RFILE scored response file default no file oo eee eesceceseecesneeceseecesaeeeeaeecesnecesaeecseaeeeeaeeceeeeeeeaeeeaes 81 SAFILE name of item step anchor file default no file 00 eee eeeeceseeeseeeeeeeeeeeeceeneeseaeeeeneeesaeeees 57 SANCHQ anchor steps interactively default N nO eeseessesseeeceseeceseeeeeeceseeeeeaeeceaeeseaeeceseeeeeaeeees 59 SDELQU delete item step categories interactively default N nO eeeeseeeeseeceseeeseeeseeeseeeneeenseeees 57 SDFILE name of item step category deletion file default no file seeneseeeseseeseereererserserereerrerese 56 SFILE step category output file default no file eeeeeeeseeeeseeseesessessessrssrssresresresresresseereserssersresees 81 SPFILE supplementary control file default no file eneeeeeeeeeeeeeseesessrssreerssresresresressersessresersresees 83 STBIAS correct for UCON estimation statistical bias default N nO cccsccccesssecceee
23. REALSE inflate S E for misfit default N no misfit allOWance cccessccceesscecessseeeeeseteeeesssseeeeeees 62 RESCOREZ response recoding with NEWSCORE or KEYn default 00 eesceeesneeeeneeeeeeees 32 RESFRM2 location of RESCORE default N before KEND cecccccccsssscceeseseeeeeeseeeeessneeeesssseeeeeees 41 RFILE scored response file default no file 0 eee eeeseeceneceeseeceeeeceneecesaeeceseecesaeessaeecesaeeeeeeeesaees 81 SAFILE name of item step anchor file default no file eee eeececeseeeeeeeceseeeesaeeeeaeeseseeeeeaeeesaes 57 SANCHQ anchor steps interactively default N nO eeescceescccssneeeseeeceseeeesaeeceeecesaeeseaeeceeaeeesaeessaes 59 SDELQU delete item step categories interactively default N nO ces eeeesseeeseceeceseceneeeseeeneeesaeeees 57 SDFILE name of item step category deletion file default no file eeeeesseeeeseereereereereerrrsrreresresre 56 SFILE step category output file default no file eee eeeeceseceseceeceeeeseesenecsaesssessseeeeesseeeeeenas 81 SPFILE supplementary control file default no file oo ceseceseceseeeseeeseeceseesseseseeeeeeneeesaeenes 83 STBIAS correct for UCON estimation statistical bias default N no cccccscccsessseceeseseeeeseseeeeeeeees 62 STEPT3 include step summary in Table 3 or 21 default Y in Table 3 oo eee eeeeeseeeseeeneeeeeeeees 76 STKEEP keep non observe
24. Statistica Neerlandica 1990 44 2 69 78 6 2 5 LOCAL locally restandardize fit statistics default N no LOCALEN accords with large sample statistical theory Standardized fit statistics test report on the hypothesis test Do these data fit the model With large sample sizes and consequently high statistical power the hypothesis cannot be accepted because all empirical data exhibit some degree of misfit to the model This can make standardized statistics meaninglessly large LOCAL N Standardized fit statistics are not rescaled Even the slightest item misfit in tests taken by many persons will be reported as very significant Columns reporting this option are headed ZSTD for theoretically standardized LOCAL Y Standardized fit statistics are rescaled to reflect their level of significance in the context of the amount of disturbance in the data being analyzed The rescaling factor is chosen to make the variance of the rescaled positive fit statistics away from the origin 1 Negative standardized statistics are rescaled by the same factor so that no fit statistic crosses over from one side of the origin to the other Thus the mean and standard deviation of the standardized statistics are also rescaled by the same factor The effect of the rescaling is make the fit statistics more useful for interpretation Columns reporting this option are headed EMP for empirically restandardized LOCAL L Instead of standardized
25. XFILE analyzed response file default no file 0 eee eeseeeseeeseeeseecseeceseceseceaeeeseeeseesseeseaeseaeeeaeee 82 XWIDE columns per response default 1 0 cee eeseeseesseecsecseeseeeseeeseecseecssecesecsseesseesseesseessaeseaeseaeees 28 1 INTRODUCTION 1 1 Rasch analysis and BIGSTEPS Rasch analysis is a method for obtaining objective fundamental linear measures qualified by standard errors and quality control fit statistics from stochastic observations of ordered category responses Georg Rasch a Danish mathematician formulated this approach in 1953 to analyze responses to a series of reading tests Rasch G Probabilistic Models for Some Intelligence and Attainment Tests Chicago MESA Press 1992 with instructive Foreword and Afterword by B D Wright BIGSTEPS implements Rasch s formulation by means of modified versions of the PROX and UCON estimation methods for more information and further references see BTD and RSA Flaw of work using BIGSTEPS BISSTEPS tee k Control File Contro Instructions ees Fem Word Procesor abals j i tem Name Labels oe Bate Entry of EHD MAWES Database Observatlons We gdh y Sove as DUS TEXT or ASCA BIGSTEPS i Statistico Pookoge Lise Courter or Reporting Procedures fred spore font Word Processor BIGSTEPS is designed to construct Rasch measurement from the responses of a set of persons to a set of items Responses may be recorded as letters or integers and
26. You want the lowest reportable person measure to be 0 and the highest to be 100 Looking at Table 20 you see the extreme values are 4 53 and 5 72 The current values in the output are USCALE 1 and UIMEAN 0 Page 66 A User s Guide to BIGSTEPS Example 5 Example 6 USCALE previous USCALE wanted range current range USCALE 1 100 0 5 72 4 53 1 100 10 25 9 76 UMEAN2 wanted low current low previous UMEAN wanted range current range 0 4 53 0 100 10 25 44 20 Double checking when previous UMEAN 0 USCALE 1 low value current low USCALE UMEAN 4 53 9 76 44 20 0 01 high value current high USCALE UMEAN 5 72 9 76 44 20 100 02 Required values are UIMEAN 44 20 USCALE 9 76 UDECIM 0 to show no decimal places in report You want the lowest reportable person measure to be 100 and the highest to be 900 Looking at Table 20 you see the extreme values are 4 53 and 5 72 Looking at the second page of output you see the current values are USCALE 1 and UMEAN 0 USCALE previous USCALE wanted range 900 100 reported range 5 72 4 53 1 800 10 25 78 05 UMEAN2 wanted low reported low previous UMEAN wanted range reported range 100 4 53 0 800 10 25 453 56 Required values are UIMEAN 453 56 USCALE 78 05 UDECIM 0 to show no decimal places in report You want norm referenced scaling
27. auto size For ease in comparing the outputs from multiple runs force consistent scaling by using MRANGE T1I and T1P Choose T1P to be the largest number of persons summarized by one from any of the separate runs Example In one run the bottom of Table 1 states that EACH IN THE PERSON COLUMN IS 250 PERSONS In another run EACH IN THE PERSON COLUMN IS 300 PERSON To make the runs visually comparable specify the bigger value T1P 300 6 3 11 ISORT column within item name for alphabetical sort in Table 15 default 1 Table 15 lists items alphabetically By default the whole item name is used To sort starting on a column after the first item name column specify ISORT Example 1 The item name is entered in the specification file as sequence number followed by identification in column 6 Sort by identification for Table 15 NI 4 TABLES 111111111111111111111111 ISORT 4 amp END 0001 Addition Item 0002 Subtraction Item 0003 Multiplication item 0004 Division item Page 76 A User s Guide to BIGSTEPS T sort column END NAMES Example 2 The item name contains several important classifiers Table 15 is required for each one TFILE 15 1 sort starts with column 1 of item name 15 6 sort starts with column 6 15 13 sort starts with column 13 t entered as place holders see TFILE x amp END MCQU Geogrp 1995 0234 T sort column T sort column T sort column END NAMES Example 3
28. rating scale Estimates calibrations for four different ordered response category structures Dichotomies are always analyzed the same way regardless of what model is specified but it is convenient to specify them as 2 category rating scales MODELS R_ default uses the Andrich Rating Scale model This is also the Masters Partial Credit model when GROUPS 0 The response category is deliberately chosen with knowledge of all others This model is a stochastic parameterization of Guttman pattern data MODELS S _ uses the Glas Verhelst Success growth model f and only if the person succeeds on the first category another category is offered until the person fails or the categories are exhausted e g an arithmetic item on which a person is first rated on success on addition then if successful on multiplication then if successful on division etc Scaffolded items often function this way Note Success Failure may not estimate correctly MODELS F uses the Linacre Failure mastery model Jf a person succeeds on the first category top rating is given and no further categories are offered On failure the next lower category is administered until success is achieved or categories are exhausted Note Success Failure may not estimate correctly When only one letter is specified with MODELS e g MODELS R all items are analyzed using that model Otherwise MODELS some combination of R s F s S s and
29. such that the person mean is 0 0 and the person standard deviation is 1 0 In the standard run according to Table 3 0 the person S D is 0 8 Then in the rescaled run USCALE wanted S D reported S D 1 0 0 8 1 25 UPMEAN 0 A User s Guide to BIGSTEPS Page 67 6 OUTPUT CONTROL 6 1 Overall reporting These options enable you to make the tabular output more meaningful 6 1 1 TITLE title for output listing default control file name Use this option to label output distinctly and uniquely Up to 60 characters of title This title will be printed at the top of each page of output Example You want the title to be Analysis of Math Test TITLE Analysis of Math Test 6699 Quote marks or are required if the title contains any blanks 6 1 2 TABLES output tables default 1110011001001000100000 If in doubt omit TABLES and obtain the default the tables A 22 character string that tells BIGSTEPS which output tables to prepare for printing The sequence number of the 1 or 0 in the TABLES string matches the table number For more elaborate table selection use TFILE 1 means prepare the corresponding table 0 or anything else means do not prepare the corresponding table See p 120 for the list of output tables Example 1 You want only Tables 2 4 6 8 10 and 20 to be prepared TABLES 01010101010000000000100 This is the same as specifying TFILE 2 4 6 8 10
30. 15 has four categories 0 1 2 3 and this particular scale is to be anchored at pre set calibrations 1 Create a file named say PC 15 2 Enter the lines 1500 Bottom categories are always at logit 0 15 1 2 0 item 15 step to category 1 anchor at 2 logits 1520 5 1531 5 3 Specify in the control file GROUPS 0 SAFILE PC 15 Example 3 A grouped rating scale analysis GROUPS 21134 has a different rating scale for each group of items Item 26 belongs to group 5 for which the scale is three categories 1 2 3 and this scale is to be anchored at pre set calibrations 1 Create a file named say GROUP ANC 2 Enter the lines 26 2 3 3 for item 26 representing group 5 step to category 2 anchored at 3 3 26 3 3 3 3 Specify in the control file GROUP 21134 SAFILE GROUP ANC Example 4 A questionnaire includes several rating scales each with a pivotal step between two categories The item difficulties are to be centered on those pivots 1 Use GROUPS to identify the item scale subsets 2 Look at the scales and identify the pivot point e g Strongly Disagree 1 Disagree 2 Page 58 A User s Guide to BIGSTEPS Neutral 3 Agree 4 Strongly Agree 5 If agreement is wanted the pivot point is the step from 3 to 4 If no disagreement is wanted the pivot point is the step from 2 to 3 3 Anchor the step corresponding to the pivot point at 0 SAFILE 23 30 23 is an item in group pivoted at agreement 34 x
31. 2 3 1 2 4 1 3 4 2 1 4 3 4 1 1 4 3 2 1 4 2 3 1 3 2 4 2 4 3 1 1 3 1 2 4 1 3 2 4 3 1 4 3 2 4 1 4 2 3 4 1 1 3 2 4 1 3 1 4 2 3 1 4 1 4 3 1 2 4 4 1 3 4 2 1 4 last 18th item name END NAMES signals end of item names Richard M 111111100000000000 first data record Tracie F 111111111100000000 more data records Elsie F 111111111101010000 Helen F 111000000000000000 last data record Page 10 A User s Guide to BIGSTEPS Example 2 Control and anchor files A control file EXAMPLE2 CON for the analysis of a test containing 18 items each item already scored dichotomously as 0 1 The person id data begins in column 1 and the item response string begins in column 11 The default tables will be appear in the printout There is user scaling Items 2 4 6 and 8 are anchored at 400 450 550 and 600 units respectively supplied in file EXAMPLE2 JAF Your data is in file EXAMPLE2 DAT This file is EXAMPLE2 CON amp INST TITLE KNOX CUBE TEST ANCHORED the title for output NI 18 the number of items ITEM 1 11 position of first response in data record NAME1 1 first column of person id in data record PERSON KID ITEM TAP DATA EXAMPLE2 DAT name of data file IAFILE EXAMPLE2 IAF EXAMPLE2 IAF is item anchor file UIMEAN 500 user scaling item mean USCALE 100 user scaling 1 logit 100 UDECIM 0 print measures without decimals amp END 1 4 item names 4 1 3 4 2 1 4 E
32. 29 F8 2 2 Measure for an expected score of 0 25 AT 25 The following fields are repeated for the remaining active categories 30 34 I5 4 Active category number CAT 35 39 I5 4 Step number STEP 40 47 F8 2 2 Step calibration MEASURE 48 55 F8 2 6 Step calibration s standard error ERROR A User s Guide to BIGSTEPS Page 81 56 63 F8 2 2 Measure for an expected score of category 0 5 score points AT 0 5 64 71 F8 2 2 Measure for an expected score of category score points ATSTEP 72 719 F8 2 6 Measure at the Thurstone threshold THRESH Since the ISFILE has the same number of category entries for every item the repeated fields are filled out with 0 for any further categories up to the maximum categories for any item The format descriptors are In Integer field width n columns Fn m Numeric field n columns wide including n m 1 integral places a decimal point and m decimal places An Alphabetic field n columns wide When CSV Y commas separate the values with quotation marks around the Item name When CSV T the commas are replaced by tab characters Example You wish to write a file on disk called ITEMSTEP FIL containing the item statistics reported in Table 2 2 for use in constructing your own tables ISFILE ITEMSTEP FIL 6 4 8 XFILE analyzed response file default no file The size and significance of differential item functioning item bias can be obtained by examination of this f
33. 3 10 and 14 Page 86 A User s Guide to BIGSTEPS 8 INTERPRETING OUTPUT TABLES The statistics reported and labeled in the tables are intended to be self explanatory or more details see BTD and RSA Some of the terms are explained here Note The tables shown here are examples not the results of one an actual analysis Table 0 is the last Table printed see p 108 8 1 Table 1 1 Distribution map controlled by MRANGE ITEM PERSON MAXPAG These tables show the distribution of the persons and items The variable is laid out vertically with the most able persons and most difficult items at the top 1 18 33 1996 3 CATEGS v2 67 TABLE 1 1 LIKING FOR SCIENCE Wright amp Masters p 18 76 PUPILS 25 ACTS ANALYZED 74 PUPILS SF OUT Dec 25 ACTS 76 PUPILS 25 ACTS Number of persons in data file and items in NI specification ANALYZED 74 PUPILS 25 ACTS 3 CATEGSNumber of persons items and categories that with non extreme scores v2 67 BIGSTEPS version number MAP OF KIDS AND TAPS MEASURE MEASURE lt more gt KIDS Sto TAPS sSSSSsery Sess p os s gt lt rare gt 5 0 X 5 0 XXX items too hard for persons 4 0 4 0 XX x 3 0 3 0 x x 2 0 XXXX 2 0 x 1 0 XXXXX 1 0 x A User s Guide to BIGSTEPS Page 87 XXXXXXXXXXXX gap 1 0 1 0 XXX x 2 0 2 0 Xx X x Xx XX 3 0 XX X 3 0 lt leSs gt KIDS TAPS lt frequent gt
34. 76 ITEM1 column number of first response required no default 00 eee eeeeceeeeeeceeeeeeeeeeseeeneeesneenas 27 ITEM title for item names default ITEM ssssessssssssssressserossseressserossseressseressserossseressseressserossseressseressee 69 ITLEN maximum length of item name default 30 eee eeecccesnceceseeceeeeeeeecesneeeeaeeceaeeesaeeseseeeeeaeeees 29 IVALUEx recoding for x type items with IREFER default none eee eeseeeseeeeereeeeneeeeenees 33 KEYFRM2 location of KEYn default 0 before amp END eee ccccccccsssseceeesseeeceeseeeeeesseeeeessseeeeeseaaes 41 KEYn scoring key default none oe eee eseeeseceseeeseecseecssecsseceseceseesseecseessaecsaessaeesseesseeeeesneeesaeenas 33 KEYSCR reassign scoring keys default 123 ooo eee eeesseceseceseceseceeesseeseecseecsaecsseesseesseeseeeseeenaeenas 35 LCONV logit change at convergence default 01 logits eee eeesceeseecsecseceeceseeeeeeseesseessaessaeeeaeees 61 LINLEN length of printed lines in Tables 7 10 16 22 default 80 0 0 eee eeeeeseeseeeseeteeeteeeeeaeeeaeees 75 LOCAL locally restandardize fit statistics default N nO eceeescecesceceseceeeeeeseecesaeecsaeecesaeeeeeeesaees 73 LOWADJ correction for bottom rating scale categories default 0 25 eee eeeeeseceeeeseeeeeeseeeeeaeeeaeees 64 MAXPAG the maximum number of lines per page default 0 no limit oo eee eeeeeeeeeeeeeeeeeeneees 69 MISSING t
35. A User s Guide to BIGSTEPS Page 79 by tab characters Example You wish to write a file on disk called ITEM CAL containing the item statistics for use in updating your item bank with values separated by commas IFILE ITEM CAL CSV Y 6 4 4 PFILE person output file default no file PFILE filename produces an output file containing the information for each person This file contains 4 heading lines unless HLINES N followed by one line for each person containing Columns Start End Format Description 1 1 Al Blank or if no responses or deleted status 2 3 2 6 I5 1 The person sequence number ENTRY 7 14 F8 2 2 Person s measure rescaled by UMEAN USCALE UDECIM MEASURE 15 17 3 3 The person s status STATUS 1 Estimated measure 2 Anchored fixed measure 0 Extreme minimum estimated using EXTRSC 1 Extreme maximum estimated using EXTRSC 2 No responses available for measure 3 Deleted by user 18 23 16 4 The number of responses used in measuring COUNT 24 30 I7 5 The raw score used in measuring SCORE 31 37 F7 2 6 Measure s standard error rescaled by USCALE UDECIM ERROR 38 44 F7 2 7 Person mean square infit IN MSQ 45 51 F7 2 8 Person infit standardized ZSTD locally standardized ZEMP or log scaled LOG 52 58 F7 2 9 Person mean square outfit OUT MSQ 59 65 F7 2 10 Person outfit standardized ZSTD locally standardized ZEMP or log scaled LOG 66 72
36. A version of Table 15 sorted on item name column 13 is to be specified on the DOS command line or on the Extra Specifications line Commas are used as separators and as place holders TFILE 15 13 6 3 12 PSORT column within person name for alphabetical sort in Table 19 default 1 Table 19 lists persons alphabetically By default the whole person name is used To sort starting on a column after the first person name column specify PSORT Example 1 The person name is entered in the data file starting in column 20 It is a 6 digit student number followed by a blank and then gender identification in column 8 of the person name Sort by gender identification for Table 19 NAME1 20 NAMLEN 8 student number gender PSORT 8 alphabetical sort on gender TABLES 111111111111111111111111 amp END END NAMES XXXXXXXXXXXXXXXXXXX123456 M 0010101101001002010102110011 XXXXXXXXXXXXXXXXXXX229591 F 1102010020100100201002010021 T sort column Example 2 The person name contains several important classifiers Table 19 is needed for each one NAME1 14 Person name starts in column 14 ITEM1 24 Response start in column 24 TFILE 19 1 sort starts with column 1 of person name 19 8 sort starts with column 8 of person name 19 6 sort starts with column 6 of person name t entered as place holders see TFILE x A User s Guide to BIGSTEPS Page 77 amp END END NAMES XXXXXXXXXXXXX1234 M 12 0
37. BDCDCDCDADCBCCBDBDCABCBDACDABCAABCAACBBBACAADDAACA ACBCACBBDADCBDCBBBCDCCDACCBCADCACCAACDBCCDADDBACDA BADCDCBDBDCDCCBACCCBBAABDBCDBCCBAADBABBADBDDABDCAA DCDBCDCDBADBCCB 090111008402 10084 CDABCDADDDCDDDDCBDCCBCCDACDBBCACDBCCCBDDACBADCAACD ACBDCCDBDADCCBCDDBBDCABACCBDBDCBCCACCBACDCADABACDA BABBDCADBDDBCDADDDCDDBCABCBDCCCAACDBACBDDBDBCCAACB DBACDBCDBADDCBC 090111008502 10085 CDABADADABCADCDDBDADBBCBACDABCCABACCCDAAACBADAAACD ACBCDCBBDADCDDCADBCCCDBADDBBBDCACAABCBDDDCADABACDA BADABBADBBADCADACDABBAACACAABDCBACDBADBACCDBACBADA BCABCBCDBADDCCC or using continuation lines for the key CODES ABCD The raw responses are ABCD and BLANK A User s Guide to BIGSTEPS Page 15 sKEYFRM omit this not needed KEY1 CDABCDBDABCADCBDBCADBABDDCDABCBABDCACBADACBADBAACD CCBDACABDADCBDCABBCACDBAABCDADCDCADBCABCDCADABACDA BADCDBADCBADCDBACBADBCAADBCBBDCBACDBACBADCDADBACDB ABDACDCDBADBCAB amp END Page 16 A User s Guide to BIGSTEPS Example 7 A partial credit analysis A 30 item MCQ Arithmetic Test is to be analyzed in which credit is given for partial solutions to the questions The analytical rating scale model is the Andrich model but each item is conceptualized to have its own rating scale structure as in the Masters Partial Credit model This file is EXAMPLE7 CON amp INST TITLE A Partial Credit Analysis page heading NAME1 1 Person id starts in column 1 ITEM1 23 Item responses start in column 23 NI 30 There ar
38. DRESSING E LOWER BODY DRESSING F TOILETING G BLADDER H BOWEL I BED TRANSFER J TOILET TRANSFER K TUB SHOWER L WALK WHEELCHAIR M STAIRS END NAMES The admission data is in file EXAM12 LO 21101 5523133322121 Patient number in cols 1 5 ratings in 7 19 21170 4433443345454 22618 4433255542141 22693 3524233421111 Page 24 A User s Guide to BIGSTEPS The discharge data is in file EXAM12 HI 21101 5734366655453 Ratings generally higher than at admission 21170 6466677777676 22618 7667656666565 22693 7776677676677 The batch job stream for BIGSTEPS to perform the three analyses is in EXAM12 BAT REM COMPARISON OF ITEM CALIBRATIONS FOR HIGH AND LOW SAMPLES BIGSTEPS EXAM12 CON EXAM12 0UT DATA EXAM12 LO EXAM12 HI TITLE ADMIT amp DISCHARGE SFILE EXAM12 SF BIGSTEPS EXAM12 CON EXAM12LO OUT DATA EXAM12 LO TITLE ADMIT SAFILE EXAM12 SF IFILE EXAM12 IFL CSV Y BIGSTEPS EXAM12 CON EXAM12HI OUT DATA EXAM12 HI TITLE DISCHARGE SAFILE EXAM12 SF IFILE EXAM12 IFH CSV Y To run this enter at the DOS prompt C gt EXAM12 Enter The shared step calibration anchor file is EXAM12 SF 3 STEP DIFFICULTY FILE FOR ADMIT amp DISCHARGE May 23 13 56 1993 CATEGORY STEP DIFFICULTY 00 2 11 1 61 25 06 1 92 2 99 N Ou PWN PB 1 H The item calibrations measures for admission and discharge are written into IFILE files with comma separated values CSV Y so that they can easily be imported into a
39. F7 2 11 Person displacement rescaled by USCALE UDECIM DISPLACE 73 79 F7 2 12 Person by test score correlation point biserial PTBS or point measure CORR 80 80 1X 16 Blank 81 110 A30 17 Person name NAME The format descriptors are In Integer field width n columns Fn m Numeric field n columns wide including n m 1 integral places a decimal point and m decimal places An Alphabetic field n columns wide Nx n blank columns When CSV Y commas separate the values with quotation marks around the Person name When CSV5T the commas are replaced by tab characters Example You wish to write a file on disk called STUDENT MES containing the person statistics Page 80 A User s Guide to BIGSTEPS for import later into a student information database PFILE STUDENT MES 6 4 5 RFILE scored response file default no file useful for reformatting data from a family of test forms linked by a network of common items into a single common structure suitable for one step item banking If this parameter is specified in the control file with RFILE filename a file is output which contains a scored keyed copy of the input data This file can be used as input for later analyses Items and persons deleted by PDFILE or the like are replaced by blank rows or columns in the scored response file The file format is 1 Person id A30 2 Responses one per item A1 if largest scored response is less than or equal to 9 A
40. FORMAT unless there is no other way A test of 165 multiple choice items with multiple data lines per data record The scoring key is formatted in the same way as the data lines This file is EXAMPLE6 CON amp INST TITLE Demonstration of KEY1 record title FORMAT 1X 10A T23 50A T23 50A T23 50A T23 15A Use the first character is ignored then 10 characters in first record are person id then starting in column 23 50 columns in first 3 records and 15 responses in fourth record In the reformatted record NAME1 1 Person id starts in column 1 ITEM 1 11 Item responses start in column 11 of reformatted record NI 165 There are 165 items CODES ABCD The raw responses are ABCD and BLANK Put character strings in if blanks are to be included KEYFRM 1 lt 1 There is a KEY1 record after amp END which is formatted exactly like a data record specifying the correct responses amp END KEY 1 formatted like your data Key 1 Record CDABCDBDABCADCBDBCADBABDDCDABCBABDCACBADACBADBAACD after amp END CCBDACABDADCBDCABBCACDBAABCDADCDCADBCABCDCADABACDA in FORMAT format BADCDBADCBADCDBACBADBCAADBCBBDCBACDBACBADCDADBACDB before item names ABDACDCDBADBCAB 7 Al First item name 1 1 A2 4 1 A164 3 6 A165 END NAMES 090111000102 10001 BDABADCDACCDCCADBCBDBCDDACADDCACCCBCCADBDABADCAADD ABDDDCABDADCBDACDBCACADABCDCCDCBDBCCABBCDCADDCDCDA BDCCDBABCDCDDDCBADCACBDCBDBACBCBCADBABAADCDCBABAAC DCBCCACABCDDCBC 090111000202 10002
41. Item statistics entry order with option counts Item statistics alphabetical order with option counts Person distribution map Horizontal histogram of person distribution with abbreviated person ids Person statistics measure order Person statistics entry order Person statistics alphabetical order Measures for all scores on a test of all calibrated items with percentiles 20 3 for persons Category probability curves Category probabilities plotted against the difference between person and item measures then the expected score and cumulative probability and expected score ogives 22 Sorted observations Data sorted by person and item measures into a Guttman scalogram pattern Recommended default selection for TABLES Option distracter counts shown if DISTRT Y the default Page 120 A User s Guide to BIGSTEPS INDEX of CONTROL VARIABLES ASCII output only ASCII characters default Y yes ceseeeeeseceseesseecsseceecseesseeeseesseeesaeesseeeeeeeeesees 69 CATREF reference category for Table 2 default 0 item difficulty 0 0 eee eee eeseeeeeeeeeeeeeeeeaeeeaeees 74 CFILE name of category label file default no file oo eee eseeseceeeceseceseceseeeseeseesseeesaeseaeeeaeee 55 CODES valid data codes default 01 oo ccc cccsscccssssscceessseceesenseeeeseeeecessseeecesssseeeessseeeesseseeeeesesaeeeees 30 CSV comma separated values in output files default N NO cee eseeseeeseecesecesec
42. Page 20 A User s Guide to BIGSTEPS Il From Test B make a response RFILE file rearranging the items with FORMAT Responses unique to Test A are filled with 15 blank responses to dummy items This file is EXAM10B CON amp INST TITLE Analysis of Test B RFILE EXAMI0B RF The constructed response file for Test B NI 35 FORMAT 25A T44 3A T42 A T51 A T100 15A T41 A T43 A T47 4A T52 9A ITEM 1 26 CODES ABCD KEYFRM 1 amp END Key 1 Record BANK 1 TEST B4 BANK 5 TESTB 11 BANK 6 TEST A 2 BANK 20 TEST A 20 BANK 21 TEST B 1 BANK 35 TEST B 20 END NAMES Person 01 B Person 12 B Blanks are imported from an unused part of the data record to the right Items start in column 26 of reformatted record Beware of blanks meaning wrong Key in data record format CDABCDBDABCADCBDBCAD BDABDDCDBBCCCCDAACBC BADABBADCBADBDBBBBBB The RFILE file EXAM10B RF is Person 01 B Person 02 B Person 11 B Person 12 B 10111 010101001000100 00000 010000000001000 00010 001000000000100 00000 000101000101000 A User s Guide to BIGSTEPS Page 21 IM Analyze Test A s and Test B s RFILE s together This file is EXAMI0C CON amp INST TITLE Analysis of Tests A amp B already scored NI 35 ITEM1 31 Items start in column 31 of RFILE CODES 01 Blanks mean not in this test DATA EXAMI0A RF EXAM10B RF Combine data files or first at the DOS prompt C gt COPY EXAM10A RF E
43. and Mike Linacre are happy to answer questions to do with the operation of BIGSTEPS or the nature of Rasch analysis More prolonged consultations can also be arranged We strongly recommend that you join the Rasch Measurement Special Interest Group of the American Educational Research Association for which AERA membership is not required The SIG publication Rasch Measurement Transactions contains instructive articles on the fundamentals of Rasch analysis as well as the latest ideas in theory and practice Share your experiences with your colleagues at SIG meetings The advance of science is an activity in which we can all participate to our mutual benefit The SIG address is Rasch Measurement SIG www rasch org rmt Page 2 A User s Guide to BIGSTEPS 2 GENERAL DESCRIPTION 2 0 Installing BIGSTEPS BIGDOS EXE To install BIGSTEPS under MS DOS These create directory C BIGSTEPS and install in it BIGSTEPS Sample control and data DAT files are also installed to help you get started KCT DAT is the Knox Cube Test data BTD p 31 see Section 1 1 The results in BTD were obtained with more approximate algorithms and do not agree exactly with BIGSTEPS results SF DAT is the Liking For Science data RSA p 18 There are EXAMPLE files described starting on p 10 of this manual Under DOS At the DOS prompt C gt A BIGDOS To Run BIGSTEPS C gt CD BIGSTEPS C gt BIGSTEPS SF DAT SF OUT Enter 2 1 How to use BIGSTEPS i Install BIGS
44. categories When STKEEP Y missing categories are retained in the rating scale so maintaining the raw score ordering but they distort the step difficulty calibrations If these are to be used for anchoring later runs compare these calibrations with the calibrations obtained by an unanchored analysis of the new data This will assist you in determining what adjustments need to be made to the original calibrations in order to establish a set of anchor calibrations that maintain the same rating scale structure Example Keep the developmentally important rating scale categories whether they are observed or not STKEEP Y 4 3 3 Weighting items There are some circumstances in which certain items are to be given more influence in constructing the measurement than others For instance certain items may be considered critical to the demonstration of competence Step 1 Analyze the data without weighting Investigate misfit construct validity etc Step 2 Weight the items Method 1 STKEEP together with RESCORE and KEYn can be used to construct integer item weighting by converting 0 1 data into say 0 3 data This rescoring makes the item maps difficult to comprehend losing the construct definition Method 2 Enter the weighted items multiple times in the data using say the FORMAT specification This maintains the construct Example 1 A 6 item MCQ test entered as A B C D is to be scored Items 1 2 3 are weighte
45. category for CATREF Table 2 most probable responses scores maps the items vertically and the most probable responses expected scores and Thurstone thresholds horizontally By default the vertical ordering is item difficulty calibration If instead a particular category is to be used as the reference for sorting give its value as scored and recoded Special uses of CATREF are CATREF 3 for item entry order CATREF 2 for item measure order CATREF 1 for items measure order with GROUPS CATREF 0 for item measure order CATREF 1 99 for item measure order based on this category Example 1 You have 4 point partial credit items entered in your data as A B C D and then scored as 1 2 3 4 You wish to list them based on the challenge of category C rescored as 3 CODES ABCD original responses NEWSCORE 1234 rescored values RESCORE 2 rescore all CATREF 3 Table 2 reference category GROUPS 0 partial credit one item per group If for an item the category value 3 is eliminated from the analysis or is the bottom category the nearest higher category is used for that item Example 2 You have 6 3 category items in Group 1 and 8 4 category items in Group 2 You wish to list them in Table 2 2 by difficulty within group and then by difficulty overall CODES 1234 NI 14 GROUPS 11111122222222 TFILE 2 2000 1 amp 1 means CATREF 1 2 20000 last 0 means CATREF 0 x 6 3 2 CURVES probability curves for Ta
46. category number a blank and the step measure scale value in logits or your rescaled units at which to anchor the step corresponding to that category see Table 3 If you wish to force category 0 to stay in an analysis anchors its calibration at 0 or If items use different rating scales i e GROUPS 0 or items are assigned to different groups e g GROUPS 122113 then type the sequence number of any item belonging to the group a blank the category number a blank and the step measure scale value in logits if USCALE 1 otherwise your rescaled units at which to anchor the step up to that category for that group If you A User s Guide to BIGSTEPS Page 57 wish to force category 0 to stay in an analysis anchor its calibration at 0 This information may be entered directly in the control file using SAFILE Example 1 A rating scale common to all items of three categories 2 4 and 6 is to be anchored at pre set calibrations The calibration of the step from category 2 to category 4 is 1 5 and of the step to category 6 is 1 5 1 Create a file named say STANC FIL 2 Enter the lines 4 1 5 step to category 4 anchor at 1 5 logits 61 5 3 Specify in the control file GROUPS the default SAFILE STANC FIL step anchor file or enter directly in the control file SAFILE 4 1 5 61 5 x Example 2 A partial credit analysis GROUPS 0 has a different rating scale for each item Item
47. chars 10 2 chars XWIDE 2 all converted to 2 columns CODES a b c d AABBCCDD a becomes a NEWSCORE 1 2341234 response values RESCORE 2 rescore all items NAME1 1 lt name starts column 1 of reformatted record ITEM1 31 items start in column 31 NI 1I5 15 items all XWIDE 2 Example 8 Items are to rescored according to Type A and Type B Other items to keep original scoring CODES 1234 IREFER AAAAAAAABBBBBBBB lt amp 3 item types IVALUEA 1223 Recode Type A items IVALUEB 1123 Recode Type B items IVALUE 1234 Recode Type item Can be omitted 4 2 2 MISSING treatment of missing data default 255 ignore MISSING says what to do with characters that are not valid response codes e g blanks and data entry errors By default any characters not in CODES are treated as missing data and assigned a value of 255 which means ignore this response This is usually what you want when such responses mean not administered If they mean I don t know the answer you may wish to assign missing data a value of 0 meaning wrong Non numeric codes included in CODES when there are no KEYn or RESCORE variables or in NEWSCORE are always assigned a value of not administered 255 Example 1 Assign a code of 0 to any responses not in CODES MISSING 0 missing responses are scored 0 A User s Guide to BIGSTEPS Page 31 Example 2 In an attitude rating scale with
48. column within item name for alphabetical sort in Table 15 default 1 oo eeeeeeeseeeeeeees 76 ITEM1 column number of first response required no default 0 eee eeeeceseceseeeseeeseesseeseeeseaeeeaeee 27 ITEM title for item names default ITEM 0 cccccsccccessscceessneeeeessneeeeesseeeeesssaeeessesseeesssseeeesseeeeeees 69 ITLEN maximum length of item name default 30 ee ceescccesnecesseeceseeceeneeesseecesaeecsaeeceeaeeeeeeeeaaees 29 IVALUEXx recoding for x type items with IREFER default none eeeeeeeceeeseeeeseeeeneeeeenees 33 KEYFRM2 location of KEYn default 0 before SEND cee cccescccesesseceeseneeeeeseeeeesssteeeesseseeeeeees 41 KEYn scoring key default NON oe eeeeeeeseeescesseeeseeceseeeseesseesseecseecssecsaeeeseeseeseecseeseaeseaeeeaeees 33 KEYSCR reassign scoring keys default 123 oo eee eeeeesseceseceseeeseeeseecseeceseceseeeseesseeseesseeseaeseaeesaeees 35 LCONV logit change at convergence default 01 lOgits oo eee eee eseeeeeeceneceseeeseeeseecseecsseeeseeeseeeseeenees 61 LINLEN length of printed lines in Tables 7 10 16 22 default 80 oo eee eeseeseeceeceeeeeeeeseeeseeeeees 75 LOCAL locally restandardize fit statistics default N no ceeceescccesseceescecesneeeeeecesaeceeaeeceseeeeeaeeesaes 73 LOWADJ correction for bottom rating scale categories default 0 25 ee eeseeseecseceeceseeeseeeseeeeees 64 MAXPAG the maximum number of lines per page
49. control and data files 00 eee eesessseeesecseeseeseeceseeseeaeeesseesesaeeesseesesseeaseessesseeaeeseeens 5 2 6 1 Using a word processor or text CCItOL 0 eee sseeeeeeseeeesceeceecseeseeesseesesaceasasseeseeaseessesseeaseeseeeees 5 2 0 2 Whe data tile icici ceveSpdscesisheysigaseth entra atine aa eaae e eaaa ies Leite eased vans 6 20 3 Thecontrol filen ee E eae aAA AA heen daar ieee 7 2 6 4 Syntax rules for assigning values to control variables key words eeeeesessseeeeeeteeseeeeeeeees 7 2 7 How long will an analysis take oo tees esesesecsecseeseeessecseesceaceceseeseeaeeessessesseeesseeseeseeasessesseeasensseens 8 2 8 How big an analysis Can dO sisne enean ae oa se o eE a Rao eN EES E nE Bae 8 2 9 If BIGSTEPS does not Work rronin aa aeea eaaa asa ea aAa oaa aai aai i aeia 8 29 1 Not enough disk sp ter iiie ne is N e BS ee a eke 9 292 Nof enou A MEOS a a a a a ed 9 EXAMPLES OF CONTROL AND DATA FILES sesssseeeeesesseeesestssesrsesrstsestststeseststsreresesesrstesesesrsreeee 10 Example 1 Simple control file with data included 00 0 cece esssesseessecseeseeeseeeseeseeeceeseeaeeaseessesseeaeesseees 10 Example 2 Control and anchor files cic ainar ee eua eai aea ieoi ieie kuii 11 Example 3 Item recoding and item deletion s sesesseseeesesesesesesesestseststststsrsrrreresssesetstsrststststsrsteeseseesrseses 12 Example 4 Selective item recoding eeeessssesseessecseeseeeceeesecseeacsessecseeacsesseeseeseeassassesaeeaseesse
50. deviation of those measures The separation and reliability computations are repeated but including any elements with extreme measures Since the measures for extreme scores are imprecise these statistics are often lower than their non extreme equivalents Conventional computation of a reliability coefficient KR 20 Cronbach Alpha includes persons with extreme scores The PERSON SEP REL of this second analysis is the conventional reliability and is usually between the MODEL and REAL values closer to the MODEL 8 4 2 Summary of rating scale steps controlled by STEPT3 STKEEP A User s Guide to BIGSTEPS Page 93 a For dichotomies SUMMARY OF MEASURED STEPS CATEGORY STEP OBSERVED AVGE INFIT OUTFIT LABEL VALUE COUNT MEASURE MNSQ MNSQ 0 0 240 3 47 1 01 83 1 1 236 3 13 1 00 62 CATEGORY LABEL is the number of the category in your data set after scoring keying STEP VALUE is the number of the step which the category represents after the scale categories have been ordinally recounted from the lowest observed category OBSERVED COUNT is the count of occurrences of this category used in the estimation AVGE MEASURE is the average of the measures that were modelled to produce the responses observed in the category The average measure is expected to increase with category value Disordering is marked by 6699 INFIT MNSQ is the average of the INFIT mean squares associated with the respons
51. eighth at 2 7 1 Create a file named say PERSON ANC 2 Enter the line 3 1 5 into this file meaning person 3 is fixed at 1 5 logits 3 Enter the line 8 2 7 meaning person 8 is fixed at 2 7 logits 4 Specify in the control file PAFILE PERSON ANC or enter directly into the control file PAFILE 31 5 8 2 7 x 4 5 5 PANCHQ anchor persons interactively default N no If your system is interactive persons to be anchored can be entered interactively by setting PANCHQ Y between the amp INST and amp END lines If you specify this you will be asked if you want to anchor any persons If you respond yes it will ask if you want to read these anchored persons from a file if you answer yes it will ask for the file name and process that file in the same manner as if PAFILE had been specified If you answer no you will be asked to enter the sequence number of each person to be anchored one at a time along with the logit or rescaled by USCALE UMEAN5 calibration When you are finished enter a zero Example You are doing a number of analyses anchoring a few but different persons each analysis This time you want to anchor person 4 Page 54 A User s Guide to BIGSTEPS Enter on the DOS control line or in the control file PANCHQ Y You want to anchor person 4 BIGSTEPS asks you DO YOU WANT TO ANCHOR ANY PERSONS respond YES Enter DO YOU WISH TO READ THE ANCHORED PERS
52. end with DOS or ASCII Carriage Return and Line Feed codes Be particularly careful to instruct your Word Processor to allow more characters within each line than are present in the longest line in your control or data files Then your Word Processor will not break long data or control lines into two or more text lines with Soft Return codes These cause BIGSTEPS to malfunction Space for a large number of characters per line is obtained by specifying a very wide paper size and or a very small type size to your Word Processor When using Word Perfect to edit control or data files select the smallest available type size often 20 cpi or 5 pt Define and use a very wide 50 inch page style It does not matter whether your printer can actually print it Always save control and data files as DOS Text or ASCH files With WordStar use Non Document mode to avoid these difficulties b Output files when importing BIGSTEPS output into a document file the following options have proved useful Base Font 17 cpi or more or 8 point or smaller or 132 characters per line or more Left Justify Page numbering Margins top 1 bottom 0 5 left 1 right 0 2 6 2 The data file If your data file is small it is easiest merely to have it at the end of your control file If your data is extensive keep it in a separate data file Your data file is expected to contain a record for each person containing a person id f
53. from the extreme ZONE is the range of measures from an expected score from score point below to the category to A User s Guide to BIGSTEPS Page 95 score point above it Measures in this range on an item of 0 difficulty are expected to be observed on average with the category value THURSTONE THRESHOLD gives the location of median probabilities At these calibrations the probability of observing the categories below equals the probability of observing the categories equal or above The Thurstone threshold is the point on the variable at which the category interval begins CATEG RESIDUL when shown is the category residual the difference between the observed and expected counts of observations in the category Only shown if gt 1 0 Indicates lack of convergence step anchoring or large data set 8 5 Tables 4 1 5 1 and 8 1 9 1 Fit plots controlled by FRANGE LOCAL OUTFIT 4 3 2 1 0 1 2 3 4 5 t t t P 5 B 5 E A R 4 4 S o 3 C 3 N D I Qos Ss e A eE F G Q M Ubiri n 2 I E H N T JIT 1 F R J1s 11 1 1 1 I 0 K 1 1 z1 21 1 1 12 W 0 T 0 2 L1 Tx y 1 1 1 1N t22 Inp r u 1 1 vvh ji o 2 _ w eg f k 2 T b d D 3 a c 3 t
54. measures however will be rescaled by UMEAN and USCALE Example Your item bank calibrations are maintained in logits but you want to report person measures in CHIPS BTD p 201 UMEAN 50 USCALE 4 55 UANCHOR N 5 3 6 User friendly rescaling A User s Guide to BIGSTEPS Page 65 Transforming logits into other units more meaningful for particular applications is discussed in Chapter 8 of BTD Example 1 CHIPs are a useful transformation in which 1 logit 4 55 CHIPs In this scaling system standard errors tend to be about 1 CHIP in size The recommended control variable settings are USCALE 4 55 UIMEAN 50 UDECIM 1 MRANGE 50 The probability structure of this relationship is Probability of Success Difference between Person Ability and Item Difficulty in CHIPs re f Example 2 WITs are one tenth the size of CHIPs enabling the elimination of decimals from your output tables USCALE 45 5 UIMEAN 500 UDECIM 0 MRANGE 500 Example 3 You want the lowest reportable person measure to be 0 and the highest to be 100 Looking at Table 20 you see the extreme values are 4 53 and 5 72 You have not used USCALE and UMEAN USCALE wanted range current range USCALE 100 0 5 72 4 53 100 10 25 9 76 UMEAN wanted low current low USCALE 0 4 53 9 76 44 20 Required values are USCALE 9 76 UIMEAN 44 20 UDECIM 0 to show no decimal places in report Example 4
55. of all responses that are missing SCR is the scored value of the response means ignored for estimation purposes 00 is a code from the CODES variable above the count of such responses gives the percent of all non missing responses that the count represents SCR gives the score value of the code 8 10 Tables 6 10 Unexpected Responses 8 10 1 Tables 6 4 10 4 Most Misfitting Response Strings These tables display the unexpected responses in the most misfitting response strings in a Guttman scalogram format MOST MISFITTING RESPONSE STRINGS ACT OUTMNSQ PUPIL 14314 452371 17667422 315231 4 4 31725 1475789534189175348742556649769502293 fia Gh se ts he eee aE EEA 23 WATCH A RAT 5 68 Al1110 0 2 2 2 1112 2211122 20 WATCH BUGS 1 99 Bl 232 7 Onan ey ee Dacre de 11 21 9 LEARN WEED NAMES T29 Di cate ia Ore ees O aaa coe EEEE A use 1 4314745237181766742243152319464531725 11475 895341 91753487 255664 7 9 02293 The items or persons are ordered by descending mean square misfit Each column corresponds to a person The entry numbers are printed vertically The responses are ordered so that the highest expected responses are to the left high the lowest to the right low The category values of unexpected 6699 observations are shown Expected values with standardized residuals less than 2 are shown by Page 100 A User s Guide to BIGSTEPS Missing values if any are left blank 8 10 2 Tables 6 5 10 5 Unex
56. of item calibrations in fit order an item is omitted only if the absolute values of both standardized fit statistics are less than FITI both mean square statistics are closer to 1 than FITI 10 and the item point biserial correlation is positive For Table 11 the diagnosis of misfitting items all items with a standardized fit greater than FITI are reported Selection is based on the OUTFIT statistic unless you set OUTFIT N in which case the INFIT Statistic is used Example 1 You wish to focus on grossly noisy items in Tables 10 and 11 FITI 4 an extreme positive value Example 2 You wish to include all items in Tables 10 and 11 FITI 1000 a value more negative than any fit statistic 6 2 2 FITP person misfit criterion default 2 0 Specifies the minimum standardized fit value at which persons are selected for reporting as misfits For Table 6 person measures in fit order a person is omitted only if the absolute values of both standardized fit Statistics are less than FITP both mean square statistics are closer to 1 than FITP 10 and the person point biserial correlation is positive For Table 7 the diagnosis of misfitting persons persons with a standardized fit greater than FITP are reported Selection is based on the OUTFIT statistic unless you set OUTFIT N in which case the INFIT Statistic is used Example 1 You wish to examine wildly guessing persons in Tables 6 and 7 FITP 3 an extreme positive valu
57. of zoned responses cseeeeeseeseeeeeeeseeseeeeeeeeeeseeaeeesseeseeaeeaeeeseees 107 8 18 The title page cis ceeds vceeteg a a a A a A E aa a aa sb ae aTa ae Aa E 108 8 19 Table 0 1 Control variables report neiseina i aa ai i 108 8 20 Table 0 2 Convergence r po ceuni aa a i a a aae S 109 Appendix 1 Order of Elements in Control File eee eeeeseesseceseeseeseeeceesecsecaeeeesessesseeaeeessessesaeeaseseees 112 Appendix 2 Diagnosing Misfit eee E NA N E KE N EAEAN NEE Ee EANN NA 113 Appendix 3 Diagnosis of Error Codes IOSTATS eeeeseeeessesesesesesesesrsesesestststsesrsrsressseeesrsrsrstsesreteeeeees 114 Appendix 4 What 18 4 Logit oinarekin aianei eee EAK A N Ee EENE A EOE EA EERENS 118 Appendix 5 Rasch two facet measurement models in BIGSTEPS ssseseseeeesessssseesesesesesesrsrsrsrsrereeeeess 119 Appendix 6 Output Talesa tenn hp anit an a E R N E a a e AE 120 INDEX of CONTROL VARIABLES ASCII output only ASCII characters default Y yes cescccescccesncceeseeceseeceeaeeeeseecesaeecsaeecesaeeeeneeeeaees 69 CATREF reference category for Table 2 default 0 item difficulty eee eeeeeseesseeeneeeneeeneeeeeeeees 74 CFILE name of category label file default no file oo eee eeeeseeeseeeseesseeceeeceaeeeseeeseeseessaeeaeenas 55 CODES valid data codes default 01 oo ardiren aair E AREKE inaa ae iia eRe Erea eiA 30 CSV comma separated values in output files default N nO seeseeseessese
58. on one line of the GROUPS control variable Instructs where to find the GROUPS information GRPFRM N GROUPS is a control variable between amp INST and amp END the default GRPFRM Y GROUPS information follows just after amp END but before the item names see p 112 It is formatted exactly like a data record It is helpful to enter GROUPS for reference where the person name would go Example An attitude survey of 10 items with 3 rating scale definitions Items 1 through 3 on Scale 1 items 4 through 6 on Scale 2 and items 7 through 10 on Scale 3 The GROUPS information is formatted like a data record and entered after amp END and before the item names The responses are in columns 1 10 and the person id in column 11 onwards amp INST NAME1 11 start of person id ITEM 1 1 start of responses A User s Guide to BIGSTEPS Page 47 NI 10 lt number of items CODES 12345 valid responses GRPFRM Y GROUPS formatted like data amp END 1112223333 GROUPS GROUPS information Item name 1 item names Item name 10 END NAMES 2213243223 John Smith first data record 4 4 Deleting or anchoring items 4 4 1 IDFILE name of item deletion file default no file Deletion or selection of items from a test for an analysis but without removing the responses from your data file is easily accomplished by creating a file in which each line contains the sequence number or numbers of items to be deleted or sel
59. output tables default 1110011001001000100000 0 eee eseeeeeeteeeeeeeeeeees 68 6 1 3 FORMFD the form feed character default MS DOS standard cccceeccessceteeeteeeeeees 68 6 1 4 MAXPAG the maximum number of lines per page default 0 no limit eee 69 6 1 5 ITEME title for item names default ITEM josronicsnenrtasesudi nn o en E 69 6 1 6 PERSON title for person names default PERSON cccecessseseeseceececeeeeeceeceeeeaeeseeeeeeneeas 69 6 1 7 ASCH output only ASCII characters default Y yes sec ceeeseseeseeessecseesceeseesseesseaeeeseees 69 6 1 8 TFILE input file listing tables to be output default none ee eeeeeeeeeeeeeeeeeeeeeseeeeeeeeees 70 G2 MISHESEIECH ON sit 5ciiell ce csthats voetes sat A AA A AEA LE eddy tute TE E ETEA 71 6 2 1 FITI item misfit criterion default 2 0 cccccccccssscssscesceseceseceseceseceseceseceseceseceseceeeseseenseenes 72 6 2 2 FITP person misfit criterion default 2 0 eee eeeecceseesceseceeeeeeeseeseceeceeeesecseceeceeeeaeeseeeeeeaeeas 72 6 2 3 OUTFIT sort misfits on infit or outfit default Y Outfit 0 cc ecccsscesceeeceeeeeeeeeeeeeeeenes 72 6 2 4 NORMAI normal distribution for standardizing fit default N chi square 0 0 0 cee 73 6 2 5 LOCAL locally restandardize fit statistics default N nO eceeeeceeseeseeeeceeeeeeeseeteeeeeeeeens 73 6 2 6 PTBIS compute point biserial correlation coefficients default Y yes eeeeeeseeeeeeeees 73 6 3 Sp
60. response string A FORMAT statement is required if 1 each person s responses take up several lines in your data file 2 if the length of a single line in your data file is more than 3000 characters 3 the person id field or the item responses are not in one continuous string of characters 4 you want to rearrange the order of your items in your data record to pick out sub tests or to move a set of connected forms into one complete matrix 5 you only want to analyze the responses of every second or nth person FORMAT contains up to 512 characters of reformatting instructions contained within which follow special rules Instructions are nA read in n characters starting with the current column and then advance to the next column after them Processing starts from column 1 of the first line so that SA reads in 5 A User s Guide to BIGSTEPS Page 37 nX TLe TRc n gt characters and advances to the sixth column means skip over n columns E g 5X means bypass this column and the next 4 columns go to column c T20 means get the next character from column 20 T55 means tab to column 55 not tab passed 55 columns which is TR55 go c columns to the left TL20 means get the next character the column which is 20 columns to the left of the current position go c columns to the right TR20 means get the next character the column which is 20 columns to the right of the current position means go to colu
61. s data record file is 80 characters long and takes up one line in your data file The person id is in columns 61 80 The 56 item responses are in columns 5 60 Codes are A B C D No FORMAT is needed Data look like XXXXDCBDABCADCDBACDADABDADCDADDCCDADDCAABCADCCBBDADCACDBBADCZarathrustra Xerxes Without FORMAT XWIDE 1 response width the default ITEM1 5 start of item responses NI 56 lt number of items NAME I1 61 start of name NAMLEN 20 length of name CODES ABCD valid response codes With FORMAT Reformatted record will look like DCBDABCADCDBACDADABDADCDADDCCDADDCAABCADCCBBDADCACDBBADCZarathrustra xerxes Page 38 XWIDE 1 response width the default FORMAT 4X 56A 20A skip unused characters ITEM 1 1 start of item responses NI 56 lt number of items NAME1 57 start of name A User s Guide to BIGSTEPS NAMLEN 20 length of name CODES ABCD valid response codes Example 2 Each data record is one line of 80 characters The person id is in columns 61 80 The 28 item responses are in columns 5 60 each 2 characters wide Codes are A B C D No FORMAT is necessary Data look like XXXX CDBACBCAADDDDCDDCACDCBACCBA CZarathrustra Xerxes Without FORMAT XWIDE 2 response width ITEM1 5 start of item responses NI 28 number of items NAME1 61 start of name NAMLEN 20 length of name CODES ABCD valid response codes
62. scores Look at the location of the E s in the tails of the test ogive in Table 20 If they look too far away increase EXTRSC by 0 1 If they look too bunched up reduce EXTRSC by 0 1 The measure corresponding to an extreme perfect or zero score is not estimable but the measure corresponding to a score of 0 5 score points less than perfect or 0 5 score points more than zero is estimable and is often a useful measure to report Extreme score estimates are not included in the summary Statistics in Table 3 Rasch programs differ in the way they estimate measures for extreme scores Adjustment to the value of EXTRSC can enable a close match to be made to the results produced by other programs There is no correct answer to the question How large should EXTRSC be The most conservative value and that recommended by Joseph Berkson is 0 5 Some work by John Tukey indicates that 0 167 is a reasonable value The smaller you set EXTRSC the further away measures corresponding to extreme scores will be located from the other measures Treatment of Extreme Scores Tables Output files A User s Guide to BIGSTEPS Page 63 Placed at extremes of map 1 12 16 Positioned by estimated measure 13 17 22 Positioned by other criteria 14 15 18 19 IFILE ISFILE PFILE RFILE Omitted 2 3 4 5 6 7 8 9 SFILE XFILE 10 11 20 21 Example You wish to estimate conservative finite measures for extreme scores by subtr
63. set MUCON 0 LCONV 0 RCONV 0 2 You stop the iterative process a If you press Ctrl with F during PROX iterations PROX iteration will cease as soon extreme scores have been identified and point biserial correlations have been calculated UCON iterations then start b If you press Ctrl with F during UCON iterations UCON iteration will cease at the end of this iteration Fit statistics will then be calculated and output tables written to disk c If you press Ctrl with F during the output phase Output will cease at the end of the current output operation Acknowledgement of your Ctrl with F instruction is shown by the replacement of by in the horizontal bar drawn across you screen which indicates progress through the current phase of analysis 3 You cancel BIGSTEPS execution immediately Press Ctrl and Break keys together or Press Ctrl and C keys together No more analysis or output is performed 2 6 The BIGSTEPS control and data files 2 6 1 Using a word processor or text editor a Input files all lines in your control and data files follow DOS text conventions This means that files created with a Word Processor such as Word Perfect must be saved as DOS text or ASCIT files 1 Lines must not contain tabs or word processor codes 2 Lines cannot overflow onto the next line except for data records which are processed using the FORMAT control variable A User s Guide to BIGSTEPS Page 5 3 Lines must
64. small delete files but rather just enter the numbers at the terminal so specify PDELQU Y You want to delete persons 23 and 50 BIGSTEPS asks you DO YOU WANT TO DELETE ANY PERSONS respond YES Enter DO YOU WISH TO READ THE DELETED PERSONS FROM A FILE respond NO Enter INPUT PERSON TO DELETE 0 TO END respond 23 Enter the first person to be deleted INPUT PERSON TO DELETE 0 TO END 50 Enter INPUT PERSON TO DELETE 0 TO END 0 Enter to end deletion 4 5 3 PSELECT person selection criterion default all persons Persons to be selected may be specified by using the PSELECT instruction to match characters within the person name PSELECT uses the same format as MS DOS file selection Persons deleted by PDFILE or PDELQU are never selected by PSELECT For multiple selections specify only and then follow it by a list Control characters to match person name matches any character i matches any string of characters Other alphanumeric characters match only those characters Example 1 Select for analysis only persons with M in the 5th column of person name Person name starts in column 6 of the data record NAME1 6 Person name field starts in col 6 NAMLEN 8 Person name field is 8 characters long PSELECT M amp Column 5 of person name is sex END NAMES xxxxxBPL M J 01101000101001 selected xxxxxMEL F S 01001000111100 omitted 1234 selection column Example 2 Select for analysis all p
65. spreadsheet Here is the comparison Arthritis Patient Sample Good Stairs Looks like the same voriable Much harder at Acimission foo risky fo try ischarge FIM Item Colibrotions a a 2 1 D 1 2 3 Admission FIM Hem Coalibrotians Using shared anchored roting scale caolibrotions A User s Guide to BIGSTEPS Page 25 Example 13 Paired comparisons as the basis for measurement Paired comparisons can be modeled directly with the Facets computer program For BIGSTEPS a dummy facet of occasion must be introduced On each occasion in this example each column there is a winner 1 a loser 0 or a draw D recorded for the two players In column 1 of the response data in this example Browne 1 defeated Mariotti 0 In column 2 Browne D drew with Tatai D Specifying PAIRED YES adjusts the measures for the statistical bias introduced by this stratagem Each player receives a measure and fit statistics Occasion measures are the average of the two players participating Misfitting occasions are unexpected outcomes Point biserial correlations have little meaning Check the occasion summary statistics in Table 3 to verify that all occasions have the same raw score This common control file is EXAM13 TXT amp INST TITLE Chess Matches at the Venice Tournament 1971 PERSON PLAYER ITEM MATCH Example of paired comparison CODES 0D1 0 loss D draw 1 win NEWSCORE 012 NAME 1
66. the data do not conform to the basic Rasch model specification that randomness in the data be normally distributed A User s Guide to BIGSTEPS Page 111 Appendix 1 Order of Elements in Control File Condition for Occurrence optional with GRPFRMEN the default optional with MODFRMEN the default optional with RESFRMEN the default optional if KEYFRM omitted the default optional if KEYFRM omitted the default optional if KEYFRM omitted the default n 1 to 99 number of largest key required if GRPFRM Y required if MODFRM Y required if RESFRM Y required if KEYFRM 1 or more required if KEYFRM 2 or more and so on up to required if KEYFRM n required if INUMBEN the default required if INUMBEN the default required if DATA the default Page 112 A User s Guide to BIGSTEPS Appendix 2 Diagnosing misfit Diagnosing Misfit Classification INFIT OUTFIT Explanation Investigation Lack of convergence Final values in Table 0 large Loss of precision Many categories Large logit range Anchoring Displacements reported F Bad item Ambiguous or negative wording Hard Item Debatable or misleading options Only answered by top people At end of test Qualitatively different item Different process or content Incompatible anchor value Anchor value incorrectly applied Biased DIF item Stratify residuals by person group Curriculum interaction Are there alternative cu
67. 010101101001002010102110011 XXXXXXXXXXXXX2295 F 09 1102010020100100201002010021 T sort column T sort column T sort column Example 3 A version of Table 19 sorted on person name column 6 is to be specified on the DOS command line or on the Extra Specifications line Commas are used as separators and as place holders TFILE 19 6 6 3 13 PRCOMP principal components analysis of item residuals in Table 10 default N no Principal components analysis of item response residuals can help identify structure in the misfit patterns across items Specifying principal components analysis with PRCOMP automatically produces Table 10 PRCOMP S Analyze the standardized residuals observed expected model standard error Simulation studies indicate that PRCOMP S gives the most accurate reflection of secondary dimensions in the items PRCOMP R Analyze the raw score residuals observed expected for each observation PRCOMP L Analyze the logit residuals observed expected model variance PRCOMP O Analyze the observations themselves Example 1 Perform a Rasch analysis and then see if there is any meaningful other dimensions in the residuals PRCOMP S Standardized residuals Example 2 Analysis of the observations themselves is more familiar to statisticians PRCOMP O Observations 6 4 Special purpose output files These output files are useful if you want to continue analysis with other computer programs such a
68. 11 20 are all ratings on another Andrich scale Group 2 Items 3 20 have response codes A B C D E or a b c d e This file is EXAMPLE9 CON amp INST TITLE Grouping and Modeling ITEM 1 11 Item responses start in column 11 NI 20 There are 20 items RESCORE 2 All responses are to be rescored CODES YNABCDEabcde Response codes to all items NEWSCORE 101234512345 Rescored 0 through 5 MODELS RRSRRRRRRRRRRRRRRRRR_ The models in item order GROUPS 00011111112222222222 The groups in item order DATA EXAMPLE9 DAT Location of data file amp END RO Prompt 1 Item id s remind us of models and groups Models and groups are shown in item measure Tables 10 13 14 15 RO Prompt 2 SO Logic R1 Grammar 1 R1 Grammar 2 R1 Grammar 3 R1 Grammar 4 R1 Grammar 5 R1 Grammar 6 R1 Grammar 7 R2 Meaning 1 R2 Meaning 2 R2 Meaning 3 R2 Meaning 4 R2 Meaning 5 R2 Meaning 6 R2 Meaning 7 R2 Meaning 8 R2 Meaning 9 R2 Meaning 10 END NAMES The data is in file EXAMPLE9 DAT Richard M OObcDCDddcDddddcDccE Tracie F OOBCBABBCccbBbbBbBBBb James M OOccaBbabBAcbacbaBbb Joe M 10BdBBBBccBccbbccbcc A User s Guide to BIGSTEPS Page 19 Example 10 Combining tests with common items Test A in file EXAMI0A CON and TEST B in EXAMI10B CON are both 20 item tests They have 5 items in common but the distracters are not necessarily in the same order The responses must be
69. 2 3 logits 3 Specify in the control file ITAFILE ANC FIL or place directly in the control file TAFILE 31 5 42 3 x Example 2 The calibrations from one run are to be used to anchor subsequent runs The items have the same numbers in both runs This is convenient for generating tables not previously requested 1 Perform the calibration run say C gt BIGSTEPS SF DAT SOME OUT IFILE ANCHORS SF TABLES 111 2 Perform the anchored runs say C gt BIGSTEPS SF DAT MORE OUT IAFILE ANCHORS SF TABLES 0001111 C gt BIGSTEPS SF DAT CURVES OUT IAFILE ANCHORS SF CURVES 111 4 4 4 IANCHQ anchor items interactively default N no Items to be anchored can be entered interactively by setting IANCHQ Y If you specify this you are asked if you want to anchor any items If you respond yes it will ask if you want to read these anchored items from a file if you answer yes it will ask for the file name and process that file in the same manner as if IAFILE had been specified If you answer no you will be asked to enter the sequence number of each item to be anchored one at a time along with its logit or rescaled by USCALE UMEANS value When you are finished enter a zero Example You are doing a number of analyses anchoring a few but different items each analysis You don t want to create a lot of small anchor files but rather just enter the numbers at the terminal so specify IANCHQ Y You want to anchor i
70. 2 if largest scored response is more than 9 The width of the responses is not determined by XWIDE 6 4 6 SFILE step category output file default no file If SFILE filename is specified a file is output which contains the item and category information needed for anchoring steps It has 4 heading lines unless HLINES N and has the format 1 The item sequence number 16 CATEGORY 2 The category value 13 STEP 3 Step calibration F7 2 rescaled by USCALE DIFFICULTY If CSV Y these values are separated by commas When CSV T the commas are replaced by tab characters 6 4 7 ISFILE item step output file default no file ISFILE filename produces an output file containing the step category measure information for each item All measures are added to the corresponding item s calibration and rescaled by USCALE and UDECIM This file contains 4 heading lines unless HLINES N followed by one line for each item containing Columns Start End Format Description 1 1 Al Blank or if no responses or deleted status 2 3 2 6 I5 1 The item sequence number ENTRY 7 11 I5 2 The item s status ST 1 Estimated calibration 2 Anchored fixed calibration 0 Extreme minimum estimated using EXTRSC 1 Extreme maximum estimated using EXTRSC 2 No responses available for calibration 3 Deleted by user 12 5 I5 3 Number of active categories MAX 17 5 I5 4 Lowest active category number LOW 22
71. 20 x Example 2 You want only Tables 1 4 TABLES 1111 6 1 3 FORMFD the form feed character default MS DOS standard Do not change FORMFD unless you have problems printing the tables or importing them into some other program The form feed character indicates the start of a new page of print out The DOS standard is Ctrl L ASCII 12 which is what represented by Shift 6 The DOS standard is understood by most word processing software and PC printers as the instruction to skip to the top of a new page i e form feed The ASA FORTRAN form feed character is 1 Page 68 A User s Guide to BIGSTEPS Example 1 You want your EPSON LQ 500 printer to form feed automatically at each new page of output You have already set the printer to use compressed print at 15 cpi because output lines contain up to 132 characters FORMFD the default Example 2 Your main frame software understands a 1 in the first position of a line of print out to indicate the top of a new page FORMFD 1 6 1 4 MAXPAG the maximum number of lines per page default 0 no limit For no page breaks inside Tables leave MAXPAG 0 If you prefer a different number of lines per page of your output file to better match the requirements of your printer or word processor give that value see Using a Word Processor or Text Editor in Section 2 If you prefer to have no page breaks and all plots at their maximum size leave MAXPAG 0 Example You plan
72. 3 0 x X x XX XX xx X Page 88 A User s Guide to BIGSTEPS xx X 2 0 XXX X 2 0 xx X xx XX X xx XX xx x XXX XXX X X 1 0 XXXXXX X X 1 0 XXX X XXXXX XX X XX XXX XX X XXXXXXXXXXX XXX X XXXX X X 0 XXXXXXK X X X 0 x x XX X Xx Xx X X XX XXX X x X X 1 0 xX X X X 1 0 Xx XX X x x x xIx x 2 0 X X X 2 0 x x x 3 0 X X 3 0 x 4 0 X 4 0 lt less gt PUPILS ACTS BOTTOM ACTS CENTER ACTS TOP lt frequ gt Observe that the top pupil left column is well above the top category of the most difficult act item right most column but that all pupils are above the top category of the easiest item bottom X in right most column Above means with greater than 50 chance of exceeding 8 2 Tables 1 0 1 2 1 3 12 and 16 Distribution maps controlled by MRANGE MAXPAG NAMLMP In Table 12 the item names are shown located at their calibrations along with the person distribution Table 1 2 is printed if the item map can be squeezed into one page In Table 16 the person names are shown with an item distribution Table 1 3 is printed if the person map can be squeezed into one page If person and item maps can be squeezed into one page Table 1 0 is printed You can use NAMLMP to control the number of characters of each name reported QSMSQ summarize the distributions An M marker represents
73. 4 item test Items 1 and 2 are success growth development items in which a score at a higher level is only possible if success at all lower levels has been achieved Items 3 and 4 are failure mastery items at which successively lower levels of competence are tested only if success has not been attained at higher levels This file is EXAMPLE8 CON amp INST TITLE Success and Failure scales at top of page NAME1 8 Person id starts in column 8 ITEM 1 1 Item responses start in column 1 NI 4 There are 4 items CODES 0123456789 lt Numeric responses are 0 through 9 MODELS SSFF The four scale models in order as applied to the 4 items In practice choice of Success Failure or Rating scale models has little effect on person measures unless theoretical structure is very clear Success and Failure models can be difficult to explain GROUPS 0 lt 1 Each item has its own scale TABLES 111111111111111111111111 More than enough 1 s for every Table NORMAL Y Also reports gross data to model fit amp END Maze Passengers Blocks Egg race END NAMES 5553 Richard 5975 Tracie 5333 David 5777 Thomas Page 18 A User s Guide to BIGSTEPS Example 9 Grouping and modeling items Note Success Failure may not estimate correctly A 20 item test Items 1 2 are dichotomous items coded Y N Item 3 is a Success item Items 4 10 are all ratings on one Andrich scale Group 1 and items
74. 4 point rating scale GROUPS Example 2 An arithmetic test in which partial credit is given for an intermediate level of success on some items There is no reason to assert that the intermediate level is equivalent for all items O No success 1 Intermediate success or complete success on items with no intermediate level 2 Complete success on intermediate level items CODES 012 valid codes GROUPS 0 each item has own scale Example 3 An attitude survey consists of four questions on a 0 1 2 scale group 1 followed by three 0 1 items group 2 and ends with one 0 1 2 3 4 5 question grouped by itself 0 NI 8 amp number of items CODES 012345 valid codes for all items GROUPS 11112220 the item groups When XWIDE 2 use two columns for each GROUPS code Each GROUPS code must be one character a letter or number specified once in the two columns e g 1 or 1 mean 1 and 0 or 0 mean 0 Example 4 You wish most items on the Liking for Science Survey to share the same rating scale in Group A Items about birds 1 10 21 are to share a separate scale in Group B Items 5 cans and 18 picnic each has its own scale Group 0 NI 25 number of items XWIDE 2 CODES 000102 valid codes for all items GROUPS BAAAQOAAAABAAAAAAANDAABAAAA t item groups 4 3 5 MODFRM2 location of MODELS default N before SEND Only use this if you have too many items to put conv
75. 5 4 6 1 CFILE name of category label file default no file eee eeeeeeeeteeseeeeeeeetseeeeeeees 55 4 6 2 SDFILE name of item step category deletion file default no file eee eeeeeeeeeee 56 4 6 2 SDELQU delete item step categories interactively default N NO eeeeeeeeseeseeeeeeeees 57 4 6 3 SAFILE name of item step anchor file default no file 0 eee eee eseeseeeeeeeeeeeeeeeeeeeeeeees 57 4 6 4 SANCHQ anchor steps interactively default N NO ec ceeeeesseseeseceeeeeeeseeeeceeeeeeeaeceeeeeeeeees 59 ANALYSIS CONTROIG 2 so foes ite dein estat See I ee A Gan AS ee ats 61 Sle Convergence control aro shee pide stnett cb cin Manel akiadle a a e cuitied aide tinal 61 5 1 1 MPROX maximum number of PROX iterations default 10 ccc cecccsscstecetseeeeeeeeenseeees 6l 5 1 2 MUCON maximum number of UCON iterations default 0 no limit eee eeeeeeeees 6l 5 1 3 LCONV logit change at convergence default 01 logits sseseseseseessesesserersrsrsrsrsrsresrersesees 61 5 1 4 RCONV score residual at convergence default 0 5 oo cceeceesssseesecesceseeeeeceeeeeeeaeceeeeeeneens 6l 5 2 Es m te adjustments ss eeen are E aaa a a ren a aia aoa ass 62 5 2 1 REALSE inflate S E for misfit default N no misfit allowance ccceccececesecetseeeeeeeeees 62 5 2 2 STBIAS correct for UCON estimation statistical bias default N nO cccecesseeteeeeeees 62 5 2 3 TARGET estimate using inf
76. BIGSTEPS Page 109 3 37 0292 53 5 1 3 10 0091 4 26 0206 53 21 1 2 74 0079 5 20 0154 53 21 0 1 90 0056 6 15 0113 53 21 0 1 42 0042 7 11 0083 53 21 0 1 05 0030 4 Standardized Residuals N O 1 Mean 06 S D 1 05 Look for scores and residuals in last line to be close to 0 and standardized residuals to be close to mean 0 0 S D 1 0 The meanings of the columns are PROX normal approximation algorithm for quick initial estimates ITERATION number of times through your data to calculate estimates ACTIVE COUNT number of elements participating in the estimation process after elimination of deletions and perfect zero scores PERSONS person parameters ITEMS item parameters CATS scale categories shows 2 for dichotomies EXTREME 5 RANGE PERSONS The current estimate of the spread between the average measure of the top 5 persons and the average measure of the bottom 5 persons ITEMS The current estimate of the spread between the average measure of the top 5 items and the average measure of the bottom 5 items MAX LOGIT CHANGE MEASURES maximun logit change in any person or item estimate This i expected to decrease gradually until convergence i e less than LCONV STEPS maximum logit change in any step difficulty estimate for your information need not be as small as MEASURES UCON unconditional maximum likelihood algori
77. DO YOU WANT TO ANCHOR ANY STEPS respond YES Enter DO YOU WISH TO READ THE ANCHORED STEPS FROM A FILE respond NO Enter INPUT STEP TO ANCHOR 1 TO END respond 2 Enter the first step to be anchored INPUT VALUE AT WHICH TO ANCHOR STEP respond 0 Enter the first anchor value INPUT STEP TO ANCHOR 1 TO END 4 Enter INPUT VALUE AT WHICH TO ANCHOR STEP 1 5 Enter INPUT STEP TO ANCHOR 1 TO END 6 Enter INPUT VALUE AT WHICH TO ANCHOR STEP 1 5 Enter INPUT STEP TO ANCHOR 1 TO END 1 Enter to end anchoring A User s Guide to BIGSTEPS Page 59 Example 2 You wish to enter the step difficulty measures for several scales each comprising a group of items S ANCHQ Y BIGSTEPS asks you DO YOU WANT TO ANCHOR ANY STEPS YES Enter DO YOU WANT TO READ THE ANCHORED STEPS FROM A FILE NO Item 1 represents the first group of items sharing a common scale INPUT AN ITEM REPRESENTING A GROUP 0 TO END 1 INPUT STEP TO ANCHOR 1 TO END 0 bottom category INPUT VALUE AT WHICH TO ANCHOR STEP 0 NONE INPUT AN ITEM REPRESENTING A GROUP 0 TO END 1 INPUT STEP TO ANCHOR 1 TO END 1 INPUT VALUE AT WHICH TO ANCHOR STEP 0 5 INPUT AN ITEM REPRESENTING A GROUP 0 TO END 1 INPUT STEP TO ANCHOR 1 TO END 2 INPUT VALUE AT WHICH TO ANCHOR STEP 0 5 Item 8 represents the second group of items sharing a common scale INPUT AN ITEM REPRESENTING A GROUP 0 TO END 8 INPUT STEP TO ANCHOR 1 TO END 0 bot
78. E RESPONSE MODE BETWEEN 0 AND 1 IS 0 ETC NUM ITEM 5 4 3 2 1 0 1 2 3 4 5 5 FIND BOTTLES 0 1 2 2 20 WATCH BUGS 0 1 2 2 8 LOOK IN SIDE 0 1 2 2 7 WATCH ANIMAL 0 1 2 2 17 WATCH WHAT A 0 1 2 2 deliberate space gt 21 WATCH BIRD M 0 1 2 2 10 LISTEN TO BI 0 1 2 2 12 GO TO MUSEUM 0 1 2 2 18 GO ON PICNIC 0 1 2 2 5 4 3 2 1 0 1 2 3 4 5 1 PERSON 1 2 1121 174135336222232221 11 1 1 14 Q 5 M S Q In the Expected Score Table the default or selected with CURVES 010 answers the question what is the average rating that we expect to observer for persons of a particular measure This rating information is expressed in terms of expected scores with at the half score points Extreme scores are located at expected scores 25 score points away from the extremes EXPECTED SCORE MEAN INDICATES HALF SCORE POINT 5 4 3 2 1 0 1 2 3 4 5 4 4 5 FIND BOTTLES 0 1 2 23 WATCH A RAT 0 1 2 9 LEARN WEED N 0 1 2 21 WATCH BIRD M 0 1 2 11 FIND WHERE A 0 1 2 19 GO TO zoo 0 p 1 2 18 GO ON PICNIC 0 1 2 4 5 4 3 2 1 0 1 2 3 4 5 The Thurstone Threshold Table selected with CURVES 001 answers the questi
79. ELQU N PSEL PAFILE PANCHQ N Step Cat Delete Anchor SDFILE SDELQU N SAFILE SANCHQ N HLINES Y IFILE SF IF ISFILE PFILE SF PF RFILE SF RF SFILE SF SF XFILE SF XF Data Reformat UMEAN 000 RCONV 500 FORMAT USCALE 1 000 TARGET N GRPFRM N UDECIM 2 2 2 22 KEYFRM 0 UANCH Y Scale Structure MODFRM N MRANGE 000 GROUPS RESFRM N MODELS R SPFILE Adjustment STKEEP N 2 2 22 This is used to check that your specifications were understood correctly 8 20 Table 0 2 Convergence report controlled by LCONV RCONV MPROX MUCON CUTLO CUTHI CONVERGENCE TABLE 4 PROX ACTIVE COUNT EXTREME 5 RANGE MAX LOGIT CHANGE ITERATION PERSONS ITEMS CATS PERSONS ITEMS MEASURES STEPS 4 5 1 76 25 3 3 78 3 20 3 8918 0740 2 74 25 3 4 53 3 67 7628 6167 3 74 25 3 4 73 3 85 2143 0991 4 74 25 3 4 82 3 90 0846 0326 4 UCON MAX SCORE MAX LOGIT LEAST CONVERGED CATEGORY STEP ITERATION RESIDUAL CHANGE PERSON ITEM CAT RESIDUAL CHANGE 1 3 01 4155 60 24 2 27 64 0184 2 50 0258 53 24 1 6 88 0198 A User s Guide to
80. ES LISTING 4 4 Tables selected by the TABLES specification are flagged with next to the Table number 8 19 Table 0 1 Control variables report On the second page are listed the settings of all the control variables for this analysis Those not specified in your control file have been set to their default values TITLE CONTROL FILE sf dat OUTPUT FILE sf out DATE Nov 24 CONTROL VARIABLES Input Data Format DATA NAME1 51 NAMLEN 30 ITEM1 1 Page 108 LIKING FOR SCIENCE Wright amp Masters p 18 9 18 1996 EXTRSC 300 0 HIADJ 250 Item Delete Anchor LOWADJ 250 IDFILE REALSE N IDELQU N STBIAS Y IAFILE A User s Guide to BIGSTEPS ITLEN 30 NI 25 XWIDE 2 INUMB N Data Scoring CODES 000102 MISSING 255 RESCOR NEWSCORE KEY1 KEYSCR CUTHI 000 CUTLO 000 Output Tables TITLE LIKING FOR SCIEN TABLES 1111111111111111 TFILE FORMFD A MAXPAG 0 ITEM ACT PERSON PUPIL ASCII Y User Scaling Misfit Selection FITI 2 000 FITP 2 000 OUTFIT Y LOCAL N NORMAL N PTBIS Y Special Table Control CATREF 0 CURVES 111 DISTRT Y FRANGE 000 LINLEN 80 NAMLMP 20 STEPT3 Y T1I 0 T1P 0 ISORT 1 PSORT 1 Convergence Cont rol MPROX 10 MUCON 0 LCONV 010 TANCHQ N Person Delete Anchor PDFILE PD
81. G s e g MODELS RRSF Items are assigned to the model for which the serial location in the MODELS string matches the item sequence number The item grouping default becomes each item with its own scale GROUPS 0 XWIDE 2 requires two columns per MODELS code e g S or S mean S Example 1 All items are to be modelled with the Success model MODELS S amp the Success model Example 2 A competency test consists of 3 success items followed by 2 failure items and then 10 dichotomies The dichotomies are to be reported as one group NI 1I5 fifteen items MODELS SSSFFRRRRRRRRRR matching models GROUPS 000001111111111 lt dichotomies grouped 4 3 2 STKEEP keep non observed steps categories default N no Unobserved categories are normally dropped from rating scales and the remaining category steps recounted A User s Guide to BIGSTEPS Page 43 during estimation For intermediate categories only recounting can be prevented and unobserved categories retained in the analysis This is useful when the unobserved categories are important steps in the rating scale logic or are usually observed even though they happen to have been unused this time Steps for which anchor calibrations are supplied are always maintained wherever computationally possible even when there are no observations in the current data set STKEEP N _ Eliminate unused categories and close up the observed categories STKEEP Y Retain unused non extreme
82. I 50 ITEM1 63 IDFILE ITEM DEL TABLES 1110111 amp END or specify in the control file NI 50 ITEM1 63 IDFILE 5 10 17 14 x TABLES 1110111 amp END Example 2 The analyst wants to delete the most misfitting items reported in Table 10 1 Set up a standard control file 2 Specify IDFILE x 3 Copy the target portion of Table 10 4 Paste it between the 5 Delete characters before the entry numbers 6 Type after the entry numbers to make further numbers into comments amp INST TITLE Example of item deletion list from Table 10 IDFILE Delete the border character before the entry number ENTRY RAW INFIT OUTFIT 3 NUM SCORE COUNT MEASURE ERROR MNSQ ZSTD MNSQ ZSTD PTBIS ACTS Bog 2 4 00 1 03 1 48 1 8 1 50 1 8 A 83 FIND BOTTLES AND CANS 8 2 4 00 1 03 1 40 1 61 43 1 6 B 71 LOOK IN SIDEWALK CRACKS 4 3 4 00 6211 33 711 49 9 C 21 WATCH GRASS CHANGE 9 4 4 00 74 1 51 811 57 9 D 59 LEARN WEED NAMES 20 1 4 00 1 0311 12 511 14 6 E 05 WATCH BUGS A User s Guide to BIGSTEPS coo oO Om Page 49 24 6 4 30 1 03 1 15 611 13 5 F 15 FIND OUT WHAT FLOWERS LIVE ON 0 Enter the to make other numbers into comments x Example 3 The analyst want to delete item 4 and items 18 to 23 on the DOS control or Extra Specifications line Extra specifications IDFILE 4 18 23 Enter or C gt BIGSTEPS CONTROL FIL OUTPUT FIL IDFILE 4 18 23 4 4 2 IDELQU del
83. IT SCORE COUNT MEASURE ERROR MNSQ ZSTD MNSQ ZSTD 4 5 5 5 5 5 5 5 5 5 5 5 5 5 5 5 5 5 5 5 MEAN 26 4 16 8 56 99 5 74 1 01 2 82 3 S D 11 9 5 7 23 67 1 33 65 1 4 78 1 2 4 5 5 5 5 5 5 5 5 5 5 5 5 5 5 5 5 5 5 5 5 5 5 REAL RMSE 6 47 ADI SD 22 77 SEPARATION 3 52 KID RELIABILITY 93 MODEL RMSE 5 90 ADJ SD 22 93 SEPARATION 3 89 KID RELIABILITY 94 S E OF KID MEAN 2 77 WITH 1 EXTREME KIDS 75 KIDS MEAN 60 13 S D 24 37 REAL RMSE 6 80 ADJ SD 24 47 SEPARATION 3 60 KID RELIABILITY 93 MODEL RMSE 6 26 ADJ SD 24 61 SEPARATION 3 93 KID RELIABILITY 94 4 5 5 5 5 5 5 5 5 5 5 5 5 5 5 5 5 5 5 5 5 5 5 MAXIMUM EXTREME SCORE 1 KIDS LACKING RESPONSES 1 KIDS VALID RESPONSES 67 1 CUTLO 1 0 CUTHI 0 SUMMARY OF 25 MEASURED NON EXTREME TAPS fa ee RAW MODEL INFIT OUTFIT SCORE COUNT MEASURE ERROR MNSQ ZSTD MNSQ ZSTD 4 5 5 ee MEAN 78 2 49 7 50 00 3 48 1 06 0 89 1 S D 43 0 22 5 27 56 1 04 36 1 3 43 4 5 5 ee REAL RMSE 4 03 ADJ SD 27 27 SEPARATION 6 76 TAP RELIABILITY 98 MODEL RMSE 3 63 ADJ SD 27 32 SEPARATION 7 53 TAP RELIABILITY 98 S E OF TAP MEAN 5 63 For valid observations used in the estimation SCORE is the raw score number of correct responses COUNT is the number of responses made MEASURE is the estimated measure for persons or calibration for items ERROR is the standard error
84. M 1 1 item 1 in column 1 of reformatted record NI 100 for 100 items NAME1 101 person id starts in column 101 NAMLEN 10 person id starts in 10 columns wide 4 1 3 NI number of items required no default The total number of items to be read in including those to be deleted by IDFILE or IDELQU NIS is limited to about 3000 for one column responses or about 1500 for two column responses in the standard program NI is usually the length of your test or the total number of items in all test forms to be combined into one analysis A User s Guide to BIGSTEPS Page 27 Example If there are 230 items in your test enter NI 230 230 items 4 1 4 XWIDE columns per response default 1 The number of columns taken up by each response in your data file 1 or 2 If possible enter your data one column per response If there are two columns per response make XWIDE 2 If your data includes responses entered in both 1 and 2 character width formats use FORMAT to convert all to XWIDE 2 format When XWIDE 2 these control variables require two columns per item or per response code CODES KEYn KEYSCR NEWSCORE IVALUE Either 1 or 2 columns can be used for RESCORE GROUPS RESCORE and IREFER Example 1 The responses are scanned into adjacent columns in the data records XWIDE 1 Observations 1 column wide Example 2 Each response is a rating on a scale from 1 to 10 and so requires two columns in the date record XWIDE 2 l
85. Math Co Processor does not work correctly Disable your Math Co processor by keying at the DOS prompt C gt SET NO87 DISABLE Enter Page 8 A User s Guide to BIGSTEPS and run BIGSTEPS again b There may not be enough disk space for work files see Not enough disk space c There may not be sufficient RAM memory to execute See Not enough memory 2 9 1 Not enough disk space Files with names BIGSTEPS and WST TMP and files in the TEMP directory are work files These can be deleted You need about the twice as much work file space on your disk as the size of your data file The work files are placed temporarily in the current directory reported on your screen when BIGSTEPS starts Delete unwanted files to give yourself more disk space or log onto a different disk drive with more available space before executing BIGSTEPS Type C gt CHKDSK F Enter to verify that you have enough available space on your current disk drive The program control file output file and work files can be dispersed onto four different drives D gt C BIN BIGSTEPS A KCT DAT B KCT OUT Enter The work files will be placed on drive D 2 9 2 Not enough memory BIGSTEPS does not use extended or expanded memory but makes use of all free conventional RAM memory To free up more memory remove all DOS Shells Device Drivers and Terminate and Stay Resident TSR programs This can be done by editing your AUTOEXEC BAT and CONFIG S
86. N in each item name line will be used but you may enter longer names in the control file for your own record keeping 4 The item names must be listed in exactly the same order as their responses appear in your data records 5 There should be the same number of item names as there are items specified by NI If there are too many or too few names a message will warn you and sequence numbers will be used for the names Page 28 A User s Guide to BIGSTEPS of any unnamed items You can still proceed with the analysis 6 Type END NAMES starting in column 1 of the line after the last item name Example An analysis of 4 items for which you supply identifying labels amp INST these lines can start at any column NI 4 four items ITEM1 10 responses start in column 10 INUMB N item names supplied the default amp END My first item name must start at column 1 My second item label My third item identifier My fourth and last item name END NAMES must start at column 1 in capital letters Person A 1100 data records Person Z 1001 4 1 6 ITLEN maximum length of item name default 30 ITLEN specifies the maximum number of columns in the control file that are to be used as item names Example 1 Example 2 You only wish to use the first five columns of item information to identify the items on the output amp INST NI 4 ITLEN 5 amp END AX123 This part is not shown on the output BY246 Trial item A
87. ND NAMES End of this file The anchoring information is contained in file EXAMPLE2 IAF and contains the following lines starting in column 1 2400 item 2 anchored at 400 units if logits are rescaled then anchor values must also be rescaled 4450 item 4 anchored at 450 units 6550 item 6 anchored at 550 units 8600 item 8 anchored at 600 units Item calibration files IFILE from prior runs can be used as item anchor files IAFILE of later runs Your data is in the separate file EXAMPLE2 DAT with person id starting in column 1 and item responses starting in column 11 Richard M 111111100000000000 Tracie F 111111111100000000 El si e F 111111111101010000 Helen F 111000000000000000 End of this file A User s Guide to BIGSTEPS Page 11 Example 3 Item recoding and item deletion The test has 25 items specified in EXAMPLE3 CON The item response string starts in column 12 Person id s start in column 1 the default value Original item codes are 0 1 2 and X All items are to be recoded and the original to new code assignments will be 090 192 21 and X93 since the responses are on a rating scale Items 5 8 and 20 through 25 are to be deleted from the analysis and are specified in the control The misfit criterion for person or item behavior is 3 0 Tables 1 2 3 4 5 6 7 8 9 11 13 15 17 19 20 and 21 are to appear in your output file EXAMPLE3 OUT Sequence numbers are used as ite
88. ONS FROM A FILE respond NO Enter INPUT PERSON TO ANCHOR 0 TO END respond 4 Enter the first person to be anchored INPUT VALUE AT WHICH TO ANCHOR PERSON respond 1 45 Enter the first anchor value INPUT PERSON TO ANCHOR 0 TO END 0 Enter to end anchoring 4 6 Categories and steps labeling deleting and anchoring 4 6 1 CFILE name of category label file default no file Rating scale output is easier to understand when the categories are shown with their substantive meanings These meanings can be specified using CFILE and a file name or CFILE and placing the labels in the control file Each category number is listed one per line followed by its descriptive label If the observations have been rescored NEWSCORE or keyed KEYn then use the final category value in the CFILE specification When there are different category labels for different GROUPS of items specify an example item from the group followed immediately by and the category number Blanks or commas can be used a separators between category numbers and labels Example 1 Identify the three LFS categories 0 Dislike 1 Don t know 2 Like CODES 012 CFILE 0 Dislike 1 Don t know 2 Like x The labels are shown in Table 3 as CATEGORY OBSERVED AVGE INFIT OUTFIT STEP LABEL COUNT MEASURE MNSQ MNSQ MEASURE 0 378 87 1 08 1 19 NONE Dislike 1 620 13 85 69 85 Don t know 2 852 2 23 1 00 1 46 85
89. Output TUES ensenis aat a e E adsl iie a a aaa 78 6 4 1 CSV comma separated values in output files default N 10 eeeeseeseeeeeeeeeeeeeeeeeeees 78 6 4 2 HLINES heading lines in output files default Y yes tcc eeeeeseesceeeeeeneeseeeseeeseeseeseeessees 79 6 4 3 IFILE item output file default no file eee eeseeseeseceececeeseesececeeeesecseceeeneeaeeseseeseneeas 79 6 4 4 PFILE person output file default no file ee eeseesseseeeeeeeseeseeeceeseeseeesesseeseeaeeeseees 80 6 4 5 RFILE scored response file default no file se ssseesssessessssessssssessesesessessssrseseesesesesess 81 6 4 6 SFILE step category output file default no file ee ee eeeeseeeeeeseeeeseeeseessesseeaeeeseees 81 6 4 7 ISFILE item step output file default no file eee eeeeeeeteeseeseeeseecsetseeeseesseeaeeaeeeseees 81 6 4 8 XFILE analyzed response file default no file ee eeeeeeereeseeeseeceeaeeeeeesseeseeaeeeseees 82 6 4 9 GRFILE probability curve coordinate output file default no file eee eeseeeeeeeees 83 6 4 10 Automating file SeleCtion cece eessssssseeseeseeseeeseecseeseeecseeseeseeassesseeseeaeassesseeaeeasesseeaseaseesaees 83 6 5 Supplemental control files ee eeeeeseeseseeseeeeseeseeceseeseesceecseesecseeseeessesseeseeassassesseeaseasessesaseaseseeees 83 6 5 1 SPFILE supplementary control file default no file oo eee eeeeeeeseeseeeeeeeteeseeeeeeee
90. STD OUTFIT MISFIT OVER 2 0 23 WATCH A RAT 2 00 5 8 A 8 1 RESPONSE 1 02111 22020 0101101000 01100 Z RESIDUAL x 2 3 3 2 RESPONSE 26 12021 m0011 10100310000 00021 Z RESIDUAL 3 62 2 2 4 This letter on fit plots 5 FIND BOTTLES AND CANS 2 21 5 2 B 6 5 RESPONSE 1 12001 20010 02001 10000 01101 Z RESIDUAL x 2 4 6 2 8 13 Table 20 Complete score to measure table on test of all items A measure and standard error is estimated for every possible score on a test composed of all non extreme items included in the analysis The measures corresponding to extreme scores all items right or all items wrong are marked by E and estimated using the EXTRSC criterion A graph of the score to measure conversion is also reported indicates the conversion and the one S E confidence interval Since the S and F models specify that not all item levels are encountered measures complete tests are only approximated here In the Table of Measures on Complete Test SCORE raw score on a complete test containing all calibrated items MEASURE measure corresponding to score S E standard error of the measure TABLE OF MEASURES ON COMPLETE TEST 4 4 4 4 SCORE MEASURE S E SCORE MEASURE S E SCORE MEASURE S E SCORE MEASURE S E 4 4 4 4
91. T SF OUT Enter 2 4 1 Reading extra control variables from the DOS prompt line BIGSTEPS expects to find the control variables in your control file You may however specify one or more control variables on the DOS prompt line These variables will be processed after all the variables in the control file and will supersede any conflicting instructions there This is useful for making temporary changes to the control variables These extra control variables must not contain any blanks Example 1 You want to verify that your data is correctly formatted so you only want to do one UCON iteration this time C gt BIGSTEPS SF DAT SF OUT MUCON 1 Enter where MUCON 1I specifies only one UCON iteration is to be performed Page 4 A User s Guide to BIGSTEPS C gt BIGSTEPS SF DAT SF OUT MUCON 1 Enter is invalid since there are blanks included in MUCON 1 Example 2 You want to produce an extra copy of the fit plot in Table 4 with specially chosen ranges on the axes C gt BIGSTEPS SF DAT SF OUT TABLES 0001 MRANGE 3 FRANGE 4 Enter 2 5 Stopping BIGSTEPS The BIGSTEPS program ceases execution when 1 The program stops itself The estimation procedure has reached an acceptable level of convergence and all output has been produced This happen when a The estimates are within the convergence criteria LCONV and RCONV b The maximum number of iterations has been reached MPROX and then MUCON To instruct BIGSTEPS to run indefinitely
92. TEPS ii Make sure it works use one of the example data sets see the following instructions ili Choose an example that is similar to your data see 3 EXAMPLES p 10 iv Choose a word processor e g WordPerfect or text editor e g EDIT or WordPad that can write or save DOS TEXT or ASCII files v Open the example file with your word processor or text editor vi Edit the example file to match your data This is now your BIGSTEPS control file Don t worry if your specifications are not exactly right BIGSTEPS will tell you of anything it doesn t understand vii Key in your data at the end of the example file or reference your data file with a DATA specification viii Write or save your edited control as a DOS TEXT or ASCII file with the name of your choice e g MYFILE CON ix If your data is in a separate file check to make sure it has the format you specified and is in DOS TEXT or ASCII characters i e it is not in SPSS or DBASE format When writing a file from SPSS the syntax is FORMATS ITEM ITEM2 ITEM3 F1 i e FORMATS varlist format varlist The procedure is FORMATS and then the variable list Enclosed in parentheses is the format type F signifies numeric while 1 signifies the width F2 would signify a numeric with a width of 2 A User s Guide to BIGSTEPS Page 3 columns for XWIDE 2 See pages 216 and 217 of the SPSS Reference Guide 1990 x Run BIGSTEPS Specify MYFILE CON as the con
93. TEPS Example Some low achievers have guessed wildly on a MCQ test You want to reduce the effect of their lucky guesses on their measures and on item calibrations TARGET Y How Targeting works a for each observation calculate probability of each category 0 1 for dichotomies calculate expected score probability of 1 for dichotomy calculate variance information probability of 1 probability of 0 for dichotomies so maximum value is 0 25 when ability difficulty b for targeting weighted observation variance observation weighted expected score variance expected score c sum these across persons and items and steps d required targeted estimates are obtained when for each person item step sum weighted observations sum weighted expected scores e for calculation of fit statistics and displacement weights of 1 are used but with the targeted parameter estimates Displacement size and excessive misfit indicate how much off target aberrant behavior exists in the data For targeting there are many patterns of responses that can cause infinite measures e g all items correct except for the easiest one The convergence criteria limit how extreme the reported measures will be 5 2 4 EXTRSC extreme score correction for extreme measures default 0 5 This is the fractional score point value to subtract from perfect scores and to add to zero scores in order to estimate finite values for extreme
94. XAM10B RF EXAMI0AB DAT Enter then in EXAM10C CON z DATA EXAM10AB DAT PFILE EXAM10C PF Person measures for combined tests IFILE EXAM10C IF Item calibrations for combined tests tfile List of desired tables 3 Table 3 for summary statistics 10 Table 10 for item structure x PRCOMP S Principal components analysis of items amp END BANK 1 TESTA3B4 BANK 35 TEST B 20 END NAMES Shortening FORMAT statements If the required FORMAT statement exceeds 512 characters consider using this technique A Relocate an entire item response string but use an IDFILE to delete the duplicate items i e replace them by blanks E g for Test B instead of FORMAT 25A T44 3A T42 A T51 A T100 15A T41 A T43 A T47 4A T52 9A NI 35 Put Test 2 as items 21 40 in columns 51 through 70 FORMAT 25A T44 3A T42 A T51 A T100 15A T41 20A NI 40 Blank out delete the 5 duplicated items with an IDFILE containing 24 26 22 31 Page 22 A User s Guide to BIGSTEPS Example 11 Item responses two characters wide The Liking for Science data see RSA is in file EXAM11 CON Each observation is on a rating scale where 0 means dislike 1 means don t care don t know and 2 means like The data has been recorded in two columns as 00 01 and 02 This file is EXAM11 CON amp INST TITLE LIKING FOR SCIENCE Wright amp Masters p 18 XWIDE 2 lt Responses are 2 columns wide CODES 000102 Code
95. YS files and then rebooting your computer A User s Guide to BIGSTEPS Page 9 3 EXAMPLES OF CONTROL AND DATA FILES Rather than attempting to construct a control file from scratch it is usually easier to find one of these examples that is similar to your problem and modify it Example 1 Simple control file with data included A control file EXAMPLE1 CON for an analysis of the Knox Cube Test see BTD a test containing 18 items each item is already scored dichotomously as 0 1 The person id data begins in column 1 and the item string begins in column 11 No items will be deleted recoded or anchored The default tables will appear in your output file EXAMPLE1 OUT The number of data lines is counted to determine how many children took the test For an explanation of the output obtained see section 6 Run this example with Control file EXAMPLE1 CON Output file EXAMPLE1 0UT Extra specifications Enter This file is EXAMPLE1 CON amp INST start of control variable instructions TITLE KNOX CUBE TEST at top of each printout page NI 18 number of items ITEM 1 11 location of response to first item NAME1 1 start of person id TABLES 1110001001001 Output default Tables PERSON KID person label is KID ITEM TAP item label is TAP PFILE EXAMPLE1 PF also write person measures to a file IFILE EXAMPLE1 IF also write item calibrations to a file amp END end of control variables 1 4 first item name
96. ZA76 This item may be biased ZZ234 Hard item at end of test END NAMES Your item names may be up to 50 characters long amp INST NI 4 ITLEN 50 amp END This item demonstrates ability for constructive reasoning This item flags rigid thinking This item detects moral bankruptcy This item is a throw away END NAMES 4 1 7 NAME1 first column of person id default 1 NAME gives the column position where the person id information starts in your data file or in the new record formatted by FORMAT Example 1 The person id starts in column 10 NAMEI1 10 starting column of person id A User s Guide to BIGSTEPS Page 29 Example 2 The person id starts in column 23 of the second record FORMAT 80A 80A lt concatenate two 80 character records NAME1 103 starts in column 103 of combined record 4 1 8 NAMLENE length of person id default calculated Use this if too little or too much person id information is printed in your output tables NAMLEN2 allows you define the length of the person id name with a value in the range of 1 to 30 characters This value overrides the value obtained according to the rules which are used to calculate the length of the person id These rules are 1 Maximum person id length is 30 characters 2 Person id starts at column NAME1 3 Person id ends at ITEM1 or end of data record 4 If NAME1 equals ITEM 1 then length is 30 characters Example The 9 characters including and fol
97. a separate DOS TEXT file called FINAL SPC C gt BIGSTEPS CONTROL FIL OUTPUT FIL SPFILE FINAL SPC Keyn is a particularly useful application of SPFILE Put the KEY1 instruction for each test form in its own DOS TEXT file then reference that file rather than including the key directly in the control file Here is FORMA KEY NI 23 CODES ABCD KEY 1 ABCDDADBCDADDABBCAADBBA Here is the control file amp INST TITLE FORM A READING RASCH ANALYSIS ITEM 1 20 SPFILE FORMA KEY TABLES 111011011 l A User s Guide to BIGSTEPS 7 THE ITERATION SCREEN While BIGSTEPS is running information about the analysis is displayed on the screen Here is an example based on the Liking for Science data The analysis was initiated with C gt BIGSTEPS SF DAT SF OUT Enter The is a horizontal bar chart which moves from left to right to show progress through the work file during each phase of the analysis The screen display includes BIGSTEPS Version 2 58 Program running shows version number Reading Control Variables Processing your control variables Reading keys groups etc Processing special scoring instructions Input in process Reading in your data 76 person records input Total person records found writing response file if RFILE specified To stop iterations Press Ctrl with S press Ctrl with C to cancel program CONVERGENCE TABLE reported in Table 0 Control sf dat Output sf out
98. acting 0 4 score points from each perfect score and adding 0 4 score points to each zero person score EXTRSC 0 4 5 2 5 HIADJ correction for top rating scale categories default 0 25 The Rasch model models the measure corresponding to a top rating scale category as infinite This is difficult to think about and impossible to plot Consequently graphically in Table 2 2 and numerically in Table 3 1 a measure is reported corresponding to a top category This is the measure corresponding to an imaginary rating HIADJ rating points below the top category Example The standard spread in Table 2 2 is based on HIADJ 0 25 You wish the top category number to be printed more to the right further away from the other categories HIADJ 0 1 5 2 6 LOWADJ correction for bottom rating scale categories default 0 25 The Rasch model models the measure corresponding to a bottom rating scale category as infinite This is difficult to think about and impossible to plot Consequently graphically in Table 2 2 and numerically in Table 3 1 a measure is reported corresponding to a bottom category This is the measure corresponding to an imaginary rating LOWADJ rating points above the bottom category Example The standard spread in Table 2 2 is based on LOWADJ 0 25 You wish the bottom category number to be printed more to the right close to the other categories LOWADJ 0 4 5 2 7 PAIRED correction for paired comparison data default N Paired comparison da
99. active categories to be deleted can be entered interactively by setting SDELQU Y between the amp INST and amp END lines If you specify this you will be asked if you want to delete any categories steps If you respond yes it will ask if you want to read these deleted categories from a file if you answer yes it will ask for the file name and process that file in the same manner as if SDFILE had been specified If you answer no you will be asked to enter a the sequence number of each item representing a group as described under SDFILE This question is omitted if all items are in one group b the score value of one category to be deleted from that item and its group Enter these deletions one at a time When you are finished enter a zero 4 6 3 SAFILE name of item step anchor file default no file The SFILE of one analysis may be used unedited as the SAFILE of another The anchoring option facilitates test form equating The steps in the rating scales of two test forms or in the item bank and in the current form can be anchored at their other form or bank values Then the common rating scale calibrations are maintained In order to anchor item steps an anchor file must be created of the following form 1 Use one line per item step to be anchored 2 Ifall items use the same rating scale i e GROUPS the default or you assign all items to the same group e g GROUPS 222222 then type the
100. alue of a response is calculated where possible so that both 01 and 1 are analyzed as 1 Data look like 02 1 20102 1 2 01 XWIDE 2 two characters wide CODES 1 2 3 401020304 two characters per response Example 5 The valid responses are percentages in the range 00 to 99 XWIDE 2 two columns each percent Page 30 A User s Guide to BIGSTEPS uses continuation lines CODES 0001020304050607080910111213141516171819 2021 222324252627282930313233343536373839 4041 424324454647484950515253545556575859 606162636465666768697071 7273747576777879 808182838485868788899091 9293949596979899 Example 6 The valid responses to an attitude survey are a b c and d These responses are to be recoded 1 2 3 and 4 Data look like adbdabcd CODES abcd four valid response codes NEWSCORE 1234 new values for codes RESCORE 2 rescore all items Typically abcd data implies a multiple choice test Then KEY 1 is used to specify the correct response But in this example abcd always mean 1234 so that the RESCORE and NEWSCORE options are easier to use Example 7 Five items of character width abcd then ten items of 2 character width AA BB CC DD These are preceded by person id of 30 characters Data look like George Washington Carver III dabcdBBAACCAADDBBCCDDBBAA FORMAT 30A1 5A1 10A2 lt Name 30 characters 5 1
101. ard control file 2 Specify PDFILE x 3 Copy the target portion of Table 6 4 Paste it between the 5 Delete characters before the entry numbers 6 Type after the entry numbers to make further numbers into comments amp INST TITLE Example of person deletion list from Table 6 IDFILE Delete the border character before the entry number ENTRY RAW INFIT OUTFIT NUM SCORE COUNT MEASURE ERROR MNSQ ZSTD MNSQ ZSTD PTBIS PUP 73 21 22 14 37 95 311 03 2 B 19 SAN 75 316 22 56 39 95 3 1 03 2 C 19 PAU Page 52 A User s Guide to BIGSTEPS t Enter the to make other numbers into comments closure of IDFILE 4 5 2 PDELQU delete persons interactively default N no Persons to be deleted or selected can be entered interactively by setting PDELQU Y If you specify this you will be asked if you want to delete any persons If you respond yes it will ask if you want to read these deleted persons from a file if you answer yes it will ask for the file name and process that file in the same manner as if PDFILE had been specified If you answer no you will be asked to enter the sequence number or numbers of persons to be deleted or selected one line at a time following the rules specified for IDFILE When you are finished enter a zero Example You are doing a number of analyses deleting a few but different persons each analysis You don t want to create a lot of
102. assigns items to groups default all in one group oe eee eee eeseeeseeeneceseeteeeteeeeeeeseaeeeaeees 46 GRPFRM location of GROUPS default N before amp END e cc ccccccccesssscceeseneeeesesneeeesesneeeeessneeeeeees 47 HIADJ correction for top rating scale categories default 0 25 eee eeeeceseceseceseeeseesseeseeessaeeeseesaeees 64 HLINES heading lines in output files default Y yes 0 0 ce eeeesesseecseecseecsseceseeeseceseesseesseessaeseseeeaeen 719 IAFILE name of item anchor file default no file eecccccceessseccessneeeeeeseeeesseeeeeeseeeeesseeeeeeees 51 IANCHQ anchor items interactively default N no 0 ceeceeescccssncceeseeceseeceeaeeceseecesaeeesaeeeesaeeeeeeeesaees 51 IDELQU delete items interactively default N no eceeescecescccssneceeseeceseecesaeeeeeecesaeeceaeecesaeeesaeeeeaees 50 IDFILE name of item deletion file default no file eee cessscccessnecceseneeeeessneeeeeseeeeessseeeesseeeeeeees 48 IFILE item output file default no file 0 eee eeeeeeececesneeeeeeessneceeaeeceeeceeaeecenecesaeeseaeecesaeeeeeeseaees 719 INUMB name items by sequence numbers default N names after amp END eee eeeeeeeeeeeeeeeeeeeneees 28 IREFER identifying items for recoding with IVALUE default none eee eeeeeneeeneeeeeees 33 ISFILE item step output file default no file oo eeeescceseeeneeeseeceeeceseceaeeeseeeseeseesseessaeseaeesaeee 81 ISORT
103. asures in the matching zone on the x axis The degree to which the data support this is given by the COHERENCE statistics in Table 3 A User s Guide to BIGSTEPS Page 105 CATEGORY PROBABILITIES MODES Step measures at intersections P 4 R 1 0 00000000 22222222 o 0000000 2222222 B 8 000 222 A 000 222 B 6 00 22 I 5 00 111111111 22 L 4 111 00 22 111 I 111 00 22 111 T 2 1111 22 00 1111 Y 1111111 22222 00000 1111111 20 4 HHI II 222222222222 000000000000000 4 t 5 4 3 2 1 0 1 2 3 4 5 PERSON MINUS ITEM MEASURE EXPECTED SCORE OGIVE MEANS E A ere ne E atana edd G aama aiai Sos eek gad E danian sree tell seach alee and sf res C aain mated E amadanan Gu ded 4 x 2 222222222222 P 2222222 E 1 5 2222 C 111 T 111 E 1 111 D 111 5 111 S 0000 Cc 0000000 X o 0 000000000000 R es Sar et pheaca i cinch choad se amc hac ead E chan leat Ae eae e C ade ar fe ae T nie meee Pree FE E 5 4 3 2 1 0 1 2 3 4 5 PERSON MINUS ITEM MEASURE The third graph is of the zone curves which indicate the probability of an item score at or below the stated category for any particular difference between person measure and item calibration The area to the left of the 0 ogive corresponds to 0 The right most area corre
104. atic action is possible IOSTAT followed by a number or in some other manner OMAANDNFWNK CO 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 36 38 39 10000 10001 10002 10003 10004 10005 Error not detected Processing has stopped itself Invalid function File not found Path not found Too many open files Access denied Invalid internal file identifier Storage control blocks destroyed Insufficient memory Invalid block address Environment incorrect Incorrect program format Invalid access code Invalid data Insufficient memory Invalid drive Current directory cannot be removed File cannot be moved to a different disk drive No more files Media is write protected Specified drive cannot be found The drive is not ready The device does recognize the command Data error Command length is incorrect Drive seek error The specified disk cannot be accessed The specified sector cannot be found The printer is out of paper Cannot write to specified device Cannot read from specified device Device is not responding The file is already open by another process Another process has locked the file The wrong disk is the drive Too many files open for sharing Reached end of file The disk is full File not open for read File not open for write File not found Record length negative or 0 Buffer allocation failed Bad i
105. ble 21 and Table 2 default 110 Ist and 2nd Interpreting rating scale structure is an art and sometimes counter intuitive See the examples in RSA Page 74 A User s Guide to BIGSTEPS Table 21 provides three curves for each rating scale definition Table 2 provides the equivalent response locations for each item The first curve shows the probability of response in each category The second curve shows the expected score ogive The third curve shows the cumulative probability of response in each category or below The Thurstone thresholds are at the intersections of these curves with the 5 probability line The control indicators of 1 and 0 in the Ist 2nd or 3rd position of the CURVES variable select or omit output of the corresponding curve CURVES 000 indicates no curves are to be drawn Table 21 will be skipped unless STEPT3 N in which case only the step summaries are output CURVES 111 draw all 3 curves in Table 21 and 3 versions of Table 2 CURVES 001 draw only the 3rd cumulative probability score curve 6 3 3 DISTRT output option counts in Tables 10 13 15 default Y output This variable controls the reporting of counts of option distracter or category usage The default is DISTRT Y if more than two values are specified in CODES DISTRT N Omit the option or distracter information DISTRT Y Include counts for each item for each of the values in CODES and for the number of responses counted as MISSING
106. calibration 0 Extreme minimum estimated using EXTRSC 1 Extreme maximum estimated using EXTRSC 2 No responses available for calibration 3 Deleted by user 18 23 16 4 The number of responses used in calibrating COUNT 24 30 17 5 The raw score used in calibrating SCORE 31 37 F7 2 6 Calibration s standard error rescaled by USCALE UDECIM ERROR 38 44 F7 2 7 Item mean square infit IN MSQ 45 51 F7 2 8 Item infit standardized ZSTD locally standardized ZEMP or log scaled LOG 52 58 F7 2 9 Item mean square outfit OUT MS 59 65 F7 2 10 Item outfit standardized ZSTD locally standardized ZEMP or log scaled LOG 66 72 F7 2 11 Item displacement rescaled by USCALE UDECIM DISPLACE 73 79 F7 2 12 Item by test score correlation point biserial PTBS or point measure CORR 80 80 1X 13 Blank 81 81 Al 14 Group to which item belongs G 82 82 1X 15 Blank 83 83 Al 16 Model used for analysis R Rating S Success F Failure M 84 84 1X 17 Blank 85 114 A30 18 Item name NAME The format descriptors are In Integer field width n columns Fn m Numeric field n columns wide including n m 1 integral places a decimal point and m decimal places An Alphabetic field n columns wide nX n blank columns When CSV Y commas separate the values which are squeezed together without spaces between Quotation marks surround the Item name e g 1 2 3 4 Name When CSV T the commas are replaced
107. cceesceceseeeeeeeceseecseaeecsenecesaeecsaeecesaeeesaeeceaaees 65 Wser triendly Tescala ver dshatoscactubvateg aestancten werdabe taeda A E Tact ana ene 66 XFILE analyzed response file default no file oo eee ee eeeeeseesseecseeceseseseeeeeeseeseessaeesseseseeeeessees 82 XWIDE columns per response default 1 0 eee eeecesecsseceseceeceseesseecseecsseesseeseeeseecssecsaeeeseeeseenseeenees 28 Page 122 A User s Guide to BIGSTEPS
108. ceessseeeeessseeeeseanes 72 PAFILE name of person anchor file default no file eee eeeececesceceseeeeeeeeseeceeaeeceaeecesneeeeeeeeaaees 54 PAIRED correction for paired comparison data default N ce seeseeseeesseceseceseceeeeseesseesseessaesnaeeeaeee 64 PANCHQ anchor persons interactively default N O ccescccssecceeseeceseeceeeeeeeeecesaeeesaeecesaeeeeeeeeaaees 54 PDELQU2 delete persons interactively default N 1O cecceescccesecceeseeceseeceeaeeeeeecesaeecsaeecesaeeeeeeeeaees 53 PDFILE name of person deletion file default no file eee eeeseeceseeceeneeeeeeecesaeeceaeecesaeeeeeeeesaees 52 PERSON title for person names default PERSON eeccccesccceseeceeseeceseeceeaeeeeeecesaeecsaeeceeaeeeeeesesaees 69 PFILE person output file default no file oo ee eeeeesseceneceseesneeseecseecsseceseceseeeseeeeeseeseaeseaeesaeees 80 PRCOMP principal components analysis of item residuals in Table 10 default N oe ee eeeereee 78 PSELECT person selection criterion default all persons cece eeeeeeeeseeceseceseceseeeseeseeeseesseesnaeeeaeee 53 PSORT column within person name for alphabetical sort in Table 19 default 1 ee eeeereees 77 PTBIS compute point biserial correlation coefficients default Y yes ceeeeseeseeeseeseeeeeeeeseeeaeees 73 RCONVSE score residual at convergence default 0 5 eeesccescccssnecesseeceseeceeaeeeeeeecesaeeceaeecesaeeeeneeeeaees 61
109. cess files Cannot open an existing file with STATUS NEW Command not allowed for unit type MRWE is required for that feature Bad specification for window Endian specifier not BIGSENDIAN or LITTLESENDIAN Cannot ENDIAN convert entire structures Attempt to read past end of record Attempt to read past end of record in non advancing I O Illegal specifier for ADVANCE Illegal specifier for DELIM Illegal specifier for PAD SIZE specified with ADVANCE YES EOR specified with ADVANCE YES Cannot DEALLOCATE disassociated pointer or unallocated array Cannot DEALLOCATE a portion of an original allocation An allocatable array has already been allocated Internal or unknown runtime library error Unknown data type passed to runtime library Illegal DIM argument to array intrinsic Size of SOURCE argument to RESHAPE smaller than SHAPE array SHAPE array for RESHAPE contains a negative value Unallocated or disassociated array passed to inquiry function The ncopies argument to REPEAT is negative The S argument to NEAREST is negative A User s Guide to BIGSTEPS Page 115 10056 10057 10058 10059 10060 10061 10062 10063 10064 10065 10066 10067 10068 10069 10070 The ORDER argument to RESHARE contains an illegal value Result of TRANSFER with no SIZE is smaller than source SHAPE array for RESHAPE is zero sized array VECTOR argument to UNPACK contains insufficient values Attempt to write a record longer than
110. ch way from UMEAN 6 3 7 NAMLMP2 id length for Tables 12 16 default calculated The id fields are truncated for Tables 12 and 16 The name length for maps variable NAMLMP2 overrides A User s Guide to BIGSTEPS Page 75 the calculated truncation Example The 9 characters including and following NAMEI are the person s Social Security number and are to be used as the person id on the maps NAMLMP 9 6 3 8 STEPT3 include step summary in Table 3 or 21 default Y in Table 3 The step summary statistics appear by default or if there is only one GROUPS in Table 3 For grouped analysis this part of Table 3 can become long in which case it can be moved to Table 21 Example Don t output partial credit step summaries in Table 3 Move them to Table 21 GROUPS 0 each item has own scale STEPT3 N report scale statistics in Table 21 6 3 9 TII number of items summarized by symbol in Table 1 default auto size For ease in comparing the outputs from multiple runs force consistent scaling by using MRANGE T1I and T1P Choose TII to be the largest number of items summarized by one from any of the separate runs Example In one run the bottom of Table 1 states that EACH IN THE ITEM COLUMN IS 20 ITEMS In another run EACH IN THE ITEM COLUMN IS 15 ITEMS To make the runs visually comparable specify the bigger value T1 20 6 3 10 TIP number of persons summarized by symbol in Table 1 default
111. comes infinite The information weighted fit statistic infit and the outlier sensitive fit statistic outfit are described in BTD and RSA Possible values and hence interpretation of these statistics is influenced by the observed distribution the person and item statistics This is particularly true of their standardized values which are designed to follow standard normal 0 1 distributions The local significance of these statistics is best interpreted in terms of their means and standard deviations reported in Table 3 Start investigating the misfit causing the most extreme values of these statistics and stop your investigation when the observed responses become coherent with your intentions The fit statistics reported will not exactly match those printed in BTD or RSA or those produced by another program This is because the reported values of these statistics are the result of a continuing process of development in statistical theory and practice Neither correct fit statistics nor correct values exist but see p 113 for guidance Report measure in Tables 6 and 10 if any of A User s Guide to BIGSTEPS Page 71 Less than Greater than FITP or FITI FITP or FITI To include every person specify FITP 0 For every item FITI 0 6 2 1 FITI item misfit criterion default 2 0 Specifies the minimum standardized fit value at which items are selected for reporting as misfits For Table 10 the table
112. control file LIKING FOR SCIENCE wright amp Masters p 18 fo Sao SSS Sea ess So SS Sa ee eS SSE Ss See Se Seo eae Sees 76 KIDS IN 74 KIDS MEASURED INFIT OUTFIT SCORE COUNT MEASURE ERROR IMNSQ ZSTD OMNSQ ZSTD MEAN 26 4 16 8 56 99 5 74 1 01 2 82 3 S D 11 9 5 7 23 67 1 33 65 1 4 78 1 2 REAL RMSE 5 90 ADJ SD 22 93 SEPARATION 3 89 KID RELIABILITY 94 A e h A Me Se a a Bo Ea ty D E D a ape re ee SN ok a ada E a Re a a a A a 25 TAPS IN 25 TAPS MEASURED INFIT OUTFIT MEAN 78 2 49 7 50 00 3 48 1 06 0 89 1 S D 43 0 22 5 27 56 1 04 36 1 3 43 7 REAL RMSE 3 63 ADJ SD 27 32 SEPARATION 7 53 TAP RELIABILITY 98 n E A sae A A AN NUM SCORE COUNT MEASURE ERROR IMNSQ ZSTD OMNSQ ZSTD CORR KID 73 15 14 33 18 4 75 3 46 4 5 4 52 5 4 A 16 SANDBERG RYN 71 24 19 47 29 4 32 3 25 4 8 5 19 5 1 B 14 STOLLER DAVE 14 15 11 37 24 5 48 2 09 2 1 1 77 1 3 C 45 HWA NANCY MA 32 21 13 50 17 5 77 1 14 3 2 04 1 2 D 13 ROSSNER JACK NUM SCORE COUNT MEASURE ERROR IMNSQ ZSTD OMNSQ ZSTD CORR TAPS 23 8 10 103 87 5 68 2 06 2 1 2 18 2 3 A 19 WATCH A RAT 9 49 37 62 83 3 12 1 67 2 5 1 81 1 4 B 47 LEARN WEED NA 16 53 40 60 93 3 01 1 73 2 8 1 54 1 0 C 38 MAKE A MAP 7 40 29 66 62 3 51 1 20 7 1 31 6 D 47 WATCH ANIMAL t Summary statistics from Table 3 and the largest misfitting persons and items Output written to SF OUT name of output file For details of the numbers see the description of output Tables 0
113. d 1 items 4 5 are weighted 3 and item 6 is weighted 7 Person name starts in column 1 Item responses start in column 21 Data looks like Mary N Simmons BCDABC Method 1 Rescoring NI 6 six items CODES ABCD codes as entered KEY 1 BAD key for weight 1 items 1 3 KEY3 CA key for weight 3 items 4 5 KEY7 D key for weight 7 item 6 STKEEP Y keep the defined structure NAMEI 1 ITEM 1 21 Method 2 Response replication Page 44 A User s Guide to BIGSTEPS Data looks like Mary N Simmons BCDABC becomes Mary N Simmons BCDABABABCCCCCCC NI 16 six items replicates NAME1 1 person starts at column 1 ITEM 1 21 item starts at column 21 FORMAT 20A 3A 3 T24 3A 6 T26 1A replicate items KEY1 BADCACACADDDDDD Keyl matches replicates amp END Item name 1 Item 2 name 2 Item 3 name 3 Item 4 name 4 Item 5 name 5 Item 4 name 6 Item 5 name 7 Item 4 name 8 Item 5 name 9 Item 6 name 10 Item 6 name 16 END NAMES Example 2 A ten item test all scored 0 1 Items 9 and 10 are to be given double weight Name starts in column 1 Item responses start in column 31 Method 1 Rescoring NI 10 ten items CODES 01 coded 0 1 NEWSCORE 02 items 9 10 rescored 0 2 RESCORE 000000001 1 rescore items 9 10 STKEEP Y keep the 0 2 structure Method 2 Response replication NAME1 1 person starts at column 1 ITEM 1 31 item starts at co
114. d steps categories default N nO eseeseeseesessrsseesresrrsresresresresresrrssersresresrese 44 T1I number of items summarized by symbol in Table 1 default auto size ee eee eee eeeeeeeeeee 76 T1P number of persons summarized by symbol in Table 1 default auto size eee ee eeeeeeee 76 TABLES output tables default 1110011001001000100000 0 ee eee eeeeceeeeseeceseseseeeeeeeeeseeesaeeees 68 TARGET estimate using information weighting default N nO 0 0 cee eeeeseeeseeesecseeeeeeseesseesneeesaeenss 62 TFILE input file listing tables to be output default none eee ee eee eeeeseeeeeeceeeceaeeeseeeeeeneeeneeesaeenss 70 TITLE title for output listing default control file name eee eee eeeeeeeeeeeeeseeeaeeeseeeseeseeeseeeeaeeeas 68 UANCHOR2 anchor values supplied in user scaled units default Y oo ee eeeeesseeeseceseeeneeeneeeseeesneenes 65 UDECIM number of decimal places reported default 2 eee eeeeesceseeeseeeseeesseessessseeeeesseecneeesaeenas 65 UIMEANs the mean or center of the item scale default 0 0 ececcccsssseeccesseeeeeesseeceseseeecenssseeeeseaees 65 UPMEAN them mean of center of the person scale default not used cece eeeeeescecesteeeeeeeeteeeeeneeees 65 USCALE the scale value of 1 logit default 1 oo cece ceessccescecesneeeeseeceeaeeeeaeecesaeeeeaeeceaeeeeaeeceeeeeeaeeees 65 User friendly rescaling ines Sets totes oR de ee cane ee Saeed eee eden ets 66
115. default 0 no limit eee eeeeeeeeceseeeeeeeneeeeees 69 MISSING treatment of missing data default 255 ignore 0 eee ee eeeeeeeeceseceseeeneeeseeeseessaeeeseeeseeeseeenees 31 MODELS assigns model types to items default R rating Scale eee eeseeseeeseeeeeeceeeeeseeeeeneeesees 43 MODFRM location of MODELS default N before amp END cccccccccsessccesesseeeeseseeeesesneeeesenseeees 47 MPROX maximum number of PROX iterations default 10 ecccsscccsesseccesesseeeeesseeeeseseeeesseseeees 61 MRANGE half range of measures on plots default 0 auto size oo eee eeseeneeeseeeeeeeeseeeseeeeeeeeeaees 75 MUCON maximum number of UCON iterations default 0 no Limit eee eceeccsesneeeeestteeeeneeeeeees 61 A User s Guide to BIGSTEPS Page 121 NAME I first column of person id default 1 eeeceeescecssceceseeeeneecesaeeeseecesaeceseeceeaeceeaeeceseeeeeaeeraes 29 NAMLEN length of person id default calculated eee eeeeeesesseeeececsaeeeseeeneeeeecseecsaeeeaeesseenseeenees 30 NAMLMP id length for Tables 12 16 default calculated eee eeescceesceceseceeneeceeeeeeeaeeceseeeeeeeeeaes 76 NEWSCORE recoding values with RESCORE default none ssessssssessssssssssssssssssssessssessee 33 NI number of items required no default seneseeeseeeseeesssssesssessressessetsserssersserssersseresseesseeseeesseessees 27 NORMAL normal distribution for standardizing fit default N chi square
116. e Example 2 You wish to include all persons in Table 6 FITP 0 includes all in Table 6 but not all in Table 7 6 2 3 OUTFIT sort misfits on infit or outfit default Y Outfit Other Rasch programs may use infit outfit or some other fit statistic There is no correct statistic Use the one you find most useful Specifies whether standardized infit or standardized outfit is used as your output sorting and selection criterion for the diagnosis of misfits Page 72 A User s Guide to BIGSTEPS OUTFIT Y Outfit is used as the fit statistic for sorting and selection the default OUTFIT N Infit is used as the fit statistic for sorting and selection 6 2 4 NORMAL normal distribution for standardizing fit default N chi square The default generally matches the statistics used in BTD and RSA Specifies whether distribution of squared residuals is hypothesized to accord with the chi square or the normal distribution Values of standardized fit statistics are obtained from squared residuals by means of one of these distributions NORMALEN _ Standardized fit statistics are obtained from the squared residuals by means of the chi square distribution and the Wilson Hilferty transformation the default NORMAL Y _ Standardized fit statistics are obtained from the squared residuals by means of an asymptotic normal distribution F A G Windmeijer The asymptotic distribution of the sum of weighted squared residuals in binary choice models
117. e treated as missing Some items in a test have two correct answers so two keys are used Since both answers are equally good KEY1 and KEY2 have the same value specified by KEYSCR But some items have only one correct answer so in one key a character not in CODES is used to prevent a match CODES 1234 KEY 1 23313141324134242113 KEY2 31 324321 3142314 is not in CODES KEYSCR 11 both KEYs scored 1 More than 9 KEYn lines together with KEYSCR are required for a complex scoring model for 20 items but the original data are only one character wide Original data Person name columns 1 10 20 Item responses columns 21 40 Looks like M Stewart 1321233212321232134 Solution reformat from XWIDE 1 to XWIDE 2 amp INST TITLE FORMAT from XWIDE 1 to 2 FORMAT 10A1 10X 20A1 10 of Name skip 10 20 of responses NI 20 NAME1 1 ITEM1 11 Responses in column 11 of reformatted record XWIDE 2 CODES 1 23 4 Original response now response blank KEY1 12132123143211111211 Keying 20 items KEY2 21211211211123322 21 KEY10 333233423 44444444 KEYSCR 1 232223414 Renumbering 10 KEYn amp END A User s Guide to BIGSTEPS 4 2 9 Disjoint strings of responses When the responses are not arranged in one continuous string in the record instructs to skip over or ignore the gaps Example The 18 item string is in columns 40 to 49 and then 53 to 60 of your data file The person
118. e 30 items CODES ABCD Scores entered as A through D KEY 1 CDABCDBDABCADCBDBCADBABDDCDABG Fully correct KEY2 ABCDABCCBADDABACABBBACCCCDABAB Partially correct KEY3 DABABAAACC CCABAC lt a Some partially correct if no matching response use a character not in CODES e g KEYSCR 211 KEY 1 fully correct 2 points KEY2 KEY3 partially correct 1 point each GROUPS 0 Each item is its own group MODELS R lt Each item has an Andrich rating scale STKEEP Y Keep in category 1 even if never observed CURVES 111 Print all 3 item curves in Tables 2 and 21 CATREF 2 Use category 2 for ordering in Table 2 STEPT3 N Move step calibrations from Table 3 to Table 21 SFILE EXAMPLE7 SF Write step calibrations for anchoring later ISFILE EXAMPLE7 ISF Write item step calibrations for plotting sSIDELQU Y lt 1 Comment only are any items are to be deleted IDELQU Y PDELQU Y and SDELQU Y are useful to place as Extra Specifications to initiate interactive deletion e g Extra specifications IDELQU Y Enter amp END 7 Al 1 1 A2 3 1 A29 2 7 A30 END NAMES 090111000102 10001 BDABADCDACCDCCADBCBDBCDDACADDC 090111000202 10002 BDCDCDCDADCBCCBDBDCABCBDACDABC 090111005302 10053 BDABCDDDCCCDDCDBACBABABCMCDBDC 090111005402 10054 BDADCDCCDACBCCABBDADBBCBBDDDDC A User s Guide to BIGSTEPS Page 17 Example 8 Items with various rating scale models Note Success Failure may not estimate correctly A
119. e of the RESCORE control variable Instructs where to find the RESCORE information RESFRM N RESCORE is a control variable between amp INST and amp END the default RESFRM Y RESCORE information follows after amp END but before the item names if any see p 112 and is formatted exactly like a data record It is helpful for reference to enter the label RESCORE where the person name would go in your data record Example KEY1 and KEY2 information follows the RESCORE information all are formatted like your data No item names are provided amp INST NAME1 5 start of person id ITEM1 14 start of responses NI 10 ten items INUMB Y use item sequence numbers as names CODES ABCDE valid codes RESFRM Y rescore information in data record format KEYFRM 2 two keys in data record format amp END RESCORE 0000110000 lt RESCORE looks like data KEY1 AB amp KEY1 looks like data KEY2 CA 4 KEY2 looks lie data record George ABCDABCDAB first data record l lt subsequent data records 4 2 12 KEYFRM location of KEYn default 0 before amp END Only use this if you have too many items to put conveniently on one line of the KEYn control variable Instructs where to find the KEYn information KEYFRM 0 KEY 1 through KEY99 if used are between amp INST and amp END KEYFRM 1 KEY 1 information follows after KEND but before the item names see p 112 The key is formatt
120. each recorded response may be of one or two characters Alphanumeric characters not designated as legitimate responses are treated as missing data This causes these observations but not the corresponding persons or items to be omitted from the analysis The responses to an item may be dichotomous right wrong yes no or may be on a rating scale good better best disagree neutral agree or may have partial credit or other hierarchical structures The items may all be grouped together or be grouped into subsets of one or more items which share the same response structure BIGSTEPS begins with a central estimate for each person measure item calibration and rating scale category step calibration unless pre determined anchor values are provided by the analyst An iterative version of the PROX normal approximation algorithm is used reach a rough convergence to the observed A User s Guide to BIGSTEPS Page 1 data pattern The UCON method is then iterated to obtain more exact estimates standard errors and fit Statistics This implementation of the UCON unconditional maximum likelihood joint maximum likelihood method used proportional curve fitting rather than the Newton Raphson method for finding improved estimates Output consists of a variety of useful plots graphs and tables suitable for import into written reports The statistics can also be written to data files fo
121. ecial table control neesii tie eeins steer sit a a sheeted Ee s Eeee hte uated EEEE EEE aiio 74 6 3 1 CATREF reference category for Table 2 default 0 item difficulty 0 ee eeneeeeeeeees 74 6 3 2 CURVES probability curves for Table 21 and Table 2 default 110 1st and 2nd 75 6 3 3 DISTRT output option counts in Tables 10 13 15 default Y output eeeeeeeeeeeeeeeeeeeseeee 75 6 3 4 LINLEN length of printed lines in Tables 7 10 16 22 default 80 oo ee eeeeeneeeeeeeees 75 6 3 5 FRANGE half range of fit statistics on plots default 0 auto size eeeeeeeeeseeeeeeeeeeseeeseeees 75 6 3 6 MRANGE half range of measures on plots default 0 auto size seeeeeseeeeeeeeeeereeeeeeeseeees 75 6 3 7 NAMLMPz id length for Tables 12 16 default calculated 0 0snnseeseesesesseseseesssessssseseses 76 6 3 8 STEPT3 include step summary in Table 3 or 21 default Y in Table 3 oe eee 76 6 3 9 T1I number of items summarized by symbol in Table 1 default auto size 76 6 3 10 T1P number of persons summarized by symbol in Table 1 default auto size 76 6 3 11 ISORT column within item name for alphabetical sort in Table 15 default 1 76 6 3 12 PSORT column within person name for alphabetical sort in Table 19 default 1 77 6 3 13 PRCOMP principal components analysis of item residuals in Table 10 default N 78 O 4 Special purpose
122. ected Specify this file by means of the control variable IDFILE or enter the deletion list in the control file using IDFILE a Delete an item enter the item number E g to delete item 5 enter 5 b Delete a range of items enter the starting and ending item number on the same line separated by a blank or dash E g to delete items 13 through 24 13 24 or 13 24 c Select an item for analysis enter a plus sign then the number E g to select item 19 from a previously deleted range 19 d Select a range of items for analysis enter a plus sign the starting number a blank or dash then the ending number E g to select items 17 through 22 17 22 or 17 22 e If a selection is the first entry in the deletion file then all items are deleted before the first selection is undertaken so that the items analyzed will be limited to those selected e g if 10 20 is the only line in the item deletion file for a 250 item test it means 1 250 delete all 250 items 10 20 reinstate items 10 through 20 f You may specify an item deletion list in your control with IDFILE List x e g IDFILE Page 48 A User s Guide to BIGSTEPS 17 delete item 17 2 delete item 2 x Example 1 You wish to delete the fifth and tenth through seventeenth items from an analysis but then keep item fourteen 1 Create a file named say ITEM DEL 2 Enter into the file the lines 5 10 17 14 3 Specify in the control file N
123. ed exactly like a data record It is helpful to place the name of the key e g KEY 1 where the person name would usually go for reference KEYFRM n KEY1 then KEY2 and so on up to KEYn where n is a number up to 99 follow amp END but placed before the item names see p 112 Each key is formatted exactly like a A User s Guide to BIGSTEPS Page 41 data record It is helpful to place the name of the key e g KEY2 where the person name would usually go Example KEY 1 and KEY2 information are to follow directly after amp END amp INST NAMEI 1 start of person id the default ITEM1 10 start of response string NI 7 number of items CODES abcd valid codes KEYFRM 2 two keys in data record format amp END KEY 1 bacddba keys formatted as data KEY2 cdbbaac Item 1 name item names Item 7 name END NAMES Mantovanibbacdba first data record l subsequent data records 4 2 13 CUTHI cut off responses with high probability of success default 0 no Use this if careless responses are evident CUTHI cuts off the top left hand corner of the Scalogram in Table 22 Eliminates cuts off observations where examinee ability is CUTHI logits or more as rescaled by USCALE higher than item difficulty so the examinee has a high probability of success Removing off target responses takes place after PROX has converged After elimination PROX is restarted followed by UCON estimation and fi
124. eeeeeeeeees 27 4 1 3 NI number of items required no default oe eee eeceseeseeseceeceeceseeseceeceeeesecseceeeeaeeaeeeeeeeeaees 27 4 1 4 XWIDE columns per response default 1 ee eseecsesecseeseeeeseceeseeeesesseeseeseeessesseeaeeeseeeaes 28 4 1 5 INUMB name items by sequence numbers default N names after amp END 0 eee 28 4 1 6 ITLEN maximum length of item name default 30 ceeeceecceseeeeceeeeeeesececeeeeneeaeeeeeeeenees 29 4 1 7 NAME 1 first column of person id default 1 cecseseeeeceseeeeseceececeesesseceeeeeeeaeseeeeeeaees 29 4 1 8 NAMLEN length of person id default calculated tie eeeseseeseeseeeseeeseeseeeseesseeaeeeseesees 30 4 2 Specifying how data is to be recoded eeeeesesseeeceeeseeseeeceeseeseeaceesseeseeseeasseesesaeeaseessesseeaseasseees 30 4 2 1 CODES valid data codes default 01 oo cece cecccsscssscsssceseceseesseceseceseceseceseceseenseeeseenseenseesseens 30 5 4 2 2 MISSING treatment of missing data default 255 ignore eeseseeeeeseeseseseseseerersesrsrsrsrsrseee 31 4 2 3 RESCORE response recoding with NEWSCORE or KEYn default oc 32 4 2 4 NEWSCORE recoding values with RESCORE default none cece eeeseeeeeeeeees 33 4 2 5 IREFER identifying items for recoding with IVALUE default none eee 33 4 2 6 IVALUEx recoding for x type items with IREFER default none essences 33 4 2 7 KEYn scoring key default
125. eeeeseesseeeseesseeesaeeeneee 78 CURVES probability curves for Table 21 and Table 2 default 110 Ist and 2nd oo ee eeeee 75 CUTHI cut off responses with high probability of success default 0 NO cee ceeeeeeeeeeesseeeeeeeeeeeneees 42 CUTLO cut off responses with low probability of success default 0 NO cece eeeeeeeeeeseeeeeeeeeeeeaeees 43 DATA name of data file default data at end of control file cece ccccsessccesesteeeeesseeeessseeeesesaeeees 27 DISTRT output option counts in Tables 10 13 15 default Y output 0 ee eee eeeecerecereeeseeeseeeeees 75 EXTRSC extreme score correction for extreme measures default 0 5 cccccccccssscccsesseeesesteeeessseeees 63 FITI item misfit criterion default 2 0 0 ccccccccsssccesssseeeesssseeeesssseeeesesaeeeessseecesesaeeseseaeesesseseeseneeaeeees 72 FITP person misfit criterion default 2 0 cccsssccssscsesescssscsesrcccnssssesncsesscssnssseseeseeenscesstaconsaseonenesnaes 72 FORMAT reformat data default NONE cecccccccesseecesesseeeesesseeeeseseeeesesseeeesssaeesesesaeeseseaeeseneeaeeees 37 FORMFD2 the form feed character default MS DOS standard cc ccccccsessecesesteeeesssteeeenenseeees 68 FRANGE half range of fit statistics on plots default 0 aUtO SIZE 0 eee eee eseeeseeeeeeeeseeeseeeseeneeesees 75 GRFILE probability curve coordinate output file default no file 0 eee eeeeseeeeeeeseeeeeeeeaeeeaeees 83 GROUPS
126. ees 83 T THE TIERA TION SCREEN a a Aea Aaea edie ENE ENOVA E ANE steading tie 85 8 INTERPRETING OUTPUT TABLES 0000 ccecesceessesseceseeeeseeeescseseeseseeacsesacseseesceeeacseeacsesesaceetaceeeeeaceeeaees 87 8 1 Fable 1 1 Distribution maps ses easiness einan eE ins NEE E o ETEA AE ANENE 87 8 2 Tables 1 0 1 2 1 3 12 and 16 Distribution maps essssessssssesssssssssesesessesessesessessseseseesesessesesseses 89 8 3 Table 2 Most probable response expected score Thurstone threshold plots 0 0 0 cesses 90 8 4 Tabl 3 Summary Statist CS a eeleiiaci id tial ele aap t aE a e E ed fe tach clint Seinas 92 8 4 1 Summaries of persons and items 0 0 eee esceeeeeseeseeseeeeseeseeseescessesseeseeasessesseeaseessesseeaeeaseseees 92 8 4 2 Summary of rating scale steps 0 ee eeeeseeseeseseseeeseeseeesseeseeseeaceessesseeaeeasasseesseaseessesseeseeaseeseees 94 8 5 Tables 4 1 5 1 and 8 1 9 1 Fit plots nan a n a a Nenin 96 8 6 ables o 2 and 9 2 Bit plots sti n nea p a ea a e tleeslneateetin at deetias 97 8 7 Tables 6 1 10 1 13 1 14 1 15 1 17 1 18 1 19 1 Person and item statistics eee ceeeeeeeees 97 8 7 1 Tables 6 1 10 1 Person and item fit Selection ccceeccccesesscccesscecesssscecesssceccsssscecesssseesessaees 99 8 8 Tables 6 2 10 2 13 2 14 2 15 2 17 2 18 2 19 2 Person and item Statistics ccceeeeceeeeeeeees 99 8 9 Tables 10 3 13 3 14 3 15 3 Item option frequencies eee eeeeeeeeeseeeeeeececeesee
127. eniently on one line of the MODELS control variable It is easier to us continuation lines Page 46 A User s Guide to BIGSTEPS Instructs where to find the MODELS information MODFRM N MODELS is a control variable between amp INST and amp END the default MODFRM Y MODELS information follows just after amp END but before the item names see p 112 It is formatted exactly like a data record It is helpful to enter MODELS where the person name would go Example A test consists of 10 two step items The highest level answer is scored with KEY2 The next level with KEY1 Some items have the Success structure where the higher level is administered only after success has been achieved on the lower level Some items have the Failure structure where the lower level is administered only after failure at the higher level The MODELS KEY1 KEY2 are formatted exactly like data records The data records are in a separate file amp INST NAME1 5 start of person id ITEM1 20 start of responses NI 10 ten items CODES ABCDE valid codes MODFRM Y lt 1 MODELS in data format KEYFRM 2 two keys in data format DATA DATAFILE location of data amp END MODELS SSSFFFSSSS data format KEY1 BCDABCDABC KEY2 ABCDDBCBAA Item name 1 first item name Item name 10 END NAMES 4 3 6 GRPFRM location of GROUPS default N before SEND Only use this if you have too many items to put conveniently
128. er item CODES 1234 valid codes KEY 1 31432432143142314324 correct answers Example 2 A 20 item MCQ test with responses entered as a b c d CODES abcd valid responses KEY 1 cadcbdcbadcadbcadcbd correct answers Example 3 A 20 item multiple choice exam with two somewhat correct response choices per item One of the correct choices is more correct than the other choice for each item so the less correct choice will get a score of 1 using KEY1 and the more correct choice will get a score of 2 using KEY2 All other response choices will be scored 0 CODES 1234 valid responses KEY 1 23313141324134242113 assigns 1 to these responses KEY2 31432432143142314324 assigns 2 to these responses 0 is assigned to other valid responses Example 4 A 100 item multiple choice test key CODES ABCD KEY 1 BCDADDCDBBADCDACBCDADDCDBBADCDACBCDADDCA DBBADCDACBCDADDCDBBADCDACBCDADDCDBBADCCD ACBCDADDCDBBADCDACBC continuation lines 4 2 8 KEYSCR reassign scoring keys default 123 This is only needed for complicated rescoring Specifies the score values assigned to response choices which match KEY1 etc To assign responses matching key to the missing value of 255 make the corresponding KEYSCR entry blank or some other non numeric character When XWIDE 1 each value can only take one position so that only KEY1 through KEY9 can be reassi
129. ere are three lines per person In the first line from columns 31 to 50 are 10 item responses each 2 columns wide Person id is in the second line in columns 5 to 17 The third line is to be skipped Codes are A B C D Data look like XXXXXXXXXXXXXXXXXXXXXXXXXXXXXX A C B D A D C B A DXXXXXXXX xxxxJoseph Carl0SXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX becomes Columns 1 2 3 4 5 6 7 8 9 0 1234567890123 ACBDAD CBA DJoseph Carlos FORMAT T31 10A2 T5 13A1 ITEM 1 1 start of item responses NI 10 number of items XWIDE 2 lt 4 2 columns per response NAME1 11 starting A of person name NAMLEN 13 length of person name CODES A BC D valid response codes If the third line isn t skipped format a redundant extra column in the skipped last line Replace the first control variable in this with FORMAT T31 10A2 T5 13A1 A1 last Al unused Example 7 Pseudo random data selection Page 40 A User s Guide to BIGSTEPS You have a file with 1 000 person records This time you want to analyze every 10th record beginning with the 3rd person in the file i e skip two records analyze one record skip seven records and so on The data records are 500 characters long FORMAT 500A 4 4 2 11 RESFRM location of RESCORE default N before SEND Only use this if you have too many items to put conveniently on one lin
130. ersons with code A 4 in columns 2 4 of their names Person name A User s Guide to BIGSTEPS Page 53 starts in column 23 so target characters starts in column 24 NAME1 23 person name starts in column 23 PSELECT A 4 lt 1 quotes because a blank is included A is in col 2 etc Selects ZA 4PQRS Example 3 Select all persons with M anywhere in their person name field which signifies Male PSELECT M lt 4 M matches the first blank M Selects 123456 MPQRS Selects 123 4M56 MPQRS Does not select 123 45MPQRS Example 4 Select Males M in column 8 in School 23 023 in column 14 16 Selects 1234567MABCDE023X YZ 4 5 4 PAFILE name of person anchor file default no file The PFILE from one run can be used unedited as the PAFILE of another Person anchoring can also facilitate test form equating The persons common to two test forms can be anchored at the values for one form Then the measures constructed from the second form will be equated to the scale of the first form In order to anchor persons an anchor file must be created of the following form 1 Use one line per person to be anchored 2 Type the sequence number of the person a blank and the scale value in logits if USCALE 1 otherwise your rescaled units at which to anchor the person Or this information may be specified in the control file using PAFILE Example The third person in the test is to be anchored at 1 5 logits and the
131. es in each category The expected values for all categories are 1 0 OUTFIT MNSQ is the average of the OUTFIT mean squares associated with the responses in each category The expected values for all categories are 1 0 This statistic is sensitive to grossly unexpected responses RESIDUL when shown is the residual difference between the observed and expected counts of observations in the category Only shown if 21 0 Indicates lack of convergence step anchoring or large data set b For rating scales the step calibration table lists SUMMARY OF MEASURED STEPS FOR GROUP 0 MODEL R ACT NUMBER 1 WATCH BIRDS ACT DIFFICULTY OF 87 ADDED TO MEASURES San pe a a a a a ENE CATEGORY OBSERVED AVERAGE EXP COHERENCE INFIT OUTFIT STEP LABEL COUNT MEASURE EXP OBS MNSQ MNSQ CALIBRATN SERRE DTACA SARRERA Ene A Aa A E A ease 0 3 94 42 0 0 74 73 NONE 1 35 14 32 69 82 70 52 1 64 2 36 1 61 1 39 81 72 75 77 1 64 a A a a aa Sa at a a AAA e CATEGORY STEP STEP SCORE TO MEASURE THURSTONE LABEL MEASURE S E AT CAT ZONE THRESHOLD ee ee a a ee ee ee SS 53 335 SSeS 5SS SS 4 55553545 0 NONE IC 3 64 INF 2 61 1 2 51 61 87 2 61 87 2 55 Page 94 A User s Guide to BIGSTEPS 2 77 26 C 1 90 87 INF 81 DIFFICULTY ADDED TO MEASURES is shown when the rating scale applies to only one item e g when GROUPS 0 Then all measures i
132. esessrssresresresresresressessessresersresees 78 CURVES probability curves for Table 21 and Table 2 default 110 Ist and 2nd eee eee 75 CUTHI cut off responses with high probability of success default 0 nO eseesseseeeeeseeresseesesereerrerese 42 CUTLO cut off responses with low probability of success default 0 no eseeseeseeeeeeeeeeeseeeeserererese 43 DATA name of data file default data at end of control file eee cceesecccessseceeessneeeessseeeeeseneeeeeees 27 DISTRT output option counts in Tables 10 13 15 default Y output 0 eee eee eeeeeeeeeeeeeeeeeneees 75 EXTRSC extreme score correction for extreme measures default 0 5 ccccessccccsssseceeesteeeeeesseeeeeees 63 FITI item misfit criterion default 2 0 cccsscccesssscceesssecceessneeeeesseeeeesseeeeessseeeeeseseesesssseeeeeseeeeeses 72 FITP person misfit criterion default 2 0 0 nenn a aS GEEAE TOS EA ENS 72 FORMAT reformat data default ONG eceesscccesssecccessneeeeesseeeeesseeeeeseseeeesseseeeeeeeseeeeeeeseeeeeses 37 FORMFD2 the form feed character default MS DOS standard cecccccccessccceessneceesenseeeessneeeeeees 68 FRANGE half range of fit statistics on plots default 0 aUtO SIZE eee eee eeseceeeeeeeeeeeeteeeteeeeeaeeeaeees 75 GRFILE probability curve coordinate output file default no file eee eeseesseeeseeeneeeeeeeeaeeees 83 GROUPS assigns items to groups default all in one
133. ete items interactively default N no Use this if you have one or two items to delete or will be running repeatedly with different deletion and selection patterns otherwise use IDFILE If your system is interactive items to be deleted or selected can be entered interactively by setting IDELQU Y If you specify this you will be asked if you want to delete any items If you respond yes it will ask if you want to read these deleted items from a file if you answer yes it will ask for the file name and process that file in the same manner as if IDFILE had been specified If you answer no you will be asked to enter the sequence number or numbers of items to be deleted or selected one line at a time following the rules specified for IDFILE When you are finished enter a zero Example You are doing a number of analyses deleting a few but different items each analysis You don t want to create a lot of small delete files but rather just enter the numbers directly into the program using amp INST NI 60 ITEM1 30 IDELQU Y amp END You want to delete items 23 and 50 through 59 BIGSTEPS asks you DO YOU WANT TO DELETE ANY ITEMS respond YES Enter DO YOU WISH TO READ THE DELETED ITEMS FROM A FILE respond NO Enter INPUT ITEM TO DELETE 0 TO END respond 23 Enter the first item to be deleted INPUT ITEM TO DELETE 0 TO END 50 59 Enter INPUT ITEM TO DELETE 0 TO END 0 Enter to end deletion If you ma
134. gned KEY10 through KEY99 can also be used but keep their default values of 10 through 99 When XWIDE 2 each value takes two positions and the values corresponding to all keys KEY 1 through KEY99 can be reassigned Example 1 Three keys are used and XWIDE 1 Response categories in KEY 1 will be coded 1 Response categories in KEY2 will be coded 2 Response categories in KEY3 will be coded 3 KEYSCR 123 default Example 2 Three keys are used and XWIDE 1 Response categories in KEY 1 will be coded 2 Response categories in KEY2 will be coded 1 Response categories in KEY3 will be coded 1 KEYSCR 211 A User s Guide to BIGSTEPS Page 35 Example 3 Example 4 Example 5 Example 6 Page 36 Three keys are used and XWIDE 2 Response categories in KEY 1 will be coded 3 Response categories in KEY2 will be coded 2 Response categories in KEY3 will be coded 1 KEYSCR 030201 or KEYSCR 3 2 1 Three keys are used and XWIDE 1 Response categories in KEY3 will be coded 1 Response categories in KEY6 will be coded missing Response categories in KEY9 will be coded 3 KEY3 BACDCACDBA response keys KEY6 ABDADCDCAB KEY9 CCBCBBBBCC KEYSCR xx1xxXxx3 scores for keys The x s correspond to unused keys and so will be ignored The X corresponds to specified KEY6 but is non numeric and so will cause responses matching KEY6 to be ignored i
135. group eee eee eeeeeeeeeeeeeeaeeeseeeseeeeesseeeaeenes 46 GRPFRM location of GROUPS default N before amp END cecccccccssscceesstececeeseeeeeesseeeeessnseeeeseaaes 47 HIADJ correction for top rating scale categories default 0 25 eee eeeeseesseceseceseeeeeeneesseecneeeseeenes 64 HLINES heading lines in output files default Y yes 0 cece eeeceseceseeeseeeseesseesssecsaeesseseeeseeesaeeaeenss 719 TAFILE name of item anchor file default no file eee eeecccccssncceeesneeecessneeeeesseeeeeeseeeeeseeeeeeees 51 IANCHQ anchor items interactively default N no eeceessessscesseceeeeceeecesneeeeaeeceaeeesaeeceseeeseareess 51 IDELQU delete items interactively default N no ecceeseessccceseeceeseeceseeceeaeeceseecesaeeseaeecesaeeeeeeeeaaees 50 IDFILE name of item deletion file default no file 0 eee toniet nin EEEE 48 IFILE item output file default no file eee eeeeeeseecesneecsseeesseeeeeaeeceeecesaeecseecesaeessaeecesaeeeeaeeeesaees 719 INUMB name items by sequence numbers default N names after amp END eee eeseeseeeseeeeeeeeeeeees 28 IREFER identifying items for recoding with IV ALUE default none eee eeeeeeneeeeeeeeeeeees 33 ISFILE item step output file default no file eee eeeeceseceseceseesseesseecseecsaessseesseeeeesseesseeesaeenas 81 ISORT column within item name for alphabetical sort in Table 15 default 1 eeeeeeesteeeeeees
136. hich would have been obtained if the value had been estimated INFIT is a standardized information weighted mean square statistic which is more sensitive to unexpected behavior affecting responses to items near the person s ability level MNSQ is the mean square infit statistic with expectation 1 Values substantially less than 1 indicate dependency in your data values substantially greater than 1 indicate noise See p 113 ZSTD is the infit mean square fit statistic standardized to approximate a theoretical mean 0 and variance 1 distribution When LOCAL Y then EMP is shown indicating a local 0 1 standardization When LOCAL LOG then LOG is shown and the logarithms of the mean squares are reported OUTFIT is a standardized outlier sensitive mean square fit statistic more sensitive to unexpected behavior by persons on items far from the person s ability level MNSQ is the mean square outfit statistic with expectation 1 Values substantially less than 1 indicate dependency in your data values substantially greater than 1 indicate the presence of unexpected outliers See p 113 ZSTD is the infit mean square fit statistic standardized to approximate a theoretical mean 0 and variance 1 distribution When LOCAL Y then EMP is shown indicating a local 0 1 standardization When LOCAL L then LOG is shown and the logarithms of the mean squares are reported PTBIS is the point biserial correlation fpbis between the individual item or person
137. id is in columns 11 30 Data look like xxxxxxxxxxPocahontas Smith Jrxxxxxxxxx1001001110xxx11001110 Method a Delete unwanted items in columns 50 51 52 using an item delete file IDFILE NAME1 11 in original record NAMLEN 20 length in original record ITEM1 40 in original record NI 21 include deleted items IDFILE DEL5052 file of deletions The contents of DEL5052 are 11 13 Cols 50 52 are items 11 13 Method b Rescore items in columns 50 51 52 as missing values with RESCORE NAME 11 in original record NAMLEN 20 ITEM1 40 NI 21 include rescored items RESCORE 000000000011100000000 rescore 50 52 CODES 01 the default NEWSCORE XX non numerics specify missing Method c Make the items form one continuous string in a new record created with FORMAT Then the item string starts in the 21st column of the new record Reformatted record looks like Pocahontas Smith Jr100100111011001110 FORMAT T11 20A T40 10A T53 8A reformatting NAME 1 in the formatted record ITEM 21 in the formatted record NI 18 the actual number of items 4 2 10 FORMAT reformat data default none Enables you to process awkwardly formatted data FORMAT enables you to reformat one or more data record lines into one new line in which all the component parts of the person information are in one person id field and all the responses are put together into one continuous item
138. ield and a string of responses to some items Your data can be placed either at the end of your control file or in a separate disk file BIGSTEPS reads up to 30 columns of person id information Normally the person id is assumed to end when the response data begin or when the end of your data record is reached However an explicit length of 30 characters or less can be given using the NAMLEN amp control variable By the term response is meant a data value which can be a category label or value score on an item or a multiple choice option code The responses can be one or two characters wide Every record must contain responses or missing data codes to the same items The response or missing data code for a particular item must be in the same position in the same format in every record If every person was not administered every item then mark the missing responses blank or make them some otherwise unused code so that the alignment of item responses from record to record is maintained A table of valid responses is entered using the CODES character string Any other response found in your data is treated as missing By using the CODES KEYn NEWSCORE and VALUE options virtually any type of response e g 01 1234 123 4 abcd a b c d can be scored and analyzed Missing responses are usually ignored but the MISSING control variable allows such responses to be treated as say wrong When wri
139. ile If XFILE filename is specified in the control file a file is output which enables a detailed analysis of individual response anomalies This file contains 4 heading lines unless HLINES N followed by one line for each person by item response used in the estimation Each line contains 1 Person number 17 PERSON 2 Item number 17 ITEM 3 Original response value after keying scoring I4 OBS 4 Observed response value after recounting 14 ORD 5 Expected response value F7 3 EXPECT 66 Modelled variance of observed values around the expected value F7 3 VAR This is also the statistical information in the observation Square root modelled variance is the observation s raw score standard deviation 7 Standardized residual Observed Expected Square root Variance F7 3 ZSCORE 8 Score residual Observed Expected F7 3 RESID If CSV Y these values are separated by commas When CSV T the commas are replaced by tab characters This file enables a detailed analysis of individual response anomalies The response residual can be analyzed in three forms 1 in response level score units from observed value expected value 2 in logits from observed value expected value variance 3 in standard units observed value expected value square root of variance Example You wish to write a file on disk called MYDATA XF containing response level information for use in examining particularly response
140. in the data is merely a reflection of the stochastic nature of the model This is a best case reliability which reports an upper limit to the reliability of measures based on this set of items for this sample REAL RMSE is computed on the basis that misfit in the data is due to departures in the data from model specifications This is a worst case reliability which reports a lower limit to the reliability of measures based on this set of items for this sample ADJ S D is the standard deviation of the estimates after subtracting from their observed variance the error variance attributable to their standard errors of measurement ADJ S D S D of MEASURE RMSE The ADJ S D is an estimate of the true standard deviation from which the bias caused by measurement error has been removed PERSON or ITEM SEP is the ratio of the ADJ S D to RMSE It provides a ratio measure of separation in RMSE units which is easier to interpret than the reliability correlation PERSON or ITEM SEP REL is a separation reliability equivalent to KR 20 Cronbach Alpha and the Generalizability Coefficient The relationship between separation SEP and reliability REL is REL SEP 1 SEP or SEP vV REL 1 REL S E OF MEAN is the standard error of the mean of the person or item sample WITH 1 EXTREME KIDS 75 KIDS MEAN is the mean of the measures including measures corresponding to extreme scores S D is the standard
141. it to disambiguate it an equal sign and the value e g TABLES 11011011100 or TAB 1101101110 this is enough of TABLES to make clear what you mean 2 You must use one line for each assignment but continuation lines are permitted To continue a line put a at the end of the line Then put a at the start of the text in the next line The two lines will be joined together so that the signs are squeezed out e g TITLE Analysis of medical research data is interpreted as TITLE Analysis of medical research data Continuation lines are helpful to make control files fit on your screen CODES 01020304 05060708 is interpreted as CODES 0102030405060708 3 The control variables may be listed in any order 4 Character strings must be enclosed in single quotes or double quotes when they contain blanks e g TITLE Rasch Analysis of Test Responses or TITLE Rasch Analysis of Test Responses Quotes are not required for single words containing no blanks e g PFILE kct pf 5 The control variables may be in upper or lower case or mixed e g Pfile Person Dat A User s Guide to BIGSTEPS Page 7 6 Blanks before or after control variables and before or after equal signs are ignored e g TITLE Test Report and TITLE Test Report are equally correct 7 Commas at the end of lines are ignored so equally correct are NAME 33 and NAME 33 8 Control variable
142. ith responses a b c d and a scoring key for 69 items Your data are in the control file This file is EXAMPLES CON amp INST TITLE An MCQ Test the title NI 69 69 items ITEM1 10 response string starts in column 10 NAME1 1 person id starts in column 1 CODES abcd valid response codes MISSING 255 default blanks are ignored KEYL dcbbbbadbdcacacddabadbaaaccbddddcaadccccdbdcccbbdbcccbdcddbacaccbcddb scoring key of correct answers ITEM TOPIC items are topics PERSON STDNT respondents are students TABLES 1111111111111111111111 output all tables NAMLMP 2 first 2 characters on maps e g nl PFILE EXAMPLES PF write out person measures CSV Y separate values by commas in PFILE HLINES N write out no heading lines in PFILE Many spreadsheets and statistics programs expect a file of numbers separated by commas Use IFILE or PFILE with CSV Y and HLINES N MUCON 0 allow as many UCON iterations as necessary EXTRSC 0 5 amp most conservative extreme measures wanted XFILE EXAMPLES XF write out individual response residual file amp END nl01 Month nl02 Sign sb02 newspaper sb03 newspaper END NAMES IM CAT a dcacc ccabbcaa NM KAT b badad accaaba aa c dd ab c NH RIC ddb b dbdcbcaadba ba acd bad db c d cc IL HOL a a da d d ccbddd bcd dc ca last data record Page 14 A User s Guide to BIGSTEPS Example 6 Keys in data record FORMAT Do not use
143. ke a mistake it is simple to start again reinstate all items with INPUT ITEM TO DELETE 0 TO END 1 999 Enter where 999 is the length of your test or more and start selection again 4 4 3 IAFILE name of item anchor file default no file The IFILE from one analysis can be used unedited as the IAFILE of another Anchoring facilitates equating test forms and building item banks The items common to two test forms or in the item bank and also in the current form can be anchored at their other form or bank calibrations Then the scale constructed from the current data and the measures reported will be equated to the scale of the other form or bank Page 50 A User s Guide to BIGSTEPS In order to anchor items a data file must be created of the following form 1 Use one line per item to be anchored 2 Type the sequence number of the item a blank and the scale value at which to anchor the item in logits if USCALE 1 or in your rescaled units otherwise Further values in each line are ignored An IFILE works well as an IAFILE Or this information may be specified in the control file using AFILE Example 1 The third item is to be anchored at 1 5 logits and the fourth at 2 3 logits 1 Create a file named say ANC FIL 2 Enter the line 3 1 5 into this file which means item 3 is to be fixed at 1 5 logits 3 Enter a second line 4 2 3 into this file which means item 4 is to be fixed at
144. l value to the center the scale for persons Previous UIMEAN values are ignored Example You want to used norm referenced scaling With person mean of 0 UPMEAN 0 5 3 3 USCALE the scale value of 1 logit default 1 Specifies the number of reported units per logit Example 1 You want to rescale 1 logit into 45 5 units so that differences of 100 50 0 50 100 correspond to success rates of 10 25 50 75 90 USCALE 45 5 Example 2 You want to reverse the scale directions since the data matrix is transposed so that the items are examinees and the persons are test questions USCALE 1 KEYn RESCORE GROUPS will still apply to the columns not the rows of the data matrix Centering will still be on the column measures 5 3 4 UDECIM number of decimal places reported default 2 This is useful for presenting your output measures and calibrations in a clear manner by removing meaningless decimal places from the output Range is 0 12345 to 4 1 2345 Example 1 You want to report measures and calibrations to the nearest integer UDECIM 0 Example 2 You want to report measures and calibrations to 4 decimal places because of a highly precise though arbitrary pass fail criterion level UDECIM 4 5 3 5 UANCHOR anchor values supplied in user scaled units default Y This simplifies conversion from a logit measures to user scaled measures UANCHOREN specifies that the anchor values are in logits Reported
145. lowing NAMEI are the person s Social Security number and are to be used as the person id NAMLEN 9 4 2 Specifying how data is to be recoded 4 2 1 CODES valid data codes default 01 Says what characters to recognize as valid codes in your data file If XWIDE 1 the default use one column character per legitimate code If XWIDE 2 use two columns characters per legitimate code Characters in your data not included in CODES are given the MISSING value Example 1 A test has four response choices These are 1 2 3 and 4 Each response uses 1 column in your data file Data look like 1342123421323212343221 XWIDE 1 one character wide the default CODES 1234 four valid 1 character response codes Example 2 There are four response choices Each response takes up 2 columns in your data file and has leading O s so the codes are 01 02 03 and O04 Data look like 0302040103020104040301 XWIDE 2 amp two characters wide CODES 01020304 four valid 2 character response codes Example 3 There are four response choices entered on the file with leading blanks so that codes are 1 2 3 and 4 Data look like 3242132 XWIDE 2 lt two characters wide CODES 1 23 4 required blanks in 2 character responses Example 4 Your data is a mixture of both leading blanks and leading 0 s in the code field e g 01 1 2 02 etc The numerical v
146. lumn 31 NI 12 ten items 2 duplicated FORMAT 30A 10A TL2 2A enter items 9 amp 10 twice amp END Item 1 name Item 10 name Item 9 name again Item 10 name again END NAMES 4 3 4 GROUPS assigns items to groups default all in one group Items in the same group share the same rating scale definition For tests comprising only dichotomous A User s Guide to BIGSTEPS Page 45 items or for tests in which all items share the same rating scale definition all items belong to one group For tests using the Partial Credit rating scale model each item comprises its own group For tests in which some items share one rating scale definition and other items another scale definition there can be two or more groups GROUPS default if only one model specified with MODELS All items belong to one group GROUPS 0 default if MODELS specifies multiple models Each item has a group of its own so that a different rating scale is defined for each item as in the partial credit model GROUPS some combination of numbers and letters 0 s 1 s 2 s A s B s etc Items are assigned to the group label whose location in the GROUPS string matches the item sequence number Each item assigned to group 0 is allocated to a partial credit group by itself Items in groups labelled 1 etc share the scale definition with other items in the same labelled group Example 1 Responses to all items are on the same
147. m names Data are in file EXAMPLE3 DAT This file is EXAMPLE3 CON amp INST TITLE A test with 25 recoded items informative title NI 25 25 items ITEM1 12 item responses start in column 12 CODES 012X valid codes NEWSCORE 0213 corresponding response score RESCORE 2 specifies rescore all items TABLES 1111111110101010101110 selected tables FITP 3 0 person misfit cut off for reporting FITI 3 0 item misfit cut off for reporting DATA EXAMPLE3 DAT name of data file IDFILE list of items to delete Or IDFILE EXAMPLE2 IDF 5 delete item 5 8 delete item 8 20 25 delete items 20 through 25 end of list INUMB Y use item sequence numbers as names amp END The data is in file EXAMPLE3 DAT 101F20FIDP 21XX2XXxXXxX111x1200001Xx2x 102M20PFNP X2210122211222211222120x2 175F FBDP 1Xx00x00000200012x02220100 176F23FEDP 21121022012002121 2202000 person id s contain demographic information Page 12 A User s Guide to BIGSTEPS Example 4 Selective item recoding The test has 18 items specified in file EXAMPLE4 CON The response string starts in column 1 Person id s start in column 41 Original codes are 0 1 in data file EXAMPLE4 DAT Items 2 3 and 4 are to be recoded as 1 0 All tables are to appear in report file EXAMPLE4 OUT in a standardized form An adjustment is to be made for UCON estimation bias This file is EXAMPLE4 CON amp INST TITLE An example of selective ite
148. m recoding page title NI 18 18 items ITEM 1 1 item responses start in column 1 NAME1 41 person id starts in column 41 NAMLEN 9 person id has 9 characters Richard M NAMLMP 7 lt 1 7 characters to appear on maps Richard CODES 01 the observed response codes NEWSCORE 10 the new scores for items 2 3 4 RESCORE 01 1 100000000000000 rescore items indicated by 1 TABLES 1111111111111111111111 output all tables DATA EXAMPLE4 DAT name of data file STBIAS Y adjust for UCON estimation bias MRANGE 5 plots and maps have half range of 5 logits FRANGE 4 fit plots have half range of 4 ZSTD units LINLEN 0 allow maximum line length LINLEN 0 puts distractors for same item or mapped elements with same measure on one output line MAXPAG 0 no maximum page length MAXPAG 0 prints tables without breaks for new page headings ASCII Y Default Replace PC characters by ASCII etc Use ASCII N if report file is to be printed by PC DOS not Windows REALSE Y Inflate standard errors by misfit LOCAL Y Local standardizing of fit statistics amp END 1 4 R2 3 R reminds us item coding was reversed R1 2 4 2 1 4 4 1 3 4 2 1 4 END NAMES The data file EXAMPLE4 DAT is 100011100000000000 Richard M 100011111100000000 Tracie F 100100000000000000 Helen F A User s Guide to BIGSTEPS Page 13 Example 5 Scoring key for items A multiple choice adaptive test in file EXAMPLES CON w
149. mean squares associated with the responses in each category The expected values for all categories are 1 0 High infit signals this category was chosen when nearby categories were expected OUTFIT MNSQ is the average of the OUTFIT mean squares associated with the responses in each category The expected values for all categories are 1 0 High outfit mean square signals that this category was observed when a far away category choices were expected STEP CALIBRATN the calibrated difficulty of this step the transition from the category below to this category relative to the prior step The bottom step has no prior step and so that the difficulty is shown as NONE The step calibration is expected to increase with category value Disordering is marked by STEP MEASURE the calibrated difficulty of this step adjusted for item difficulty The bottom step has no prior step and so that the difficulty is shown as NONE The step measure is expected to increase with category value Disordering is marked by STEP S E is an approximate standard error of the step difficulty SCORE TO MEASURE AT CAT is the measure on an item of 0 logit difficulty corresponding to an expected score equal to the category label which for the rating scale model is where this category has the highest probability is printed where the matching calibration is infinite The value shown corresponds to the measure 25 score points or LOWADJ and HIADJ away
150. mn 1 of the next line in your data file repeat the string of instructions within the exactly n times a comma is used to separate the instructions Set XWIDE 2 and you can reformat your data from original 1 or 2 column entries Your data will all be analyzed as XWIDE 2 Then nA2 Al read in n pairs of characters starting with the current column into n 2 character fields of the formatted record For responses with a width of 2 columns read in n 1 character columns starting with the current column into n 2 character fields of the formatted record Always use nA1 for person id information Use nA1 for responses entered with a width of 1 character when there are also 2 character responses to be analyzed When responses in 1 character format are converted into 2 character field format compatible with XWIDE 2 the 1 character response is placed in the first left character position of the 2 character field and the second right character position of the field is left blank For example the 1 character code of A becomes the 2 character field A Valid 1 character responses of A B C D must be indicated by CODES A BCD with a blank following each letter ITEM1 must be the column number of the first item response in the formatted record created by the FORMAT statement NAME1 must be the column number of the first character of the person id in the formatted record Example 1 Each person
151. mple 1 Read the observations from file A PROJECT RESPONSE DAT DATA A PROJECT RESPONSE DAT Example 2 Read scanned MCQ data from file DATAFILE in the current directory DATA DATAFILE You may specify that several data files be analyzed together in one run by listing their file names separated by signs The list e g FILE1 DAT MORE DAT YOURS D can be up to 200 characters long The layout of all data files must be identical Example 3 A math test has been scanned in three batches into files BATCH 1 BATCH 2 and BATCH 3 They are to be analyzed together DATA BATCH 1 BATCH 2 BATCH 3 4 1 2 ITEM1 column number of first response required no default Specifies the column position where the response string begins in your data file record or the column where the response string begins in the new record formatted by FORMAT If you have the choice put the person identifiers first in each record and then the item responses with each response taking one column Example 1 The string of 56 items is contained in columns 20 to 75 of your data file ITEM1 20 response to item 1 in column 20 NI 56 for 56 items XWIDE 1 one column wide the default Example 2 The string of 100 items is contained in columns 30 to 69 of the first record 11 to 70 of the second record followed by 10 character person i d XWIDE 1 one column wide the default FORMAT T30 40A T11 60A 10A two records per person ITE
152. ms XWIDE 2 two characters wide CODES 0 1 original codes NEWSCORE 1 0 new values RESCORE 1000001000 rescore items 1 amp 7 or RESCORE 10000001000 no blanks allowed Example 4 The original codes are 0 1 and 2 You want to make 090 1 gt 1 and 291 for all items XWIDE 1 one character wide default CODES 012 valid codes NEWSCORE 01 1 desired scoring Example 5 The original codes are 0 1 and 2 You want to make 092 191 and 290 for even numbered items in a twenty item test NI 20 twenty items CODES 012 three valid codes NEWSCORE 210 desired scoring RESCORE 01010101010101010101 rescore even items Example 6 The original codes are 0 1 2 3 and some others You want to make all non specified codes into 0 but to treat codes of 2 as missing CODES 0123 four valid codes NEWSCORE 01X3 response code 2 will be ignored MISSING 0 treat all invalid codes as 0 Example 7 The original codes are 0 1 2 3 You want to rescore some items selectively using KEY1 and KEY2 and to leave the others unchanged their data codes will be their rating values For items 5 and 6 090 190 291 392 for item 7 090 190 290 391 Responses to other items are already entered correctly as 0 1 2 or 3 CODES 0123 valid codes RESCORE 00001 11000 rescore items 5 6 7 KEY1
153. mum number limitation set MUCON 0 Iteration always be stopped by Ctrl with F see Stopping BIGSTEPS in Section 2 Example 1 To allow up to 4 iterations in order to obtain rough estimates of a complex rating scale MUCON 4 lt 4 UCON iterations maximum Example 2 To allow up to as many iterations as needed to meet the other convergence criteria MUCON 0 Unlimited UCON iterations Example 3 Perform no UCON iterations since the PROX estimates are good enough MUCON 1 No UCON iterations 5 1 3 LCONV logit change at convergence default 01 logits Measures are only reported to 2 decimal places so a change of less than 01 logits will probably make no visible difference Specifies what value the largest change in any logit estimate for a person measure or item calibration must be less than in the iteration just completed for iteration to cease The current largest value is listed in Table 0 and displayed on your screen Example To set the maximum change at convergence to be less or equal to 001 logits LCONV 001 5 1 4 RCONV score residual at convergence default 0 5 Scores increment in integers so that 0 5 is about as precise a recovery of observed data as can be hoped for Specifies what value the largest score residual corresponding to any person measure or item calibration must be less than in the iteration just completed for iteration to cease The current largest value is listed in Table 0 and displayed on y
154. n DOS needs a larger file area Increase the number in your FILES statement in your CONFIG SYS file You tried to write to a read only or locked file In WINDOWS does another program still have your file open You ve run out of disk space see Section 2 11 2 See 1 above Your analysis requires more RAM memory see Section 2 11 1 Temporary file names exhausted Delete all files BIGSTEPS If your FORMAT control value is somewhat awry 6205 6502 6503 6504 6980 6981 6982 6983 6984 6985 A edit descriptor problem e g ZA instead of 2A or for 1 or O for 0 Repeat field not a positive integer e g 7 3A1 instead of 7A1 Multiple repeat field e g 2 3 instead of 6 Invalid number e g 7D mistyped for 70 Integer expected e g T2Z mistyped for T22 Initial left parenthesis expected FORMAT must start with Positive integer required e g OX is not allowed Invalid repeat count e g 3T22 is not allowed only T22 An integer must precede X e g X is not allowed use 1X B must be followed by N or Z Page 116 A User s Guide to BIGSTEPS 6986 Parentheses can go no more than 16 levels deep 6987 _F D E G fields must contain a e g F6 instead of F6 0 6988 Not enough in format 6989 Unknown or missing character in format e g two commas together 6990 Too many leading zeroes specified in I format e g 16 7 instead of 16 6 6991 Numerical value greater than 32766 in FORMAT statement 6099 amp 6104 You
155. n of 1 s and 0 s Only items corresponding to 1 s are recoded with NEWSCORE or scored with KEYn When KEYn is specified NEWSCORE is ignored If some but not all items are to be recoded or keyed assign a character string to RESCORE in which 1 means recode key the item and 0 or blank means do not recode key the item The position of the 0 or 1 in the RESCORE string must match the position of the item response in the item string Example 1 The original codes are 0 and 1 You want to reverse these codes i e 190 and 091 for all items XWIDE 1 one character wide responses the default CODES 01 valid response codes are 0 and 1 the default NEWSCORE 10 desired response scoring RESCORE 2 rescore all items this line can be omitted Example 2 Your data is coded 0 and 1 This is correct for all 10 items except for items 1 and 7 which have the reverse meaning i e 190 and 091 Page 32 A User s Guide to BIGSTEPS NI 10 ten items CODES 01 default shown here for clarity NEWSCORE 10 revised scoring RESCORE 1000001000 only for items and 7 If XWIDE 2 use one or two columns per RESCORE code e g 1 or 1 mean recode key 0 or 0 mean do not recode key Example 3 The original codes are 0 and 1 You want to reverse these codes i e 190 and 091 for items 1 and 7 of a ten item test NI 10 ten ite
156. n these tables are adjusted by the estimated item difficulty CATEGORY LABEL the number of the category in your data set after scoring keying OBSERVED COUNT the count of occurrences of this category used in the estimation AVERAGE MEASURE is the empirical average of the measures that were modelled to produce the responses that were observed in the category i e 2 B Dj 21 for all observations in a category The average measure is expected to increase with category value Disordering is marked by AVERAGE EXP MEASURE is the expected optimum value of the average measure for these data Compare this with AVERAGE MEASURE to discover the amount of discrepancy in the data COHERENCE EXP is the percentage of all observations whose expectations fell in this category that were actually observed in this category i e EXP Observations in cat with expectations also in cat all expectations in cat This indicates to what extent measures corresponding to this category predict ratings in it 100 is optimum Less than 50 is inferential insecure COHERENCE OBS is the percentage of all observations in this category whose expectations are also in this category i e OBS Observations in cat with expectations also in cat all observations in cat This indicates to what extent ratings in this category measures corresponding to it 100 is optimum Less than 50 is inferential insecure INFIT MNSQ is the average of the INFIT
157. ns 4000 4019 then in 4021 50 Data look like XXXXXXXXXXXXXXIOMN Smi thxxxx XxxDCBACDADABCADCBCDABDxXBDCBDADCBDABDCDDADCDADBBDCDABB becomes on reformatting John Smi thDCBACDADABCADCBCDABDBDCBDADCBDABDCDDADCDADBBDCDABB FORMAT T15 10A T4000 20A 1X 30A A User s Guide to BIGSTEPS Page 39 NAME1 1 start of person name in formatted record NAMLEN 10 length of name automatic ITEM1 11 start of items in formatted record NI 50 amp 50 item responses CODES ABCD valid response codes Example 5 There are five records or lines in your data file per person There are 100 items Items 1 20 are in columns 25 44 of first record items 21 40 are in columns 25 44 of second record etc The 10 character person id is in columns 51 60 of the last fifth record Codes are A B C D Data look like XXXXXXXXXXXXXXXXXXXXXXXXACDBACDBACDCABACDACD XXXXXXXXXXXXXXXXXXXXXXXXDABCDBACDBACDCABACDA XXXXXXXXXXXXXXXXXXXXXXXXACDBACDBACDCABACDACD XXXXXXXXXXXXXXXXXXXXXXXXDABCDBACDBACDCABACDA XXXXXXXXXXXXXXXXXXXXXXXXABCDBACDBACDCABACDADXXxXxXxxXMary Jones becomes ACDBACDBACDCABACDACDDABCDBACDBACDCABACDAACDBACDBACDCABACDACDDABCDBACDBACDCABACDAABCDBACDBACDCABACDADMary Jones FORMAT 4 T25 20A T25 20A T51 10A ITEM1 1 start of item responses NI 100 number of item responses NAME1 101 start of person name in formatted record NAMLEN 10 length of person name CODES ABCD valid response codes Example 6 Th
158. of the estimate INFIT is an information weighted fit statistic which is more sensitive to unexpected behavior affecting responses to items near the person s ability level MNSQ is the mean square infit statistic with expectation 1 Values substantially below 1 indicate dependency in your data values substantially above 1 indicate noise ZSTD is the infit mean square fit statistic standardized to approximate a theoretical mean 0 and Page 92 A User s Guide to BIGSTEPS variance 1 distribution When LOCAL Y then EMP is shown indicating a local 0 1 standardization When LOCAL L then LOG is shown and the logarithms of the mean squares are reported OUTFIT is an outlier sensitive fit statistic more sensitive to unexpected behavior by persons on items far from the person s ability level MNSQ is the mean square outfit statistic with expectation 1 Values substantially less than 1 indicate dependency in your data values substantially greater than 1 indicate the presence of unexpected outliers ZSTD is the outfit mean square fit statistic standardized to approximate a theoretical mean 0 and variance 1 distribution When LOCAL Y then EMP is shown indicating a local 0 1 standardization When LOCAL L then LOG is shown and the logarithms of the mean squares are reported RMSE is the root mean square standard error computed over the persons or over the items MODEL RMSE is computed on the basis that the data fit the model and that all misfit
159. olist specifier Page 114 Errors codes may be reported as A User s Guide to BIGSTEPS 10006 10007 10008 10009 10010 10011 10012 10013 10014 10015 10016 10017 10018 10019 10020 10021 10022 10023 10024 10025 10026 10027 10028 10029 10030 10031 10032 10033 10034 10035 10036 10037 10038 10039 10040 10041 10042 10043 10044 10045 10046 10047 10048 10049 10050 10051 10052 10053 10054 10055 Error in format string Illegal repeat count Hollerith count exceeds remaining format string Format string missing opening Format string has unmatched parentheses Format string has unmatched quotes Non repeatable format descriptor Attempt to read past end of file Bad file specification Format group table overflow Illegal character in numeric input No record specified for direct access Maximum record number exceeded Illegal file type for namelist directed I O Illegal input for namelist directed I O Variable not present in current namelist Variable type or size does not match edit descriptor Illegal direct access record number Illegal use of internal file RECL only valid for direct access files BLOCK only valid for unformatted sequential files Unable to truncate file after rewind backspace or endfile Can t do formatted 1 0 on an entire structure Illegal negative unit specified Specifications in re open do not match previous open No implicit OPEN for direct ac
160. on whereabouts in the category ordering is a person of a particular measure located This information is expressed in terms of Thurstone thresholds the point at which the probability of scoring in or above the category is 5 THURSTONE THRESHOLD MEDIAN 6 4 2 0 2 4 6 1 1 I 1 I 1 I 1 I l 1 I 1 1 1 I 1 I l 1 1 1 1 1 i 1 1 i 1 1 1 1 1 1 1 1 1 1 1 t 1 t 1 1 1 1 1 1 1 I 1 I 5 FIND BOTTLES 0 2 23 WATCH A RAT 0 2 9 LEARN WEED N 0 2 21 WATCH BIRD M 0 1 2 2 0 2 0 2 0 2 11 FIND WHERE A 1 2 19 GO TO ZOO 1 2 18 GO ON PICNIC 1 2 6 4 2 0 2 4 6 A User s Guide to BIGSTEPS Page 91 8 4 Table 3 Summary statistics 8 4 1 Summaries of persons and items controlled by STBIAS REALSE UMEAN USCALE This table summarizes the person item and step information Extreme scores zero and perfect scores have no exact measure under Rasch model conditions so they are dropped from the main summary statistics Using a Bayesian technique however reasonable measures are reported for each extreme score see EXTRSC Totals including extreme scores are also reported but are necessarily less inferentially secure than those totals only for non extreme scores SUMMARY OF 74 MEASURED NON EXTREME KIDS 4 5 5 5 5 5 5 5 5 5 5 5 5 5 55 5 5 5 5 5 RAW MODEL INFIT OUTF
161. on distribution maps if small Most probable response plot The most probable categories for each item listed in difficulty order are plotted against the range of person measures If there are rating scales this is followed by a similar plot of the expected scores and Thurstone thresholds Summary statistics Person item and category step distribution and fit statistics Person infit plot Person infits plotted against person measures Nn Person outfit plot Person outfits plotted against person measures Infit vs outfit plot if both Tables 4 and 5 requested Person statistics fit order Only misfitting persons are shown see FITP Unexpected responses in Guttman format TE Misfitting Persons Lists response details for persons with standardized fit greater than FITP Item infit plot Item infits plotted against item calibrations Item outfit plot Item outfits plotted against item calibrations Infit vs outfit plot if both Tables 8 and 9 requested 10 Item statistics fit order Only misfitting items are shown see FITI with option counts Unexpected responses in Guttman format If PRCOMP specified shows largest item correlations and principal components analysis Misfitting Items Response details for items with standardized fit greater than FITI Item distribution map Horizontal histogram of item distribution with abbreviated item names Item statistics difficulty order with option counts
162. or saving the file as an ASCII or DOS text file REM the MYBATCH BAT batch file to automate BIGSTEPS BIGSTEPS 1 1 OUT PFILE 1 PF IFILE 1 IF To execute this type at the DOS prompt C gt MYBATCH ANAL Press Enter Key This outputs the tables in ANAL1 OUT PFILE in ANAL1 PF and IFILE in ANALLIF You can also edit the files BIGBATCH BAT or WINBATCH BAT These can be executed from the DOS prompt or from their Windows icons 6 5 Supplemental control files 6 5 1 SPFILE supplementary control file default no file There is often a set of control instructions that you wish to make a permanent or temporary part of every control file Such files can be specified with SPFILE Multiple SPFILE specifications can be included in one control file Supplemental files called with SPFILE can also include SPFILE specifications Example 1 The analyst has a standard set of convergence criteria and other control instructions to include in every control file a Enter these into a standard DOS TEXT ASCH file e g SUPPL CON The analyst s SUPPL CON contains LCONV 01 ITEM TASK PERSON CLIENT TABLES 101110011 b Specify this supplemental file in the main control file say MAIN CON A User s Guide to BIGSTEPS Page 83 Example 2 Example 3 Page 84 amp INST TITLE NEW YORK CLIENTS SPFILE SUPPL CON ITEM1 37 NI 100 The analyst has a set of control instructions that are used only for the final run These are coded in
163. ore S E standard error of the measure Statistics for this sample NORMED measures linearly rescaled so that the mean person measure for this sample is 500 and the sample measure standard deviation is 100 S E standard error of the normed measure FREQUENCY count of sample with measures at or near for missing data the complete test measure percentage of sample included in FREQUENCY CUM FREQ count of sample with measures near or below the test measure the cumulative frequency o percentage of sample include in CUM FREQ PERCENTILE mid range percentage of sample below the test measure constrained to the range 1 99 8 14 Table 20 3 Complete score to calibration table for tests based on whole sample This Table which must be selected explicitly with TFILE shows an estimated item calibration for all possible rates of success by persons i e the item difficulty corresponding to every observable p value for Page 104 A User s Guide to BIGSTEPS the entire sample To select this Table enter into your control file TFILE 20 3 x TABLE OF ITEM MEASURES ON COMPLETE SAMPLE FOR GROUP 0 MODEL R ACT NUMBER 1 WATCH BIRDS Fisa a ss Porta aE TEA A 0 8 71E 1 81 50 1 91 22 100 51 22 1 7 47 1 02 51 1 86 22 101 56 22 2 6 72 74 52 1 82 22 102 61 22 3 6 26 62 53 1 77 22 103 66 23 A n a Sooo see soe S 8 15 Table 21 Probability curves controlled by MRANGE CURVES The probabili
164. ormation weighting default N no eeeeeeeeseseeerseseseererreerreeeees 62 5 2 4 EXTRSC extreme score correction for extreme measures default 0 5 eeeceeseeteeeteeees 63 5 2 5 HIADJ correction for top rating scale categories default 0 25 eeeeseeseeeeeereeeeeeeeees 64 5 2 6 LOWADJ correction for bottom rating scale categories default 0 25 ci eeeeeeeseeseeeeeees 64 5 2 7 PAIRED correction for paired comparison data default N 0 0 eeesseeseeseeseeeeeeeeeeseeeeeeesees 64 DB SOF SCAM BG EA A E idee berkidtes 65 5 3 1 UIMEAN the mean or center of the item scale default 0 oo cece ceccesscetsceeceseeeeeenseeeeeenes 65 5 3 2 UPMEAN them mean of center of the person scale default not Used eeeeeeeereeeeees 65 5 3 3 USCALE the scale value of 1 logit default 1 ececeeceeseeteeeeceeceeeeseeseeeeceeeeaeeseeeeeeneeas 65 5 3 4 UDECIM number of decimal places reported default 2 oo eee eeeeeeeeeneeeeeeeeeeeeseeeeeeseees 65 5 3 5 UANCHOR anchor values supplied in user scaled units default Y ce eeeeeeeeseeeeeeeees 65 5 3 6 User friendly rescaling desistir ie Was E E cdi waits aie dina 66 6 OUTPUT CONTR OIDs ietettes tees ctteletictustectcateiatetestane e eia aa eisa an oa aa oae aaa aaia raa Nat eoit s 68 6 1 Overall reporti gooien n a a Ai a a R i aa aiie 68 6 1 1 TITLE title for output listing default control file name eseseseseseseeeeesseeeesrsesrsrersrrseessesees 68 6 1 2 TABLES
165. our screen In large data sets the smallest meaningful logit change in estimates A User s Guide to BIGSTEPS Page 61 may correspond to score residuals of several score points Example To set the maximum score residual when convergence will be accepted at 5 score points Your data consists of the responses of 5 000 students to a test of 250 items NI 250 250 items RCONV 5 score residual convergence at 5 score points 5 2 Estimate adjustments 5 2 1 REALSE inflate S E for misfit default N no misfit allowance The modelled REALSE N standard errors are the smallest possible errors These always overstate the measurement precision Controls the reporting of standard errors in all tables REALSE N Report modelled asymptotic standard errors the default REALSE Y Report the modelled standard errors inflated by the square root of the maximum of the infit mean square and its inverse This inflates the standard error to include uncertainty due to overall lack of fit of data to model 5 2 2 STBIAS correct for UCON estimation statistical bias default N no Other Rasch programs may or may not attempt to correct for estimation bias When comparing results from other programs try both STBIAS Y and STBIAS N to find the closest match The UCON unconditional maximum likelihood estimation algorithm produces estimates that have a usually small statistical bias This bias increases the spread of measures and calibrations but u
166. patterns XFILE MYDATA XF 6 4 9 GRFILE probability curve coordinate output file default no file If GRFILE filename is specified Table 21 is produced and a file is output which contains a list of measures Page 82 A User s Guide to BIGSTEPS x axis coordinates and corresponding expected scores and category probabilities y axis coordinates to enable you to use your own plotting program to produce plots like those in Table 21 This file contains 1 The rating scale table sub heading number I5 matches Table 21 2 The measure F7 2 rescaled by USCALE 3 Expected score F7 2 4 Probability of observing lowest category F7 2 5 etc Probability of observing higher categories F7 2 If CSV Y values are separated by commas When CSV T values are separated by tab characters Example You wish to write a file on disk called MYDATA GR containing x and y coordinates for plotting your own category response curves GRFILE MYDATA GR 6 4 10 Automating file selection Assigning similar names to similar disk files can be automated using DOS commands For example suppose you want to analyze your data file and always have your output file have suffix OUT the PFILE have suffix PF and the IFILE have suffix IF Key in your control file and data say ANALI omitting PFILE and IFILE control variables Then key in the following DOS Batch file called say MYBATCH BAT using EDIT or your word process
167. pectation or not Observations within 0 5 rating points of their expectation are deemed to be in their expected categories and are reported with their category values e g 1 2 etc These ratings support the overall inferential relationship between observations and measures Observations more than 0 5 rating points away from their expectations i e in a wrong category are marked with a letter equivalent A 0 B 1 C 2 etc These contradict observation to measure inferences The proportion of in and out of category observations are reported by the COHERENCE statistics in Table 3 A User s Guide to BIGSTEPS GUTTMAN SCALOGRAM OF ZONED RESPONSES 41 17 45 40 70 33 38 43 74 60 65 He PUPIL ACT 111111 2 1221 121 22 8901231125427634569780435 22222222222222222222222B2 22222222222222222222222BA 22222222222222222222CC1AA 2222222222222B222121A11AA 222222222222222BBCBCAACAA 4222222222222221C1A11CACAA 4 222222222222B21111111C11A 422222222222221C111CCAAAAL 4 2222222222B22CCCCCACAAAAA 4 222222B2222BB1C111C1111A1 4 222222122BA1111AACA1A110C 4 222222122CC11A1AA11CA010B 1111111221221631219782425 1890123 1 5427 456 03 Page 107 8 18 The title page This page contains the authorship and version information the TITLE control file output file and the date of the analysis A list of tables is provided to remind you of what you can request using TABLES Those you did request are marked by
168. pected observations These tables display the unexpected responses in Guttman scalogram format The Guttman Scalogram of unexpected responses shows the persons and items with the most unexpected data points those with the largest standardized residuals arranged by measure such that the high value observations are expected in the top left of the data matrix near the high and the low values are expected in the bottom of the matrix near the low The category values of unexpected observations are shown standardized residuals less than 121 are shown by Missing values if any are left blank MOST UNEXPECTED RESPONSES PUPIL 41 FATALE NATASHA 17 SCHATTNER GAIL 71 STOLLER DAVE 53 SABOL ANDREW MEASURE ACT 1111112 122 1 22 89203112542669784035 high ATE EER A eatin 1 SSOP cues na e eee ae cole 0 96 B 0 10 0 222 S159 Esters caeee tens S low 11111122122619784225 8920311 542 6 03 8 11 Table 10 Item Dependency Diagnosis 8 11 1 Table 10 6 Largest Residual Correlations for Items This Table shows items that may be locally dependent Specify PRCOMPER for score residuals or PRCOMPS3S for standardized residuals or PRCOMP L for logit residuals to obtain this Table Residuals are those parts of the data not explained by the Rasch model High correlation of residuals for two items indicates that they may not be locally independent either because they d
169. pm is always reported when CUTLO or CUTHI are specified DISPLACE approximates the displacement of the estimate away from the statistically better value which would result from the best fit of your data to the model Only meaningfully large values are displayed They indicate lack of convergence or the presence of anchored or targeted values The best fit value can be approximated by adding the displacement to the reported measure or calibration 8 7 1 Tables 6 1 10 1 Person and item fit selection controlled by FITI FITP These tables list the person measures and item calibrations in fit order Report measure in Tables 6 and 10 if any of Less than Greater than an 1 FITP or FITD 10 mean square OUTFIT 1 FITP or FITD 10 1 FITP or FITD 10 point biserial correlation negative To include every person specify FITP 0 For every item FITI 0 Tables 6 and 10 are sorted by Outfit by default or Infit if OUTFIT N 8 8 Tables 6 2 10 2 13 2 14 2 15 2 17 2 18 2 19 2 Person and item statistics controlled by USCALE UMEAN UDECIM LOCAL PUPIL FIT GRAPH OUTFIT ORDER E SSeS Seo See Sie Se ee ese ee eee Ss Sce se ENTRY MEASURE INFIT MEAN SQUARE OUTFIT MEAN SQUARE NUMBR 0 O27 1 1 3 210 0 7 1 1 3 2 PUPIL aasar 72 A JACKSON SOLOMON 47 l J VAN DAM ANDY 53 l K SABOL ANDREW 32 l w
170. r analysis has exceeded computational abilities reduce the size of Other codes your analysis Since these codes may indicate program malfunction please report them to www winsteps com Error Messages and Common Diagnoses 1 Message VARIABLE UNKNOWN OR VALUE INVALID WPC Diagnosis Your control file is in WordPerfect format The control file must be DOS Text or ASCII Return to WordPerfect and save your control file as a DOS Text file Then rerun 2 Message VARIABLE UNKNOWN OR VALUE INVALID L Diagnosis Your control file is in Word for Windows format The control file must be DOS Text or ASCII files Return to Word for Windows save your control file as MS DOS Text Then rerun 3 Message PROBLEM BLANK LINE NEEDED AT END OF filename Diagnosis The last line of file filename does not end with a line feed LF code This can be fixed by adding a blank line to the end of that file 4 Message BEYOND CAPACITY Diagnosis Your data file contains more person records than can be analyzed in one run Some records have been bypassed Data sets that exceed program to analyze in one run offer opportunities as well as challenges There are several strategies I Analyze a sample of the data Use this to produce anchor values Then using the anchor values run all the data one section at a time I Analyze a sample of the data Analyze another sample Compare results to identify instability and compute reasonable anchor value
171. r import into other software Measures are reported in Logits log odds units unless rescaled Fit statistics are reported as mean square residuals which have approximate chi square distributions These are also reported standardized N 0 1 References BTD means Wright B D amp Stone M H Best Test Design Chicago MESA Press 1979 RSA means Wright B D amp Masters G N Rating Scale Analysis Chicago MESA Press 1982 1 2 About the User s Guide You don t need to know about every BIGSTEPS option in order to use the program successfully Glance through the examples in Section 3 and find one similar to yours Adapt the example to match your requirements Then fine tune your analysis as you become familiar with further options 1 3 Type style Most of this Guide is in proportionately spaced type When it is important to be precise about blanks or spaces or about column alignment fixed space type is used when it is important to show everything that appears on a long line small type is used Suggestions that we have found helpful are shown like this 1 4 Getting further help BIGSTEPS is a powerful weapon in the struggle to wrest meaning from the chaos of empirical data As you become skilled in using BIGSTEPS you will find that it helps you to conceptualize what you are measuring and to diagnose measurement aberrations You may also find that you can use a word of advice on occasion The authors of BIGSTEPS Ben Wright
172. reatment of missing data default 255 ignore 0 eee eeeeeeeeeeceeceseceseeeeeseesseeseeesnaeeeaeee 31 MODELS assigns model types to items default R rating Scale ee eee eeseceseeeseeeseeeeeeseeeeeaeeeaeees 43 MODFRM location of MODELS default N before SEND cccccccessccceessceeessneeeesssneeeesseseeeeeees 47 MPROX maximum number of PROX iterations default 10 ccccccccessseccessseeeessneeeeeseneeeeeseseeeeeees 61 MRANGE half range of measures on plots default 0 auto size 0 ele eee esse ceeeceseeeseeeeeeeeeeeeaeeeaeee 75 MUCON maximum number of UCON iterations default 0 no Limit eee eeesseeceeeeteeeeeeeneeeeeees 61 NAME I first column of person id default 1 oo cee eeecccesseeceseeeeseeeeeaeeceeeceeaeeeseecesaeeceaeecssaeeeeeeeeaees 29 NAMLEN length of person id default calculated eee esesseeeseeeseeceseceseceseceseeeseeseesseessaeenaeeeaeees 30 NAMLMP id length for Tables 12 16 default calculated cee eesccesneceeneeceeeeceeeeeeeecesaeeeeneeeeaees 76 NEWSCORE recoding values with RESCORE default nON 0 0 eeeeeseeceeneeeeteeeeeneeeeneeeeenees 33 NI number of items required no default eee seeseecsecsseceeeeseesseecsseceseceaeceaeeseeeeesseeseaessaeeeaeees 27 NORMAL normal distribution for standardizing fit default N chi square es ceeeseeeeseeeeeeeereeeeee 73 OUTFIT sort misfits on infit or outfit default Y Outfit cecccccssnecccesseeecesssee
173. response scores and the total person or item test score less the individual response scores Extreme scores are omitted from the computation Negative values for items often indicate mis scoring or rating scale items with reversed direction Letters indicating the identity of persons or items appearing on the fit plots appear under PTBIS For adaptive tests an rppis near zero is expected The formula for this product moment correlation coefficient is L x x y y 2 2 Z f Ly y where x observation for this item or person y total score for person omitting this item or for item omitting this person F pois Conventional computation of fpbis includes persons with extreme scores These correlation can be Page 98 A User s Guide to BIGSTEPS obtained by forcing observations for extreme measures into the analysis The procedure is 1 Perform a standard analysis but specify PFILE and IFILE 2 Perform a second analysis setting PAFILE to the name of the PFILE of the first analysis and JAFILE to the name of the IFILE The PTBIS of this second analysis is the conventional rppi Which includes extreme scores RPM is reported instead of PTBIS when PTBIS N or PTBIS RPM is specified RPM is the point measure correlation rpm It is computed in the same way as the point bi serial except that Rasch measures replace total scores Since the point biserial loses its meaning in the presence of missing data r
174. rricula Item Similar items Redundant item One item answers another Item correlated with other variable Extreme category overuse Poor category wording Combine or omit categories Middle category overuse Wrong model for scale Processing error Scanner failure Person i Clerical error Form markings misaligned Idiosyncratic person Qualitatively different person Rating scale Unexpected wrong answers Unexpected errors at start Unexpected errors at end High Person Guessing Unexpected right answers Low Person Response set Systematic response pattern Special knowledge Content of unexpected answers Plodding Did not reach end of test Caution Only answered easy items Noisy Extreme category overuse Extremism Defiance Muted Middle category overuse Conservatism Resistance Apparent unanimity Collusion Person Judge Rating Judge Rating INFIT information weighted mean square sensitive to irregular inlying patterns OUTFIT usual unweighted mean square sensitive to unexpected rare extremes Muted unmodelled dependence redundance error trends Noisy unexpected unrelated irregularities See BTD p 2 4 66 82 23 24 121 125 165 190 RSA p 19 23 94 105 108 111 for basics RSA p 132 135 147 151 171 178 190 198 for examples A User s Guide to BIGSTEPS Page 113 Appendix 3 Diagnosis of Error Codes IOSTAT Error codes are reported for which no autom
175. s Remember that small random changes in item calibrations have negligible effect on person measures To select a sample of your data use the FORMAT statement See the example on pseudo random person selection A User s Guide to BIGSTEPS Page 117 Appendix 4 What is a Logit A logit log odds unit is a unit of interval measurement which is well defined within the context of a single homogeneous test When logit measures are compared between tests their probabilistic meaning is maintained but their substantive meanings may differ This is often the case when two tests of the same construct contain items of different types Consequently logit scales underlying different tests must be equated before the measures can be meaningfully compared This situation is parallel to that in Physics when some temperatures are measured in degrees Fahrenheit some in Celsius and others in Kelvin As a first step in the equating process plot the pairs of measures obtained for the same elements e g persons from the two tests You can use this plot to make a quick estimate of the nature of the relationship between the two logit scales If the relationship is not close to linear the two tests may not be measuring the same thing Logit to Probability Conversion Table Logit difference between ability measure and item calibration 5 0 4 6 4 0 3 0 2 2 2 0 14 1 1 1 0 0 8 0 5 0 4 0 2 0 1 0 Page 118 Probability of succes
176. s spreadsheet graphing or statistical programs 6 4 1 CSV comma separated values in output files default N no To facilitate importing the IFILE ISFILE PFILE SFILE and XFILE files into spreadsheet and database programs the fields can be separated by commas and the character values placed inside marks CSV N Use fixed field length format the default CSV Y or CSV Separate values by commas with character fields in marks 6699 CSV T Separate values by tab characters with character fields in marks Page 78 A User s Guide to BIGSTEPS 6 4 2 HLINES heading lines in output files default Y yes To facilitate importing the IFILE PFILE SFILE and XFILE files into spreadsheet and database programs the heading lines can be omitted from the output files HLINES Y Include heading lines in the output files the default HLINES N Omit heading lines 6 4 3 IFILE item output file default no file IFILE filename produces an output file containing the information for each item This file contains 4 heading lines unless HLINES N followed by one line for each item containing Columns Start End Format Description 1 1 Al Blank or if no responses or deleted status 2 3 2 6 I5 1 The item sequence number ENTRY 7 14 F8 2 2 Item s calibration rescaled by UMEAN USCALE UDECIM MEASURE 15 17 13 3 The item s status STATUS 1 Estimated calibration 2 Anchored fixed
177. s are 00 01 and 02 ITEM1 1 Items start in column 1 NI 25 25 Items NAMEI1 51 Person id starts in column 51 NAMLMP 20 lt Show 20 characters of id on maps TABLES 11111111111111111111111 CURVES 111 Print all curves in Tables 2 and 21 IFILE EXAM11 IF Output item measure file PFILE EXAM11 PF Output person measure file SFILE EXAM11 SF Output step calibration file RFILE EXAM11 RF Output reformatted response file XFILE EXAM11 XF Output observation and residual file UIMEAN 455 User scaling mean 455 USCALE 94 LINLEN 0 Print with minimum of split lines MAXPAG 0 Print with minimum of headings amp END WATCH BIRDS READ BOOKS ON ANIMALS FIND OUT WHAT FLOWERS LIVE ON TALK W FRIENDS ABOUT PLANTS END NAMES 01020101010002000102020202000201010202000201000200ROSSNER MARC DANIEL 02020202020202020202020202020202020202020202020202ROSSNER LAWRENCE F 02020200000202000002020202020202020202000202000102PATRIARCA RAY 01020000010100010102010202020201000202000200000100PAULING LINUS BLANK RECORD A User s Guide to BIGSTEPS Page 23 Example 12 Comparing high and low samples with rating scales Rasch estimates are constructed to be as sample independent as is statistically possible but you must still take care to maintain comparability of measures across analyses For instance if a rating scale is used and a high low ability split is made then the low rating scale categories may not appear in
178. s can be made into comments and so be ignored by entering a semi colon in column 1 e g FITP 3 is ignored 9 Comments can appear on the same line as a control variable as long as they are separated by at least one blank from the control value e g FITP 3 this is a comment or CODES ABCD12345 this is a comment there is a code of blank 10 When all control variables required or optional have been assigned values type amp END in upper or lower case on the next line e g amp INST This is optional Title A 30 Item test NI 30 this is a comment person names in columns 1 20 ITEM1 21 amp END 2 7 How long will an analysis take A PC with a math co processor processes about 1 000 000 observations per minute Most analyses have reached convergence within 20 iterations so a rule of thumb is length of analysis in minutes number of persons length of test 2 100 000 2 8 How big an analysis can I do The upper limit is 32 000 persons by 3 000 items Rating scale categories must be in the range 0 99 Since memory and disk space are assigned dynamically large analyses may cause a report of Not enough disk space or Not enough memory 2 9 If BIGSTEPS does not work 1 Repeat the installation process It will not delete any of your data files 2 Check that the BIGSTEPS runs 3 If the program will not run or produces implausible results a There are some computers in which the
179. s on a dichotomous item 99 99 98 95 90 88 80 75 73 70 62 60 55 52 50 Logit difference between ability measure and item calibration Probability of success on a dichotomous item 1 1 2 5 10 12 20 25 27 30 38 40 45 48 50 A User s Guide to BIGSTEPS Appendix 5 Rasch two facet measurement models in BIGSTEPS Observed Type of Data Categories Dichotomy no Rasch yes Rating scale Andrich Partial credit Masters Rank order Success Glas amp Verhelst Failure B is the person row ability D is the item column difficulty Ist item right Ist item wrong Ordinal Step Measurement Model Interpretation Score for Log Ppi Paij 1 less 0 B Di more 1 j l least i F j 1 m most highest i Fj j 1 m lowest first complete partial j 1 m last Ist item wrong i Fj 2nd tem wrong oe j 1 Mi mth item wrong mth item right mth item right i Fj m 1th item right j 1 m F is the additional difficulty of the step into category j from category j A User s Guide to BIGSTEPS Page 119 Appendix 6 Output Tables Table Description Control Variables and Convergence report Lists the control variables and shows the convergence of the PROX and UCON estimation procedures after each iteration Joint distribution map Horizontal histograms of person and item distributions Item and Pers
180. saeeaseasseees 13 Example 5 Scoring k ey t ritemS iraan a a a E a E ia 14 Example 6 Keys in data record FORMAT sseeeseseseseeseseseseresessseseststseststsretesesseeseststststseststetstesseeeeeseses 15 Example 7 A partial credit analysis 0 0 eee ssseseeceecseeseeeceeeseeseescessecsseaesesseesesaeeassessesaeeasassesaeeaseaseeees 17 Example 8 Items with various rating scale MOdelS eee esseeeeeseeseeseeeeeeseeseeecsesseeseeaeeesseeaeeasaseeens 18 Example 9 Grouping and modeling items 0 0 ieee esesseeeeseesecseeeeseesecseeaeeeseeseeseeassaesesaeeaseesseeaeeaseaseeens 19 Example 10 Combining tests with COMMON items cesses eseeeeeeeteeseeseeesseeseeseeasseesesseeaeeeseeeseeaeeaseeees 20 Example 11 Item responses two characters Wide 00 cesesesessessseseeeesecseeseeesseeseeseeessessesseeasesseeaeeaseaseeees 23 Example 12 Comparing high and low samples with rating Scales 0 0 eeeesesseeseeeseeeeseeeseeseeaeeeeseens 24 Example 13 Paired comparisons as the basis for measurement cee eeeeeseeseeeeeeeeeeseeeeeeeeeeseeaeeeeseees 26 READING YOUR DADA eeen a eaa Aaa eaa aa iaaa a naai iGo penaa 27 4 1 Specifying the layout of your data oo ec esseesecseeseeessessesseeeceessecseeaceessecsesaeeassassesaeeaseessesseeaseaseeees 27 4 1 1 DATA name of data file default data at end of control file cece eceeseeseeeseeeteeeteeeseees 27 4 1 2 ITEM1 column number of first response required no default 0 0 eee eseseeeeeete
181. scored on an individual test basis Also the validity of each test is to be examined separately Then one combined analysis is wanted to equate the tests and obtain bankable item difficulties For each file of original test responses the person information is in columns 1 25 the item responses in 41 60 The combined data file specified in EXAM10C CON is to be in RFILE format It contains Person information 30 characters always Item responses Columns 31 64 The identification of the common items is Test Item Number Location in item string Combined 6 20 A 2 4 6 10 20 B I From Test A make a response RFILE file rearranging the items with FORMAT This file is EXAM10A CON amp INST TITLE Analysis of Test A RFILE EXAMI0A RF The constructed response file for Test A NI 20 FORMAT 25A T43 A T41 A T47 3A T42 A T44 3A TSO0 11A ITEM 1 26 Items start in column 26 of reformatted record CODES ABCD Beware of blanks meaning wrong Use your editor to convert all wrong blanks into another code e g so that they will be scored wrong and not ignored as missing KEYFRM 1 Key in data record format amp END Key 1 Record CCBDACABDADCBDCABBCA BANK 1 TEST A 3 BANK 20 TEST A 20 END NAMES Person 01 A BDABCDBDDACDBCACBDBA Person 12 A BADCACADCDABDDDCBACA The RFILE file EXAMIOA RF is Person 01 A 00001000010010001001 Person 02 A 00000100001110100111 Person 12 A 00100001100001001011
182. seceececeeseeseceeeeeeeas 100 8 10 Tables 6 10 Unexpected Responses eeecssesssecsesesesseeseeessecseeseeeceeseeseeaseesseeseeaeeasesseeaeeaseeseees 100 8 10 1 Tables 6 4 10 4 Most Misfitting Response Strings 00 ieee eeseseeeeeeeeeeeeseeeseesseeaeeaseesseens 100 8 10 2 Tables 6 5 10 5 Unexpected observations ecieceeeseeseeseeeseeeseeseeseeecseesesseeaseessesaeeaseaseeens 101 8 11 Table 10 Item Dependency Diagnosis 000 ee eseesseesesseeseeeseeseeseeeceeseeseeaeesseeseeaeeessessesaeeaseeeeees 101 8 11 1 Table 10 6 Largest Residual Correlations for Items 0 ei eeeeeseeseeeeeeeseeseeeeeeeeeeaeeaeeeseeees 101 8 11 2 Table 10 7 Principal Components Analysis of Residuals cccseeeseseeseeseeeeeeeeeseeeseeees 102 8 11 3 Table 10 8 9 Principal Components Plots of Item Loading cee sseeeeeeeeeneeseeeeeeens 102 8 12 Tables 7 and 11 Misfitting responses 00 0 eee eeeeesesseeseeesseeseeseeecseeseesceasessesseeaeassesseeaeeaseeseees 103 8 13 Table 20 Complete score to measure table on test of all items oe eee eeeeeteeseeeeeeeteeseteeeeeeees 103 8 14 Table 20 3 Complete score to calibration table for tests based on whole sample 103 8 15 Table 21 Probability CUrVES cee eeeseeseessesseeseeesseesesseeseessesseeseeasseesesseeaeseesesseeaeeassessesaeeaseeseees 105 8 16 Table 22 1 Sorted observed data matrix scalogram ce eeesesseeeceeeeteeseceececeesetseceececeeaeeseeeeeeaeeas 107 8 17 Table 22 2 Guttman scalogram
183. seeeeesseeeesenees 62 STEPT3 include step summary in Table 3 or 21 default Y in Table 3 o e 76 STKEEP keep non observed steps categories default N nO eeseessessssresessrssrssresrrsresresressesressresersresees 44 T1I number of items summarized by symbol in Table 1 default auto size eee eee eeeeeeneee 76 T1P number of persons summarized by symbol in Table 1 default auto size eeeeeceeeeeneenee 76 TABLES output tables default 1110011001001000100000 ssssssssessessessrssrssrrsrssresressessessrssessresresss 68 TARGET estimate using information weighting default N no eeeeesseesesessserssresresresresresressrrsrrsresresee 62 TFILE input file listing tables to be output default none s eesseseessesresessssrseresresresresresresressresersresresee 70 TITLE title for output listing default control file name oe eee eeee cee ceeeceeeceaeeeseeeeeseeseeeesaeeeaeee 68 UANCHOR2 anchor values supplied in user scaled units default Y 0 ele eeeeesceseeeseeeeeeeeeeeeaeeeaeees 65 UDECIM number of decimal places reported default 2 oo esc eieescesseecseecseceseceseeeeeeeesseessaeesaeeeaeee 65 UIMEANs the mean or center of the item scale default 0 0 cece cccsscccesssseccessneeeeeseneeeesssneeeeeseeeeesees 65 UPMEAN them mean of center of the person scale default not used eee eeeeceeeneeeeteeeeeneeeeneeeeaees 65 USCALE the scale value of 1 logit default 1 0 cee eee ceess
184. sing data but without removing these responses from your data file is easily accomplished by creating a file in which each line contains if there is more than one group the sequence number of an item representing a group followed by a blank the number of the category to be deleted from that item and its group Specify this file by means of the control variable SDFILE or this information may be specified in the control file using SDFILE Since rating scales may be shared by groups of items deletion of categories is performed by group a If no GROUPS control variable is specified no item need be entered since specifying deletion of a category deletes that category for all items Page 56 A User s Guide to BIGSTEPS b If a GROUPS control variables is specified then specifying deletion of a category for one item deletes that category for all items in the same group c If GROUPS 0 is specified only the specified category for the specified item is deleted Example You wish to delete particular categories for the fifth and tenth partial credit items for this analysis 1 Create a file named say CAT DEL 2 Enter into the file the lines 53 item 5 category 3 10 2 item 10 category 2 3 Specify in the control file SDFILE CAT DEL GROUPS 0 or enter in the control file SDFILE 53 102 x GROUPS 0 4 6 2 SDELQU delete item step categories interactively default N no If your system is inter
185. specified record length ADVANCEZ specified for direct access or unformatted file NAMELIST name is longer than specified record length NAMELIST variable name exceeds maximum length PAD specified for unformatted file NAMELIST input contains multiple strided arrays Expected amp or as first character for NAMELIST input NAMELIST group does not match current input group Pointer or allocatable array not associated or allocated NAMELIST input contains negative array stride Runtime memory allocation fails 1 amp 6405 amp 6501 End of file has been reached unexpectedly Are all control variables item names 6100 6101 6103 6209 6311 6405 6413 6414 6416 6417 6421 6422 6501 6700 6701 and data records present e g END NAMES see Appendix 2 You are trying to set an integer control variable outside the range 32 676 An integer is expected but something else has been found Are there person item or category numbers in your deletion and anchor files A number is expected but something else has been found Are your anchor values specified correctly You assigned two files with the same name You are trying to process more than 32 676 persons See 1 above You have specified two different files with the same name e g your output file and your IFILE must have different names You tried to access a protected file or a directory index An input file cannot be found or an output file cannot be writte
186. sponds to the highest category The P 0 5 intercepts are the Thurstone thresholds THURSTONE THRESHOLDS MEDIANS Cumulative probabilities P 4 R 1 0 222222222222222 o 10 1111111 22222 B 8 0 111 222 A 10 111 22 B 6 0 11 22 I 0 111 222 4 0 11 22 I 10 11 222 T 2 0 111 222 Y 10 11111 2222222 0 0 ies es roa eG Be 4 5 4 3 2 1 0 1 2 3 4 5 PERSON MINUS ITEM MEASURE Page 106 A User s Guide to BIGSTEPS 8 16 Table 22 1 Sorted observed data matrix scalogram The observations are printed in order of person and item measures with most able persons listed first the left This scalogram shows the extent to which a Guttman pattern is easiest items printed on the approximated GUTTMAN SCALOGRAM OF RESPONSES 41 17 45 40 65 PERSON ITEM 1111112 1 221 1 21 22 8920311251427643569784035 2222222222222222222222212 2222222222222222222222210 2222222222222222222221200 2222222222222122212101100 2222222211011101020101122 2222222221211001011201001 1111112211221613219784225 18920311 5 427 4 56 03 8 17 Table 22 2 Guttman scalogram of zoned responses The scalogram is that of Table 22 1 but with each observation marked as to whether it conforms with its ex
187. statistics the natural logarithm of the mean square fit statistic is reported This is a linearized form of the ratio scale man square Columns reporting this option are headed LOG for mean square logarithm 6 2 6 PTBIS compute point biserial correlation coefficients default Y yes specify PTBIS N whenever missing data makes the conventional point biserial meaningless PTBIS Y Compute and report conventional point bi serial correlation coefficients rppis These are reported not only for items but also for persons In Rasch analysis fpbis is a useful diagnostic indicator of data miscoding or item miskeying negative or zero values indicate items or persons with response strings that contradict the variable A User s Guide to BIGSTEPS Page 73 PTBIS N or PTBIS RPM Compute and report point measure correlation coefficients rpm These are reported for items and persons They correlate an item s or person s responses with the measures of the encountered persons or items rpm maintains its meaning in the presence of missing data Negative or zero values indicate response strings that contradict the variable PTBIS N is set whenever CUTLO or CUTHI is specified 6 3 Special table control See TFILE options on p 70 for greater control over individual output tables 6 3 1 CATREF reference category for Table 2 default 0 item difficulty If a particular category corresponds to a criterion level of performance choose that
188. sually less than the standard error of measurement The bias quickly becomes insignificantly small as the number of persons and items increases For paired comparisons and very short tests estimation can double the apparent spread of the measures artificially inflating test reliability STBIAS Y causes an approximate correction to be applied to measures and calibrations A useful correction for bias is to multiply the usual measures by L 1 L where L is the smaller of the average person or item response count so for paired comparisons multiply by Example You have a well behaved test of only a few items for which you judge the statistical bias correction to be useful STBIAS Y 5 2 3 TARGET estimate using information weighting default N no TARGET Y lessens the effect of guessing on the measure estimates but increases reported misfit A big discrepancy between the measures produced by TARGET N and TARGET Y indicates much anomalous behavior disturbing the measurement process Unwanted behavior e g guessing carelessness can cause unexpected responses to off target items The effect of responses on off target items is lessened by specifying TARGET Y This weights each response by its statistical information during estimation Fit statistics are calculated as though the estimates were made in the usual manner Reported displacements show how much difference targeting has made in the estimates Page 62 A User s Guide to BIGS
189. t 4 2 columns per datum Example 3 Some responses take one column and some two columns in the data record Five items of l character width code a b c or d then ten items of 2 character width coded AA BB CC DD These are preceded by person id of 30 characters XWIDE 2 lt Format to two columns per response FORMAT 30A1 5A1 10A2 lt Name 30 characters 5 1 chars 10 2 chars CODES abcdAABBCCDD a becomes a NEWSCORE 1 2341234 response values RESCORE 2 rescore all items NAME1 1 person id starts in column 1 ITEM1 31 item responses start in column 31 NI 1I5 15 items all now XWIDE 2 4 1 5 INUMB name items by sequence numbers default N names after amp END Are item names provided or are they to default to sequence numbers INUMB Y a name is given to each item based on its sequence number in your data records The names are 10001 10002 and so on for the NI items This is a poor choice as it produces uninformative output INUMB N the default Your item names are entered by you after the amp END at the end of the control variables Entering detailed item names makes your output much more meaningful to you The rules for supplying your own item names are 1 Item names are entered one per line generally directly after amp END see p 112 2 Item names begin in column 1 3 Only the first 30 characters or ITLE
190. t calculation using only the reduced set of responses Example Eliminate responses where examinee ability is 3 or more logits higher than item difficulty to eliminate careless wrong responses CUTHI 3 This produces a scalogram with eliminated responses blanked out RESPONSES SORTED BY MEASURE KID ITAP 111111111 123745698013245678 a 15 111 11100000 observations for extreme scores remain 14 111 1110000000 28 111 111010000000 30 1111 1111000000000 27 111111100000000000 4 2 14 CUTLO cut off responses with low probability of success default 0 no Use this if guessing or response sets are evident CUTLO cuts off the bottom right hand corner of the Scalogram in Table 22 Eliminates cuts off observations where examinee ability is CUTLO logits or more rescaled by USCALE lower than item difficulty so that the examinee has a low probability of success The Page 42 A User s Guide to BIGSTEPS elimination of off target responses takes place after PROX has converged After elimination PROX is restarted followed by UCON estimation and point biserial and fit calculation using only the reduced set of responses Example Disregard responses where examinees are faced with too great a challenge and so might guess wildly i e where examinee ability is 2 or more logits lower than item difficulty CUTLO 2 4 3 Specifying the structure of rating scales 4 3 1 MODELS assigns model types to items default R
191. ta is entered as only two observations in each row or each column The raw score of every row or column is identical In the simplest case the winner receives a 1 the loser a 0 and all other column or rows are left blank indicating missing data Example Data for a chess tournament is entered Each row is a player Each column a match The winner is scored 2 the loser 0 for each match For draws each player recieves a 1 PAIRED YES paired comparisons CODES 012 valid outcomes NI 56 number of matches 5 3 User scaling The conventional unit for Rasch analysis is the Logit log odds unit The conventional origin is set at the center of the item difficulties UIMEAN 0 USCALE 1 You can recenter and rescale the measures calibrations and standard errors from logits into some other unit The rescaled values are reported in the tables and in the IFILE ISFILE PFILE and SFILE files Measures in anchor files must also be rescaled Page 64 A User s Guide to BIGSTEPS 5 3 1 UIMEAN the mean or center of the item scale default 0 Assigns your chosen numerical value to the center the scale for items Previous UPMEAN values are ignored Example You want to recenter the item difficulties at 10 logits and so add 10 logits to all reported measures to avoid reporting negative measures for low achievers UIMEAN 10 5 3 2 UPMEAN them mean of center of the person scale default not used Assigns your chosen numerica
192. tems 4 and 8 BIGSTEPS asks you DO YOU WANT TO ANCHOR ANY ITEMS respond YES Enter A User s Guide to BIGSTEPS Page 51 DO YOU WISH TO READ THE ANCHORED ITEMS FROM A FILE respond NO Enter INPUT ITEM TO ANCHOR 0 TO END respond 4 Enter the first item to be anchored INPUT VALUE AT WHICH TO ANCHOR ITEM respond 1 45 Enter the first anchor value INPUT ITEM TO ANCHOR 0 TO END 8 Enter INPUT VALUE AT WHICH TO ANCHOR ITEM 0 23 Enter INPUT ITEM TO ANCHOR 0 TO END 0 Enter to end anchoring 4 5 Deleting or anchoring persons 4 5 1 PDFILE name of person deletion file default no file Deletion or selection of persons from a test to be analyzed but without removing their responses from your data file is easily accomplished by creating a file in which each line contains the sequence number of a person or persons to be deleted or selected according to the same rules given under IDFILE and then specifying this file by means of the control variable PDFILE or enter the deletion list in the control file using PDFILE Example 1 You wish to delete the fifth and tenth persons from this analysis 1 Create a file named say PERSON DEL 2 Enter into the file the lines 5 10 3 Specify in the control file PDFILE PERSON DEL or enter directly into the control file PDFILE 5 10 x Example 2 The analyst wants to delete the most misfitting persons reported in Table 6 1 Set up a stand
193. the data for the high ability sample and vice versa To compare item calibrations for the two samples requires the rating scale to be calibrated on both samples together and then the scale calibrations to be anchored for each sample separately Comparison of patient measures from separate analyses requires both the rating scale calibrations and the item calibrations to share anchor calibrations 35 arthritis patients have been through rehabilitation therapy Their admission to therapy and discharge from therapy measures are to be compared They have been rated on the 13 mobility items of the Functional Independence Measure FIM Each item has seven levels At admission the patients could not perform at the higher levels At discharge all patients had surpassed the lower levels Data courtesy of C V Granger amp B Hamilton UDS A generic control file is in EXAM12 CON The admission ratings are in EXAM12 LO and the discharge ratings in EXAM12 HI Three analyses are performed 1 joint analysis of the admission low and discharge high data to obtain rating scale calibrations 2 amp 3 separate runs for the admission low and discharge high data to obtain item calibrations This common control file is EXAM12 CON amp INST TITLE GENERIC ARTHRITIS FIM CONTROL FILE ITEM 1 7 Responses start in column 7 NI 13 13 mobility items CODES 1234567 7 level rating scale amp END A EATING B GROOMING C BATHING D UPPER BODY
194. the location of the mean measure S markers are placed one standard deviation away from the mean Q markers are placed two standard deviations away A User s Guide to BIGSTEPS Page 89 Look for the hierarchy of item names to spell out a meaningful construct from easiest at the bottom to hardest at the top KIDS MAP OF TAPS 120 FIND BOTTLES AND CAN 110 E Q Q WATCH A RAT 100 gt X 90 XXX XXX LOOK IN SIDEWALK CRA WATCH BUGS 80 XXXXX S S WATCH GRASS CHANGE XXX 70 XXXX XXXXX LEARN WEED NAMES WATCH ANIMAL MOVE 60 XXXXX MAKE A MAP XXX M LOOK AT PICTURES OF LOOK UP STRANGE ANIM TALK W FRIENDS ABOUT 50 XXXXXXXXXXXX M READ BOOKS ON PLANTS XXXXX FIND OUT WHAT ANIMAL WATCH WHAT ANIMALS E 40 XXXX FIND OUT WHAT FLOWER WATCH BIRDS XXXXXXXX S READ ANIMAL STORIES 8 3 Table 2 Most probable response expected score Thurstone threshold plots controlled by MRANGE CATREF CURVES Each plot answers a different question What category is most likely The maximum probability mode plot What is the average or expected category value The expected score mean plot What part of the variable corresponds to the category The Thurstone median plot The left side of this table lists the items in descending order of difficulty Anchored items are indicated by an between the sequence number and name A particular category can be used as the reference for sorting the items by specifying the CATREF variable
195. thm for precise estimates ITERATION number of times through your data to calculate estimates It is unusual for more than 100 iterations to be required MAX SCORE RESIDUAL maximum score residual difference between integral observed core and decimal expected score for any person or item estimate used to compare with RCONV This number is expected to decrease gradually until convergence acceptable indicates to which person or item the residual applies MAX LOGIT CHANGE maximum logit change in any person or item estimate used to compare with LCONV This number is expected to decrease gradually until convergence is acceptable LEAST CONVERGED element numbers are reported for the person item and category farthest from meeting the convergence criteria indicates whether the person or the item is farthest from convergence CATEGORY RESIDUAL maximum count residual difference between integral observed count and decimal expected count for any scale category for your information This number is expected to decrease gradually Values Page 110 A User s Guide to BIGSTEPS less than 0 5 have no substantive meaning STEP CHANGE maximum logit change in any step calibration Not used to decide convergence but only for your information This number is expected to decrease gradually Standardized Residuals These are modeled to have a unit normal distribution Gross departures from mean of 0 0 and standard deviation of 1 0 indicate that
196. three categories 0 1 2 you want to assign a middle code of 1 to missing values MISSING 1 missing responses scored 1 Example 3 You want blanks to be treated as wrong answers but other unwanted codes to be ignored items on a questionnaire with responses Y and N CODES YN blank included as valid response NEWSCORE 100 new response values RESCORE 2 rescore all items MISSING 255 ignore missing responses default Example 4 Your optical scanner outputs an if two bubbles are marked for the same response You want to ignore these for the analysis but you also want to treat blanks as wrong answers CODES 1234 blank is the fifth valid code KEY1 31432432143142314324 correct answers MISSING 255 applies to default Example 5 Unexpected codes are scored wrong but 2 s to mean not administered CODES 012 NEWSCORE 01X X is non numeric matching 2 s ignored MISSING 0 all non CODES responses scored 0 4 2 3 RESCORE response recoding with NEWSCORE or KEYn default The responses in your data file may not be coded as you desire The responses to some or all of the items can be rescored or keyed using RESCORE RESCORE and NEWSCORE are ignored when KEYn is specified except as below RESCORE or 2 or is omitted All items are recoded using NEWSCORE RESCORE 2 is the default when NEWSCORE is specified RESCORE some combinatio
197. ting a file from SPSS the syntax is FORMATS ITEM ITEM2 ITEM3 F1 i e FORMATS varlist format varlist The procedure is FORMATS and then the variable list Enclosed in parentheses is the format type F Page 6 A User s Guide to BIGSTEPS signifies numeric while 1 signifies the width F2 would signify a numeric with a width of 2 columns for XWIDE 2 See pages 216 and 217 of the SPSS Reference Guide 1990 2 6 3 The control file The control file tells what analysis you want to do The easiest way to start is to look at one of the examples in the next section of this manual or on the program diskette The control file contains control variables These are listed in the index of this manual Only two control variables must have values assigned for every analysis NI and ITEM1 Almost all others can be left at their automatic default values which means that you can defer learning how to use most of the control variables until you know you need to use them When in doubt don t specify control variables then they keep their default values 2 6 4 Syntax rules for assigning values to control variables key words Do not worry about these unless BIGSTEPS does not respond to your control file the way you expected If possible compare your control file with what is shown in Table 0 of your output file in order to isolate the problem 1 Values are assigned to control variables by typing the name of the control variable or enough of
198. to print your output file on standard paper with 60 lines per page pages are 11 inches long less 1 inch for top and bottom margins at 6 lpi MAXPAG 60 set 60 lines per page FORMFD default Word Processor form feed 6 1 5 ITEM title for item names default ITEM Up to 6 characters to use in table headings to describe the kind of items e g ITEM MCQ Choose a word which makes its plural with an s e g MCQS since an S is added to whatever you specify 6 1 6 PERSON2 title for person names default PERSON Up to 6 characters to use in table headings to describe the persons e g PERSON KID Choose a word which makes its plural with an s e g KIDS 6 1 7 ASCH output only ASCII characters default Y yes Tables include graphic characters such as and which some printers can t print These graphics characters can be replaced by the ASCII characters and ASCI N use graphics characters ASCI Y replace graphics characters with ASCII characters the default Example ASCIEN produces what follows or else accented letters e g aaa OVERVIEW TABLES ITEM CALIBRATIONS 1 PERSON AND ITEM DISTRIBUTION MAP 12 ITEM MAP BY NAME ASCI Y always produces Fa celal aaa i mii en alanine OVERVIEW TABLES ITEM CALIBRATIONS Oe a Ee ees pee E ae ee eR a eee 1 PERSON AND ITEM DISTRIBUTION MAP 12 ITEM MAP BY NAME A User s Guide to BIGSTEPS Page 69 TFILE Parameters enter
199. tom category When all are anchored enter 0 to end INPUT AN ITEM REPRESENTING A GROUP 0 TO END 0 Page 60 A User s Guide to BIGSTEPS 5 ANALYSIS CONTROL 5 1 Convergence control 5 1 1 MPROX maximum number of PROX iterations default 10 Specifies the maximum number of PROX iterations to be performed PROX iterations will always be performed so long as inestimable parameters have been detected in the previous iteration because inestimable parameters are always dropped before the next iteration At least 2 PROX iterations will be performed PROX iteration ceases when the spread of the persons and items no longer increases noticeably The spread is the logit distance between the top 5 and the bottom 5 persons or items If you wish to continue PROX iterations until you intervene with Ctrl and S set MPROX 0 UCON iterations will then commence Example To set the maximum number of PROX iterations to 20 in order to speed up the final UCON estimation of a symmetrically distributed set of parameters MPROX 20 5 1 2 MUCON maximum number of UCON iterations default 0 no limit UCON iterations may take a long time for big data sets so initially set this to 1 for no UCON iterations Then set MUCON to 10 or 15 until you know that more precise measures will be useful Specifies the maximum number of UCON iterations to be performed Iteration will always cease when both LCONV and RCONV criteria have been met To specify no maxi
200. trol file MYFILE OUT as the output file xi Look at the BIGSTEPS output file using your word processor or text editor Be prepared to rerun BIGSTEPS several times before you get results that make good sense 2 4 Starting BIGSTEPS in DOS A typical analysis requires two components control information and data These can be in separate computer files or can be combined in one file The results of the analysis are written to an output file on disk To change to directory BIGSTEPS at the DOS prompt type C gt CD BIGSTEPS Press Enter Key To launch BIGSTEPS C BIGSTEPS gt BIGSTEPS Enter Now BIGSTEPS asks you for the names of your input and output files C gt BIGSTEPS Enter Please enter name of BIGSTEPS control file KCT DAT Enter Please enter name of report output file NEWFILE Enter No check is done as to whether an output file already exists If there is already a file with the same name as your output file it will be overwritten You can streamline starting BIGSTEPS Enter BIGSTEPS then the name of the control file C gt BIGSTEPS KCT DAT Enter Please enter name of report output file NEWFILE Enter Or enter BIGSTEPS then the names of your control and output files C gt BIGSTEPS CONTROL OUTPUT Press Enter Key where CONTROL is the name of the input file containing your control specifications and data or data location and OUTPUT is the name of the file to hold your output report Example C gt BIGSTEPS SF DA
201. ty of each response is shown across the measurement continuum The measure to be used for determining the probability of any particular response is the difference between the measure of the person and the calibration of the item For dichotomies only one curve is shown plotting the probability of scoring a 1 correct and also of scoring a 0 incorrect for any measure relative to item difficulty For S and F models these curves are approximations DICHOTOMOUS CURVES P 4 77 SSSSS H45 PER e 7 pepe SSS ras R 1 0 0000000000000000 1111111111111111 o 000000 111111 B 8 000 111 A 00 11 B 6 00 11 I kik L 4 11 00 I 11 00 T 2 111 000 Y 111111 000000 0 1111111111111111 0000000000000000 4 4 4 4 4 4 6 4 2 0 2 4 6 PERSON MINUS ITEM MEASURE When there are more than two categories the probability of each category is shown For scales with three or more categories two further graphs can be drawn The second graph depicts the expected score ogive The vertical characters correspond to integer expected scores and the P characters correspond to half score point expected scores For the purposes of inference measures in the zone on the x axis between l and l correspond on average to the rating given on the y axis 1 Similarly ratings on the y axis can be thought of as corresponding to me
202. ub table and other control parameters separated by blanks or commas Unused control values are specified with The list may be entered directly into the control file with TFILE see Example 2 Page 70 A User s Guide to BIGSTEPS Example 1 The analyst wishes to select and print several tables TFILE TABLES TF TABLES TF is a DOS ASCII file with the following lines Table Low High Columns number Range Range per Unit 2 print Tables 2 1 2 2 2 3 10 2 0 5 1 5 print Table 10 2 with fit bars at 0 5 and 1 5 8 5 5 print Table 8 with range 5 to 5 logits 9 2 7 10 range 2 to 7 logits 10 columns per logit 9 5 5 10 print Table 9 again different range 15 4 print Table 15 sorted on column 4 of item name or enter directly into the control file TFILE 2 10 2 8 55 9 2710 9 5510 15 4 x Example 2 Analyst wishes to specify on the DOS control line Table 15 sorted on item name column 4 Values are separated by commas because blanks act as end of line separators C gt BIGSTEPS SF DAT SF OUT TFILE 15 4 6 2 Misfit selection Rasch measurement does not make any presumptions about the underlying distribution of the parameters Maximun likelihood estimation expects errors in the observations to be more or less normally distributed around their expected values Since all observations are integral values this expectation can be met only asymptotically as the number of persons and items be
203. ucture The principal components analysis detects local patterns in inter item correlations based on residuals or their transformations Letters C I etc relate items to misfit plots and reports PRINCIPAL COMPONENT ANALYSIS OF LOGIT RESIDUAL CORRELATIONS FOR ACTS INFIT OUTFIT ENTRY FACTOR LOADING MEASURE MNSQ MNSQ NUMBER ACT ETETEN 1 84 2 13 2 41 4 09 A 23 WATCH A RAT l 1 82 1 79 1 33 1 82 C 20 WATCH BUGS 1 74 2 36 2 30 3 60 B 5 FIND BOTTLES AND CANS to Sa ala E 1 42 1 45 78 57 k 10 LISTEN TO BIRD SING 1 41 1 26 1 22 94 I 13 GROW GARDEN l A A A A AE A 2 l 52 83 84 65 j 21 WATCH BIRD MAKE NEST A A E A E EE l 2 47 69 1 18 1 17 E 9 LEARN WEED NAMES 2 40 41 82 75 g 14 LOOK AT PICTURES OF PLANTS E A eae a oe ee E E 8 11 3 Table 10 8 9 Principal Components Plots of Item Loadings These plots show the factor structure by plotting the linearized Fisher z loading on each factor against the item calibration A random pattern with few high loadings is expected Letters C T etc relate items to misfit plots and reports In this example based on the LFS data the second factor reflects idiosyncratic choice of the hard to like items A H PRINCIPAL COMPONENTS LOGIT RESIDUAL MISFIT FACTOR PLOT 4 3 2 1 0 1 2 3
204. unused parameters with 1 Table number 2 5 7 subtable Distribution Lowest Highest Rows per unit Persons Items per map measure measure per H 1 0 1 1 Response Lowest Highest Columns per Reference category High Low plot measure measure marked division for sorting rating rating 2 adjust adjust ment ment Person fit Lowest Highest Columns per plots measure measure marked division 4 5 Person Item Low fit High fit list 6 10 bar bar 13 14 15 17 18 19 Item fit plots Lowest Highest Columns per 8 9 measure measure marked division Item map Lowest Sort column within Items per 12 1 2 measure Item list alphabetical 15 Person map Lowest measure Person list alphabetical 19 Score table 20 Lowest measure Category curves 21 Highest Rows per unit measure Highest measure Rows per unit Columns per marked division Highest measure Columns per marked division item name H Sort column within item name Persons per Sort column within person name Sort column within person name 6 1 8 TFILE input file listing tables to be output default none TABLES selects the tables in a fixed sequence and prints only one copy TFILE allows the analyst to print multiple copies of tables to different specifications TFILE specifies the name of an input ASCII file Each line of the file contains a table number or table number s
205. uplicate some feature of each other or because they both incorporate some other shared dimension Letters C T etc relate items to misfit plots and reports chao eR hat ibaa 0 RESIDUL ENTRY CORRELN NUMBER Site pee ea 76 C 20 62 I 13 57 M 2 56 1 12 55 M 2 541K 4 52 M 2 50 k 10 aha nes pane 47 M2 arent eaten es LARGEST LOGIT RESIDUAL CORRELATIONS USED TO IDENTIFY DEPENDENT ACTS ENTRY ACT NUMBER ACT ee Ste ae eS ae ee S WATCH BUGS A 23 WATCH A RAT GROW GARDEN G 19 GO TO ZOO READ BOOKS ON ANIMALS d 11 FIND WHERE ANIMAL LIVES GO TO MUSEUM F 18 GO ON PICNIC READ BOOKS ON ANIMALS e 15 READ ANIMAL STORIES WATCH GRASS CHANGE Ic 20 WATCH BUGS READ BOOKS ON ANIMALS j 21 WATCH BIRD MAKE NEST LISTEN TO BIRD SING G 19 GO TO ZOO Be ES PS 2D BS eee ee EE spe Fe So NS a ars en READ BOOKS ON ANIMALS A 23 WATCH A RAT 8 11 2 Table 10 7 Principal Components Analysis of Residuals This Table decomposes the matrix of item correlations based on residuals to identify possible other factors dimensions that may be affecting response patterns Specify PRCOMPSS or R or L to obtain this Table A User s Guide to BIGSTEPS Expected values with Page 101 The first factor dimension is that reported in Table 1 as the Rasch dimension Residuals are those parts of the observations not explained by the Rasch dimension According to Rasch specifications these should be random and show no str

Download Pdf Manuals

Related Search

Related Contents

ダウンロード - ソニー製品情報  AEROMOTIVE Part # 18651 & 18652 Stealth Fuel Tank Sump    Piano della sicurezza  2014年10月号（No.91） [15104KB pdfファイル]  Samsung ES73 Felhasználói kézikönyv  Generac 004721-0, 004722-0, 004723-0, 004724-0, 004725-0, 004725-1, 004725-2, 004725-3, 004726  トリートメントテーブル  04 ISC 161 Manual Tecnico  Manual B-UPR 504-754 Noviembre 2012  

Copyright © All rights reserved. Failed to retrieve file