Home

User manual for Meta-Essentials: Workbooks for meta

1. 45 User manual for Meta Essentials 5 Adapting plots for reporting 5 Adapting plots for reporting In Meta Essentials extensive use is made of the graphical displays available in Microsoft Excel In order to fully benefit from these displays in a report it might be useful to edit them For instance in a publication grey scaled figures might be preferred In Excel it is fairly simple to make changes according to one s preferences Here a number of recommendations are discussed First it is recommended to edit the displays in Excel before copying them to a word processing program In Excel 2010 and later a Chart Tools function will appear when a display is left clicked Features such as colours properties of the axes size titles and labels can be changed By right clicking on a chart and then selecting Select data it is possible to change the items that are displayed in a graph By default the effect sizes of individual studies with their confidence intervals and the combined effect size with its confidence interval and prediction interval are displayed One can adapt the size of the forest plot on the Forest Plot sheet as well as on the Subgroup Analysis sheet These charts show 200 studies by default which is the current maximum number of studies that can be entered into a meta analysis in Meta Essentials Usually a large part of the graph will be empty It is recommended to change the axis and size of the
2. No of a combination that is selected in the Valid options chosen row see Figure 47 36 User manual for Meta Essentials 4 Specific features of individual workbooks Meta analysis model options Weighting method Inverse variance Model Effect Size Measure Risk Ratio Presentation Effect Size Measure Risk Ratio Valid options chosen Yes Presentation Sort by Entry number Order Ascending Figure 47 Example of additional selection options on the Forest Plot sheet of Workbook 2 Differences between independent groups binary data xlsx 41 2 3 Statistical procedures Some non standard solutions are used in this workbook for conversion of statistics for Odds Ratio to statistics for Risk Difference particularly for the standard error which affects the calculation of the confidence and prediction interval The basic premise of this procedure is that the statistical significance of the various statistics is equal See the working paper on the website for this method by Van Rhee amp Suurmond 2015 Note that not all the heterogeneity measures are scale free and that they are based on the effect size measure of the model not the effect size measure of the presentation This means that the scale of the heterogeneity measures depends on the choice of the effect size measure in the model 4 1 3 Subgroup Analysis sheet In the Subgroup Analysis it is not possible to make separate choices of effect size measure for the model and t
3. Egger Regression Estimate SE CI LL Cl UL Intercept 9 09 9 61 30 23 12 05 Slope 2 95 1 93 1 30 7 19 ttest 0 95 p value 0 366 Figure 21 Example of Egger regression part of the Publication Bias Analysis sheet The Begg and Mazumdar rank correlation test uses the correlation between the ranks of effect sizes and the ranks of their variances Begg amp Mazumdar 1994 p 1088 This sheet presents a the difference between concordant and discordant ranks Ax y b the rank correlation Kendall s Tau a c a z value as well as d a p value for this correlation see Figure 22 for an example Begg amp Mazumdar s rank correlation test Ax y 5 Kendall s Tau a 0 08 z value 0 34 p value 0 366 Figure 22 Example of Begg and Mazumdar s rank correlation test part 3 5 3 Standardized Residual Histogram The Standardized Residual Histogram is based on the idea that the z scores of individual studies also known as standardized residuals are expected to follow a normal distribution around the combined effect size Sutton et al 2000 p 41 To assess whether there are outliers in the effect sizes one could put the residuals in bins and plot them against a standard normal distribution The standardized residuals are arranged in 9 bins and the proportion of residuals in that bin determines the height of the bar see Figure 23 for an example 21 User manual for Meta Essentials 3 Work with the workbooks Standardized Residual H
4. outcome Table 1 Overview of the Meta Essentials workbooks Workbook 1 Effect size data xlsx can be considered the generic one This workbook can be used when the user has 1 the point estimate of the effect size and 2 its standard error The effect sizes of the different studies must be comparable or in other words they must be sizes on the same scale Workbook 1 can only be used for effect sizes on a continuous scale on which the intervals have the same weight or meaning at every point on the scale This scale can be an unstandardized one such as millimetres minutes grams dollars regression weights etc or a standardized one Cohen s d Hedges g Intervals between standardized regression weights and between correlation coefficients are not the same in this sense and hence the generic workbook 1 cannot be used for meta analysing that type of effect size Workbooks 5 6 and 7 can be used for meta analysing correlation coefficients and results of a multiple regression analysis Workbooks 2 to 7 are basically extended versions of workbook 1 They perform calculations and transformations that precede the meta analysis proper These include calculations of effect sizes of studies that do not report them and transformations of effect sizes to more suitable scales Each one of the workbooks 2 to 7 does this for a specific type of effect size To decide which workbook you should use you must first determine whether your effect size
5. 120 120 Yes AA 22 11 kkkk Yes 37 32 45 45 Yes BB 17 12 111 Yes 8 42 5 45 Yes BB 18 Figure 45 Input sheet of Workbook 2 Differences between independent groups binary data xlsx 4 1 2 Forest Plot sheet A unique feature of this workbook is an additional forest plot that presents the effect sizes on a logarithmic scale see red rectangle in Figure 46 for an example Note that the lowest value on the x axis shows 0 13 instead of 0 125 because of rounding This makes it easier to interpret the results of the meta analysis when the odds ratio or risk ratio is selected as the effect size measure It is recommended to always use the logarithmic forest plot for the presentation of a meta analysis of odds ratios or risk ratios and to use the normal forest plot for risk differences only 35 User manual for Meta Essentials 4 Specific features of individual workbooks Effect Size 0 13 0 25 0 50 1 00 2 00 4 00 8 00 16 00 32 00 64 00 r T T T T T T T T T 1 H e 1 2 H gt 1 3 H e 4 H e i 5 e 6 f o 1 7 e 1 8 H e 1 9 H D 1 10 H e 11 H 12 e 13 o _ JE 14 Figure 46 Logarithmic forest plot in Forest Plot sheet of Workbook 2 Differences between independent groups binary data xlsx 41 21 Weighting methods The user can choose between three weighting methods the standard inverse variance method the Mantel Haenszel method Mantel amp Haenszel 1959 or the Peto Odds
6. 14 7 8888 Yes 120 0 46 0 50 Yes AA 19 8 hhhh Yes 130 0 50 0 70 Yes AA 13 9 iiii Yes 80 0 52 0 40 Yes BB 19 10 jjjj Yes 240 0 48 2 10 Yes AA 22 11 kkkk Yes 90 0 41 0 40 Yes BB 17 12 1111 Yes 97 0 53 0 50 Yes BB 18 Figure 56 Input sheet of Workbook 4 Differences between dependent groups continuous data xlsx Note the difference between the columns in the middle parts of these two figures which represents the difference in study design 4 2 1 1 Sufficient data Possible sufficient options are amongst others studies mentioned refer to Figure 55 Workbook 3 e Means standard deviations and sample sizes for both groups o Mh Ma S S2 nj and nz as in study aaaa e Means for both groups the pooled standard deviation and sample sizes for both groups o Mmi Ma Spooled N1 n2 study bbbb e The difference in means and the standard deviations and sample sizes of both groups o Mz Mi 1 S2 n1 and nz study cccc e The difference in means the pooled standard deviation and sample sizes o Mz Mi Spooled 1 and nz study dddd e t value and sample sizes o t value n and nz study eeee e F value and sample sizes o F value n and nz study gggg e One of the effect sizes directly along with sample sizes o Cohen s d OR Hedges g with n and nz studies kkkk and iiii respectively Note that in each option the sample sizes of both groups are required input As a comparison between Figure 55 and Figure 56 shows exactly
7. 49 68 15 25 0 004 73 77 0 01 0 10 0 63 0 19 Combined effect size Combined effect size 1 44 3 22 362 77 0 000 95 31 1 28 1 13 3 14 4 92 Figure 11 Example of Table with studies and subgroups of the Subgroup Analysis sheet Furthermore two types of forest plots are available one with studies subgroups and combined effect see Figure 12 and one with subgroup and combined effects only which enhances the comparison of subgroups see Figure 13 In these plots blue dots represent individual studies red dots represent subgroups and the green dot represents the combined effect size Also the prediction 14 User manual for Meta Essentials 3 Work with the workbooks intervals are shown for the subgroups and combined effect size in their respective colours whereas the confidence interval is shown in black Note that because the confidence interval of the first subgroup in the example of Figures 9 and 10 is so small that it disappears almost entirely behind the red dot Effect Size 3 00 2 00 1 00 0 00 1 00 2 00 3 00 4 00 5 00 6 00 hb 8 i OoOn nun fF WN O H 1 Figure 12 Example of Forest plot with studies and subgroups part of the Subgroup Analysis sheet Effect Size 4 00 3 00 2 00 1 00 0 00 1 00 2 00 3 00 4 00 5 00 6 00 0 1 2 S 3 nn eee 90 0 Figure 13 Example of Forest plot with subgroups part of the Subgroup Analysis sheet 3 3 1 Options The user must choose how to distribute weights to stu
8. 6 ffff Random effects model 9596 221 2 43 2 33 0 45 8 3596 8 17 8 49 8 35 1 39 1 37 1 77 0 35 onfidence level ort By Entry number Order Ascending 7 gees 2 00 Combined Effect Size 8hhhh 180 1 39 2 21 8 35 Effect Size 107 9 iii 0 40 0 05 0 85 8 30 Standard error 034 10 jjj 240 178 242 845 A Lowerlimit 0 33 11 kkkk 0 40 0 82 0 02 8 33 Q Upper limit 180 12 1111 0 50 0 90 0 10 8 36 PI Lowerlimit 1 56 13 PI Upper limit 3 69 14 Z value 15 16 One tailed p value 17 Two tailed p value 0 001 18 19 Number of incl studies 12 20 21 Heterogeneity 2 a 36277 23 Pa 0 000 24 IE 097 25 Le 131 26 T 114 27 Figure 6 Example of the Forest Plot sheet 3 2 Forest Plot sheet The Forest Plot sheet which you can open by clicking on the regarding tab as shown in Figure 7 consists of three parts On the left side a number of statistics is presented that are generated by Meta Essentials Four important pieces of information are a the combined effect size b the lower and upper limits of its confidence interval Cl c the lower and upper limits of its prediction interval PI and d several heterogeneity statistics In the middle a table is given with the individual study results see the red rectangle labelled Table in Figure 6 and a graphical representation of the weights assigned to the studies in the meta analysis Finally on the
9. graph in Excel before copying it into a text document Right click on the vertical axis click Format Axis and change the parameter Maximum under Axis options to Auto or manually insert the number of studies included plus one for the combined effect size and plus the number of subgroups in case of the plot for Subgroup Analysis Then scroll down towards the bottom of the figure and reduce the size of the chart area by drag and drop 46 User manual for Meta Essentials 6 References 6 References Alo A M 2014 An empirical investigation of partial effect sizes in meta analysis of correlational data The Journal of General Psychology 141 1 47 64 dx doi org 10 1080 00221309 2013 853021 Alo A M amp Becker B J 2012 An effect size for regression predictors in meta analysis Journal of Educational and Behavioral Statistics 37 2 278 297 dx doi org 10 3102 1076998610396901 Anzures Cabrera J amp Higgins J P T P T 2010 Graphical displays for meta analysis An overview with suggestions for practice Research Synthesis Methods 1 1 66 80 dx doi org 10 1002 jrsm 6 Begg C B amp Mazumdar M 1994 Operating characteristics of a rank correlation test for publication bias Biometrics 50 4 1088 1101 dx doi org 10 2307 2533446 Borenstein M 2009 Effect sizes for continuous data In H Cooper L V Hedges amp J C Valentine Eds The handbook of research synthesis and
10. is assumed that it is possible or likely that different true effects underlie the effect sizes from different studies The aim of the meta analysis is to estimate and then explain the variance of these true effects and the prediction interval is interpreted as an estimate of that variance or dispersion for a more detailed discussion of these models see e g Hedges amp Vevea 1998 In Meta Essentials the random effects model is used by default because the assumptions underlying the fixed effects model are very rarely met especially in the social sciences Furthermore when a fixed effects model would make sense to use i e when there is little variance in effect sizes the random effects model converges automatically into a fixed effects model 3 2 2 Prediction Interval The Meta Essentials software does not only generate a confidence interval for the combined effect size but additionally a prediction interval Most other software for meta analysis will not generate a prediction interval although it is in our view the most essential outcome in a random effects model i e when it must be assumed that true effect sizes vary If a confidence level of 9596 is chosen the prediction interval gives the range in which in 9596 of the cases the outcome of a future study will fall assuming that the effect sizes are normally distributed of both the included and not yet included studies This in contrast to the confidence inter
11. is of the difference family or of the correlation family The difference family or d family regards effect sizes that are based on differences between or within groups you can use workbook 2 or 3 or 4 The correlation family or User manual for Meta Essentials 2 Select the appropriate workbook r family regards effect sizes based on the association between two or more continuous variables you can use workbook 5 6 or 7 If your effect size is of the d family you can find guidance about how to make a choice between workbooks 2 3 and 4 in the following section If your effect size is of the r family you can find guidance about how to make a choice between workbooks 5 6 and 7 in the section thereafter 2 1 Effect sizes of the d family Research designs in the d family can be categorized along two dimensions 1 The dependent variable can be categorical or continuous This difference results in different types of effect size namely a difference between proportions if the dependent variable is categorical and a difference between means if the dependent variable is continuous 2 The difference that is studied can be a difference between different groups or a within group difference Examples of the first type independent groups are experiments with separate groups and non experimental differences between separate categories e g between men and women or between different types of companies An example of the second typ
12. method Peto et al 1977 p 31 41 22 Meta analysis model and presentation effect size From a statistical perspective meta analysing Log Odds Ratios is preferable because the Odds Ratio is less prone to heterogeneity compared to Risk Difference in particular On the down side however the Odds Ratio is rather hard to interpret In this workbook the user can select an effect size for the meta analysis model i e the effect size measure used in the calculations and another one for presentation in the forest plot All calculations can be inspected on the Calculations sheet Note that conversion is performed only from Odds Ratio to Risk Ratio or Risk Difference downstream not the other way around because there is no use for the opposite direction For the inverse variance weighting method the user can also choose between using the Odds Ratio Risk Ratio and Risk Difference for both the model and the presentation If you choose the Odds Ratio or Risk Ratio for the model the meta analysis will actually be run in Log Odds Ratio and Log Risk Ratio respectively For the Peto weighting method a slightly different Odds Ratio is available called the Peto Odds Ratio whereas all other options are available as well Note that a weighting method must be chosen before the effect size measure because not all options for effect size measure are available for all weighting methods The user is informed about the validity Yes or
13. right side the forest plot pictures the effect size with confidence interval of each study and below them a the combined 11 User manual for Meta Essentials 3 Work with the workbooks effect size with its confidence interval in black colour and its prediction interval in green colour These are the basic outcomes of any meta analysis Input Forest Plot Subgroup Analysis Moderator Analysis Publication Bias Analysis Calculations 0 Figure 7 The tab to access the Forest Plot sheet of Meta Essentials 3 2 1 Choose options In the top left corner of the sheet see the red rectangle labelled Choose options here in Figure 6 the user can make some choices regarding the meta analysis itself random effects versus fixed effects and confidence level and regarding the ordering of studies on the output sheets sorting criterion and sorting order The user can choose between a fixed effects model and a random effects model In the fixed effects model it is assumed that all differences between effect sizes observed in different studies are due to sampling error only In other words the unobserved true effect is assumed to be the same for each study and the studies are functionally equivalent The aim of the meta analysis is to estimate that true effect and the combined effect size and its confidence interval are interpreted as an estimate of the true effect In the random effects model it
14. sets Psychological Methods 3 1 46 54 dx doi org 10 1037 1082 989x 3 1 46 49
15. subjects Sufficient data Subgroup Moderator 1 aaaa Yes 2 bbbb Yes 3 cccc Yes 4 dddd Yes 5 eeee Yes 6 ffff Yes 7 8888 Yes 8 hhhh Yes 9 iii Yes 10 jjjj Yes 11 kkkk Yes 12 111 Yes Required data Figure 57 Input sheet of Workbook 5 Correlational data xlsx 4 3 3 Forest Plot sheet With the Sort By function the user can additionally choose from options that include Fisher transformed values such as for the effect size correlation coefficient and variance see Figure 58 for an example they are indicated with z Its values are then automatically displayed in column J but not used in the plot or in the table for the combined effect size Entry number number Study name Correlation Correlation z Correlation Number of subjects Confidence interval LL Variance z Confidence interval UL Standard error 2 Prediction interval Weight fixed Prediction interval UL 1 00 Figure 58 Example of Sort By function on Input sheet of Workbook 5 Correlational data xlsx 4 3 4 Moderator Analysis sheet For the moderator analysis Fisher s transformed correlation coefficients are used and displayed This is a difference with some other sheets where after the necessary computations the Fisher s transformed correlation coefficients are conversed back for presentation into normal correlation coefficients 4 3 5 Publication Bias Analysis sheet For the publication bias analysis Fisher s transf
16. type Yes or No 10 User manual for Meta Essentials 3 Work with the workbooks Study name Include study Effect size 1 aaaa Yes Y 2 20 2 bbbb Yes 1 80 3 cccc 1 90 4 dddd Yes 2 05 B eeee Yes 0 05 6 ffff Yes 0 60 Figure 5 Example of Include Study on the Input sheet If No is selected for a particular study this study will be omitted from all analyses including the subgroup analysis moderator analysis and publication bias analyses The cells in the column will automatically indicate a number for the order of entry in the input sheet It will only indicate a number if both Include study and Sufficient data are Yes The cells in the column Sufficient data automatically indicate whether sufficient data is entered for inclusion of the study in the meta analysis In workbook 1 Effect size data xlsx it is set to Yes indicating sufficiency whenever the effect size and standard error for a particular study have been entered In the other workbooks different criteria for sufficiency are applied These will be discussed for each of them separately in the section that describes features that are specific to a certain workbook Table Weights Choose options here Forest plot Effect Size 1 00 Study Effect a a E name size WE CAESUS A eds Weighting 0 05 2 00 1 00 0 00 0 04 Model 2 20 1 80 1 90 2 05 0 05 0 60 1aaaa 2 bbbb 3 cccc 4 dddd 5 eeee
17. 0 3 90 Yes BB 14 7 gegg Yes 0 20 0 05 ve 120 Yes AA 19 8 hhhh Yes 0 22 0 04 8 130 Yes AA 13 9 iiii Yes 0 05 0 11 4 80 Yes BB 19 10 jjjj Yes 0 15 10 240 Yes AA 22 11 kkkk Yes 0 03 8 90 Yes BB ati 12 1111 Yes 0 05 5 100 Yes BB 18 Figure 59 Input in Workbook 6 Partial correlational data xlsx Semi partial SE of Semi partial Number of Number of Study name Include study Beta SEofBeta t value R squared Sufficient data Subgroup Moderator correlation correlation predictors in model observations 1 aaaa Yes 0 40 0 10 100 Yes AA 15 2 bbbb Yes 0 30 0 08 130 Yes AA 16 3 cccc Yes 0 02 0 20 80 Yes AA 13 4 dddd Yes 2 10 0 07 6 300 Yes AA 18 5 eeee Yes 1 80 0 11 6 95 Yes BB 20 6 ffff Yes 0 50 0 18 3 90 Yes BB 14 7 8888 Yes 0 20 0 05 0 06 Zi 120 Yes AA 19 8 hhhh Yes 0 22 0 04 0 08 8 130 Yes AA 13 9 iii Yes 0 05 0 11 0 07 4 80 Yes BB 19 10 jijj Yes 0 15 0 30 10 240 Yes AA 22 11 kkkk Yes 0 03 0 03 8 90 Yes BB 17 12 1111 Yes 0 05 0 06 5 100 Yes BB 18 Figure 60 Input in Workbook 7 Semi partial correlational data xlsx 4 4 3 Forest Plot sheet If the number of observations is not inserted on the input sheet the confidence intervals of individual studies cannot not be generated because they rely on a Student s t distribution for which 44 User manual for Meta Essentials 4 Specific features of individual workbooks an appropriate degrees of freedom needs to be specified This applies to the Subgroup Analysis sheet as well
18. 111 0 50 100 0 20 24 61 0 74 0 90 0 10 8 36 1 57 0 50 0 40 0 40 Figure 31 Example of Forest Plot part of the Calculations sheet 3 6 2 Subgroup Analysis The subgroup analysis section of the calculations sheet contains the necessary calculations to construct the Subgroup Analysis sheet It begins with repeating the information from the input section see Figure 32 for an example The first section to the left gives information per study The Display studies as well as the information in the second table of the following picture are for plotting purposes The weights in a random model if that was chosen in the Subgroup Analysis sheet depend on subgroup estimates of heterogeneity either separate per subgroup or pooled over subgroups as specified in the Subgroup Analysis sheet 26 User manual for Meta Essentials 3 Work with the workbooks Figure 32 Example of first part of Subgroup Analysis part of the Calculations sheet Subgroup Analysis The second section refers to subgroups and starts with a display number for plotting purposes see Figure 33 for an example It gives the name of the subgroup the subgroup number and the number of studies in the subgroups followed by heterogeneity measures and subgroup combined effect sizes estimates with standard errors confidence and prediction interval limits and finishes with weights for fixed and random effects models Figure 33 Example of second part of Subgroup Analy
19. Calculations O Figure 1 The tabs to access the six sheets of Meta Essentials The first sheet is the Input sheet This is the sheet where you enter the information from the studies that you want to include in your meta analysis The next four sheets are output sheets one for the forest plot one for the subgroup analysis one for the moderator analysis and one for the publication bias analyses The sixth sheet contains the calculations that are performed for producing the four output sheets For basic use of the workbooks there is no need for you to look at or work with this sheet The six sheets of a Meta Essentials workbook will be discussed below with screenshots The examples used in these screenshots come from analyses in workbook 1 Effect size data xlsx with fictitious data All six sheets of all seven workbooks are essentially the same Features that are specific to a certain workbook are discussed in a separate section Different colours mark different purposes of cells Cells in which the user can give input or change settings are always coloured in pink calculations in dark grey and output in lighter grey see Table 3 Purpose Cell colour User s input choice Calculation Output Table 3 Purpose of cell colours The user is free to make changes in the files but we advise the novice user not to change any cells except the ones coloured in pink which are designed for user input It is advised in particular not to in
20. S G Deeks J J amp Altman D G 2003 Measuring inconsistency in meta analyses British Medical Journal 327 7414 557 560 dx doi org 10 1136 bmj 327 7414 557 Higgins J P T Thompson S G amp Spiegelhalter D J 2009 A re evaluation of random effects meta analysis Journal of the Royal Statistical Society Series A Statistics in Society 172 1 159 dx doi org 10 1111 j 1467 985x 2008 00552 x L Abb K A Detsky A S amp O Rourke K 1987 Meta analysis in clinical research Annals of Internal Medicine 107 2 224 233 dx doi org 10 7326 0003 4819 108 1 158 2 Mantel N amp Haenszel W 1959 Statistical aspects of the analysis of data from retrospective studies of disease Journal of the National Cancer Institute 22 4 719 748 inci oxfordjournals org content 22 4 719 full pdf html Orwin R G 1983 A fail safe N for effect size in meta analysis Journal of Educational Statistics 8 2 157 159 dx doi org 10 2307 1164923 Peto R Pike M C Armitage P Breslow N E Cox D R Howard S V Smith P G 1977 Design and analysis of randomized clinical trials requiring prolonged observation of each patient Il Analysis and examples British Journal of Cancer 35 1 1 39 dx doi org 10 1038 bjc 1977 1 Rosenthal R 1979 The file drawer problem and tolerance for null results Psychological Bulletin 86 3 638 64 dx doi org 10 1037 0033 2909 86 3 638 S nchez Meca J 8 Mar
21. SE 0 05 Number of imputed studies 2 Cl Lower limit 1 35 Choose Trim and Fill options here Cl Upper limit 1 58 PI Lower limit 1 43 PI Upper limit 4 36 Figure 20 Example of funnel plot part of the Publication Bias Analysis sheet The user can turn the trim and fill procedure On or Off can decide whether to search for studies missing in the meta analysis on the Left or Right side of the combined effect size and can choose between two estimators Linear also described as Lo or Leftmost Rightmost Run also described as Ro Once the trim and fill is turned on Meta Essentials will calculate an adjusted combined effect size with CI and PI represented on the red horizontal line in Figure 11 as well as adjusted heterogeneity measures These adjusted statistics are based upon the set of initially included studies expanded with the imputed data points orange open circles in the plot see Figure 20 3 5 2 Egger regression and Begg and Mazumdar rank correlation test The Egger regression gives the degree of funnel plot asymmetry as measured by the intercept from regression of standard normal deviates against precision Egger Smith Schneider amp Minder 1997 p 629 The output on this sheet consists of this intercept and its confidence interval as well as of the outcome of a t test t value and p value see Figure 21 for an example 20 User manual for Meta Essentials 3 Work with the workbooks
22. User manual for Meta Essentials Workbooks for meta analyses Henk van Rhee Robert Suurmond amp Tony Hak Version 1 0 February 2015 This is the user manual for Meta Essentials a set of workbooks for meta analyses The workbooks as well as this manual are licensed under the Creative Commons Attribution NonCommercial ShareAlike 4 0 International License Preferred citation Van Rhee H J Suurmond R amp Hak T 2015 User manual for Meta Essentials Workbooks for meta analyses Version 1 0 Rotterdam The Netherlands Erasmus Research Institute of Management Retrieved from www erim eur nl research support meta essentials Contact e Henk van Rhee vanrhee rsm nl Rotterdam School of Management Burgemeester Oudlaan 50 3062PA Rotterdam The Netherlands e Robert Suurmond suurmond rsm nl e Tony Hak thak rsm nl 1 5 6 Short Table of Contents uidere Uedfojee C RM 5 11 Aimofthisuser man al e eerte ie ine o TEE Pn e AN EN Ova sew da 5 1 2 Wee EE 5 1 3 Compatibility lu ER 5 Select the appropriate WorkboO0k cccoconococoonnononononononnnnnnnonononononnnnnnnonnnnnnonnnnnnnnnnnone no nnnnnnnncnnanonos 6 Work with the Workbooks eeeeseeeeeeeeeseseeeeenene nenne nenne nnnm nn nennen entr cnn rra 9 3 1 Input y ee A EAE ted ed o mut et al 10 3 2 Frest Plot tias dt 11 3 3 S bgroup Analysis sheet isse daa 12 3 4 Moderator Analysis sheet ccccccccccscssssssssecece
23. a meta analysis represented by blue dots in a space defined by effect size on the x axis scale displayed on top of the plot and standard error on the y axis It also presents the combined effect size green dot with its confidence interval black and prediction interval green The plot also shows a vertical line also in red that runs through the adjusted combined effect size and the corresponding lower and upper limits of the confidence interval red diagonal lines The adjusted combined effect size and accompanying confidence and prediction intervals in this plot represents the results of a trim and fill procedure as proposed by Duval and Tweedie 2000a 2000b 19 User manual for Meta Essentials 3 Work with the workbooks Effect Size Study name Effect Size Standard Error 1 00 j 1 00 2 00 aaaa 2 20 0 25 bbbb 1 80 0 21 CCCC 1 90 0 27 dddd 2 05 0 14 eeee 0 05 0 20 ffff 0 60 0 21 g888 2 00 0 22 hhhh 1 80 0 21 iiii 0 40 0 22 i jij 2 10 0 16 e kkkk 0 40 0 21 Ht 0 50 0 20 5 S 0 35 e Studies 9 Combined Effect Size Adjusted CES O Inputed Data Points Combined effect size Observed Heterogeneity Adjusted Effect Size 1 15 Q 580 86 SE 0 06 Pa 0 000 Cl Lower limit 102 I 0 98 Cl Upper limit qos ie 1 79 PI Lower limit 1 37 T 1 34 PI Upper limit 3167 Trim and Fill On Combined effect size Adjusted Search from mean Right Effect Size 1 47 Estimator for missing studies Leftmost Run Rightmost Run
24. and a table on top with the lower middle and upper values of the bins see Figure 41 for an example Figure 41 Example of Standardized Residual Histogram part of the Calculations sheet 3 6 4 6 Galbraith plot For the Galbraith plot the calculations section contains the inverse standard error and z score for plotting purposes see Figure 42 for an example The second table contains information for the regression lines in the plot Figure 42 Example of Galbraith Plot part of the Calculations sheet 3 6 4 7 Normal Quantile Plot The calculations for the normal quantile plot contain information on the ranks the normal and sample quantile for plotting purposes and some information for the calculation of the standard error User manual for Meta Essentials 3 Work with the workbooks of the regression estimates see Figure 43 for an example The second table is used for plotting the regression line Figure 43 Example of Normal Quantile Plot part of the Calculations sheet 3 6 4 8 Failsafe N tests For the Failsafe numbers the z score its p value and a og value of the p value are given see Figure 44 for an example Note that because Excel cannot cope with more than 15 digits in its calculations under the hood it will round the very small p values to zero Therefore the natural logarithm of that value would give an error since the natural logarithm of zero is undefined To overcome this problem Meta Essentials automatically
25. ate of the association between the moderator and a study s effect size This is also visualized in the plot also shown in Figure 16 where the effect sizes of the studies are plotted against their moderator values and a regression line through these points Note that the size of the dots represents their relative weight However since in the example all studies receive about the same weight the dot sizes appear to be equal 16 User manual for Meta Essentials 3 Work with the workbooks Regression of moderator on effect size 2 50 1 00 Effect Size 0 50 0 00 0 50 1 00 10 00 16 00 18 00 Moderator 12 00 14 00 20 00 22 00 Intercept 1 45 2 22 3 43 6 33 Slope 0 02 0 13 0 31 0 26 0 06 Model 0 03 1 0 860 0 03 Residual 9 92 10 0 448 0 99 Total 9 95 11 0 535 Combined effect size 1 07 qr method of moments estimation 1 45 R 0 31 Figure 16 Example of right part of the Moderator Analysis sheet 3 5 Publication Bias Analysis sheet 24 00 26 00 0 66 0 512 0 18 0 860 0 03 0 863 Publication bias analysis is not a core meta analysis feature and for some of the methods rather strong assumptions apply which means they should be used with caution see Hak et al 2015a Multiple procedures or statistics regarding publication bias analysis are provided by Meta Essentials funnel plot Egger regression Begg and Mazumdar s rank correlation test standardized residual histogram Galbraith plot normal q
26. ceesesesneseeeeecesseseaeaeeeeeceseeeesaeaeeeeeceseeseaeaeeeesens 16 3 5 Publication Bias Analysis Sheet cccconononoconnnonononononannnnnnncnononononnnncnnnnnnnno nono nnnnnncnnnnnnnnnnnnnnnns 17 316 Calculationis sheet eme retener AO atea 25 3 7 Statistical proced res aciei et e a ica ee ra as eo e iced des 33 Specific features of individual WOrkb0OKS ccccconococoonccnnnnconononnnncnncnnnanonnnnononnnnnnnnnnnnnnnnnnnanannnns 35 4 1 Workbook 2 Differences between independent groups binary data xlsx 35 42 Workbooks 3 Differences between independent groups continuous data xlsx and 4 Differences between dependent groups continuous data xlsx sess 40 4 3 Workbook 5 Correlational data xlsx esses ener nnne nnne 42 44 Workbooks 6 Partial correlational data xlsx and 7 Semi partial correlational data xlsx 43 Adapting plots for reporting References cnccccccccncnoncncnncnnnnonananicnnns Detailed Table of Contents 1 oideero U sidfejoc RE 5 11 Aim ofthisuserman al e terere daa id 5 1 2 Wee ELS 5 1 3 Compatibility xiii dioit etre oe te y ERE EXER Ex Ren eT evo ru vend iran vay xi vens tn oss 5 2 Select the appropriate WOrkboO0k cccoconococconccnonononononnnncnnonononononnnnnnncnnnnnononnnnnnnnnnnnne no nnns sns nn nan 6 2 1 1 Effect sizes of the d family ccoo dia A eee rA e LEER En 7 2 1 2 Effect sizes in the r family esineen eisi
27. culated semi partial correlations or can choose to let the Workbook calculate them In the latter case the three main input formats in Workbook 6 are e t value number of predictors and number of observations study ddda e Beta standard error of Beta number of predictors and number of observations study 9990 e Partial correlation number of predictors and number of observations study jjjj e Partial correlation standard error of partial correlation and number of observations see study aaaa in Figure 31 Please note that Fisher s transformation is not possible with this input as indicated in the Sufficient data column In Workbook 7 the possible input formats are similar to that of Workbook 6 however providing the R squared is mandatory for all input options except when providing the semi partial correlation the standard error of the semi partial correlation and the number of observations SE of Partial Number of Number of Po Study name Include study Partial correlation Beta SE of Beta t value predictors in Sufficient data Subgroup Moderator correlation observations model 1aaaa Yes 0 40 0 10 100 Yes Insufficient data for Fisher s r to z transformation AA 15 2 bbbb Yes 0 30 0 08 130 Yes Insufficient data for Fisher s r to z transformation AA 16 3 cccc Yes 0 02 0 20 80 Yes Insufficient data for Fisher s r to z transformation AA 13 4 dddd Yes 2 10 6 300 Yes AA 18 B ecee ves 1 80 6 95 MES BB 20 6 ffff Yes 0 5
28. dies between subgroups and within subgroups see the red rectangle labelled Choose options here in Figure 9 For the Between subgroup weighting the user can choose from a Fixed effects and Random effects default model For the Within subgroup weighting the user can choose between Fixed effects Random effects Tau separate for subgroups default and Random effects Tau pooled over subgroups models If the latter option is selected the variance components Tau of each subgroup will be pooled averaged and used for every subgroup Note that these defaults are not always appropriate to use Theory will 15 User manual for Meta Essentials 3 Work with the workbooks have to tell which option to use in general using pooled variance components is more appropriate when you have very few studies included in your meta analysis or in any particular subgroup Borenstein Hedges amp Higgins 2009 pp 149 ff 3 3 2 Heterogeneity The Heterogeneity part of the sheet is more complex than the one in the Forest Plot sheet because it contains measures on three levels within between and total See Assess heterogeneity in Figure 9 The total heterogeneity variance is the heterogeneity among all studies ignoring the structure of the data i e the subgroups The heterogeneity within subgroups states how much of the total variance is within the subgroups The heterogeneity between subgroups stat
29. e dependent groups is a difference in time for instance before and after a therapy or other intervention Four types of studies with a d design can be distinguished based on these two dimensions see Table 2 Workbooks 2 3 and 4 each fill one of the cells in table The cell for categorical dependent variable with dependent groups is empty because this type of design is very rare Should you want to meta analyse effect sizes of such type you can use workbook 1 Effect size data xlsx Independent groups Dependent groups Categorical dependent 2 Differences between variable independent groups binary data xlsx Numerical dependent 3 Differences between 4 Differences between variable independent groups dependent groups continuous data xlsx continuous data xlsx Table 2 Overview of the Meta Essentials workbooks of the d family Workbook 2 Differences between independent groups binary data xlsx can be used for meta analysing studies that compare two groups typically an experimental group and a control group when the outcome of interest is categorical e g success versus failure This is a common research design in clinical studies but could be applied in social sciences as well For instance the relationship of the gender of an entrepreneur with the one year survival survival versus bankruptcy of a start up could in one study be evaluated with a two by two table Typical statistics to grasp the size of difference in
30. e for an estimation of the individual study confidence intervals because the degrees of freedom of the Student s T distribution are based on them l e the number of observations is not necessary for calculating a meta analytical effect size nor for any of the additional analysis but is necessary for calculation of confidence intervals for the individual studies as presented in the forest plot Not required but probably useful are the following inputs e Entering a name or other identifier of a study Study name The study name can be any name you choose it works best if you use a unique name for each study e Assigning membership to a subgroup Subgroup The subgroup must be a categorical variable which can be used in the subgroup analysis You can enter the categories of this variable in any way you want numerical textual or combinations thereof e Entering a score for another feature of the population studied or for the study Moderator The moderator is a continuous variable which might be used in the moderator analysis The moderator must be a numerical variable which is assumed to have at least an interval scale e Deciding whether a study will be included in a meta analyses Include study The study will be included by default Yes This can be changed by using a dropdown menu which can be accessed by clicking on the cell and then clicking on the small arrow next to the cell Figure 5 You can also
31. ea ae EEEE E RE aeaii EEEE 8 3 Work with the workbookS rettet nere eoa ec rnnt tt enit aede Ree theo E ERE dg aae neo idad 9 3 1 IniD tshieet oou UENIT E 10 3 2 B insduelle c 11 3 2 1 Choose options esses nennen nennen nennt nini sss s ins n neni asas assesses si nass anna 12 3 2 2 Prediction Interval oies ecrit terri etx PER I Ree ee Eee Ea ERR Ve reae khe ARR en eV RE 12 3 3 Subgroup Analysis sheet ecce eR the eee te dae te Me eas vore ico ra a deo ecu 12 3 3 1 reete RN eee 15 3 3 2 H teroBenlelty os fomir A ee fion duds d 16 3 4 Moderator Analysis SHE t sia ies 16 3 5 Publication Bias Analysis Sheet ccccoconococooncnnononononononnnonoconanononnnnnnnnnnnnnonnnnnnnnnnnnnnnr nn nnns nnne 17 3 5 1 PUM El Plot 19 3 5 2 Egger regression and Begg and Mazumdar rank correlation test 20 3 5 3 Standardized Residual Histogram esses eene nnne nennen 21 3 5 4 Galbraith plot oo 22 3 5 5 Normal Quantile Plot ER 23 3 5 6 Failsafe N tests niii eter tette pe dinge aae ne e Due QU aa ace pe eiia iiaiai 24 3 6 Calculations SHC ees oerte tp ER HERRERA ar ERR E RERIR IEEE ER E PUT ENIM IS 25 3 6 1 dero c 26 3 6 2 Subgroup Analysis ta atadas 26 3 6 3 Moderator Analysis oreet Ene Pee nani sheds 28 3 6 4 Publication Bias Analysis coconococcccnccncononoonancnncn
32. ection 4 1 Workbook 2 Differences between independent groups binary data xlsx 4 1 1 Inputsheet The required input for this workbook is not a point estimate with a standard error such as in Workbook 1 Instead the user must enter either the number of cases with either outcome in each group see cells a b c and d in the two by two table on the right side of Figure 45 Or any other combination of information that makes it possible to calculate these four numbers In practice this means that the user must fill at least four of the six cells in this two by two table Each of the rows in Figure 45 represents a study with sufficient information according to this principle It is a unique feature of this workbook that an effect size e g an odds ratio by can be converted into another one e g a risk difference It is not possible to insert any one of these effect sizes directly in this workbook because this conversion is not possible without the full information from the 2x2 table Study name Include study a b c d n nm Sufficient data Subgroup Moderator 1 aaaa Yes 10 4 50 50 Yes AA 15 Outcome 1 Outcome 2 Total 2 bbbb Yes 20 12 65 65 Yes AA 16 Group 1 a b ni 3 cccc Yes 15 25 18 22 Yes AA 13 Group 2 c d n2 4 dddd Yes 30 130 150 150 Yes AA 18 5 eeee Yes 18 36 48 47 Yes BB 20 6 ffff Yes 17 28 10 35 Yes BB 14 7 8888 Yes 51 10 60 60 Yes AA 19 8 hhhh Yes 50 6 65 65 Yes AA 13 9 iiii Yes 5 35 4 36 Yes BB 19 10 jjjj Yes 110 115
33. es how much of the total variance is explained by assigning subgroups to the studies Higgins et al 2003 discuss how to interpret values for heterogeneity statistics for subgroup analyses 3 4 Moderator Analysis sheet If you entered a score in the Moderator column of the Input sheet then a weighted regression will be run with Moderator as a predictor of the effect size of a study In Meta Essentials it is not possible to run a multivariate regression analysis so only one moderator can be assessed at a time You can access the Moderator Analysis sheet by clicking on the regarding tab as shown in Figure 14 Input Forest Plot Subgroup Analysis Moderator Analysis Publication Bias Analysis Calculations Figure 14 The tab to access the Moderator Analysis sheet of Meta Essentials On the left of the sheet displayed in Figure 15 the user can choose between a fixed effects model and a random effects model The user can also set the confidence level As in other sheets the random effects model is set as default Also a table is provided with some essential statistics per study Meta analysis model Study name Effect size Moderator Weight Model Random effects Confidence level 9596 Choose options here Table Figure 15 Example of part of the left part of the Moderator Analysis sheet The most important result of this regression is the coefficient B of the slope see red rectangle in Figure 16 which is an estim
34. esent in case of small sample sizes It is now customary to correct for this bias but some still refer to it as Cohen s d while Hedges g would be a clearer name for it Others such as Cumming 2012 refer to the latter as dunbiasea In Meta Essentials Cohen s d refers to the standardized mean difference as proposed by Cohen and Hedges g refers to the bias adjusted standardized mean difference as proposed by Hedges 4 3 Workbook 5 Correlational data xlsx 4 3 1 Fisher s transformation The main difference between this workbook and the workbooks discussed so far is the use of a so called Fisher s r to z transformation Fisher 1921 which will automatically be applied because the transformed correlation z will faster tend to normality For this transformed correlation a standard error is estimated based on the number of subjects the sample size 4 3 2 Input sheet Required input in this workbook is only a the correlation coefficient and b the sample size see Figure 57 The meta analysis is run with the Fisher transformed values which are transformed back into normal correlation coefficients for presentation The subscript z is used throughout this workbook to indicate transformed values Please note that a correlation coefficient is equal to the standardized bivariate regression coefficient 42 User manual for Meta Essentials 4 Specific features of individual workbooks Study name Include study Correlation Number of
35. failsafe number estimates the number of such additional studies that are required to turn the effect size from the included and additional studies combined insignificant i e that the new combined effect size is essentially zero 3 5 6 1 Rosenthal In order to calculate a Failsafe N first described by Rosenthal 1979 a test of combined significance is conducted The failsafe number is the number of missing studies averaging a z value of zero that should be added to make the combined effect size statistically insignificant see Figure 26 for an example The ad hoc rule refers to the one by Rosenthal 1979 for deciding whether the number estimated is small TRUE or large FALSE 24 User manual for Meta Essentials 3 Work with the workbooks Rosenthal Overall Z score 18 63 Failsafe N 1527 Ad hoc rule FALSE Figure 26 Example of Rosenthal s Failsafe N of the Publication Bias Analysis sheet 3 5 6 2 Gleser Olkin Gleser and Olkin 1996 provide an estimate for the number of unpublished results see Figure 27 for an example It uses the assumption that the studies in the meta analysis have the largest significance i e smallest p values from a population of effect sizes The size of the largest p value in the meta analysis determines the number of estimated unpublished studies There is no method to assess whether this number is small or large but a comparison could be made with the number of studies that actually are included i
36. ficient of correlation deduced from a small sample Metron 1 3 32 hdl handle net 2440 15169 Fisher R A 1932 Statistical methods for research workers Fourth Ed Edinburgh UK Oliver amp Boyd www worldcat org oclc 4971991 Galbraith R F 1988 Graphical display of estimates having differing standard errors Technometrics 30 3 271 281 dx doi org 10 1080 00401706 1988 10488400 47 User manual for Meta Essentials 6 References Gleser L J amp Olkin 1996 Models for estimating the number of unpublished studies Statistics in Medicine 15 23 2493 2507 dx doi org 10 1002 sici 1097 0258 19961215 15 23 3C2493 aid sim381 3E3 0 co 2 c Hak T Van Rhee H J amp Suurmond R 2015 How to interpret results of meta analysis Rotterdam The Netherlands Erasmus Rotterdam Institute of Management www erim eur nl research support meta essentials downloads Hedges L V 1981 Distribution theory for Glass s estimator of effect size and related estimators Journal of Educational and Behavioral Statistics 6 2 107 128 dx doi org 10 2307 1164588 Hedges L V amp Vevea J L 1998 Fixed and random effects models in meta analysis Psychological Methods 3 4 486 dx doi org 10 1037 1082 989x 3 4 486 Higgins J P T amp Thompson S G 2002 Quantifying heterogeneity in a meta analysis Statistics in Medicine 21 11 1539 1558 dx doi org 10 1037 1082 989x 3 4 486 Higgins J P T Thompson
37. he presentation Meta analysis model Weighting Method Inverse Variance Effect Size Measure Odds Ratio Valid options chosen Yes Between subgroup weighting Random effects Within subgroup weighting Random effects Tau pooled over subgroups Confidence level 9596 Figure 48 Options for Subgroup Analysis in Workbook 2 Differences between independent groups binary data xlsx 4 1 4 Moderator Analysis sheet The moderator regression for binary data can be run in Log Odds Ratio Log Risk Ratio or Risk Difference The logarithmic values of the Odds Ratio and Risk Ratio are used instead of the normal values because they tend to normality faster see Figure 49 37 User manual for Meta Essentials 4 Specific features of individual workbooks Weighting method Inverse Variance Effect Size Measure Log Risk Ratio Valid options chosen Yes Model Random effects Confidence level 9596 Figure 49 Options for Moderator Analysis in Workbook 2 Differences between independent groups binary data xlsx 4 1 5 Publication Bias Analysis sheet Procedures for assessing publication bias for binary data can be run in Log Odds Ratio Log Risk Ratio or Risk Difference see Figure 50 Weighting method Inverse Variance Effect Size Measure Log Risk Ratio Valid options chosen Yes Model Fixed effects Confidence level 9596 Figure 50 Options for Publication Bias Analysis in Workbook 2 Differences between independent groups binary da
38. istogram Probability o o n h o9 N 0 25 0 00 0 08 0 00 0 08 0 00 0 00 0 16 0 00 0 68 0 00 0 16 0 42 0 00 0 08 0 00 0 08 0 00 Figure 23 Example of Standardized Residual Histogram part 3 5 4 Galbraith Plot The basic idea of the Galbraith plot or radial plot Galbraith 1988 is to run an unweighted regression of z scores on the inverse of the standard error with the intercept constrained to zero see Figure 24 This plot can be used to look for outliers in the effect sizes The expectation is that 95 of the studies is within the area defined by the two lighter coloured confidence interval lines Meta Essentials gives a table with studies a plot and a table with regression estimates see Figure 24 22 User manual for Meta Essentials 3 Work with the workbooks Inverse name standard Z value Galbraith Plot nam error 2 00 4 00 6 00 Inverse standard error Regression estimate Estimate SE CI LL CI UL Intercept fixed at 0 0 00 Slope 1 15 0 06 1 02 1 28 Figure 24 Example of Galbraith Plot part of the Publication Bias Analysis sheet 3 5 5 Normal Quantile Plot Normal Quantile plots or Q Q plots are also used to assess the normality of data Wang amp Bushman 1998 The expectation is that all data points are approximately on a straight line which would indicate that the dispersion of the data follows a standard normal distribution This part in Meta Essentials see Figure 25 con
39. k 3 and Figure 56 for Workbook 4 Hence an important feature of these workbooks is that it they function as effect size generators For instance the user can insert raw group data means 40 User manual for Meta Essentials 4 Specific features of individual workbooks standard deviations and sample sizes or tests of differences t value F value or already calculated effect sizes Cohen s d Hedges g Study name Include study M M M M S Sz Sy nm nj tvalue F value Cohen s d Hedges g Sufficient data Subgroup Moderator 1 aaaa Yes 10 00 8 00 1 00 1 20 50 50 Yes AA 15 2 bbbb Yes 11 00 8 00 1 20 65 65 Yes AA 16 3 cccc Yes 0 02 0 70 0 50 40 40 Yes AA 13 4 dddd Yes 0 70 0 30 150 150 Yes AA 18 5 eeee Yes 48 47 1 60 Yes BB 20 6 ffff Yes 45 45 0 30 Yes BB 14 7 8888 Yes 60 60 0 50 Yes AA 19 8 hhhh Yes 65 65 0 70 Yes AA 13 9 iii Yes 40 40 0 40 Yes BB 19 10 jj Yes 120 120 2 10 Yes AA 22 11 kkkk Yes 45 45 0 40 Yes BB iy 12 1111 Yes 50 47 0 50 Yes BB 18 Figure 55 Input sheet of Workbook 3 Differences between independent groups continuous data xlsx Study name Include study M M M M S Sz Saiz N r t value F value Cohen s d Hedges g Sufficient data Subgroup Moderator 1 aaaa Yes 10 00 8 00 1 00 1 20 100 0 45 Yes AA 15 2 bbbb Yes 11 00 8 00 1 20 130 0 50 Yes AA 16 3 cccc Yes 0 02 0 70 0 50 80 0 47 Yes AA 13 4 dddd Yes 0 70 0 30 300 0 51 Yes AA 18 5 eeee Yes 95 0 59 1 60 Yes BB 20 6 ffff Yes 90 0 52 0 30 Yes BB
40. k for the meta analysis Then this manual discusses how to insert data how to perform a basic meta analysis and to generate a forest plot how to run a subgroup analysis a moderator analysis and various publication bias analyses Also the calculations behind the sheets and the applied statistical methods are discussed however knowledge or understanding of these methods is not required for using Meta Essentials Next the manual discusses those instructions that apply only to specific workbooks This manual concludes with discussing guidance for how output of Meta Essentials can be adapted for inclusion in a report 1 3 Compatibility The workbooks of Meta Essentials are compatible with Microsoft Excel 2010 and 2013 Older versions of Excel might work fine in some cases but several formulas and formatting features are not supported by these older versions The screen prints in this manual are made with Microsoft Excel 2013 but are not too different from how it would look in earlier versions 2 Selectthe appropriate workbook Meta Essentials is a set of seven different workbooks each for meta analysing a different type of effect size which are explained shortly hereafter Although the workbooks look the same the calculations behind them are different From the user s perspective the most noticeable difference is that the workbooks require different inputs An overview of the different workbooks is given in User manual for Meta Esse
41. meta analysis Second Ed pp 221 235 New York NY Russell Sage Foundation www worldcat org oclc 264670503 Borenstein M Hedges L V Higgins J P T P T 2009 Introduction to meta analysis Chichester UK John Wiley amp Sons dx doi org 10 1002 9780470743386 Cohen J 1969 Statistical power analysis for the behavioral sciences New York NY Academic Press www worldcat org oclc 34549 Cumming G 2012 Understanding the new statistics Effect sizes confidence intervals and meta analysis New York NY Routledge dx doi org 10 4324 9780203807002 DerSimonian R amp Laird N 1986 Meta analysis in clinical trials Controlled Clinical Trials 7 3 177 188 dx doi org 10 1016 0197 2456 86 90046 2 Duval S amp Tweedie R 2000a A nonparametric trim and fill method of accounting for publication bias in meta analysis Journal of the American Statistical Association 95 449 89 98 dx doi org 10 1080 01621459 2000 10473905 Duval S amp Tweedie R 2000b Trim and fill A simple funnel plot based method of testing and adjusting for publication bias in meta analysis Biometrics 56 2 455 463 dx doi org 10 1111 j 0006 341x 2000 00455 x Egger M Smith G D Schneider M amp Minder C 1997 Bias in meta analysis detected by a simple graphical test British Medical Journal 315 7109 629 634 dx doi org 10 1136 bmj 315 7109 629 Fisher R A 1921 On the probable error of a coef
42. n Mart nez F 2008 Confidence intervals for the overall effect size in random effects meta analysis Psychological Methods 13 1 31 48 dx doi org 10 1037 1082 989x 13 1 31 Sch nemann H J Oxman A D Vist G E Higgins J P T P T Deeks J J Glasziou P amp Guyatt G H 2011 Confidence intervals In J P T Higgins amp S Green Eds Cochrane handbook for systematic reviews of interventions version 5 1 0 Section 12 The Cochrane Collaboration handbook cochrane org chapter 12 12 4 1 confidence intervals htm Sterne J A Gavaghan D amp Egger M 2000 Publication and related bias in meta analysis Power of statistical tests and prevalence in the literature Journal of Clinical Epidemiology 53 11 1119 1129 dx doi org 10 1016 s0895 4356 00 00242 0 48 User manual for Meta Essentials 6 References Sutton A J Abrams K R Jones D R Jones D R Sheldon T A amp Song F 2000 Methods for meta analysis in medical research Chichester U K Wiley www worldcat org oclc 44167986 Van Rhee H J amp Suurmond R 2015 Working paper Meta analyze dichotomous data Do the calculations with log odds ratios and report risk ratios or risk differences Rotterdam The Netherlands Erasmus Rotterdam Institute of Management www erim eur nl research support meta essentials downloads Wang M C amp Bushman B J 1998 Using the normal quantile plot to explore meta analytic data
43. n the meta analysis Gleser amp Olkin Failsafe N 0 Figure 27 Example of Gleser and Olkin s Failsafe N of the Publication Bias Analysis sheet 3 5 63 Orwin Orwin 1983 uses a slightly different approach by looking at effect sizes rather than at p values For this method the user sets a criterion value for the combined effect size The user can set any value that would make the result of the meta analysis arbitrary ESc see Figure 28 for an example Secondly the user sets the mean of the studies that are imputed ES s Then the failsafe number will give the number of studies with average effect size ES s that would reduce the combined effect to the criterion value ESc Orwin Criterion value ESc 0 05 Mean fail safe studies ESps 0 Failsafe N 265 Figure 28 Example of Orwin s Failsafe N of the Publication Bias Analysis sheet 3 5 6 4 Fisher The fourth and final failsafe number method provided by Meta Essentials proposed by Fisher 1932 is also based on a test of the combined significance see Figure 29 for an example It is based on the sum of the natural logarithm of the p values from the studies in the meta analysis The number can be tested with a Chi Square distribution with degrees of freedom of two times the number of studies in the meta analysis Fisher Failsafe N 8549 p Chi square test 0 000 Figure 29 Example of Fisher s Failsafe N of the Publication Bias Analysis sheet 3 6 Calculations sheet The Calculation
44. nnnnononnnncnnnnnnnnonnnnnnnnnnnannno nn nnnnnnnnnnnnnnnnnnons 29 3 7 Statistical Procedures ia ada 33 4 Specific features of individual WOrkbO0OKS cccconococconcnncnncononoonnncnncnnonannnnnononnnnnnanononnnnnnnnnnnnnonos 35 4 1 Workbook 2 Differences between independent groups binary data xlsx 35 4 1 1 NDE sheet o sn etti it tests 35 4 1 2 Forest Plot Sheet triada 35 4 1 3 Suberoup Analysis Scene inde ad dd E dado 37 4 1 4 Moderator Analysis shiGet eee dias 37 4 1 5 Publication Bias Analysis Sheet ccccononococnnonononanonannnnnononononononnnnnnnnnanononnnnnnnnnnnnnnnannnnons 38 4 1 6 Calculations sheet sot een rete A t nee ente ia 39 4 2 Workbooks 3 Differences between independent groups continuous data xlsx and 4 Differences between dependent groups continuous data xlsx sss 40 4 2 1 o eno er EE re Ri 40 4 2 2 Effect size measures s titt terne eet eti Gea dans etos de ee edes ten sees eaaa ai 42 4 3 Workbook 5 Correlational data xlsx seen nennen 42 4 3 1 Fisher s transformation eie cinstersececsepessesaiesidactdgessnshssssinevisataucanendincedy iia 42 4 3 2 Input sheet oae A etes te aee ais 42 4 3 3 Forest Plot Sheet ii rette nte ec p ltda 43 4 3 4 Moderator Analysis sheet ccccccccccssssssssececececsesessnaeeeceesseeseaeaeeeeeeessesaaeseeeessessesaaaees 43 4 3 5 Publication Bias Analysis Sheet ccccononocoon
45. nonononanononnnnnnnononanonononnnonnnanononnnnnnnnnnnnananononnns 43 44 Workbooks 6 Partial correlational data xlsx and 7 Semi partial correlational data xlsx 43 4 4 1 Fisher s transformation chs ee id 44 4 4 2 Input sheet ee etai ie neue 44 4 4 3 Forest Plot Sheet e ettet AA A 44 Adapting plots for reporting ccccscccccccssssssssssecececesseseeseeececeseeseeaeaeeeeecesseseaaeaeeeeseesseaaeaeeeesens 46 RENCOR 47 User manual for Meta Essentials 1 Introduction 1 Introduction Meta Essentials is a set of workbooks that facilitate the integration and synthesis of effect sizes from different studies and provide figures tables and statistics that might be helpful for interpreting them Meta Essentials generates overall or meta statistical information regarding a set of studies of the same phenomenon based on the statistical information from each separate study The workbooks and a pdf version of this user manual can be downloaded from www erim eur nl research support meta essentials 1 1 Aim of this user manual This user manual is a guide for the usage of the software tool It is not a guide on how you should search for studies which studies you should include nor for how the results of the meta analysis should be interpreted We have written a separate text on these matters see Hak Van Rhee amp Suurmond 2015b 1 2 Structure The first step when using Meta Essentials is to choose the appropriate workboo
46. ntials 2 Select the appropriate workbook Table 1 File name Type of effect Example 1 Effect size data xlsx Any as long as directly Abnormal returns of bank loan comparable announcements 2 Differences between Difference between two Counts of start ups that did survive and did independent groups independent groups with not survive after three years per gender of binary data xlsx binary outcome entrepreneur 9 Did survive Did not survive T Male A B 5 Female C D s 5X Differences between Difference between two The difference between the average sales S independent groups independent groups with of a team that received training and that of S continuous data xlsx continuous outcome a team that did not receive training 4 Differences between Difference between two The difference between the average sales dependent groups dependent groups with of a team before and after receiving a continuous data xlsx continuous outcome training 5 Correlational Correlation between two Relationship between investments in z data xlsx variables computer technology and business tr performance E 6 Partial correlational Relation between two Idem but controlled for type of technology m data xlsx variables controlled for 2 d other variable s in both e predictor and outcome 2 7 Semi partial Relation between two Idem but controlled for average industry g correlational variables controlled for performance data xlsx other variable s in
47. o variables that is the correlation between two variables controlled for other variables Or more formally the part of the predictor that is related with the outcome variable after a portion of the effect the portion that is explained by other additional variables is partialled out This effect size can be used when you are interested in the relation between two variables while controlling for other variables in both the predictor and the dependent variable The workbook can calculate partial correlations from commonly reported multiple regression results Workbook 7 Semi partial correlational data xslx is designed to meta analyse the semi partial correlation between two variables but removes only the variance explained by additional variables from the outcome and not from the focal predictor The semi partial correlation is sometimes referred to as part correlation This effect size can be used when you are interested in the relation between two variables while controlling for other variables in only the predictor The workbook can calculate semi partial correlations from commonly reported multiple regression results User manual for Meta Essentials 3 Work with the workbooks 3 Work with the workbooks Each workbook of Meta Essentials consists of six sheets each of which can be accessed on screen by clicking a tab at the bottom of the page see Figure 1 Input Forest Plot Subgroup Analysis Moderator Analysis Publication Bias Analysis
48. odel to fixed effects or look it up on the calculations sheet where Q reported at the heterogeneity part corresponds to the Q esiduar in a fixed effects model Higgins and Thompson 2002 provide guidance on the execution of a regression analysis and its interpretation As one of their most important remarks they note that the applicability of regression analysis might be low due to limited data availability For a discussion of the methods applied in the Publication Bias Analysis sheet their application and how they should be interpreted see Sterne Gavaghanb and Egger 2000 and Anzures Cabrera and Higgins 2010 Specifically for the Trim and Fill plot Meta essentials uses an iterative procedure for trimming the set of studies from the right or left re estimate a combined effect size and finally filling the plot with symmetric results on the other side of the mean Meta Essentials runs three iterations of the procedure which is shown to be sufficient for many real life cases Duval amp Tweedie 20004 34 User manual for Meta Essentials 4 Specific features of individual workbooks 4 Specific features of individual workbooks The basic features of Meta Essentials have been discussed above The user will be able to navigate through the different worksheets of a Meta Essentials workbook However each of these workbooks has unique features that must be understood before they can be properly used The features will be discussed in this s
49. ormed correlation coefficients are used and displayed This is a difference with some other sheets where after the necessary computations the Fisher s transformed correlation coefficients are conversed back for presentation into normal correlation coefficients 4 4 Workbooks 6 Partial correlational data xlsx and 7 Semi partial correlational data xlsx Both partial and semi partial correlations are used to compare results of studies that have used different regression models Alo amp Becker 2012 Alo 2014 Although partial and semi partial correlations have the same scale and statistical characteristics as zero order correlation it is 43 User manual for Meta Essentials 4 Specific features of individual workbooks recommended not to mix these three types of correlation because they are essentially different effect size measures For a brief description these effect size measures see the section in which we guide you in selecting the appropriate workbook 4 4 1 Fisher s transformation In every worksheet of Workbook 6 the user can choose to apply Fisher s transformation Note that as yet the distributional behaviours of partial correlations and of Fisher s transformed values are not well known e g Alo amp 2014 p 48 It is recommended to run both analyses and compare the results 4 4 2 Input sheet There are various input options Workbooks 6 and 7 see Figure 59 and Figure 60 The user can either insert pre cal
50. rdized or standardized Unstandardized regression weights will almost never be meta analysed because this would require that all studies would use exactly the same measurement instruments with the same scales for both the independent and dependent variable However in the exceptional case that the user has this type of data the user could also use the generic workbook 1 assuming that the standard errors are available as well Workbook 5 Correlational data xlsx is designed to meta analyse bivariate correlations Generally when people refer to correlations they mean this type of correlation which is sometimes also referred to as Pearson s correlation All workbooks discussed so far 2 5 are used to meta analyse effect sizes for bivariate effects However it is very common in studies with effect sizes of the r family that the effect of a set of multiple independent variables on an independent variable is studied A problem for meta analysis is that it is very rare that the same set of independent variables with the same method of measurement is used across all studies This means that the regression weights generated in different studies cannot be compared directly because they are controlled for different sets of other independent variables The remaining workbooks 6 and 7 provide two slightly different solutions for this situation Workbook 6 Partial correlational data xlsx is designed to meta analyse partial correlations of tw
51. replaces p values of zero with 10 which natural logarithmic is 704 59 shown in several instances in the example Figure 44 Example of Failsafe N part of the Calculations sheet 3 7 Statistical procedures Meta Essentials applies the inverse variance weighting method with in the random effects model an additive between studies variance component based on the DerSimonian Laird estimator 33 User manual for Meta Essentials 3 Work with the workbooks DerSimonian amp Laird 1986 Note that in Workbook 2 Differences between independent groups binary data xlsx you can choose between three weighting methods The confidence intervals are estimated using the weighted variance method for random effects models see S nchez Meca and Mar n Mart nez 2008 This means that the standard error and thus the confidence and prediction intervals of the combined effect size calculated by Meta Essentials might be different from one calculated by another meta analysis program but has been shown to be better Specifically for the moderator analysis the Q statistics in Meta Essentials depend on whether the fixed or random effects model is chosen in contrast to some other tools for meta analysis that report Q statistics that are based on the fixed effects model only even if the random effects model is selected If the user is interested in the Q statistics that are based on the fixed effects model the user can temporarily set the meta analysis m
52. s sheet of Meta Essentials contains all calculations underlying the output in the other sheets It can be accessed by clicking on the regarding tab as shown in Figure 17 It is a rather extensive sheet in which different parts are clearly indicated by a header This sheet has no other function than documenting the intermediary outputs of an analysis or equivalently the transformations between input and output Because reading this sheet is not necessary for running a 25 User manual for Meta Essentials 3 Work with the workbooks meta analysis or for other types of analysis in Meta Essentials and because column names are hopefully self explanatory this sheet will be discussed only very briefly in this manual Input Forest Plot Subgroup Analysis Moderator Analysis Publication Bias Analysis Figure 30 The tab to access the Publication Bias Analysis sheet of Meta Essentials 3 6 1 Forest Plot The first part contains the necessary calculations to construct the Forest Plot sheet see Figure 31 for an example The first two columns give ranks for the presentation functions Sort By and Order on the Input sheet Effect sizes variances standard errors the weights in both fixed and random effects models are provided along with the confidence interval limits Finally the relative weight the study receives in the model is given based on the choice between fixed and random effects The second table repeats the estimates of effect
53. s size as well as the lengths of the confidence interval bars for plotting purposes On the right side the same is done for the combined effect size Note that cells showing N A are meant to show these errors since it is the only way to let Microsoft Excel ignore them when making the plots unfortunately 1 E ea cl A ES amp a Output it Bim Hm diis Rise EE We lehti weien Lower Upper Weight Residual Forest Bar Bar CES forest plot number name size observations error fixed random ER 96 limit Limit plot LL UL aL dl 1aaaa 2 0 100 0 25 15 73 0 73 1 70 2 70 8 2296 1 13 2 20 0 50 0 50 Display CES 13 21872 2 bbbb 1 80 130 0 21 23 33 0 74 1 39 2 21 8 3596 0 73 1 80 0 41 0 41 CI Bar width LL 0 74 3 3 3 cccc 1 90 80 0 27 13 97 0 73 1 37 2 43 8 1796 0 83 1 90 0 53 0 53 Cl Bar width UL 0 74 4 4 4 dddd 2 05 300 0 14 49 33 0 75 1 77 2 33 8 49 0 98 2 05 0 28 0 28 PI Bar width LL 2 62 5 5 5 eeee 0 05 95 0 20 24 06 0 74 0 35 0 45 8 3596 1 02 0 05 0 40 0 40 PI Bar width UL 2 62 6 6 6 ffff 0 60 90 0 21 21 89 0 74 1 02 0 18 8 33 1 67 0 60 0 42 0 42 Size of bubble 0 08 7 7 7 gggg 2 00 120 0 22 20 17 0 74 1 56 2 44 8 30 0 93 2 00 0 44 0 44 8 8 8 hhhh 1 80 130 0 21 23 33 0 74 1 39 2 21 8 3596 0 73 1 80 0 41 0 41 DEMO 9 iii 0 40 80 0 22 19 98 0 74 0 05 0 85 8 30 0 67 0 40 0 45 0 45 10 10 10 jjij 2 10 240 0 16 38 84 0 75 1 78 2 42 8 45 1 03 2 10 0 32 0 32 11 11 11 kkkk 0 40 90 0 21 22 43 0 74 0 82 0 02 8 33 1 47 0 40 0 42 0 42 12 12 12 1
54. sections for each procedure or statistic regarding publication bias analysis 3 6 4 1 Funnel plot The funnel plot section contains information for weighting ranks for the trim and fill plot discussed later in the first table and gives estimates of the funnel lines confidence and prediction interval bars of observed and adjusted combined effect sizes for plotting purposes in the rest of the tables see Figure 36 for an example Figure 36 Example of Funnel Plot part of the Calculations sheet 3 6 4 2 Trim and fill plot The first table in the trim and fill plot section gives the differences between the study s effect size and the combined effect size denoted by X its absolute and the weight the study receives see Figure 37 for an example This is given three times because of the before mentioned three iterations The ranks in the funnel plot section are derived by ranking the absolutes of X and 2 wo User manual for Meta Essentials 3 Work with the workbooks multiplying by minus one if Xi is negative for pragmatic reasons these are included in the Funnel Plot part shown before in Figure 36 The second part gives the combined effect size iterations with the combined effect size heterogeneity and estimated number of missing studies per iteration Figure 37 Example of first and second part of Funnel Plot part of the Calculations sheet The third part contains information for the imputed data points the estimated effect
55. sert or delete any columns or cells in the calculations sheet because this might distort the calculations In case you run into trouble you can try running a meta analysis in a fresh workbook of Meta Essentials You can easily do this by copying the data that you have filled in the Input sheet and paste that in the fresh workbook It is recommended to use the option paste values which is available under Paste options when right clicking see the red rectangle in Figure 2 7 cut EB lr Paste Options ai ai Po po po ai e Al amp e d Paste Special Ie opy Insert Copied Cells Malata Figure 2 The right click menu for pasting values User manual for Meta Essentials 3 Work with the workbooks 3 1 Input sheet By default the sheet that you will see when you open a workbook is the Input sheet If not you can access it by clicking on the regarding tab as shown in Figure 3 The Input sheet of workbook 1 Effect size xlsx has nine columns Input is required only in the columns for Effect size and Standard error see Required data in Figure 4 Forest Plot Subgroup Analysis Moderator Analysis Publication Bias Analysis Calculations Figure 3 The tab to access the Forest Plot sheet of Meta Essentials l H Study name stu Optional Required Optional Optional Figure 4 Example of the Input sheet In workbook 1 you need to insert the number of observations i e the sample siz
56. shown in Figure 8 SL Input Forest Plot ire ara ne Publication Bias Analysis Calculations Figure 8 The tab to access the Subgroup Analysis sheet of Meta Essentials The left side of this sheet is similar to the left side of the Forest Plot sheet see Figure 9 For the sake of clarity we make us of a feature of Microsoft Excel that offers the opportunity to hide certain columns These parts can be accessed by clicking the plus sign at the top of the column see Figure 10 When the first plus is clicked a table appears with individual study results combined effect sizes per subgroup and the overall combined effect size see Figure 11 Random effects Random effects Tau pooled over subgroups 9596 1555 18 36 33 Figure 9 Example of the left part of the Subgroup Analysis sheet 13 User manual for Meta Essentials 3 Work with the workbooks Random effects Random effects Tau pooled over subgroups Table with studies and subgroups Forest Plot with subgroups only Forest Plot with studies and subgroups Figure 10 Example of plus signs in the Subgroup Analysis sheet that can be clicked to unhide a set of columns 2 70 9 59 2 21 13 36 2 43 8 65 2 33 23 39 2 44 11 85 2 21 13 36 2 42 19 79 Subgroup AA 1 99 1 88 2 10 50 32 3 14 0 791 0 00 0 01 0 10 177 2 21 0 35 21 07 1 02 19 50 0 05 18 08 0 82 19 90 3 0 90 21 4596 Supgroup BB 1 14
57. sis part of the Calculations sheet The third and final section of the Subgroup Analysis part of the Calculation sheet contains information for the combined effect size as well as heterogeneity measures see Figure 34 for an example Below these estimates the between and within subgroup weighting methods input options are given in text for reference purposes 27 User manual for Meta Essentials 3 Work with the workbooks Figure 34 Example of third part of Subgroup Analysis part of the Calculations sheet 3 6 3 Moderator Analysis This part of the Calculations sheet contains the necessary calculations for the Moderator Analysis sheet see Figure 35 for an example The first table repeats information from the input section Moderator and Effect Size for plotting purposes The first table furthermore contains information from a fixed effects model the second table from a random effects model and the third for the combined effect size and heterogeneity measures in a fixed effects model Below the third table the regression line is given for plotting purposes 28 User manual for Meta Essentials 3 Work with the workbooks Figure 35 Example of Moderator Analysis part of the Calculations sheet 3 6 4 Publication Bias Analysis This part of the Calculations sheet contains the necessary calculations for the Publication Bias Analysis sheet and is divided a similar fashion as the regarding sheet That is the sheet is divided in
58. sists of four sections a table with studies a plot regression estimates and an input option for the calculation of sample quantiles The table presents the study names the estimated normal quantile and the sample quantile The plot gives these normal and sample quantiles as well as a regression line through them With the input option the user can choose to base the sample quantiles on either Standardized residuals or Z scores see red rectangle in Figure 25 23 User manual for Meta Essentials 3 Work with the workbooks Study name Normal quantile Sample quantile N orma Qua ntile P ot aaaa 0 79 4 62 bbbb 0 10 3 69 cccc 0 10 3 19 dddd 1 61 7 56 eeee 0 53 5 20 ffff 1 10 8 10 g888 0 53 4 34 hhhh 0 31 3 69 iiii 0 31 3 08 jjjj 1 10 6 91 kkkk 0 79 7 22 Ht 1 61 8 11 2 D c c 3 o S a E p Un o o 0 00 Normal quantile Regression estimate Estimate SE CI LL CI UL Intercept 0 19 0 64 zd 1 60 Slope 6 06 0 71 4 49 7 64 Base sample quantiles on Standardized residuals Figure 25 Example of Normal Quantile Plot part of the Publication Bias Analysis sheet 3 5 6 Failsafe N tests The final part of the Publication Bias Analysis sheet contains several estimates of Failsafe numbers To illustrate this imagine that for any study a number of other studies is not published Assume that these additional studies have insignificant results i e their effect sizes are essentially zero Then the
59. size and standard error and repeats that information in the fourth table for plotting purposes see Figure 38 for an example Figure 38 Example of third and fourth part of Trim and Fill part of the Calculations sheet Ww 0 User manual for Meta Essentials 3 Work with the workbooks 3 6 4 3 Egger regression The calculations for the Egger regression are mostly executed directly on the Publication Bias Analysis sheet except for some estimates needed for the calculations of the standard errors of the regression estimates see Figure 39 for an example Figure 39 Example of Egger Regression part of the Calculations sheet 3 6 4 4 Begg amp Mazumdar rank correlation test For the Begg amp Mazumdar rank correlation test an adjusted effect size and variance are first derived followed by their respective ranks all denoted with a star to indicate that they are adjusted estimates see Figure 40 for an example Under x the count of concordant ranks is given and under y the count of discordant ranks is given The remainder of the calculations is executed directly on the Publication Bias Analysis sheet Figure 40 Example of Begg amp Mazumdar Rank Correlation part of the Calculations sheet 31 User manual for Meta Essentials 3 Work with the workbooks 3 6 4 5 Standardized Residual Histogram The calculations for the standardized residual histogram consist of one table below for the calculation of the width of the bins
60. such studies are the odds ratio risk ratio and the risk difference Workbook 3 Differences between independent groups continuous data xlsx is designed to meta analyse studies of which the outcome is a difference between the means of two independent groups For instance to test whether a training has a positive effect on the sales of sales personnel a study might be designed that gives one group of salespersons a training and another group no training The effect size of interest would then be the difference between the average sales of the persons that received training compared to that of the persons that did not receive training Workbook 4 Differences between dependent groups continuous data xlsx is designed to meta analyse studies of which the outcome is a difference between the means of two measurements in the same group In comparison to the previous example this is the effect size in a study of a difference in sales in the same group of persons before and after training This is often referred to as User manual for Meta Essentials 2 Select the appropriate workbook a pre posttest study design On the face of it there are few differences between workbooks 3 and 4 However the calculations behind the workbooks are different 2 2 Effectsizes in the r family There are two common types of effect size in the r family correlation coefficients which are unit free by definition and regression weights which can be unstanda
61. ta xlsx 41 51 L Abb plot One additional plot is provided for binary data the L Abb plot L Abb Detsky amp O Rourke 1987 see Figure 51 This plot gives the Group 2 e g control risk on the x axis and the Group 1 e g treatment risk on the y axis A reference line of zero effect the diagonal is provided in red along with a blue dotted line that gives the ratio between the risks of group 2 and group 1 the combined Risk Ratio The size of the point estimates blue dots corresponds to the study weights The study weights depend on the chosen model fixed versus random effects and on the chosen weighting method 38 User manual for Meta Essentials 4 Specific features of individual workbooks L Abbe plot 0 70 0 60 0 50 o A o Group 1 risk o w o 0 20 0 10 0 00 0 00 0 05 0 10 0 15 0 20 0 25 0 30 0 35 0 40 0 45 0 50 Group 2 risk amp Risks Zero effect Observed effect 1 35 Figure 51 L Abb Plot on the Publication Bias Analysis sheet of Workbook 2 Differences between independent groups binary data xlsx 4 1 6 Calculations sheet The calculations sheet for binary data begins with a repetition of the cell counts and the Add 0 5 asks whether any of the cells has a count of zero in which case 5 should be added to all the cell counts because the effect sizes are not calculable otherwise In this tab you will see additional columns with log effect sizes for calc
62. the same input options are available in Workbook 4 41 User manual for Meta Essentials 4 Specific features of individual workbooks which is used when the effect size is a difference between two measurements in the same group e g a pre test and a post test Sample size is also required in this workbook of only one group by definition as is the correlation coefficient r describing the association between pairs of Observations in the regarding study However since this correlation is often not reported and cannot be derived from other provided statistics the researcher will need to use data from other sources to estimate this correlation If the correlation is not known precisely one could work with a range of plausible correlations and use a sensitivity analysis to see how these affect the results Borenstein 2009 pp 227 228 If more than sufficient information is entered Meta Essentials will automatically use the simplest option effect sizes first where g is preferred over d than means with standard errors and finally t values and F values Effect sizes will automatically be calculated as standardized mean differences On the output sheets the user can select either Cohen s d or Hedges g as effect size measure 4 2 2 Effect size measures Cohen s d and Hedges g are both standardized mean differences Cohen s d was first developed by Cohen 1969 and then Hedges 1981 found a bias particularly pr
63. uantile plot and several failsafe N tests They can be accessed by clicking on the regarding tab as shown in Figure 17 17 User manual for Meta Essentials 3 Work with the workbooks Input Forest Plot Subgroup Analysis Moderator Analysis Publication Bias Analysis f Calculations E Figure 17 The tab to access the Publication Bias Analysis sheet of Meta Essentials Because most of the publication bias analyses only make sense for a fixed effects model we have set that as default for this sheet You might however change it to random effects model in the table on the left of the sheet where you can set the confidence level for confidence and prediction intervals as well see red rectangle labelled Choose options here in Figure 18 Meta analysis model Model Fixed effects Confidence level 95 Choose options here Figure 18 Example of left part of Publication Bias Analysis sheet As in the Subgroup Analysis sheet the user must click on the plus sign to open a particular procedure see red rectangle in Figure 19 18 User manual for Meta Essentials 3 Work with the workbooks Funnel Plot with trim and fill Standardized Residual Histogram Galbraith Plot Normal Quantile Plot Failsafe N tests Egger Regression and Begg amp Mazumdar Rank Correlation Figure 19 Example of right part of Publication Bias Analysis sheet 3 5 1 Funnel plot A funnel plot see Figure 20 is a scatter plot of the studies in
64. ulation purposes Two additional headers and thus chapters of the tab are provided Effect Sizes and Weighting Methods In Effect Sizes four parts describing the calculations for different effect sizes are given odds ratio Peto odds ratio risk difference and risk ratio see Figure 52 for an example In Weighting Methods the three weighting methods are given Inverse Variance Mantel Haenszel and Peto see Figure 53 for an example along with some information for the conversion of one effect size measure into the other see Figure 54 for an example Odds Ratio Risk Difference Risk Ratio X E oO E xs 3 e o 2 ET a 39 User manual for Meta Essentials 4 Specific features of individual workbooks Figure 52 Example of Effect Sizes part of Calculations tab of Workbook 2 Differences between independent groups binary data xlsx Figure 53 Example of Weighting Method part of Calculations tab of Workbook 2 Differences between independent groups binary data xlsx Figure 54 Example of Conversion to Other Effect Size Measures part of Calculations tab of Workbook 2 Differences between independent groups binary data xlsx 42 Workbooks 3 Differences between independent groups continuous data xlsx and 4 Differences between dependent groups continuous data xlsx 4 2 1 Inputsheet Workbooks 3 and 4 have a rather large number of different input formats see Figure 55 for Workboo
65. val which is often interpreted as indicating a range within which we can be 95 certain that the true effect lies This statement is a loose interpretation but is useful as a rough guide The strictly correct interpretation is that i f a study were repeated infinitely often and on each occasion a 95 confidence interval calculated then 9596 of these intervals would contain the true effect Sch nemann Oxman Vist Higgins Deeks Glasziou amp Guyatt 2011 Section 12 4 1 As this is a user manual for the software of Meta Essentials and not an introduction to the aims and best practices of meta analysis we cannot expand here on the importance of the prediction interval vis vis the confidence interval but see e g Hak Van Rhee amp Suurmond 2015a Higgins Thompson amp Spiegelhalter 2009 3 3 Subgroup Analysis sheet When the user has entered a category in the Subgroup column of the Input sheet then the Subgroup Analysis sheet will present meta analytic results for each subgroup separately For instance if the user has coded the origin of the data used in a study as either USA or Non USA this sheet will give a combined effect size for the USA studies and another combined effect size for the 12 User manual for Meta Essentials 3 Work with the workbooks Non USA studies as well as an combined effect size for all included studies You can access the sheet by clicking on the regarding tab as

User manual for Meta-Essentials: Workbooks for meta

Contents

Download Pdf Manuals

Related Search

Related Contents