forked from MathiasHarrer/Doing-Meta-Analysis-in-R
-
Notifications
You must be signed in to change notification settings - Fork 5
/
Copy path12-Effectsizeconverter.Rmd
338 lines (218 loc) · 14.5 KB
/
12-Effectsizeconverter.Rmd
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
# Effect size calculators
Although the `meta` package can calculate all **individual effect sizes for every study** if we use the `metabin` or `metacont` function, a frequent scenario is that **some papers do not report the effect size data in the right format**. Especially older articles may often only report results of $t$**-tests**, **ANOVAs**, or $\chi^2$**-tests**. If enough data is reported, we can also use **such outcome formats to calculate effect sizes**. This way, we can calculate the **effect size (e.g., Hedges' g)** and the **Standard Error (SE)**, which we can then use in a meta-analysis with **pre-calculated effect sizes** using the `metagen` function (see [Chapter 4.1.1](#pre.calc)).
```{block,type='rmdinfo'}
**Hedges' g**
When dealing with **continuous outcome data**, it is conventional to calculate the **Standardized Mean Difference** (SMD) as an outcome for each study, and as your **summary measure** [@borenstein2011].
A common format to to calculate the SMD in single trials is **Cohen's d** [@cohen1988statistical]. Yet, this summary measure has been **shown to have a slight bias in small studies, for which it overestimates the effect** [@hedges1981distribution].
**Hedges *g* ** is a similar summary measure, but it **controls for this bias**. It uses a slightly different formula to calculate the pooled variance $s_{pooled}$, $s*_{pooled}$. The transformation from *d* to *g* is often performed using the formula by Hedges and Olkin [@hedges1985statistical].
$$g \simeq d\times(1-\frac{3}{4(n_1+n_2)-9}) $$
```
```{block,type='rmdachtung'}
Hedges' g is **commonly used in meta-analysis**, and it's the standard output format in **RevMan**. Therefore, we highly recommend that you also use this measure in you meta-analysis.
In `meta`'s `metabin` and `metacont` function, Hedges' g is automatically calculated for each study if we set `sm="SMD"`. If you use the `metgen` function, however, you should calculate Hedges' g for each study yourself first.
```
To calculate the effect sizes, we will use Daniel Lüdecke's extremely helpful `esc` package [@esc]. So, please **install this package first** using the `install.packages("esc")` command, and then load it in you library.
```{r,message=FALSE}
library(esc)
```
<br><br>
**Here's an overview of all calculators covered in this guide**
1. [Calculating Hedges' *g* from the Mean and SD](#a)
2. [Calculating Hedges' *g* from a regression coefficient](#b)
3. [Calculating an Odd's Ratio from *Chi-square*](#c)
4. [Calculating Hedges' *g* from a one-way ANOVA](#d)
5. [Calculating Hedges' *g* from the Mean and SE](#e)
6. [Calculating Hedges' *g* from a correlation](#f)
7. [Calculating Hedges' *g* from an independent t-test](#g)
8. [Calculating Hedges' *g* from Cohen's *d*](#h)
9. [Calculating effect sizes for studies with multiple comparisons](#i)
<br><br>
## Calculating Hedges' g from the Mean and SD {#a}
To calculate Hedges' *g* from the *Mean*, *Standard Deviation*, and $n_{group}$ of both trial arms, we can use the `esc_mean_sd` function with the following parameters.
* `grp1m`: The **mean** of the **first group** (e.g., the intervention).
* `grp1sd`: The **standard deviation** of the **first group**.
* `grp1n`: The **sample size** of the **first group**.
* `grp2m`: The **mean** of the **second group**.
* `grp2sd`: The **standard deviation** of the **second group**.
* `grp2n`: The **sample size** of the **second group**.
* `totalsd`: The **full sample standard deviation**, if the standard deviation for each trial arm is not reported
* `es.type`: the **effect measure** we want to calculate. In our case this is `"g"`. But we could also calculate Cohen's *d* using `"d"`.
<br><br>
**Here's an example**
```{r}
esc_mean_sd(grp1m = 10.3, grp1sd = 2.5, grp1n = 60,
grp2m = 12.3, grp2sd = 3.1, grp2n = 56, es.type = "g")
```
## Calculating Hedges' *g* from a regression coefficient {#b}
### Unstandardized regression coefficients
It is also possible to calculate **Hedges' *g* ** from an unstandardized or standardized regression coeffiecent [@lipsey2001practical].
For **unstardardized coefficients**, we can use the `esc_B` function with the following parameters:
* `b`: unstandardized coefficient $b$ (the "treatment" predictor).
* `sdy`: the standard deviation of the dependent variable $y$ (i.e., the outcome).
* `grp1n`: the number of participants in the first group.
* `grp2n`: the number of participants in the second group.
* `es.type`: the **effect measure** we want to calculate. In our case this is `"g"`. But we could also calculate Cohen's *d* using `"d"`.
<br><br>
**Here's an example**
```{r}
esc_B(b=3.3,sdy=5,grp1n = 100,grp2n = 150,es.type = "g")
```
<br><br>
### Standardized regression coefficents
Here, we can use the `esc_beta` function with the follwing parameters:
* `beta`: standardized coefficient $\beta$ (the "treatment" predictor).
* `sdy`: the standard deviation of the dependent variable $y$ (i.e., the outcome).
* `grp1n`: the number of participants in the first group.
* `grp2n`: the number of participants in the second group.
* `es.type`: the **effect measure** we want to calculate. In our case this is `"g"`. But we could also calculate Cohen's *d* using `"d"`.
<br><br>
**Here's an example**
```{r}
esc_beta(beta=0.7, sdy=3, grp1n=100, grp2n=150, es.type = "g")
```
## Calculating an Odd's Ratio from *Chi-square* {#c}
To calculate the **Odd's Ratio** (or any other kind of effect size measure) from $\chi^2$ using the `esc_chisq` function with the following paramters:
* `chisq`: The value of Chi-squared (or only `p`)
* `p`: the chi squared p or phi value (or only `chisq`)
* `totaln`: total sample size
* `es.type`: the summary measure (in our case, `"cox.or"`)
<br><br>
**Here's an example**
```{r}
esc_chisq(chisq=9.9,totaln=100,es.type="cox.or")
```
## Calculating Hedges' *g* from a one-way ANOVA {#d}
We can also derive the SMD from the $F$-value of a **one-way ANOVA with two groups**. Such ANOVAs can be detected if you look for the **degrees of freedom** ($df$) underneath of $F$. In a one-way ANOVA with two groups, the degrees of freedom should always start with $1$ (e.g. $F_{1,147}=5.31$). The formula for this transformation looks like this [@cohen1992power;@rosnow1996computing;@rosnow2000contrasts]:
$$d = \sqrt{ F(\frac{n_t+n_c}{n_t n_c})(\frac{n_t+n_c}{n_t+n_c-2})}$$
To calculate **Hedges' g** from $F$-values, we can use the `esc_f` function with the following parameters:
* `f`: *F*-value of the ANOVA
* `grp1n`: Number of participants in group 1
* `grp2n`: Number of participants in group 2
* `totaln`: The total number of participants (if the *n* for each group is not reported)
* `es.type`: the **effect measure** we want to calculate. In our case this is `"g"`. But we could also calculate Cohen's *d* using `"d"`.
<br><br>
**Here's an example**
```{r}
esc_f(f=5.04,grp1n = 519,grp2n = 528,es.type = "g")
```
## Calculating Hedges' *g* from the Mean and SE {#e}
When calculating **Hedges' g** from the **Mean** and **Standard Error**, we simply make use of the fact that the Standard error is not much more than the **Standard Deviation** when the sample size is taken into account [@thalheimer2002calculate]:
$$SD = SE\sqrt{n_c}$$
We can calculate **Hedges' g** using the `esc_mean` function with the following parameters:
* `grp1m`: The mean of the first group.
* `grp1se`: The standard error of the first group.
* `grp1n`: The sample size of the first group.
* `grp2m`: The mean of the second group.
* `grp2se`: The standard error of the second group.
* `grp2n`: The sample size of the second group.
* `es.type`: the **effect measure** we want to calculate. In our case this is `"g"`. But we could also calculate Cohen's *d* using `"d"`.
<br><br>
**Here's an example**
```{r}
esc_mean_se(grp1m = 8.5, grp1se = 1.5, grp1n = 50,
grp2m = 11, grp2se = 1.8, grp2n = 60, es.type = "g")
```
## Calculating Hedges' *g* from a correlation {#f}
For **equally sized groups** ($n_1=n_2$), we can use the following formula to derive the SMD from the pointbiserial **correlation** [@rosenthal1984meta].
$$r_{pb} = \frac{d}{\sqrt{d^2+4}}$$
And this formula for **unequally sized groups** [@aaron1998equating]:
$$r_{pb} = \frac{d}{\sqrt{d^2+ \frac{(N^2-2 \times N)}{n_1 n_2} }}$$
To convert $r_{pb}$ to **Hedges' g**, we can use the `esc_rpb` function with the following parameters:
* `r`: The *r*-value. Either *r* or its *p*-value must be given.
* `p`: The *p*-value of the correlation. Either *r* or its *p*-value must be given.
* `grp1n`: The sample size of group 1.
* `grp2n`: The sample size of group 2.
* `totaln`: Total sample size, if `grp1n` and `grp2n` are not given.
* `es.type`: the **effect measure** we want to calculate. In our case this is `"g"`. But we could also calculate Cohen's *d* using `"d"`.
```{r}
esc_rpb(r = 0.25, grp1n = 99, grp2n = 120, es.type = "g")
```
## Calculating Hedges' *g* from an independent t-test {#g}
The SMD can also be derived from an **independent t-test value** with the following formula [@thalheimer2002calculate]:
$$d = \frac {t(n_1+n_2)}{\sqrt{(n_1+n_2-2)(n_1n_2)}}$$
We can calculate **Hedges' g** from a **t-test** using the `esc_t` function with the following paramters:
* `t`: The t-value of the t-test. Either *t* or its *p*-value must be given.
* `p`: The *p*-value of the t-test. Either *t* or its *p*-value must be given.
* `grp1n`: The sample size of group 1.
* `grp2n`: The sample size of group 2.
* `totaln`: Total sample size, if `grp1n` and `grp2n` are not given.
* `es.type`: the **effect measure** we want to calculate. In our case this is `"g"`. But we could also calculate Cohen's *d* using `"d"`.
<br><br>
**Here's an example**
```{r}
esc_t(t = 3.3, grp1n = 100, grp2n = 150,es.type="g")
```
## Calculating Hedges' *g* from Cohen's *d* {#h}
We can also directly correct **Cohen's *d* ** and thus generate **Hedges' g** using the formula by Hedges and Olkin [@hedges1985statistical]:
$$g \simeq d\times(1-\frac{3}{4(n_1+n_2)-9}) $$
This can be done in R using the `hedges_g` function with the following parameters:
* `d`: The value of **Cohen's d**
* `totaln`: the total *N* in the study
```{r}
hedges_g(d = 0.75, totaln = 50)
```
## Calculating effect sizes for studies with multiple comparisons {#i}
Many randomized-controlled trials do not only include a single **intervention** and **control group**, but compare the effect of **two or more interventions** to a control group. It might be tempting in such a scenario to **simply include all the comparisons between the intervention groups and control within a study into one meta-analysis**. Yet, researchers should abstain from this practice, as this would mean that the control group is used twice for the meta-analysis, thus **"double-counting"** the participants in the control group. This results in a **unit-of-analysis** error, as the effect size are correlated, and thus not independent, but are treated as if they would stem from independent samples.
**There are two ways to deal with this:**
* Splitting the N of the control group: One method to control for the unit-of-analysis error to some extent would be to **split** the number of participants in the control group between the two intervention groups. So, if your control group has $N=50$ participants, you could divide the control group into two control groups with he same mean and standard deviation, and $N=25$ participants each. After this preparation step, you could calculate the effect sizes for each intervention arm. As this procedure only partially removes the unit of analysis error, it is not generally recommended. A big plus of this procedure, however, is that it makes [**investigations of hetereogeneity**](#heterogeneity) between study arms possible.
* Another option would be to **synthesize the results of the intervention arms** to obtain one single comparison to the control group. Despite its practical limitations (sometimes, this would mean synthesizing the results from extremely different types of interventions), this procedure does get rid of the unit-of-analysis error problem, and is thus recommended from a statistical standpoint. The following calculations will deal with this option.
To synthesize the **pooled effect size data** (pooled Mean, Standard Deviation and N), we have to use the following formula:
$$N_{pooled}=N_1+N_2$$
$$M_{pooled}=\frac{N_1M_1+N_2M_2}{N_1+N_2}$$
$$SD_{pooled} = \sqrt{\frac{(N_1-1)SD^{2}_{1}+ (N_2-1)SD^{2}_{2}+\frac{N_1N_2}{N_1+N_2}(M^{2}_1+M^{2}_2-2M_1M_2)} {N_1+N_2-1}}$$
As these formulae are quite lengthy, we prepared the function `pool.groups` for you, which does the pooling for you automatically. Again, R doesn't know this function yet, so we have to let R learn it by **copying and pasting** the code underneath **in its entirety** into the **console** on the bottom left pane of RStudio, and then hit **Enter ⏎**.
```{r}
pool.groups<-function(n1,n2,m1,m2,sd1,sd2){
n1 <- n1
n2 <- n2
m1 <- m1
m2 <- m2
sd1 <- sd1
sd2 <- sd2
Npooled <- n1+n2
Mpooled <- (n1*m1+n2*m2)/(n1+n2)
SDpooled <- sqrt(((n1-1)*sd1^2+(n2-1)*sd2^2+(((n1*n2)/(n1+n2))*(m1^2+m2^2-2*m1*m2)) )/(n1+n2-1))
return(data.frame(Mpooled,SDpooled,Npooled))
}
```
**To use this function, we have to specifiy the following parameters:**
* `n1`: The N in the first group
* `n2`: The N in the second group
* `m1`: The Mean of the first group
* `m2`: The Mean of the second group
* `sd1`: The Standard Deviation of the first group
* `sd2`: The Standard Deviation of the second grop
**Here's an example**
```{r}
pool.groups(n1=50,
n2=50,
m1=3.5,
m2=4,
sd1=3,
sd2=3.8)
```
```{block, type='rmdinfo'}
**What should i do when an study has more than two intervention groups**
If a study has more than one two intervention groups you want to synthesize (e.g. four arms, with three distinct intervention arms), you can **pool the effect size data for the first two interventions**, and then **synthesize the pooled data you calculated with the data from the third group**.
This is fairly straightforward if you save the output from `pool.groups` as an object, and then use the `$` operator:
```
First, pool the **first** and **second intervention group**. I will save the output as `res`.
```{r}
res<-pool.groups(n1 = 50,
n2 = 50,
m1 = 3.5,
m2 = 4,
sd1 = 3,
sd2 = 3.8)
```
Then, use the pooled data saved in `res` and **pool it with the data from the third group**, using the `$` operator to access the different values saved in `res`.
```{r}
pool.groups(n1 = res$Npooled,
n2 = 60,
m1 = res$Mpooled,
m2 = 4.1,
sd1=res$SDpooled,
sd2 = 3.8)
```