From accbde97c3f766c44835e455f3d23dfc0b20de8f Mon Sep 17 00:00:00 2001 From: Michael Rosenberg Date: Tue, 16 May 2023 10:36:36 -0400 Subject: [PATCH 1/3] Typo in references Fixed the initial of the second author in the references for Huedo-Medina et al 2006 --- resources/metawin_help.html | 12 ++++++------ src/MetaWinConstants.py | 2 +- 2 files changed, 7 insertions(+), 7 deletions(-) diff --git a/resources/metawin_help.html b/resources/metawin_help.html index cea3816..a687a67 100644 --- a/resources/metawin_help.html +++ b/resources/metawin_help.html @@ -1008,7 +1008,7 @@

Output

Dixon, P.M. (1993) The bootstrap and the jackknife: Describing the precision of ecological indices. Pp. 290—318 in Design and Analysis of Ecological Experiments, S.M. Scheiner and J. Gurevitch, eds. Chapman and Hall, New York.

Hedges, L.V. and I. Olkin (1985) Statistical Methods for Meta-analysis. Academic Press, Orlando, FL.

Higgins, J.P.T. and S.G. Thompson (2002) Quantifying heterogeneity in a meta-analysis. Statistics in Medicine 21:1539–1558.

-

Huedo-Medina, T.B., F. Sánchez-Meca, F. Marín-Martínez, and J. Botella (2006) Assessing heterogeneity in meta-analysis: Q statistic or I2 index? Psychological Methods 11:193–206.

+

Huedo-Medina, T.B., J. Sánchez-Meca, F. Marín-Martínez, and J. Botella (2006) Assessing heterogeneity in meta-analysis: Q statistic or I2 index? Psychological Methods 11:193–206.

Orwin, R.G. (1983) A fail-safe N for effect size in meta-analysis. Journal of Educational Statistics 8(2):157–159.

Rosenberg, M.S. (2005) The file-drawer problem revisited: A general weighted method for calculating fail-safe numbers in meta-analysis. Evolution 59(2):464–468.

Rosenthal, R. (1979) The “file drawer problem” and tolerance for null results. Psychological Bulletin 86(3):638–641.

@@ -1360,7 +1360,7 @@

Output

Hedges, L.V. and I. Olkin (1985) Statistical Methods for Meta-analysis. Academic Press, Orlando, FL.

Higgins, J.P.T. and S.G. Thompson (2002) Quantifying heterogeneity in a meta-analysis. Statistics in Medicine 21:1539–1558.

-

Huedo-Medina, T.B., F. Sánchez-Meca, F. Marín-Martínez, and J. Botella (2006) Assessing heterogeneity in +

Huedo-Medina, T.B., J. Sánchez-Meca, F. Marín-Martínez, and J. Botella (2006) Assessing heterogeneity in meta-analysis: Q statistic or I2 index? Psychological Methods 11:193–206.

@@ -1722,7 +1722,7 @@

Output

Hedges, L.V. and I. Olkin (1985) Statistical Methods for Meta-analysis. Academic Press, Orlando, FL.

Higgins, J.P.T. and S.G. Thompson (2002) Quantifying heterogeneity in a meta-analysis. Statistics in Medicine 21:1539–1558.

-

Huedo-Medina, T.B., F. Sánchez-Meca, F. Marín-Martínez, and J. Botella (2006) Assessing heterogeneity in +

Huedo-Medina, T.B., J. Sánchez-Meca, F. Marín-Martínez, and J. Botella (2006) Assessing heterogeneity in meta-analysis: Q statistic or I2 index? Psychological Methods 11:193–206.

@@ -1917,7 +1917,7 @@

Output

Hedges, L.V. and I. Olkin (1985) Statistical Methods for Meta-analysis. Academic Press, Orlando, FL.

Higgins, J.P.T. and S.G. Thompson (2002) Quantifying heterogeneity in a meta-analysis. Statistics in Medicine 21:1539–1558.

-

Huedo-Medina, T.B., F. Sánchez-Meca, F. Marín-Martínez, and J. Botella (2006) Assessing heterogeneity in +

Huedo-Medina, T.B., J. Sánchez-Meca, F. Marín-Martínez, and J. Botella (2006) Assessing heterogeneity in meta-analysis: Q statistic or I2 index? Psychological Methods 11:193–206.

Rosenberg, M.S. (2013) Moment and least-squares based approaches to meta-analytic inference. Pp. 108–124 in Handbook of Meta-analysis in Ecology and Evolution, J. Koricheva, J. Gurevitch and K.L. Mengersen, eds. @@ -2121,7 +2121,7 @@

Output

Hedges, L.V. and I. Olkin (1985) Statistical Methods for Meta-analysis. Academic Press, Orlando, FL.

Higgins, J.P.T. and S.G. Thompson (2002) Quantifying heterogeneity in a meta-analysis. Statistics in Medicine 21:1539–1558.

-

Huedo-Medina, T.B., F. Sánchez-Meca, F. Marín-Martínez, and J. Botella (2006) Assessing heterogeneity in +

Huedo-Medina, T.B., J. Sánchez-Meca, F. Marín-Martínez, and J. Botella (2006) Assessing heterogeneity in meta-analysis: Q statistic or I2 index? Psychological Methods 11:193–206.

@@ -2351,7 +2351,7 @@

Output

Hedges, L.V. and I. Olkin (1985) Statistical Methods for Meta-analysis. Academic Press, Orlando, FL.

Higgins, J.P.T. and S.G. Thompson (2002) Quantifying heterogeneity in a meta-analysis. Statistics in Medicine 21:1539–1558.

-

Huedo-Medina, T.B., F. Sánchez-Meca, F. Marín-Martínez, and J. Botella (2006) Assessing heterogeneity in +

Huedo-Medina, T.B., J. Sánchez-Meca, F. Marín-Martínez, and J. Botella (2006) Assessing heterogeneity in meta-analysis: Q statistic or I2 index? Psychological Methods 11:193–206.

Rosenberg, M.S., D.C. Adams, and J. Gurevitch (2000) MetaWin: Statistical Software for Meta-analysis. Sinauer Associates, Sunderland, MA.

diff --git a/src/MetaWinConstants.py b/src/MetaWinConstants.py index 618fd69..0b083b6 100644 --- a/src/MetaWinConstants.py +++ b/src/MetaWinConstants.py @@ -190,7 +190,7 @@ def resource_path(relative_path: str, inc_file: bool = False) -> str: "Higgins_Thompson_2002": ["Higgins, J.P.T. and S.G. Thompson (2002) Quantifying heterogeneity in a meta-analysis. " "Statistics in Medicine 21:1539–1558.", "Higgins and Thompson (2002)"], - "Huedo-Medina_et_2006": ["Huedo-Medina, T.B., F. Sánchez-Meca, F. Marín-Martínez, and J. Botella (2006) " + "Huedo-Medina_et_2006": ["Huedo-Medina, T.B., J. Sánchez-Meca, F. Marín-Martínez, and J. Botella (2006) " "Assessing heterogeneity in meta-analysis: Q statistic or I2 index? " "Psychological Methods 11:193–206.", "Huedo-Medina et al. (2006)"], From d6c84ced249ad9ba547eda3f8d4490254c6bcb83 Mon Sep 17 00:00:00 2001 From: Michael Rosenberg Date: Tue, 16 May 2023 10:56:35 -0400 Subject: [PATCH 2/3] I2 confidence intervals bug The formula I was using for the confidence interval for I2 was wrong (specifically, I was using the formula for the SE of ln(h) as if it was the SE of ln(h2)). --- src/MetaWinAnalysisFunctions.py | 14 +++++++------- tests/borenstein_chap18.txt | 7 +++++++ tests/test_metawin.py | 13 +++++++++++++ 3 files changed, 27 insertions(+), 7 deletions(-) create mode 100644 tests/borenstein_chap18.txt diff --git a/src/MetaWinAnalysisFunctions.py b/src/MetaWinAnalysisFunctions.py index ab2eb4d..f1b7872 100644 --- a/src/MetaWinAnalysisFunctions.py +++ b/src/MetaWinAnalysisFunctions.py @@ -368,16 +368,16 @@ def bootstrap_means(bootstrap_n, boot_data, obs_mean, pooled_var, random_effects def calc_i2(qt, n, alpha: float = 0.05): try: i2 = max(0, 100 * (qt - (n - 1))/qt) - ln_h2 = math.log(qt/(n - 1)) + ln_h = math.log(math.sqrt(qt/(n - 1))) if qt > n - 1: - se = (math.log(qt) - math.log(n-1))/(2*(math.sqrt(2*qt)-math.sqrt(2*n - 3))) + se_ln_h = (math.log(qt) - math.log(n-1))/(2*(math.sqrt(2*qt)-math.sqrt(2*n - 3))) else: - se = math.sqrt((1/(2*(n - 2))) * (1 - (1/(3*(n - 2)**2)))) + se_ln_h = math.sqrt((1/(2*(n - 2))) * (1 - (1/(3*(n - 2)**2)))) z = -scipy.stats.norm.ppf(alpha / 2) - lower_h2 = math.exp(ln_h2 - z*se) - upper_h2 = math.exp(ln_h2 + z*se) - lower_i2 = max(0, 100*(lower_h2 - 1)/lower_h2) - upper_i2 = max(0, 100*(upper_h2 - 1)/upper_h2) + lower_h = math.exp(ln_h - z*se_ln_h) + upper_h = math.exp(ln_h + z*se_ln_h) + lower_i2 = max(0, 100*(lower_h**2 - 1)/lower_h**2) + upper_i2 = max(0, 100*(upper_h**2 - 1)/upper_h**2) except ZeroDivisionError: i2, lower_i2, upper_i2 = 0, 0, 0 return i2, lower_i2, upper_i2 diff --git a/tests/borenstein_chap18.txt b/tests/borenstein_chap18.txt new file mode 100644 index 0000000..1884ae0 --- /dev/null +++ b/tests/borenstein_chap18.txt @@ -0,0 +1,7 @@ +yi vi +0.09452 0.03295 +0.27736 0.03070 +0.36654 0.04988 +0.66438 0.01051 +0.46181 0.04266 +0.18516 0.02342 diff --git a/tests/test_metawin.py b/tests/test_metawin.py index 404521d..4cb021c 100644 --- a/tests/test_metawin.py +++ b/tests/test_metawin.py @@ -348,6 +348,19 @@ def test_simple_meta_analysis_lep(): assert round(analysis_values.i2, 2) == i2_answer +def test_i2_confidence_interval(): + # not a formal test, just crosschecking values + data, _ = import_test_data("borenstein_chap18.txt") + options = MetaWinAnalysis.MetaAnalysisOptions() + options.structure = MetaWinAnalysis.SIMPLE_MA + options.effect_data = data.cols[0] + options.effect_vars = data.cols[1] + options.create_graph = False + + output, figure, chart_data, analysis_values = MetaWinAnalysis.do_meta_analysis(data, options, 4) + print_test_output(output) + + def test_simple_meta_analysis_lep_randeff(): # answers from Chapter 9, Handbook of Meta-Analysis in Ecology and Evolution qt_answer = 23.01 From 187a3b57b842aed13f5184f8caf44a7b74af7cf7 Mon Sep 17 00:00:00 2001 From: Michael Rosenberg Date: Tue, 16 May 2023 11:07:27 -0400 Subject: [PATCH 3/3] updated help to reflect correct I2 values --- resources/metawin_help.html | 16 ++++++++-------- 1 file changed, 8 insertions(+), 8 deletions(-) diff --git a/resources/metawin_help.html b/resources/metawin_help.html index a687a67..5931978 100644 --- a/resources/metawin_help.html +++ b/resources/metawin_help.html @@ -951,7 +951,7 @@

Output


 Source     I2            95% CI
 -------------------------------------
-Total    48.4444   34.9546 to 59.1366
+Total    48.4444   17.9350 to 67.6113

→ Citations: Higgins and Thompson (2002), Huedo-Medina et al. (2006) @@ -1331,7 +1331,7 @@

Output


 Source     I2            95% CI
 -------------------------------------
-Total    48.4444   34.9546 to 59.1366
+Total    48.4444   17.9350 to 67.6113

→ Citations: Higgins and Thompson (2002), Huedo-Medina et al. (2006) @@ -1652,8 +1652,8 @@

Output


        Source            I2            95% CI
 ---------------------------------------------------
-Heterocera (within)    57.8379   45.2319 to 67.5424
-Rhopalocera (within)    3.0254    0.0000 to 16.5089
+Heterocera (within)    57.8379   28.8567 to 75.0132
+Rhopalocera (within)    3.0254    0.0000 to 28.1177

→ Citations: Higgins and Thompson (2002), Huedo-Medina et al. (2006) @@ -1693,7 +1693,7 @@

Output


 Source     I2            95% CI
 -------------------------------------
-Total    48.4444   34.9546 to 59.1366
+Total    48.4444   17.9350 to 67.6113

→ Citations: Higgins and Thompson (2002), Huedo-Medina et al. (2006) @@ -1888,7 +1888,7 @@

Output


 Source     I2            95% CI
 -------------------------------------
-Total    53.3385   40.5512 to 63.3753
+Total    53.3385   24.2595 to 71.2532

→ Citations: Higgins and Thompson (2002), Huedo-Medina et al. (2006) @@ -2091,7 +2091,7 @@

Output


 Source     I2            95% CI
 -------------------------------------
-Total    54.3676   41.9334 to 64.1391
+Total    54.3676   26.1112 to 71.8182

→ Citations: Higgins and Thompson (2002), Huedo-Medina et al. (2006) @@ -2322,7 +2322,7 @@

Output


 Source     I2            95% CI
 -------------------------------------
-Total    54.3676   41.9334 to 64.1391
+Total    54.3676   26.1112 to 71.8182

→ Citations: Higgins and Thompson (2002), Huedo-Medina et al. (2006)