Investigation of within-study selective reporting in clinical research: Follow-up of applications submitted to a local research ethics committee

Seokyung Hahn, P. R. Williamson, J. L. Hutton

Research output: Contribution to journalArticle

68 Citations (Scopus)

Abstract

Rationale, aims and objectives Within-study selective reporting is widely believed to exist, although to date there have been no empirical studies to assess the extent of the problem in clinical research. The present study aimed to examine this process. Methods We undertook a pilot study, involving a single local research ethics committee (LREC), in which we compared the outcomes, analysis and sample size proposed in the original approved study protocol with the results presented in the subsequent study report. Results We received 41 (73%) replies from lead researchers of 56 projects, which were a complete cohort of clinical research applications approved in a particular time period by the LREC. Fifteen of these projects, which were completed and published at the time of our study, were further investigated. Only six (40%) stated which outcome variables were of primary interest and four (67%) of these showed consistency in the reports. Eight (53%) of the 15 studies mentioned an analysis plan. However, seven (88%) of these eight studies did not follow their prescribed analysis plan: the analysis of outcome variables or associations between certain variables were found to be missing from the report. Conclusions Our pilot study has shown that within-study selective reporting may be examined qualitatively by comparing the study report with the study protocol. Our results suggest that it might well be substantial; however, the bias can only be broadly identified as protocols are not sufficiently precise.

Original languageEnglish
Pages (from-to)353-359
Number of pages7
JournalJournal of Evaluation in Clinical Practice
Volume8
Issue number3
DOIs
StatePublished - 1 Aug 2002

Fingerprint

Research Ethics Committees
Research
Sample Size
Research Personnel

Keywords

  • Meta-analysis
  • Publication bias
  • Research ethics committee
  • Within-study selective reporting

Cite this

@article{00e053d6b7104534b01ec9a2d1199801,
title = "Investigation of within-study selective reporting in clinical research: Follow-up of applications submitted to a local research ethics committee",
abstract = "Rationale, aims and objectives Within-study selective reporting is widely believed to exist, although to date there have been no empirical studies to assess the extent of the problem in clinical research. The present study aimed to examine this process. Methods We undertook a pilot study, involving a single local research ethics committee (LREC), in which we compared the outcomes, analysis and sample size proposed in the original approved study protocol with the results presented in the subsequent study report. Results We received 41 (73{\%}) replies from lead researchers of 56 projects, which were a complete cohort of clinical research applications approved in a particular time period by the LREC. Fifteen of these projects, which were completed and published at the time of our study, were further investigated. Only six (40{\%}) stated which outcome variables were of primary interest and four (67{\%}) of these showed consistency in the reports. Eight (53{\%}) of the 15 studies mentioned an analysis plan. However, seven (88{\%}) of these eight studies did not follow their prescribed analysis plan: the analysis of outcome variables or associations between certain variables were found to be missing from the report. Conclusions Our pilot study has shown that within-study selective reporting may be examined qualitatively by comparing the study report with the study protocol. Our results suggest that it might well be substantial; however, the bias can only be broadly identified as protocols are not sufficiently precise.",
keywords = "Meta-analysis, Publication bias, Research ethics committee, Within-study selective reporting",
author = "Seokyung Hahn and Williamson, {P. R.} and Hutton, {J. L.}",
year = "2002",
month = "8",
day = "1",
doi = "10.1046/j.1365-2753.2002.00314.x",
language = "English",
volume = "8",
pages = "353--359",
journal = "Journal of Evaluation in Clinical Practice",
issn = "1356-1294",
publisher = "Wiley-Blackwell Publishing Ltd",
number = "3",

}

TY - JOUR

T1 - Investigation of within-study selective reporting in clinical research

T2 - Follow-up of applications submitted to a local research ethics committee

AU - Hahn, Seokyung

AU - Williamson, P. R.

AU - Hutton, J. L.

PY - 2002/8/1

Y1 - 2002/8/1

N2 - Rationale, aims and objectives Within-study selective reporting is widely believed to exist, although to date there have been no empirical studies to assess the extent of the problem in clinical research. The present study aimed to examine this process. Methods We undertook a pilot study, involving a single local research ethics committee (LREC), in which we compared the outcomes, analysis and sample size proposed in the original approved study protocol with the results presented in the subsequent study report. Results We received 41 (73%) replies from lead researchers of 56 projects, which were a complete cohort of clinical research applications approved in a particular time period by the LREC. Fifteen of these projects, which were completed and published at the time of our study, were further investigated. Only six (40%) stated which outcome variables were of primary interest and four (67%) of these showed consistency in the reports. Eight (53%) of the 15 studies mentioned an analysis plan. However, seven (88%) of these eight studies did not follow their prescribed analysis plan: the analysis of outcome variables or associations between certain variables were found to be missing from the report. Conclusions Our pilot study has shown that within-study selective reporting may be examined qualitatively by comparing the study report with the study protocol. Our results suggest that it might well be substantial; however, the bias can only be broadly identified as protocols are not sufficiently precise.

AB - Rationale, aims and objectives Within-study selective reporting is widely believed to exist, although to date there have been no empirical studies to assess the extent of the problem in clinical research. The present study aimed to examine this process. Methods We undertook a pilot study, involving a single local research ethics committee (LREC), in which we compared the outcomes, analysis and sample size proposed in the original approved study protocol with the results presented in the subsequent study report. Results We received 41 (73%) replies from lead researchers of 56 projects, which were a complete cohort of clinical research applications approved in a particular time period by the LREC. Fifteen of these projects, which were completed and published at the time of our study, were further investigated. Only six (40%) stated which outcome variables were of primary interest and four (67%) of these showed consistency in the reports. Eight (53%) of the 15 studies mentioned an analysis plan. However, seven (88%) of these eight studies did not follow their prescribed analysis plan: the analysis of outcome variables or associations between certain variables were found to be missing from the report. Conclusions Our pilot study has shown that within-study selective reporting may be examined qualitatively by comparing the study report with the study protocol. Our results suggest that it might well be substantial; however, the bias can only be broadly identified as protocols are not sufficiently precise.

KW - Meta-analysis

KW - Publication bias

KW - Research ethics committee

KW - Within-study selective reporting

UR - http://www.scopus.com/inward/record.url?scp=0036667853&partnerID=8YFLogxK

U2 - 10.1046/j.1365-2753.2002.00314.x

DO - 10.1046/j.1365-2753.2002.00314.x

M3 - Article

C2 - 12164983

AN - SCOPUS:0036667853

VL - 8

SP - 353

EP - 359

JO - Journal of Evaluation in Clinical Practice

JF - Journal of Evaluation in Clinical Practice

SN - 1356-1294

IS - 3

ER -