Commercially funded and United States-based research is more likely to be published; good-quality studies with negative outcomes are not. Effect of the statistical significance of results on the time to completion and publication of randomized efficacy trials. Bias in reporting clinical trials. Publication bias: evidence of delayed publication in a cohort study of clinical research projects. Publication bias and meta-analyses: a practical example. Factors associated with failure to publish large randomized trials presented at an oncology meeting. Publication bias in presentations to the Annual Scientific Congress. Positive-outcome bias and other limitations in the outcome of research abstracts submitted to a scientific meeting. A cohort study of summary reports of controlled trials.
Prior studies implying associations between receipt of commercial funding and positive (significant and/or pro-industry) research outcomes have analyzed only published papers, which is an insufficiently robust approach for assessing publication bias. In this study, we tested the following hypotheses regarding orthopaedic manuscripts submitted for review: (1) nonscientific variables, including receipt of commercial funding, affect the likelihood that a peer-reviewed submission will conclude with a report of a positive study outcome, and (2) positive outcomes and other, nonscientific variables are associated with acceptance for publication.
                All manuscripts about hip or knee arthroplasty that were submitted to The Journal of Bone and Joint Surgery, American Volume, over seventeen months were evaluated to determine the study design, quality, and outcome. Analyses were carried out to identify associations between scientific factors (sample size, study quality, and level of evidence) and study outcome as well as between non-scientific factors (funding source and country of origin) and study outcome. Analyses were also performed to determine whether outcome, scientific factors, or nonscientific variables were associated with acceptance for publication.
                Two hundred and nine manuscripts were reviewed. Commercial funding was not found to be associated with a positive study outcome (p = 0.668). Studies with a positive outcome were no more likely to be published than were those with a negative outcome (p = 0.410). Studies with a negative outcome were of higher quality (p = 0.003) and included larger sample sizes (p = 0.05). Commercially funded (p = 0.027) and United States-based (p = 0.020) studies were more likely to be published, even though those studies were not associated with higher quality, larger sample sizes, or lower levels of evidence (p = 0.24 to 0.79).
                Commercially funded studies submitted for review were not more likely to conclude with a positive outcome than were nonfunded studies, and studies with a positive outcome were no more likely to be published than were studies with a negative outcome. These findings contradict those of most previous analyses of published (rather than submitted) research. Commercial funding and the country of origin predict publication following peer review beyond what would be expected on the basis of study quality. Studies with a negative outcome, although seemingly superior in quality, fared no better than studies with a positive outcome in the peer-review process; this may result in inflation of apparent treatment effects when the published literature is subjected to meta-analysis. Medical evidence may be biased over time if completion and publication of randomized efficacy trials are delayed when results are not statistically significant.
                To evaluate whether the time to completion and the time to publication of randomized phase 2 and phase 3 trials are affected by the statistical significance of results and to describe the natural history of such trials.
                Prospective cohort of randomized efficacy trials conducted by 2 trialist groups from 1986 to 1996.
                Multicenter trial groups in human immunodeficiency virus infection sponsored by the National Institutes of Health.
                A total of 109 efficacy trials (total enrollment, 43708 patients).
                Time from start of enrollment to completion of follow-up and time from completion of follow-up to peer-reviewed publication assessed with survival analysis.
                The median time from start of enrollment to publication was 5.5 years and was substantially longer for negative trials than for results favoring an experimental arm (6.5 vs 4.3 years, respectively; P<.001; hazard ratio for time to publication for positive vs negative trials, 3.7; 95% confidence interval [CI], 1.8-7.7). This difference was mostly attributable to differences in the time from completion to publication (median, 3.0 vs 1.7 years for negative vs positive trials; P<.001). On average, trials with significant results favoring any arm completed follow-up slightly earlier than trials with nonsignificant results (median, 2.3 vs 2.5 years; P=.045), but long-protracted trials often had low event rates and failed to reach statistical significance, while trials that were terminated early had significant results. Positive trials were submitted for publication significantly more rapidly after completion than were negative trials (median, 1.0 vs 1.6 years; P=.001) and were published more rapidly after submission (median, 0.8 vs 1.1 years; P=.04).
                Among randomized efficacy trials, there is a time lag in the publication of negative findings that occurs mostly after the completion of the trial follow-up. The primary aim of the present study was to identify possible occurrence of selective reporting of the results of clinical trials to the Finnish National Agency for Medicines. Selective reporting may lead to poorly informed action or inaction by regulatory authorities.
                In 1987, 274 clinical drug trials were notified to the Finnish National Agency for Medicines. By December 1993, final reports had been received from 68 of these trials and statements that the trial had been suspended from 24 trials. The sponsors of the non-reported trials were requested to report the outcome. The outcomes, if any, of all reported and non-reported trials were classified as positive, inconclusive or negative.
                The total number of trials with positive, inconclusive or negative outcome were 111, 33 and 44, respectively; the outcomes of 86 trials could not be assessed. Final reports were received from 42/111 (38%) trials with positive, 6/33 (18%) with inconclusive and 9/44 (20%) with negative outcomes.
                Substantial evidence of selective reporting was detected, since trials with positive outcome resulted more often in submission of final report to regulatory authority than those with inconclusive or negative outcomes. To determine the extent to which publication is influenced by study outcome.
                A cohort of studies submitted to a hospital ethics committee over 10 years were examined retrospectively by reviewing the protocols and by questionnaire. The primary method of analysis was Cox's proportional hazards model.
                University hospital, Sydney, Australia.
                748 eligible studies submitted to Royal Prince Alfred Hospital Ethics Committee between 1979 and 1988.
                Time to publication.
                Response to the questionnaire was received for 520 (70%) of the eligible studies. Of the 218 studies analysed with tests of significance, those with positive results (P < 0.05) were much more likely to be published than those with negative results (P > or = 0.10) (hazard ratio 2.32 (95% confidence interval 1.47 to 3.66), P = 0.0003), with a significantly shorter time to publication (median 4.8 v 8.0 years). This finding was even stronger for the group of 130 clinical trials (hazard ratio 3.13 (1.76 to 5.58). P = 0.0001), with median times to publication of 4.7 and 8.0 years respectively. These results were not materially changed after adjusting for other significant predictors of publication. Studies with indefinite conclusions (0.05 < or = P < 0.10) tended to have an even lower publication rate and longer time to publication than studies with negative results (hazard ratio 0.39 (0.13 to 1.12), P = 0.08). For the 103 studies in which outcome was rated qualitatively, there was no clear cut evidence of publication bias, although the number of studies in this group was not large.
                This study confirms the evidence of publication bias found in other studies and identifies delay in publication as an additional important factor. The study results support the need for prospective registration of trials to avoid publication bias and also support restricting the selection of trials to those started before a common date in undertaking systematic reviews. Publication bias is widely appreciated, but considerable time and effort are needed to locate and obtain data from unpublished randomized controlled trials (RCTs), those published in non-English language journals or those reported in the gray literature; for this publication, we will call this collection of trials the "gray+literature." However, excluding such trials from systematic reviews could introduce bias and give rise to misleading conclusions.
                We aimed to explore and quantify the impact of inclusion of gray+ literature on the results of all completed individual patient data (IPD) reviews coordinated by our group (13 meta-analyses). For each IPD review, results were calculated for RCTs fully published in English language journals and RCTs fully published in English language journals and the gray+literature.
                The IPD meta-analyses based only on RCTs that were fully published in English language journals tended to give more favorable results than those that included RCTs from the gray+literature. Although in most cases the addition of gray+data gave less encouraging results, moving the estimated treatment effect toward a null result, the direction of effect was not always predictable.
                We recommend that all systematic reviews should at least attempt to identify trials reported in the gray+literature and, where possible, obtain data from them. Large clinical trials are the criterion standard for making treatment decisions, and nonpublication of the results of such trials can lead to bias in the literature and contribute to inappropriate medical decisions.
                To determine the rate of full publication of large randomized trials presented at annual meetings of the American Society of Clinical Oncology (ASCO), quantify bias against publishing nonsignificant results, and identify factors associated with time to publication.
                Survey of 510 abstracts from large (sample size, > or =200), phase 3, randomized controlled trials presented at ASCO meetings between 1989 and 1998. Trial results were classified as significant (P< or =.05 for the primary outcome measure) or nonsignificant (P>.05 or not reported), and the type of presentation and sponsorship were identified. Subsequent full publication was identified using a search of MEDLINE and EMBASE, completed November 1, 2001; the search was updated in November 2002, using the Cochrane Register of Controlled Trials. Authors were contacted if the searches did not find evidence of publication.
                Publication rate at 5 years; time from presentation to full publication.
                Of 510 randomized trials, 26% were not published in full within 5 years after presentation at the meeting. Eighty-one percent of the studies with significant results had been published by this time compared with 68% of the studies with nonsignificant results (P<.001). Studies with oral or plenary presentation were published sooner than those not presented (P =.002), and studies with pharmaceutical sponsorship were published sooner than studies with cooperative group sponsorship or studies for which sponsorship was not specified (P =.02). These factors remained significant in a multivariable model. The most frequent reason cited by authors for not publishing was lack of time, funds, or other resources.
                A substantial number of large phase 3 trials presented at an international oncology meeting remain unpublished 5 years after presentation. Bias against publishing nonsignificant results is a problem even for large randomized trials. Nonpublication breaks the contract that investigators make with trial participants, funding agencies, and ethics boards. Free papers presented to the Annual Scientific Congress (ASC) of the Royal Australasian College of Surgeons (RACS) were reviewed for the years 1994, 1995 and 1996. Reports were examined for evidence of publication bias.
                Suitable free papers were identified from the proceedings of the meetings and authors were contacted to obtain information about the research reported and any publications resulting from it.
                Responses were obtained from 302 of 576 presentations considered suitable. A total of 55% of responding authors reported publication of their paper. Basic science papers were most likely to be published. There was a significant bias in favour of publication of positive results (98 of 139 positive vs 76 of 159 inconclusive or negative reports; P < 0.01). Retrospective data were as likely to be published as prospective (51% and 57%, respectively). Reports describing studies of high-level evidence were more likely to be published in journals with a high impact factor.
                The ASC is a comprehensive meeting that attracts a wide range of free papers from most sections of the RACS. There appears to be no evidence of bias in selection of papers for inclusion in the meeting but there is bias in the subsequent publication, which favours positive reports. Studies with positive results are more likely to be published in biomedical journals than are studies with negative results. However, many studies submitted for consideration at scientific meetings are never published in full; bias in this setting is poorly studied.
                To identify features associated with the fate of research abstracts submitted to a scientific meeting.
                Prospective observational cohort, with 5-year follow-up of all research submitted for consideration to the major annual 1991 US research meeting in the specialty of emergency medicine.
                All research abstracts submitted for consideration at the meeting for possible presentation.
                Characteristics associated with acceptance for presentation at the meeting and subsequent publication as a full manuscript.
                A total of 492 research abstracts were submitted from programs in emergency medicine and other specialties affiliated with 103 US medical schools. A total of 179 (36%) were accepted for presentation and 214 (43%) were published in 44 journals. Of the 179 abstracts accepted for presentation, 111 studies were published. Scientific quality of abstracts or prestige of the journal in which the study was eventually published did not predict either of these outcomes. The best predictors (by logistic regression) of meeting acceptance were a subjective "originality" factor (odds ratio [OR], 2.07; 95% confidence interval [CI], 1.13-3.89) and positive results (OR, 1.99; 95% CI, 1.07-3.84), and, for publication, meeting acceptance (OR, 2.49; 95% CI, 1.49-4.35) and large sample size (OR, 2.26; 95% CI, 1.23-4.31). Forty-nine percent (241) of abstracts did not report on blinding, and 24% (118) did not report on randomization. Acceptance and publication were both more likely for positive outcomes (P=.03). Funnel plots showed the classic distribution of positive-outcome ("publication") bias at each of the submission, acceptance, and publication phases. Meeting acceptance predicted publication with a sensitivity of only 51%, specificity of 71%, positive predictive value of 57%, and negative predictive value of 66%.
                Positive-outcome bias was evident when studies were submitted for consideration and was amplified in the selection of abstracts for both presentation and publication, neither of which was strongly related to study design or quality. Substantial numbers of clinical trials continue to be reported only in summary reports that present insufficient methodological details to permit informed judgments about the likely validity of the conclusions. Using a cohort of 176 controlled trials reported in summary form, we tested the hypotheses that they would be more likely to be followed by full reports if, on the basis of the information provided in the summary report, (1) the trial was judged to be methodologically sound, (2) the results favored the test treatment, and (3) the sample size was relatively large. The results of univariate and multivariate analyses provided support for only the third of these hypotheses. Investigators, as well as those who fund and sanction the conduct of clinical research, should make greater efforts to ensure that clinical trials are reported properly.
Trials with positive findings are published more often, and more quickly, than trials with negative findings.