Treatment success in cancer: new cancer treatment successes identified in phase 3 randomized controlled trials conducted by the National Cancer Institute-sponsored cooperative oncology groups, 1955 to 2006. Evolution of the randomized controlled trial in oncology over three decades. Secular changes in the quality of published randomized clinical trials in rheumatology. Treatment success in pragmatic randomised controlled trials: a review of trials funded by the UK Health Technology Assessment programme. Abstracts of randomized controlled trials presented at the society for pediatric research meeting: an example of publication bias. Association of funding and conclusions in randomized drug trials: a reflection of treatment effect or adverse events? Comparison of registered and published primary outcomes in randomized controlled trials. Reported outcomes in major cardiovascular clinical trials funded by for-profit and not-for-profit organizations: 2000-2005. Reporting in randomized clinical trials improved after adoption of the CONSORT statement.
The evaluation of research output, such as estimation of the proportion of treatment successes, is of ethical, scientific, and public importance but has rarely been evaluated systematically. We assessed how often experimental cancer treatments that undergo testing in randomized clinical trials (RCTs) result in discovery of successful new interventions.
                We extracted data from all completed (published and unpublished) phase 3 RCTs conducted by the National Cancer Institute cooperative groups since their inception in 1955. Therapeutic successes were determined by (1) assessing the proportion of statistically significant trials favoring new or standard treatments, (2) determining the proportion of the trials in which new treatments were considered superior to standard treatments according to the original researchers, and (3) quantitatively synthesizing data for main clinical outcomes (overall and event-free survival).
                Data from 624 trials (781 randomized comparisons) involving 216 451 patients were analyzed. In all, 30% of trials had statistically significant results, of which new interventions were superior to established treatments in 80% of trials. The original researchers judged that the risk-benefit profile favored new treatments in 41% of comparisons (316 of 766). Hazard ratios for overall and event-free survival, available for 614 comparisons, were 0.95 (99% confidence interval [CI], 0.93-0.98) and 0.90 (99% CI, 0.87- 0.93), respectively, slightly favoring new treatments. Breakthrough interventions were discovered in 15% of trials.
                Approximately 25% to 50% of new cancer treatments that reach the stage of assessment in RCTs will prove successful. The pattern of successes has become more stable over time. The results are consistent with the hypothesis that the ethical principle of equipoise defines limits of discoverability in clinical research and ultimately drives therapeutic advances in clinical medicine. The randomized controlled trial (RCT) is the gold standard for establishing new therapies in clinical oncology. Here we document changes with time in design, sponsorship, and outcomes of oncology RCTs.
                Reports of RCTs evaluating systemic therapy for breast, colorectal (CRC), and non-small-cell lung cancer (NSCLC) published 1975 to 2004 in six major journals were reviewed. Two authors abstracted data regarding trial design, results, and conclusions. Conclusions of authors were graded using a 7-point Likert scale. For each study the effect size for the primary end point was converted to a summary measure.
                A total of 321 eligible RCTs were included (48% breast, 24% CRC, 28% NSCLC). Over time, the number and size of RCTs increased considerably. For-profit/mixed sponsorship increased substantially during the study period (4% to 57%; P < .001). There was increasing use of time-to-event measures (39% to 78%) and decreasing use of response rate (54% to 14%) as primary end point (P < .001). Effect size remained stable over the study period. Authors have become more likely to strongly endorse the experimental arm (P = .017). A significant P value for the primary end point and industry sponsorship were each independently associated with endorsement of the experimental agent (odds ratio [OR] = 19.6, 95% CI, 8.9 to 43.1, and OR = 3.5, 95% CI, 1.6 to 7.5, respectively).
                RCTs in oncology have become larger and are more likely to be sponsored by industry. Authors of modern RCTs are more likely to strongly endorse novel therapies. For-profit sponsorship and statistically significant results are independently associated with endorsement of the experimental arm. To assess the quality of published randomized clinical trials (RCTs) in rheumatology and to determine whether there has been improvement in quality between 2 time periods, 1987-1988 and 1997-1998.
                Using MEDLINE and a hand search of selected rheumatology journals, we identified RCTs of adult rheumatic diseases published in English in 1987-1988 or 1997-1998. We examined trial quality with an expanded version of the Jadad scale, which assesses the adequacy of reported random sequence generation, allocation concealment, blinding, and analysis. All trials were read by 1 reviewer, with prior standardization using a random sample read by 2 reviewers. We also evaluated "high"- versus "low"-impact journals based on citation index.
                Two hundred forty RCTs (1987-1988 119 RCTs, 1997-1998 121 RCTs) were assessed. Results showed improvement in the quality of the trials, but the rates of reported random sequence generation, allocation concealment, power, and intent-to-treat analyses were persistently low. Low rates of reports of random sequence generation, allocation concealment, and intent-to-treat analyses were present even in the high-impact journals.
                There has been improvement in the quality of reporting of RCTs in rheumatology between 1987-1988 and 1997-1998. However, methodologic problems such as lack of allocation concealment, inadequate random sequence generation, lack of reporting of power, and lack of intent-to-treat analyses remain common. Many of these problems are established sources of bias in RCTs and are easily rectifiable. Previous research reviewed treatment success and whether the collective uncertainty principle is met in RCTs in the US National Cancer Institute portfolio. This paper classifies clinical trials funded by the UK HTA programme by results using the method applied to the US Cancer Institute trials, and compares the two portfolios.
                Data on all completed randomised controlled trials funded by the HTA programme 1993-2008 were extracted. Each trial's primary results was classified into six categories; 1) statistically significant in favour of the new treatment, 2) statistically significant in favour of the control treatment 3) true negative, 4) truly inconclusive, 5) inconclusive in favour of new treatment or 6) inconclusive in favour of control treatment. Trials were classified by comparing the 95% confidence interval for the difference in primary outcome to the difference specified in the sample size calculation. The results were compared with Djulbegovic's analysis of NCI trials.
                Data from 51 superiority trials were included, involving over 48,000 participants and a range of diseases and interventions. 85 primary comparisons were available because some trials had more than two randomised arms or had several primary outcomes. The new treatment had superior results (whether significant or not) in 61% of the comparisons (52/85 95% CI 49.9% to 71.6%). The results were conclusive in 46% of the comparisons (19% statistically significant in favour of the new treatment, 5% statistically significant in favour of the control and 22% true negative). The results were classified as truly inconclusive (i.e. failed to answer the question asked) for 24% of comparisons (20/85). HTA trials included fewer truly inconclusive and statistically significant results and more results rated as true negative than NCI trials.
                The pattern of results in HTA trials is similar to that of the National Cancer Institute portfolio. Differences that existed were plausible given the differences in the types of trials -HTA trials are more pragmatic. The results indicate HTA trials are compatible with equipoise. This classification usefully summarises the results from clinical trials and enables comparisons of different portfolios of trials. Publication bias toward studies that favor new therapies has been known to occur for the past 40 years, yet its implications are not well studied in child health. The increased interest in meta-analyses has highlighted the need to identify the totality of evidence when addressing treatment questions.
                To measure the percentage of randomized controlled trials (RCTs) presented at a major pediatric scientific meeting that were subsequently published as full-length articles, to investigate factors associated with publication, and to describe the variables that change from abstract to manuscript form.
                The scientific proceedings from the Society for Pediatric Research were hand searched for RCTs (1992-1995). Subsequent publication was ascertained through a search of various electronic databases. Quality of abstracts and manuscripts was measured, and data were extracted using a structured form.
                A total of 264 (59.1%) of 447 abstracts were subsequently published. Almost 64% of RCTs that were subsequently published favored new therapy compared with 43.5% of studies that were never published (P<.001). Mean effect size for published vs unpublished RCTs was 0.74 vs 0.05 (P<.001). Median sample size was larger in published (n = 45) vs unpublished (n = 34) RCTs (P =.02). Quality was significantly lower for abstracts vs published RCTs (P<.001). For 5% of abstracts that were subsequently published, the conclusion regarding treatment efficacy changed.
                Publication bias is a serious threat to assessing the effectiveness of interventions in child health, as little more than half of RCTs presented at a major scientific meeting are subsequently published. There is a need to institute an international registry of RCTs in children so that the totality of evidence can be accessed when assessing treatment effectiveness. Previous studies indicate that industry-sponsored trials tend to draw proindustry conclusions.
                To explore whether the association between funding and conclusions in randomized drug trials reflects treatment effects or adverse events.
                Observational study of 370 randomized drug trials included in meta-analyses from Cochrane reviews selected from the Cochrane Library, May 2001. From a random sample of 167 Cochrane reviews, 25 contained eligible meta-analyses (assessed a binary outcome; pooled at least 5 full-paper trials of which at least 1 reported adequate and 1 reported inadequate allocation concealment). The primary binary outcome from each meta-analysis was considered the primary outcome for all trials included in each meta-analysis. The association between funding and conclusions was analyzed by logistic regression with adjustment for treatment effect, adverse events, and additional confounding factors (methodological quality, control intervention, sample size, publication year, and place of publication).
                Conclusions in trials, classified into whether the experimental drug was recommended as the treatment of choice or not.
                The experimental drug was recommended as treatment of choice in 16% of trials funded by nonprofit organizations, 30% of trials not reporting funding, 35% of trials funded by both nonprofit and for-profit organizations, and 51% of trials funded by for-profit organizations (P<.001; chi2 test). Logistic regression analyses indicated that funding, treatment effect, and double blinding were the only significant predictors of conclusions. Adjusted analyses showed that trials funded by for-profit organizations were significantly more likely to recommend the experimental drug as treatment of choice (odds ratio, 5.3; 95% confidence interval, 2.0-14.4) compared with trials funded by nonprofit organizations. This association did not appear to reflect treatment effect or adverse events.
                Conclusions in trials funded by for-profit organizations may be more positive due to biased interpretation of trial results. Readers should carefully evaluate whether conclusions in randomized trials are supported by data. As of 2005, the International Committee of Medical Journal Editors required investigators to register their trials prior to participant enrollment as a precondition for publishing the trial's findings in member journals.
                To assess the proportion of registered trials with results recently published in journals with high impact factors; to compare the primary outcomes specified in trial registries with those reported in the published articles; and to determine whether primary outcome reporting bias favored significant outcomes.
                MEDLINE via PubMed was searched for reports of randomized controlled trials (RCTs) in 3 medical areas (cardiology, rheumatology, and gastroenterology) indexed in 2008 in the 10 general medical journals and specialty journals with the highest impact factors.
                For each included article, we obtained the trial registration information using a standardized data extraction form.
                Of the 323 included trials, 147 (45.5%) were adequately registered (ie, registered before the end of the trial, with the primary outcome clearly specified). Trial registration was lacking for 89 published reports (27.6%), 45 trials (13.9%) were registered after the completion of the study, 39 (12%) were registered with no or an unclear description of the primary outcome, and 3 (0.9%) were registered after the completion of the study and had an unclear description of the primary outcome. Among articles with trials adequately registered, 31% (46 of 147) showed some evidence of discrepancies between the outcomes registered and the outcomes published. The influence of these discrepancies could be assessed in only half of them and in these statistically significant results were favored in 82.6% (19 of 23).
                Comparison of the primary outcomes of RCTs registered with their subsequent publication indicated that selective outcome reporting is prevalent. In surveys based on data available prior to 2000, clinical trials funded by for-profit organizations appeared more likely to report positive findings than those funded by not-for-profit organizations. Whether this situation has changed over the past 5 years or whether similar effects are present among jointly funded trials is unknown.
                To determine in contemporary randomized cardiovascular trials the association between funding source and the likelihood of reporting positive findings.
                We reviewed 324 consecutive superiority trials of cardiovascular medicine published between January 1, 2000, and July 30, 2005, in JAMA, The Lancet, and the New England Journal of Medicine.
                The proportion of trials favoring newer treatments over the standard of care was evaluated by funding source.
                Of the 324 superiority trials, 21 cited no funding source. Of the 104 trials funded solely by not-for-profit organizations, 51 (49%) reported evidence significantly favoring newer treatments over the standard of care, whereas 53 (51%) did not (P = .80). By contrast, 92 (67.2%) of 137 trials funded solely by for-profit organizations favored newer treatments over standard of care (P<.001). Among 62 jointly funded trials, 35 (56.5%), an intermediate proportion, favored newer treatments. For 205 randomized trials evaluating drugs, the proportions favoring newer treatments were 39.5%, not-for-profit; 54.4%, jointly funded; and 65.5%, for-profit trials (P for trend across groups = .002). For the 39 randomized trials evaluating cardiovascular devices, the proportions favoring newer treatments were 50.0%, not-for-profit; 69.2%, jointly funded; and 82.4%, for-profit trials (P for trend across groups = .07). Regardless of funding source, trials using surrogate end points, such as quantitative angiography, intravascular ultrasound, plasma biomarkers, and functional measures were more likely to report positive findings (67%) than trials using clinical end points (54.1%; P = .02).
                Recent cardiovascular trials funded by for-profit organizations are more likely to report positive findings than trials funded by not-for-profit organizations, as are trials using surrogate rather than clinical end points. Trials jointly funded by not-for-profit and for-profit organizations appear to report positive findings at a rate approximately midway between rates observed in trials supported solely by one or the other of these entities. To examine the extent to which the Consolidated Standards of Reporting Trials (CONSORT) reporting guidelines improved clinical trials reporting and subject attrition, which may undermine the credibility of published randomized clinical trials (RCTs).
                Published RCTs reported in two major medical journals before and after the CONSORT guidelines were systematically reviewed; one used the CONSORT statement (JAMA) and one did not (NEJM).
                The quality of RCT reporting improved for both journals, but JAMA showed more significant and consistent improvements in all aspects of RCT reporting. Subject attrition was better accounted for after the publication of CONSORT, although the attrition rates for various reasons actually increased. Attrition due to unknown reasons, as a percentage of total attrition, declined dramatically, from 68.7% pre-CONSORT to 13.0% post-CONSORT.
                Attrition of study subjects remains a serious problem in RCTs. Bias from selective attrition can undermine the presumptive scientific advantage of RCTs. The CONSORT guidelines improved RCT reporting when they were implemented but did not substantially improve reported attrition rates.
Society can expect that slightly more than half of new experimental treatments will prove to be better than established treatments when tested in RCTs, but few will be substantially better. This is an important finding for patients (as they contemplate participation in RCTs), researchers (as they plan design of the new trials), and funders (as they assess the 'return on investment'). Although we provide the current best evidence on the question of expected 'success rate' of new versus established treatments consistent with a priori theoretical predictions reflective of 'uncertainty or equipoise hypothesis', it should be noted that our sample represents less than 1% of all available randomized trials; therefore, one should exercise the appropriate caution in interpretation of our findings. In addition, our conclusion applies to publicly funded trials only, as we did not include studies funded by commercial sponsors in our analysis.