Reported methodologic quality and discrepancies between large and small randomized trials in meta-analyses. Empirical evidence for selective reporting of outcomes in randomized trials: comparison of protocols to published articles. Empirical evidence of bias. Dimensions of methodological quality associated with estimates of treatment effects in controlled trials. Issues that may determine the outcome of antipsychotic trials: industry sponsorship and extrapyramidal side effect. Assessing the quality of reports of randomized clinical trials: is blinding necessary? Discrepancy between results and abstract conclusions in industry- vs nonindustry-funded studies comparing topical prostaglandins. An outreach intervention to implement evidence based practice in residential care: a randomized controlled trial [ISRCTN67855475]. Quality of reporting of randomized controlled trials in general endocrinology literature. Evidence of improving quality of reporting of randomized controlled trials in subfertility.
To explore whether reported methodologic quality affects estimated intervention effects in randomized trials and contributes to discrepancies between the results of large randomized trials and small randomized trials in meta-analyses.
                Meta-analyses of randomized trials that included at least one large trial (>/=1000 participants) were included, regardless of the therapeutic area. Eligible meta-analyses were identified through electronic searches and bibliographies of relevant articles.
                Full-length randomized trials.
                Methodologic quality was assessed according to reported randomization, double blinding, and follow-up as separate components and by using the Jadad composite scale.
                Fourteen meta-analyses involving 190 randomized trials from eight therapeutic areas were included. Compared with large trials, intervention effects were exaggerated in small trials with inadequate allocation sequence generation (ratio of odds ratios, 0.46 [95% CI, 0.25 to 0.83]; P = 0.011), inadequate allocation concealment (ratio of odds ratios, 0.49 [CI, 0.27 to 0.86]; P = 0.014), and no double blinding (ratio of odds ratios, 0.52 [CI, 0.28 to 0.96]; P = 0.01). Large trials did not differ significantly from small trials with adequate generation of the allocation sequence, adequate allocation concealment, or adequate double blinding. No association was seen between reported follow-up and intervention effects. The Jadad scale provided no additional information because the scale and the quality components overlapped substantially.
                Inadequate generation of the allocation sequence, allocation concealment, and double blinding lead to exaggerated estimates of intervention benefit and may contribute to discrepancies between the results of large randomized trials and small randomized trials in meta-analyses. Selective reporting of outcomes within published studies based on the nature or direction of their results has been widely suspected, but direct evidence of such bias is currently limited to case reports.
                To study empirically the extent and nature of outcome reporting bias in a cohort of randomized trials.
                Cohort study using protocols and published reports of randomized trials approved by the Scientific-Ethical Committees for Copenhagen and Frederiksberg, Denmark, in 1994-1995. The number and characteristics of reported and unreported trial outcomes were recorded from protocols, journal articles, and a survey of trialists. An outcome was considered incompletely reported if insufficient data were presented in the published articles for meta-analysis. Odds ratios relating the completeness of outcome reporting to statistical significance were calculated for each trial and then pooled to provide an overall estimate of bias. Protocols and published articles were also compared to identify discrepancies in primary outcomes.
                Completeness of reporting of efficacy and harm outcomes and of statistically significant vs nonsignificant outcomes; consistency between primary outcomes defined in the most recent protocols and those defined in published articles.
                One hundred two trials with 122 published journal articles and 3736 outcomes were identified. Overall, 50% of efficacy and 65% of harm outcomes per trial were incompletely reported. Statistically significant outcomes had a higher odds of being fully reported compared with nonsignificant outcomes for both efficacy (pooled odds ratio, 2.4; 95% confidence interval [CI], 1.4-4.0) and harm (pooled odds ratio, 4.7; 95% CI, 1.8-12.0) data. In comparing published articles with protocols, 62% of trials had at least 1 primary outcome that was changed, introduced, or omitted. Eighty-six percent of survey responders (42/49) denied the existence of unreported outcomes despite clear evidence to the contrary.
                The reporting of trial outcomes is not only frequently incomplete but also biased and inconsistent with protocols. Published articles, as well as reviews that incorporate them, may therefore be unreliable and overestimate the benefits of an intervention. To ensure transparency, planned trials should be registered and protocols should be made publicly available prior to trial completion. To determine if inadequate approaches to randomized controlled trial design and execution are associated with evidence of bias in estimating treatment effects.
                An observational study in which we assessed the methodological quality of 250 controlled trials from 33 meta-analyses and then analyzed, using multiple logistic regression models, the associations between those assessments and estimated treatment effects.
                Meta-analyses from the Cochrane Pregnancy and Childbirth Database.
                The associations between estimates of treatment effects and inadequate allocation concealment, exclusions after randomization, and lack of double-blinding.
                Compared with trials in which authors reported adequately concealed treatment allocation, trials in which concealment was either inadequate or unclear (did not report or incompletely reported a concealment approach) yielded larger estimates of treatment effects (P < .001). Odds ratios were exaggerated by 41% for inadequately concealed trials and by 30% for unclearly concealed trials (adjusted for other aspects of quality). Trials in which participants had been excluded after randomization did not yield larger estimates of effects, but that lack of association may be due to incomplete reporting. Trials that were not double-blind also yielded larger estimates of effects (P = .01), with odds ratios being exaggerated by 17%.
                This study provides empirical evidence that inadequate methodological approaches in controlled trials, particularly those representing poor allocation concealment, are associated with bias. Readers of trial reports should be wary of these pitfalls, and investigators must improve their design, execution, and reporting of trials. This study presents a meta-analysis of the influence of several potentially biasing factors (eg industry support, extrapyramidal side effects) on efficacy of studies comparing second-generation antipsychotic (SGA) with first-generation antipsychotic (FGA) medications. We used the dataset from our previously published meta-analysis of 124 randomized controlled trials (RCTs) comparing SGAs with FGAs, to evaluate whether certain possible biases could influence the actual outcome on the total score of the Positive and Negative Syndrome Scale (PANSS), Brief Psychiatric Rating Scale (BPRS), and Clinical Global Impressions (CGI) scores. Industry sponsorship was determined by contact with authors or publication statement. We calculated whether (1) industry sponsorship, (2) study quality, (3) extrapyramidal symptoms (EPS) properties, or (4) prophylactic antiparkinsonian medications influenced SGA vs FGA efficacy for each drug and averaged overall by two Hedges and Olkin-based meta-analyses. The analysis found that none of the factors was significantly associated with a particular outcome. While industry-sponsored articles may conclude their medication to be more favorable than that of a competitor in an RCT, we found that the observed efficacy was not influenced by sponsorship. Many attribute the finding that SGAs appears to be more efficacious than FGAs to be a result of EPS-decreasing efficacy (or its measurement). We were unable to confirm that the drug's EPS properties or antiparkinsonian management altered actual efficacy. It has been suggested that the quality of clinical trials should be assessed by blinded raters to limit the risk of introducing bias into meta-analyses and systematic reviews, and into the peer-review process. There is very little evidence in the literature to substantiate this. This study describes the development of an instrument to assess the quality of reports of randomized clinical trials (RCTs) in pain research and its use to determine the effect of rater blinding on the assessments of quality. A multidisciplinary panel of six judges produced an initial version of the instrument. Fourteen raters from three different backgrounds assessed the quality of 36 research reports in pain research, selected from three different samples. Seven were allocated randomly to perform the assessments under blind conditions. The final version of the instrument included three items. These items were scored consistently by all the raters regardless of background and could discriminate between reports from the different samples. Blind assessments produced significantly lower and more consistent scores than open assessments. The implications of this finding for systematic reviews, meta-analytic research and the peer-review process are discussed. To investigate the relationship between industry- vs nonindustry-funded publications comparing the efficacy of topical prostaglandin analogs by evaluating the correspondence between the statistical significance of the publication's main outcome measure and its abstract conclusions.
                Retrospective, observational cohort study.
                English publications comparing the ocular hypotensive efficacy between any or all of latanoprost, travoprost, and bimatoprost were searched from the MEDLINE database. Each article was reviewed by three independent observers and was evaluated for source of funding, study quality, statistically significant main outcome measure, correspondence between results of main outcome measure and abstract conclusion, number of intraocular pressure outcomes compared, and journal impact factor. Funding was determined by published disclosure or, in cases of no documented disclosure, the corresponding author was contacted directly to confirm industry funding. Discrepancies were resolved by consensus. The main outcome measure was correspondence between abstract conclusion and reported statistical significance of the publications' main outcome measure.
                Thirty-nine publications were included, of which 29 were industry funded and 10 were nonindustry funded. The published abstract conclusion was not consistent with the results of the main outcome measure in 18 (62%) of 29 of the industry-funded studies compared with zero (0%) of 10 of the nonindustry-funded studies (P = .0006). Twenty-six (90%) of the industry-funded studies had proindustry abstract conclusions.
                Twenty-four percent of the industry-funded publications had a statistically significant main outcome measure; however, 90% of the industry-funded studies had proindustry abstract conclusions. Both readers and reviewers should scrutinize publications carefully to ensure that data support the authors' conclusions. The aim of this project was to assess whether outreach visits would improve the implementation of evidence based clinical practice in the area of falls reduction and stroke prevention in a residential care setting.
                Twenty facilities took part in a randomized controlled trial with a seven month follow-up period. Two outreach visits were delivered by a pharmacist. At the first a summary of the relevant evidence was provided and at the second detailed audit information was provided about fall rates, psychotropic drug prescribing and stroke risk reduction practices (BP monitoring, aspirin and warfarin use) for the facility relevant to the physician. The effect of the interventions was determined via pre- and post-intervention case note audit. Outcomes included change in percentage patients at risk of falling who fell in a three month period prior to follow-up and changes in use of psychotropic medications. Chi-square tests, independent samples t-test, and logistic regression were used in the analysis.
                Data were available from case notes at baseline (n = 897) and seven months follow-up (n = 902), 452 residential care staff were surveyed and 121 physicians were involved with 61 receiving outreach visits. Pre-and post-intervention data were available for 715 participants. There were no differences between the intervention and control groups for the three month fall rate. We were unable to detect statistically significant differences between groups for the psychotropic drug use of the patients before or after the intervention. The exception was significantly greater use of "as required" antipsychotics in the intervention group compared with the control group after the pharmacy intervention (RR = 4.95; 95%CI 1.69-14.50). There was no statistically significant difference between groups for the numbers of patients "at risk of stroke" on aspirin at follow-up.
                While the strategy was well received by the physicians involved, there was no change in prescribing patterns. Patient care in residential settings is complex and involves contributions from the patient's physician, family and residential care staff. The project highlights challenges of delivering evidence based care in a setting in which there is a paucity of well controlled trial evidence but where significant health outcomes can be attained. The reporting quality of randomized controlled trials (RCTs) is poor in general medicine and several areas of specialization but unknown in endocrinology.
                Our aim was to assess the reporting quality of RCTs in general endocrinology. A secondary objective was to identify predictors for better reporting quality.
                We systematically reviewed RCTs published in three general endocrinology journals between January 2005 and December 2006.
                We included parallel-design RCTs that addressed a question of treatment or prevention. Article selection and data abstraction were conducted by two reviewers independently, and disagreements were resolved by consensus.
                There were two main outcomes: 1) a 15-point overall reporting quality score (OQS) based on the Consolidated Standards for Reporting Trials (CONSORT); and 2) a 3-point key score, based on allocation concealment, blinding, and use of intention-to-treat analysis.
                Eighty nine RCTs were included. The median OQS was 10 (interquartile range = 2). Allocation concealment, blinding, and analysis by intention to treat were reported in 10, 20, and 16 of the 89 RCTs, respectively. A multivariable regression analysis showed that complete industrial funding [incidence rate ratio (IRR) = 1.014; 95% confidence interval (CI), 1.010-1.018], journal of publication (IRR = 1.068; 95% CI, 1.007-1.132), and sample size (IRR = 1.048; 95% CI, 1.026-1.070) were significantly associated with a slightly better OQS.
                The quality of RCT reporting in general endocrine literature is suboptimal. We discuss our results, highlight the areas where improvements are needed, and provide some recommendations. The quality of randomized controlled trials (RCTs) in subfertility and their suitability for inclusion in meta-analyses have been assessed in the past and found to be insufficient. Our aim was to assess whether this quality has improved over time, particularly since the publication of the Consolidated Standards of Reporting Trials (CONSORT) statement, and to assess what proportion of trials could be included in the meta-analyses of pregnancy outcomes such as those included in Cochrane Reviews.
                A selection of subfertility trials published in 1990, 1996 and 2002 was collected from the Cochrane Menstrual Disorder and Subfertility Group (MDSG) database. Only trials published in English as full journal articles, claiming to be randomized and reporting on pregnancy outcomes, were included.
                One hundred and sixty-four trials met our inclusion criteria. Twenty-four (15%) were found not to be randomized, despite claims, and only 10 trials (6%) provided adequate details on the methods of randomization and allocation concealment. Of these, only three had sufficient details extractable to allow for an intention-to-treat analysis of the outcome 'live birth'.
                Although an improvement in some subfertility-specific issues was observed, the quality of reporting of RCTs still needs to improve to make them suitable for inclusion in meta-analyses such as those in the Cochrane Library.
The included trial was a small-scaled study and there were serious limitations in the design and methodology (e.g. allocation concealment was unclear, blinding of outcome assessor(s) was not ensured, few quantitative outcome measures were used, and the results were not reported as planned). Therefore, the current evidence supporting the efficacy of EPG is not strong and there remains a need for high-quality randomised controlled trials to be undertaken in this area.