An outreach intervention to implement evidence based practice in residential care: a randomized controlled trial [ISRCTN67855475]. A critical assessment of the quality of reporting of randomized, controlled trials in the urology literature. Empirical evidence for selective reporting of outcomes in randomized trials: comparison of protocols to published articles. Randomised controlled trial of a theoretically grounded tailored intervention to diffuse evidence-based public health practice [ISRCTN23257060]. Does quality of reports of randomised trials affect estimates of intervention efficacy reported in meta-analyses? Quality of reporting of randomized, controlled trials in cerebral palsy. Reported methodologic quality and discrepancies between large and small randomized trials in meta-analyses. Evidence of improving quality of reporting of randomized controlled trials in subfertility. The quality of reporting of randomized controlled trials in solid organ transplantation.
The aim of this project was to assess whether outreach visits would improve the implementation of evidence based clinical practice in the area of falls reduction and stroke prevention in a residential care setting.
                Twenty facilities took part in a randomized controlled trial with a seven month follow-up period. Two outreach visits were delivered by a pharmacist. At the first a summary of the relevant evidence was provided and at the second detailed audit information was provided about fall rates, psychotropic drug prescribing and stroke risk reduction practices (BP monitoring, aspirin and warfarin use) for the facility relevant to the physician. The effect of the interventions was determined via pre- and post-intervention case note audit. Outcomes included change in percentage patients at risk of falling who fell in a three month period prior to follow-up and changes in use of psychotropic medications. Chi-square tests, independent samples t-test, and logistic regression were used in the analysis.
                Data were available from case notes at baseline (n = 897) and seven months follow-up (n = 902), 452 residential care staff were surveyed and 121 physicians were involved with 61 receiving outreach visits. Pre-and post-intervention data were available for 715 participants. There were no differences between the intervention and control groups for the three month fall rate. We were unable to detect statistically significant differences between groups for the psychotropic drug use of the patients before or after the intervention. The exception was significantly greater use of "as required" antipsychotics in the intervention group compared with the control group after the pharmacy intervention (RR = 4.95; 95%CI 1.69-14.50). There was no statistically significant difference between groups for the numbers of patients "at risk of stroke" on aspirin at follow-up.
                While the strategy was well received by the physicians involved, there was no change in prescribing patterns. Patient care in residential settings is complex and involves contributions from the patient's physician, family and residential care staff. The project highlights challenges of delivering evidence based care in a setting in which there is a paucity of well controlled trial evidence but where significant health outcomes can be attained. Randomized, controlled trials are the gold standard for evidence based assessment of therapeutic interventions. In 1996 the Consolidated Standards of Reporting Trials statement was published in an effort to standardize the reporting of clinical trials. To our knowledge we report the first systematic assessment of randomized, controlled trial quality in the urology literature by Consolidated Standards of Reporting Trials standards.
                All human subject randomized, controlled trials published in 4 leading urology journals in 1996 and 2004 were identified for formal review. A standardized evaluation form was developed based on the Consolidated Standards of Reporting Trials statement. Each article was evaluated by 2 independent reviewers and discrepancies were settled by consensus. A Consolidated Standards of Reporting Trials criteria summary score was calculated on a scale of 0 to 22.
                A total of 152 randomized, controlled trials met inclusion criteria. The mean+/-SEM Consolidated Standards of Reporting Trials summary score was 10.2+/-0.3 (median 10.3) and 12.0+/-0.3 (median 12.2) in 1996 and 2004, respectively, with a mean difference of 1.8 (95% CI 1.0, 2.6; p=0.001). Reporting of important methodological criteria, eg sample size justification and randomization implementation, improved from 1996 to 2004. Improvement notwithstanding, reporting of key methodological criteria remained consistently below 50% in 2004.
                This formal review suggests that randomized, controlled trial reporting in the urology literature has improved since the publication of the Consolidated Standards of Reporting Trials statement in 1996. Certain areas, such as reporting of trial methods, continue to meet Consolidated Standards of Reporting Trials criteria in fewer than half of publications. Ongoing graduate and postgraduate education in trial design and evidence based practice may result in further improvement in randomized, controlled trial reporting. Selective reporting of outcomes within published studies based on the nature or direction of their results has been widely suspected, but direct evidence of such bias is currently limited to case reports.
                To study empirically the extent and nature of outcome reporting bias in a cohort of randomized trials.
                Cohort study using protocols and published reports of randomized trials approved by the Scientific-Ethical Committees for Copenhagen and Frederiksberg, Denmark, in 1994-1995. The number and characteristics of reported and unreported trial outcomes were recorded from protocols, journal articles, and a survey of trialists. An outcome was considered incompletely reported if insufficient data were presented in the published articles for meta-analysis. Odds ratios relating the completeness of outcome reporting to statistical significance were calculated for each trial and then pooled to provide an overall estimate of bias. Protocols and published articles were also compared to identify discrepancies in primary outcomes.
                Completeness of reporting of efficacy and harm outcomes and of statistically significant vs nonsignificant outcomes; consistency between primary outcomes defined in the most recent protocols and those defined in published articles.
                One hundred two trials with 122 published journal articles and 3736 outcomes were identified. Overall, 50% of efficacy and 65% of harm outcomes per trial were incompletely reported. Statistically significant outcomes had a higher odds of being fully reported compared with nonsignificant outcomes for both efficacy (pooled odds ratio, 2.4; 95% confidence interval [CI], 1.4-4.0) and harm (pooled odds ratio, 4.7; 95% CI, 1.8-12.0) data. In comparing published articles with protocols, 62% of trials had at least 1 primary outcome that was changed, introduced, or omitted. Eighty-six percent of survey responders (42/49) denied the existence of unreported outcomes despite clear evidence to the contrary.
                The reporting of trial outcomes is not only frequently incomplete but also biased and inconsistent with protocols. Published articles, as well as reviews that incorporate them, may therefore be unreliable and overestimate the benefits of an intervention. To ensure transparency, planned trials should be registered and protocols should be made publicly available prior to trial completion. Previous studies have shown that Norwegian public health physicians do not systematically and explicitly use scientific evidence in their practice. They work in an environment that does not encourage the integration of this information in decision-making. In this study we investigate whether a theoretically grounded tailored intervention to diffuse evidence-based public health practice increases the physicians' use of research information.
                148 self-selected public health physicians were randomised to an intervention group (n = 73) and a control group (n = 75). The intervention group received a multifaceted intervention while the control group received a letter declaring that they had access to library services. Baseline assessments before the intervention and post-testing immediately at the end of a 1.5-year intervention period were conducted. The intervention was theoretically based and consisted of a workshop in evidence-based public health, a newsletter, access to a specially designed information service, to relevant databases, and to an electronic discussion list. The main outcome measure was behaviour as measured by the use of research in different documents.
                The intervention did not demonstrate any evidence of effects on the objective behaviour outcomes. We found, however, a statistical significant difference between the two groups for both knowledge scores: Mean difference of 0.4 (95% CI: 0.2-0.6) in the score for knowledge about EBM-resources and mean difference of 0.2 (95% CI: 0.0-0.3) in the score for conceptual knowledge of importance for critical appraisal. There were no statistical significant differences in attitude-, self-efficacy-, decision-to-adopt- or job-satisfaction scales. There were no significant differences in Cochrane library searching after controlling for baseline values and characteristics.
                Though demonstrating effect on knowledge the study failed to provide support for the hypothesis that a theory-based multifaceted intervention targeted at identified barriers will change professional behaviour. Few meta-analyses of randomised trials assess the quality of the studies included. Yet there is increasing evidence that trial quality can affect estimates of intervention efficacy. We investigated whether different methods of quality assessment provide different estimates of intervention efficacy evaluated in randomised controlled trials (RCTs).
                We randomly selected 11 meta-analyses that involved 127 RCTs on the efficacy of interventions used for circulatory and digestive diseases, mental health, and pregnancy and childbirth. We replicated all the meta-analyses using published data from the primary studies. The quality of reporting of all 127 clinical trials was assessed by means of component and scale approaches. To explore the effects of quality on the quantitative results, we examined the effects of different methods of incorporating quality scores (sensitivity analysis and quality weights) on the results of the meta-analyses.
                The quality of trials was low. Masked assessments provided significantly higher scores than unmasked assessments (mean 2.74 [SD 1.10] vs 2.55 [1.20]). Low-quality trials (score < or = 2), compared with high-quality trials (score > 2), were associated with an increased estimate of benefit of 34% (ratio of odds ratios [ROR] 0.66 [95% CI 0.52-0.83]). Trials that used inadequate allocation concealment, compared with those that used adequate methods, were also associated with an increased estimate of benefit (37%; ROR=0.63 [0.45-0.88]). The average treatment benefit was 39% (odds ratio [OR] 0.61 [0.57-0.65]) for all trials, 52% (OR 0.48 [0.43-0.54]) for low-quality trials, and 29% (OR 0.71 [0.65-0.77]) for high-quality trials. Use of all the trial scores as quality weights reduced the effects to 35% (OR 0.65 [0.59-0.71]) and resulted in the least statistical heterogeneity.
                Studies of low methodological quality in which the estimate of quality is incorporated into the meta-analyses can alter the interpretation of the benefit of intervention, whether a scale or component approach is used in the assessment of trial quality. In conducting reviews on the effectiveness of physiotherapy interventions on children with cerebral palsy, the assessment of trials can be hampered by problems in reporting. Therefore, we set out to evaluate trial reporting by using the Consolidated Standards of Reporting Trials (CONSORT) statement recommendations.
                Randomized, controlled trials published in 1990 or later were identified in literature searches for reviews on the effectiveness of physiotherapy interventions on children with cerebral palsy. Two independent reviewers evaluated the trial reporting by using a modified 33-item Consolidated Standards of Reporting Trials checklist.
                We identified 15 randomized, controlled trials. Almost half (48%) of the applicable items were reported adequately. Inadequate reporting was found in the following items: outcome measures, sample-size determination, details of the sequence generation, allocation concealment and implementation of the randomization, success of assessor blinding, recruitment and follow-up dates, intention-to-treat analysis, precision of the effect size, co-interventions, and adverse events.
                Only a small number of sufficiently reported trials were found. Because nearly all items had been described in at least 1 article, high-quality reporting seems feasible. Assessment of trials depends on appropriate reporting, and poor reporting jeopardizes judgments on the clinical implications. Authors of randomized, controlled trials are encouraged to follow the Consolidated Standards of Reporting Trials criteria. There is a clear need to improve the quality of reporting of trials in this field. To explore whether reported methodologic quality affects estimated intervention effects in randomized trials and contributes to discrepancies between the results of large randomized trials and small randomized trials in meta-analyses.
                Meta-analyses of randomized trials that included at least one large trial (>/=1000 participants) were included, regardless of the therapeutic area. Eligible meta-analyses were identified through electronic searches and bibliographies of relevant articles.
                Full-length randomized trials.
                Methodologic quality was assessed according to reported randomization, double blinding, and follow-up as separate components and by using the Jadad composite scale.
                Fourteen meta-analyses involving 190 randomized trials from eight therapeutic areas were included. Compared with large trials, intervention effects were exaggerated in small trials with inadequate allocation sequence generation (ratio of odds ratios, 0.46 [95% CI, 0.25 to 0.83]; P = 0.011), inadequate allocation concealment (ratio of odds ratios, 0.49 [CI, 0.27 to 0.86]; P = 0.014), and no double blinding (ratio of odds ratios, 0.52 [CI, 0.28 to 0.96]; P = 0.01). Large trials did not differ significantly from small trials with adequate generation of the allocation sequence, adequate allocation concealment, or adequate double blinding. No association was seen between reported follow-up and intervention effects. The Jadad scale provided no additional information because the scale and the quality components overlapped substantially.
                Inadequate generation of the allocation sequence, allocation concealment, and double blinding lead to exaggerated estimates of intervention benefit and may contribute to discrepancies between the results of large randomized trials and small randomized trials in meta-analyses. The quality of randomized controlled trials (RCTs) in subfertility and their suitability for inclusion in meta-analyses have been assessed in the past and found to be insufficient. Our aim was to assess whether this quality has improved over time, particularly since the publication of the Consolidated Standards of Reporting Trials (CONSORT) statement, and to assess what proportion of trials could be included in the meta-analyses of pregnancy outcomes such as those included in Cochrane Reviews.
                A selection of subfertility trials published in 1990, 1996 and 2002 was collected from the Cochrane Menstrual Disorder and Subfertility Group (MDSG) database. Only trials published in English as full journal articles, claiming to be randomized and reporting on pregnancy outcomes, were included.
                One hundred and sixty-four trials met our inclusion criteria. Twenty-four (15%) were found not to be randomized, despite claims, and only 10 trials (6%) provided adequate details on the methods of randomization and allocation concealment. Of these, only three had sufficient details extractable to allow for an intention-to-treat analysis of the outcome 'live birth'.
                Although an improvement in some subfertility-specific issues was observed, the quality of reporting of RCTs still needs to improve to make them suitable for inclusion in meta-analyses such as those in the Cochrane Library. Randomized controlled trials (RCTs) of interventions provide the highest level of evidence about efficacy but their value either alone or within a meta-analysis is dependent on its methodological quality. For this reason recent RCTs in organ transplantation were assessed for quality. RCTs published between 2004 and 2006 (n = 332) were assessed, after excluding duplicate and nonEnglish reports. Quality was evaluated using the Jadad score plus allocation concealment and intention to treat analysis. We noted journal type, journal author instructions, funding source, sample size and number and location of study centres. Around one-third of RCTs had a Jadad score of 3 or greater (indication of a methodologically good quality trial) and the other two parameters were satisfied in just over one third. Although the majority of trials were published in speciality journals the quality of those published in general journals was superior. Commercially sponsored trials were of better quality as were multicentre trials in contrast to single centre trials. Overall quality of reporting of RCTs in organ transplantation is poor and as RCTs provide the highest level of evidence in evaluations of interventions there needs to be a concerted effort within the transplant community to improve the standards of RCTs.
The quality of the evidence base currently provides very little guidance for the development of services. If randomised controlled trials are not feasible then consideration should be given to alternative study designs, such as prospective systems of audit conducted across several centres, as this has the potential to improve the current level of evidence. These studies should include baseline measurement at admission along with demographic data, and outcomes measured using a few standardised robust instruments.