Publication bias: evidence of delayed publication in a cohort study of clinical research projects. Outcome reporting bias in randomized trials funded by the Canadian Institutes of Health Research. Reported methodologic quality and discrepancies between large and small randomized trials in meta-analyses. Assessment of risk of bias among pediatric randomized controlled trials. A randomised controlled trial to assess the effectiveness of offering study results as an incentive to increase response rates to postal questionnaires [ISRCTN26118436]. Positive-outcome bias and other limitations in the outcome of research abstracts submitted to a scientific meeting. Empirical evidence for selective reporting of outcomes in randomized trials: comparison of protocols to published articles. Patient-based outcome results from a cluster randomized trial of shared decision making skill development and use of risk communication aids in general practice. Does a 3-day workshop for family medicine trainees improve preventive care? A randomized control trial.
To determine the extent to which publication is influenced by study outcome.
                A cohort of studies submitted to a hospital ethics committee over 10 years were examined retrospectively by reviewing the protocols and by questionnaire. The primary method of analysis was Cox's proportional hazards model.
                University hospital, Sydney, Australia.
                748 eligible studies submitted to Royal Prince Alfred Hospital Ethics Committee between 1979 and 1988.
                Time to publication.
                Response to the questionnaire was received for 520 (70%) of the eligible studies. Of the 218 studies analysed with tests of significance, those with positive results (P < 0.05) were much more likely to be published than those with negative results (P > or = 0.10) (hazard ratio 2.32 (95% confidence interval 1.47 to 3.66), P = 0.0003), with a significantly shorter time to publication (median 4.8 v 8.0 years). This finding was even stronger for the group of 130 clinical trials (hazard ratio 3.13 (1.76 to 5.58). P = 0.0001), with median times to publication of 4.7 and 8.0 years respectively. These results were not materially changed after adjusting for other significant predictors of publication. Studies with indefinite conclusions (0.05 < or = P < 0.10) tended to have an even lower publication rate and longer time to publication than studies with negative results (hazard ratio 0.39 (0.13 to 1.12), P = 0.08). For the 103 studies in which outcome was rated qualitatively, there was no clear cut evidence of publication bias, although the number of studies in this group was not large.
                This study confirms the evidence of publication bias found in other studies and identifies delay in publication as an additional important factor. The study results support the need for prospective registration of trials to avoid publication bias and also support restricting the selection of trials to those started before a common date in undertaking systematic reviews. The reporting of outcomes within published randomized trials has previously been shown to be incomplete, biased and inconsistent with study protocols. We sought to determine whether outcome reporting bias would be present in a cohort of government-funded trials subjected to rigorous peer review.
                We compared protocols for randomized trials approved for funding by the Canadian Institutes of Health Research (formerly the Medical Research Council of Canada) from 1990 to 1998 with subsequent reports of the trials identified in journal publications. Characteristics of reported and unreported outcomes were recorded from the protocols and publications. Incompletely reported outcomes were defined as those with insufficient data provided in publications for inclusion in meta-analyses. An overall odds ratio measuring the association between completeness of reporting and statistical significance was calculated stratified by trial. Finally, primary outcomes specified in trial protocols were compared with those reported in publications.
                We identified 48 trials with 68 publications and 1402 outcomes. The median number of participants per trial was 299, and 44% of the trials were published in general medical journals. A median of 31% (10th-90th percentile range 5%-67%) of outcomes measured to assess the efficacy of an intervention (efficacy outcomes) and 59% (0%-100%) of those measured to assess the harm of an intervention (harm outcomes) per trial were incompletely reported. Statistically significant efficacy outcomes had a higher odds than nonsignificant efficacy outcomes of being fully reported (odds ratio 2.7; 95% confidence interval 1.5-5.0). Primary outcomes differed between protocols and publications for 40% of the trials.
                Selective reporting of outcomes frequently occurs in publications of high-quality government-funded trials. To explore whether reported methodologic quality affects estimated intervention effects in randomized trials and contributes to discrepancies between the results of large randomized trials and small randomized trials in meta-analyses.
                Meta-analyses of randomized trials that included at least one large trial (>/=1000 participants) were included, regardless of the therapeutic area. Eligible meta-analyses were identified through electronic searches and bibliographies of relevant articles.
                Full-length randomized trials.
                Methodologic quality was assessed according to reported randomization, double blinding, and follow-up as separate components and by using the Jadad composite scale.
                Fourteen meta-analyses involving 190 randomized trials from eight therapeutic areas were included. Compared with large trials, intervention effects were exaggerated in small trials with inadequate allocation sequence generation (ratio of odds ratios, 0.46 [95% CI, 0.25 to 0.83]; P = 0.011), inadequate allocation concealment (ratio of odds ratios, 0.49 [CI, 0.27 to 0.86]; P = 0.014), and no double blinding (ratio of odds ratios, 0.52 [CI, 0.28 to 0.96]; P = 0.01). Large trials did not differ significantly from small trials with adequate generation of the allocation sequence, adequate allocation concealment, or adequate double blinding. No association was seen between reported follow-up and intervention effects. The Jadad scale provided no additional information because the scale and the quality components overlapped substantially.
                Inadequate generation of the allocation sequence, allocation concealment, and double blinding lead to exaggerated estimates of intervention benefit and may contribute to discrepancies between the results of large randomized trials and small randomized trials in meta-analyses. The goal was to assess the risk of bias among pediatric, randomized, controlled trials (RCTs) reported in 8 high-impact journals.
                We searched PubMed for all pediatric RCTs reported between July 1, 2007, and June 30, 2008, in 8 journals with high impact factors. Using Cochrane Collaboration methods for risk assessment, we evaluated all reports for risk of bias according to domain (ie, randomized sequence generation, allocation concealment, masking, incomplete outcome data, selective outcome reporting, and other). We used multiple logistic regression to test for associations between the presence of a high risk of bias according to domain and funding source, intervention type, trial registration, and multicenter status.
                Industry-funded RCTs were more likely to show a high risk of bias for sequence generation, compared with government-funded RCTs (adjusted odds ratio [aOR]: 6.1 [95% confidence interval [CI]: 1.70- 21.89]), and behavioral/educational trials were more likely to show a high risk of bias for sequence generation (aOR: 2.8 [95% CI: 1.06-7.36]) and allocation concealment (aOR: 4.09 [95% CI: 1.69-9.90]), compared with drug trials. Registered trials were less likely to have a high risk of bias for sequence generation, compared with nonregistered trials (aOR: 0.33 [95% CI: 0.15-0.71]).
                Overall, we found a large proportion of pediatric RCT reports with a high risk of bias for sequence generation and allocation concealment. Factors associated with a high risk of bias included industry funding and assessment of behavioral/educational interventions, whereas trial registration was associated with a lower risk of bias. Postal questionnaires are widely used to collect outcome data on participants. However, a poor response to questionnaires will reduce the statistical power of the study and may introduce bias. A meta analysis of ten trials offering study results, largely in the fields of education and marketing, was shown to be ineffective, with the odds ratio for response with offering research findings is 0.92 (95% CI 0.75 to 1.11). However uncertainty still exists as it is uncertain whether results from such trials can be extrapolated to that of a health care setting. The aim of this study was to assess whether offering participants study results increases the response rates to postal questionnaires.
                1038 women aged over 70 years were remotely randomised by computer in a 3:1 ratio. 250 participants did not receive the offer of knowing the results of the trial and 788 participants were offered the results of the trial in a postal questionnaire. The main outcome measure was response rate. Chi square test was used to evaluate the overall differences in response rate between the two groups. An adjusted analysis, adjusting for whether the participant was taking calcium and age was also undertaken.
                The response rates were not significantly different Odds Ratio 0.88 (95% confidence intervals 0.48 to 1.63) p = 0.69.
                Offering study results to women living in the community aged over 70 does not increase response rates to postal questionnaires. Although researchers have an ethical obligation to offer participants study results, since 10% of women did not wish to receive the results, investigators should give participants the option to opt out of receiving the study's results. Studies with positive results are more likely to be published in biomedical journals than are studies with negative results. However, many studies submitted for consideration at scientific meetings are never published in full; bias in this setting is poorly studied.
                To identify features associated with the fate of research abstracts submitted to a scientific meeting.
                Prospective observational cohort, with 5-year follow-up of all research submitted for consideration to the major annual 1991 US research meeting in the specialty of emergency medicine.
                All research abstracts submitted for consideration at the meeting for possible presentation.
                Characteristics associated with acceptance for presentation at the meeting and subsequent publication as a full manuscript.
                A total of 492 research abstracts were submitted from programs in emergency medicine and other specialties affiliated with 103 US medical schools. A total of 179 (36%) were accepted for presentation and 214 (43%) were published in 44 journals. Of the 179 abstracts accepted for presentation, 111 studies were published. Scientific quality of abstracts or prestige of the journal in which the study was eventually published did not predict either of these outcomes. The best predictors (by logistic regression) of meeting acceptance were a subjective "originality" factor (odds ratio [OR], 2.07; 95% confidence interval [CI], 1.13-3.89) and positive results (OR, 1.99; 95% CI, 1.07-3.84), and, for publication, meeting acceptance (OR, 2.49; 95% CI, 1.49-4.35) and large sample size (OR, 2.26; 95% CI, 1.23-4.31). Forty-nine percent (241) of abstracts did not report on blinding, and 24% (118) did not report on randomization. Acceptance and publication were both more likely for positive outcomes (P=.03). Funnel plots showed the classic distribution of positive-outcome ("publication") bias at each of the submission, acceptance, and publication phases. Meeting acceptance predicted publication with a sensitivity of only 51%, specificity of 71%, positive predictive value of 57%, and negative predictive value of 66%.
                Positive-outcome bias was evident when studies were submitted for consideration and was amplified in the selection of abstracts for both presentation and publication, neither of which was strongly related to study design or quality. Selective reporting of outcomes within published studies based on the nature or direction of their results has been widely suspected, but direct evidence of such bias is currently limited to case reports.
                To study empirically the extent and nature of outcome reporting bias in a cohort of randomized trials.
                Cohort study using protocols and published reports of randomized trials approved by the Scientific-Ethical Committees for Copenhagen and Frederiksberg, Denmark, in 1994-1995. The number and characteristics of reported and unreported trial outcomes were recorded from protocols, journal articles, and a survey of trialists. An outcome was considered incompletely reported if insufficient data were presented in the published articles for meta-analysis. Odds ratios relating the completeness of outcome reporting to statistical significance were calculated for each trial and then pooled to provide an overall estimate of bias. Protocols and published articles were also compared to identify discrepancies in primary outcomes.
                Completeness of reporting of efficacy and harm outcomes and of statistically significant vs nonsignificant outcomes; consistency between primary outcomes defined in the most recent protocols and those defined in published articles.
                One hundred two trials with 122 published journal articles and 3736 outcomes were identified. Overall, 50% of efficacy and 65% of harm outcomes per trial were incompletely reported. Statistically significant outcomes had a higher odds of being fully reported compared with nonsignificant outcomes for both efficacy (pooled odds ratio, 2.4; 95% confidence interval [CI], 1.4-4.0) and harm (pooled odds ratio, 4.7; 95% CI, 1.8-12.0) data. In comparing published articles with protocols, 62% of trials had at least 1 primary outcome that was changed, introduced, or omitted. Eighty-six percent of survey responders (42/49) denied the existence of unreported outcomes despite clear evidence to the contrary.
                The reporting of trial outcomes is not only frequently incomplete but also biased and inconsistent with protocols. Published articles, as well as reviews that incorporate them, may therefore be unreliable and overestimate the benefits of an intervention. To ensure transparency, planned trials should be registered and protocols should be made publicly available prior to trial completion. Shared decision-making (SDM) between professionals and patients is increasingly advocated from ethical principles. Some data are accruing about the effects of such approaches on health or other patient-based outcomes. These effects often vary substantially between studies.
                Our aim was to evaluate the effects of training GPs in SDM, and the use of simple risk communication aids in general practice, on patient-based outcomes.
                A cluster randomized trial with crossover was carried out with the participation of 20 recently qualified GPs in urban and rural general practices in Gwent, South Wales. A total of 747 patients with known atrial fibrillation, prostatism, menorrhagia or menopausal symptoms were invited to a consultation to review their condition or treatments. After baseline, participating doctors were randomized to receive training in (i) SDM skills; or (ii) the use of simple risk communication aids, using simulated patients. The alternative training was then provided for the final study phase. Patients were randomly allocated to a consultation during baseline or intervention 1 (SDM or risk communication aids) or intervention 2 phases. A randomly selected half of the consultations took place in 'research clinics' to evaluate the effects of more time for consultations, compared with usual surgery time. Patient-based outcomes were assessed at exit from consultation and 1 month follow-up. These were: COMRADE instrument (principal measures; subscales of risk communication and confidence in decision), and a range of secondary measures (anxiety, patient enablement, intention to adhere to chosen treatment, satisfaction with decision, support in decision making and SF-12 health status measure). Multilevel modelling was carried out with outcome score as the dependent variable, and follow-up point (i.e. exit or 1 month later for each patient), patient and doctor levels of explanatory variables.
                No statistically significant changes in patient-based outcomes due to the training interventions were found: COMRADE risk communication score increased 0.7 [95% confidence interval (CI) -0.92 to 2.32] after risk communication training and 0.9 (95% CI -0.89 to 2.35) after SDM training; and COMRADE satisfaction with communication score increased by 1.0 (95% CI -1.1 to 3.1) after risk communication, and decreased by 0.6 (95% CI 2.7 to -1.5) after SDM training. Patients' confidence in the decision (2.1 increase, 95% CI 0.7-3.5, P < 0.01) and expectation to adhere to chosen treatments (0.7 increase, 95% CI 0.04-1.36, P < 0.05) were significantly greater among patients seen in the research clinics (when more time was available) compared with usual surgery time. Most outcomes deteriorated between exit and 1 month later. There was no interaction between intervention effects.
                Patients can be more involved in treatment decisions, and risks and benefits of treatment options can be explained in more detail, without adversely affecting patient-based outcomes. SDM and risk communication may be advocated from values and ethical principles even without evidence of health gain or improvement in patient-based outcomes, but the resources required to enhance these professional skills must also be taken into consideration. These data also indicate the benefits of extra consultation time. To evaluate the impact on clinical behavior of a 3-day workshop designed to increase trainees' rates of smoking cessation counseling and reminders about Pap smears in routine consultations.
                Randomized control trial.
                Accredited teaching practices of the Royal Australian College of General Practitioners' Training Program.
                Thirty-four trainees and 1,500 consecutive adult patients ages 16-65 years.
                Trainees randomly allocated to the experimental group participated in a 3-day interactive workshop on disease prevention during their 13-week family medicine term. Audiotapes of consultations with adults conducted by trainees at the beginning and end of the rotation were analyzed blind to compare assessment of patients' smoking status and, for women, date of last Pap smear. A questionnaire mailed to each patient after the consultation also allowed identification of smokers and women overdue for a smear. Consultations with these patients at risk were analyzed for preventive counseling. Inter- and intrarater reliability was calculated for audiotape analysis.
                Preworkshop rates of questions about smoking were low, occurring in 22% of consultations. While trainees allocated to the experimental workshop were more likely to ask a routine question about smoking at the end of the term than those in the control group (P = 0.01), two-thirds of smokers remained undetected irrespective of trainee group and fewer than one in five were advised to stop smoking. Reminders about Pap smears did not change as a result of training and remained low in fewer than 20% of consultations. kappa values demonstrated high reliability of audiotape analysis.
                This direct measurement of clinical behavior revealed that low levels of preventive care provided by trainees are resistant to skills training without reinforcement in clinical practice. In view of the importance of prevention in routine consultations, we recommend continued evaluation of more intensive educational programs. Those withstanding rigorous evaluation could be considered for implementation in similar training contexts seeking to improve the frequency and quality of disease prevention in primary medical care.
Based on one small study judged to be of moderate risk of bias, we tentatively conclude that CAHTS may be well received by study participants and potentially offer time saving in practice. However, more robust studies with larger sample sizes are needed to confirm these. We cannot draw on any conclusions in relation to any other clinical outcomes at this stage.