Empirical evidence of bias. Dimensions of methodological quality associated with estimates of treatment effects in controlled trials. Empirical evidence for selective reporting of outcomes in randomized trials: comparison of protocols to published articles. A cohort study of summary reports of controlled trials. Abstracts of trials presented at the Vth World Congress of Psychiatry (Mexico, 1971): a cohort study. Publication bias: evidence of delayed publication in a cohort study of clinical research projects. Publication bias and meta-analyses: a practical example. Cumulating evidence from randomized trials: utilizing sequential monitoring boundaries for cumulative meta-analysis. Assessment of risk of bias among pediatric randomized controlled trials. [Effects of information on and reflection of women's pain experience during mammography].
To determine if inadequate approaches to randomized controlled trial design and execution are associated with evidence of bias in estimating treatment effects.
                An observational study in which we assessed the methodological quality of 250 controlled trials from 33 meta-analyses and then analyzed, using multiple logistic regression models, the associations between those assessments and estimated treatment effects.
                Meta-analyses from the Cochrane Pregnancy and Childbirth Database.
                The associations between estimates of treatment effects and inadequate allocation concealment, exclusions after randomization, and lack of double-blinding.
                Compared with trials in which authors reported adequately concealed treatment allocation, trials in which concealment was either inadequate or unclear (did not report or incompletely reported a concealment approach) yielded larger estimates of treatment effects (P < .001). Odds ratios were exaggerated by 41% for inadequately concealed trials and by 30% for unclearly concealed trials (adjusted for other aspects of quality). Trials in which participants had been excluded after randomization did not yield larger estimates of effects, but that lack of association may be due to incomplete reporting. Trials that were not double-blind also yielded larger estimates of effects (P = .01), with odds ratios being exaggerated by 17%.
                This study provides empirical evidence that inadequate methodological approaches in controlled trials, particularly those representing poor allocation concealment, are associated with bias. Readers of trial reports should be wary of these pitfalls, and investigators must improve their design, execution, and reporting of trials. Selective reporting of outcomes within published studies based on the nature or direction of their results has been widely suspected, but direct evidence of such bias is currently limited to case reports.
                To study empirically the extent and nature of outcome reporting bias in a cohort of randomized trials.
                Cohort study using protocols and published reports of randomized trials approved by the Scientific-Ethical Committees for Copenhagen and Frederiksberg, Denmark, in 1994-1995. The number and characteristics of reported and unreported trial outcomes were recorded from protocols, journal articles, and a survey of trialists. An outcome was considered incompletely reported if insufficient data were presented in the published articles for meta-analysis. Odds ratios relating the completeness of outcome reporting to statistical significance were calculated for each trial and then pooled to provide an overall estimate of bias. Protocols and published articles were also compared to identify discrepancies in primary outcomes.
                Completeness of reporting of efficacy and harm outcomes and of statistically significant vs nonsignificant outcomes; consistency between primary outcomes defined in the most recent protocols and those defined in published articles.
                One hundred two trials with 122 published journal articles and 3736 outcomes were identified. Overall, 50% of efficacy and 65% of harm outcomes per trial were incompletely reported. Statistically significant outcomes had a higher odds of being fully reported compared with nonsignificant outcomes for both efficacy (pooled odds ratio, 2.4; 95% confidence interval [CI], 1.4-4.0) and harm (pooled odds ratio, 4.7; 95% CI, 1.8-12.0) data. In comparing published articles with protocols, 62% of trials had at least 1 primary outcome that was changed, introduced, or omitted. Eighty-six percent of survey responders (42/49) denied the existence of unreported outcomes despite clear evidence to the contrary.
                The reporting of trial outcomes is not only frequently incomplete but also biased and inconsistent with protocols. Published articles, as well as reviews that incorporate them, may therefore be unreliable and overestimate the benefits of an intervention. To ensure transparency, planned trials should be registered and protocols should be made publicly available prior to trial completion. Substantial numbers of clinical trials continue to be reported only in summary reports that present insufficient methodological details to permit informed judgments about the likely validity of the conclusions. Using a cohort of 176 controlled trials reported in summary form, we tested the hypotheses that they would be more likely to be followed by full reports if, on the basis of the information provided in the summary report, (1) the trial was judged to be methodologically sound, (2) the results favored the test treatment, and (3) the sample size was relatively large. The results of univariate and multivariate analyses provided support for only the third of these hypotheses. Investigators, as well as those who fund and sanction the conduct of clinical research, should make greater efforts to ensure that clinical trials are reported properly. Systematic reviews should identify all relevant trials in order to minimize the potential for bias and the play of chance in their results. Other specialities have shown that conference proceedings are a rich source of trials, but many of these trials are never fully published.
                All clinical trials presented at a single conference (Vth World Congress of Psychiatry, Mexico, 1971) were identified by hand searching. Full publications of these abstracts were then sought on five databases by searching for the authors or relevant key words.
                Full publications were found for 46% of the abstracts. The odds of publication decreased for abstracts that were from a non-Anglophone country or that failed to mention randomization.
                Anyone wishing to undertake a systematic review of a mental-health care topic should search relevant conference proceedings for trials. To determine the extent to which publication is influenced by study outcome.
                A cohort of studies submitted to a hospital ethics committee over 10 years were examined retrospectively by reviewing the protocols and by questionnaire. The primary method of analysis was Cox's proportional hazards model.
                University hospital, Sydney, Australia.
                748 eligible studies submitted to Royal Prince Alfred Hospital Ethics Committee between 1979 and 1988.
                Time to publication.
                Response to the questionnaire was received for 520 (70%) of the eligible studies. Of the 218 studies analysed with tests of significance, those with positive results (P < 0.05) were much more likely to be published than those with negative results (P > or = 0.10) (hazard ratio 2.32 (95% confidence interval 1.47 to 3.66), P = 0.0003), with a significantly shorter time to publication (median 4.8 v 8.0 years). This finding was even stronger for the group of 130 clinical trials (hazard ratio 3.13 (1.76 to 5.58). P = 0.0001), with median times to publication of 4.7 and 8.0 years respectively. These results were not materially changed after adjusting for other significant predictors of publication. Studies with indefinite conclusions (0.05 < or = P < 0.10) tended to have an even lower publication rate and longer time to publication than studies with negative results (hazard ratio 0.39 (0.13 to 1.12), P = 0.08). For the 103 studies in which outcome was rated qualitatively, there was no clear cut evidence of publication bias, although the number of studies in this group was not large.
                This study confirms the evidence of publication bias found in other studies and identifies delay in publication as an additional important factor. The study results support the need for prospective registration of trials to avoid publication bias and also support restricting the selection of trials to those started before a common date in undertaking systematic reviews. Publication bias is widely appreciated, but considerable time and effort are needed to locate and obtain data from unpublished randomized controlled trials (RCTs), those published in non-English language journals or those reported in the gray literature; for this publication, we will call this collection of trials the "gray+literature." However, excluding such trials from systematic reviews could introduce bias and give rise to misleading conclusions.
                We aimed to explore and quantify the impact of inclusion of gray+ literature on the results of all completed individual patient data (IPD) reviews coordinated by our group (13 meta-analyses). For each IPD review, results were calculated for RCTs fully published in English language journals and RCTs fully published in English language journals and the gray+literature.
                The IPD meta-analyses based only on RCTs that were fully published in English language journals tended to give more favorable results than those that included RCTs from the gray+literature. Although in most cases the addition of gray+data gave less encouraging results, moving the estimated treatment effect toward a null result, the direction of effect was not always predictable.
                We recommend that all systematic reviews should at least attempt to identify trials reported in the gray+literature and, where possible, obtain data from them. We propose the adaptation of classical monitoring boundaries for use in cumulative meta-analysis as guidelines for deciding when accumulating evidence is statistically significant and medically convincing. The interpretation of information from a randomized controlled trial is compared with that from a meta-analysis. The concept of optimal information size for a meta-analysis is developed and used to adapt monitoring boundaries to cumulative meta-analysis. The goal was to assess the risk of bias among pediatric, randomized, controlled trials (RCTs) reported in 8 high-impact journals.
                We searched PubMed for all pediatric RCTs reported between July 1, 2007, and June 30, 2008, in 8 journals with high impact factors. Using Cochrane Collaboration methods for risk assessment, we evaluated all reports for risk of bias according to domain (ie, randomized sequence generation, allocation concealment, masking, incomplete outcome data, selective outcome reporting, and other). We used multiple logistic regression to test for associations between the presence of a high risk of bias according to domain and funding source, intervention type, trial registration, and multicenter status.
                Industry-funded RCTs were more likely to show a high risk of bias for sequence generation, compared with government-funded RCTs (adjusted odds ratio [aOR]: 6.1 [95% confidence interval [CI]: 1.70- 21.89]), and behavioral/educational trials were more likely to show a high risk of bias for sequence generation (aOR: 2.8 [95% CI: 1.06-7.36]) and allocation concealment (aOR: 4.09 [95% CI: 1.69-9.90]), compared with drug trials. Registered trials were less likely to have a high risk of bias for sequence generation, compared with nonregistered trials (aOR: 0.33 [95% CI: 0.15-0.71]).
                Overall, we found a large proportion of pediatric RCT reports with a high risk of bias for sequence generation and allocation concealment. Factors associated with a high risk of bias included industry funding and assessment of behavioral/educational interventions, whereas trial registration was associated with a lower risk of bias. At mammography, strong compression of the breasts is required to reduce the radiation dose and to obtain radiographic images of high quality. In recent times the mass media have received a number of letters with complaints about the painfulness of the examination and an unsympathetic reception by the personnel. The aim of this study was to find out what effect information and a special manner of reception ("reflection") had on the experience of pain at mammography. Reflection meant that the X-ray nurse, in an accepting manner, reflected the feelings and facts which the woman expressed. The study was performed as a 2 x 2 factorial design. One independent variable was a letter of information, with the two alternatives letter of information and no letter of information, and another was reflection, with the two alternatives reflection and no reflection. Seventy-six women were randomly allocated to these four conditions. The letter of information stated why the examination was to be performed, how it is done and how it is usually experienced. The results showed that all women in all groups experienced relatively little pain. They considered that they were well received and were satisfied with the information. The difference between the groups was small. There was a tendency for those who were both received with reflection and were given a letter of information to be more satisfied with the examination as compared with the other women. A larger number of women in this former group than in the other groups stated reasons for their assessment. This result was significant.(ABSTRACT TRUNCATED AT 250 WORDS)
It is difficult to draft conclusions on the basis of only two trials. One of the possible reason for the lack of evidence could be the difficulty to conduct trial with young people due to practical and ethic reasons.