A randomised controlled trial of the outcome of health assessment of people aged 75 years and over. Bias in reporting clinical trials. Does the inclusion of grey literature influence estimates of intervention effectiveness reported in meta-analyses? Systematic reviews of health effects of social interventions: 1. Finding the evidence: how far should you go? Positive-outcome bias and other limitations in the outcome of research abstracts submitted to a scientific meeting. A randomized trial of recruitment methods for older African American men in the Prostate, Lung, Colorectal and Ovarian (PLCO) Cancer Screening Trial. A randomised controlled trial to assess the effectiveness of offering study results as an incentive to increase response rates to postal questionnaires [ISRCTN26118436]. Multiphasic Health Checkup Evaluation: a 16-year follow-up. Reported methodologic quality and discrepancies between large and small randomized trials in meta-analyses.
To measure the outcomes of a health assessment, conducted by a nurse, of people aged 75 years and older (75+HA) living independently in their own homes.
                Randomised controlled trial (RCT).
                A convenience sample of six general practices within the Adelaide Western Division of General Practice (AWDGP). A random sample of 100 participants was drawn from practice age-sex registers. Data were collected in initial visits between 1 August 1998 and February 1999, then in follow-up visits one year later.
                Participants were aged 75 years and over on 1 August 1998 and living independently in the community. 145 eligible patients were invited to join the study, and 100 of these consented to enrol (69%).
                A 75+HA conducted in the participant's home by a nurse and reported to their usual general practitioner.
                Primary: number of problems in each group; number of participants with problems; and mortality. Secondary: physical function; psychological (including cognitive) function; falls; and admission to institution.
                There were no significant differences between the control and intervention groups at follow-up in the number of problems, the number of participants with problems, or mortality. In the intervention group, there was significant improvement in self-rated health, geriatric depression score (GDS 15), and number of falls.
                This RCT has not demonstrated improvement in health status of the intervention group which received a 75+HA compared with a control group left to usual care. The primary aim of the present study was to identify possible occurrence of selective reporting of the results of clinical trials to the Finnish National Agency for Medicines. Selective reporting may lead to poorly informed action or inaction by regulatory authorities.
                In 1987, 274 clinical drug trials were notified to the Finnish National Agency for Medicines. By December 1993, final reports had been received from 68 of these trials and statements that the trial had been suspended from 24 trials. The sponsors of the non-reported trials were requested to report the outcome. The outcomes, if any, of all reported and non-reported trials were classified as positive, inconclusive or negative.
                The total number of trials with positive, inconclusive or negative outcome were 111, 33 and 44, respectively; the outcomes of 86 trials could not be assessed. Final reports were received from 42/111 (38%) trials with positive, 6/33 (18%) with inconclusive and 9/44 (20%) with negative outcomes.
                Substantial evidence of selective reporting was detected, since trials with positive outcome resulted more often in submission of final report to regulatory authority than those with inconclusive or negative outcomes. The inclusion of only a subset of all available evidence in a meta-analysis may introduce biases and threaten its validity; this is particularly likely if the subset of included studies differ from those not included, which may be the case for published and grey literature (unpublished studies, with limited distribution). We set out to examine whether exclusion of grey literature, compared with its inclusion in meta-analysis, provides different estimates of the effectiveness of interventions assessed in randomised trials.
                From a random sample of 135 meta-analyses, we identified and retrieved 33 publications that included both grey and published primary studies. The 33 publications contributed 41 separate meta-analyses from several disease areas. General characteristics of the meta-analyses and associated studies and outcome data at the trial level were collected. We explored the effects of the inclusion of grey literature on the quantitative results using logistic-regression analyses.
                33% of the meta-analyses were found to include some form of grey literature. The grey literature, when included, accounts for between 4.5% and 75% of the studies in a meta-analysis. On average, published work, compared with grey literature, yielded significantly larger estimates of the intervention effect by 15% (ratio of odds ratios=1.15 [95% CI 1.04-1.28]). Excluding abstracts from the analysis further compounded the exaggeration (1.33 [1.10-1.60]).
                The exclusion of grey literature from meta-analyses can lead to exaggerated estimates of intervention effectiveness. In general, meta-analysts should attempt to identify, retrieve, and include all reports, grey and published, that meet predefined inclusion criteria. There is little guidance on how to identify useful evidence about the health effects of social interventions. The aim of this study was to assess the value of different ways of finding this type of information.
                Retrospective analysis of the sources of studies for one systematic review.
                Case study of a systematic review of the effectiveness of interventions in promoting a population shift from using cars towards walking and cycling.
                Only four of the 69 relevant studies were found in a "first-line" health database such as Medline. About half of all relevant studies were found through the specialist Transport database. Nine relevant studies were found through purposive internet searches and seven relevant studies were found by chance. The unique contribution of experts was not to identify additional studies, but to provide more information about those already found in the literature.
                Most of the evidence needed for this review was not found in studies indexed in familiar literature databases. Applying a sensitive search strategy across multiple databases and interfaces is very labour intensive. Retrospective analysis suggests that a more efficient method might have been to search a few key resources, then to ask authors and experts directly for the most robust reports of studies identified. However, internet publications and serendipitous discoveries did make a significant contribution to the total set of relevant evidence. Undertaking a comprehensive search may provide unique evidence and insights that would not be obtained using a more focused search. Studies with positive results are more likely to be published in biomedical journals than are studies with negative results. However, many studies submitted for consideration at scientific meetings are never published in full; bias in this setting is poorly studied.
                To identify features associated with the fate of research abstracts submitted to a scientific meeting.
                Prospective observational cohort, with 5-year follow-up of all research submitted for consideration to the major annual 1991 US research meeting in the specialty of emergency medicine.
                All research abstracts submitted for consideration at the meeting for possible presentation.
                Characteristics associated with acceptance for presentation at the meeting and subsequent publication as a full manuscript.
                A total of 492 research abstracts were submitted from programs in emergency medicine and other specialties affiliated with 103 US medical schools. A total of 179 (36%) were accepted for presentation and 214 (43%) were published in 44 journals. Of the 179 abstracts accepted for presentation, 111 studies were published. Scientific quality of abstracts or prestige of the journal in which the study was eventually published did not predict either of these outcomes. The best predictors (by logistic regression) of meeting acceptance were a subjective "originality" factor (odds ratio [OR], 2.07; 95% confidence interval [CI], 1.13-3.89) and positive results (OR, 1.99; 95% CI, 1.07-3.84), and, for publication, meeting acceptance (OR, 2.49; 95% CI, 1.49-4.35) and large sample size (OR, 2.26; 95% CI, 1.23-4.31). Forty-nine percent (241) of abstracts did not report on blinding, and 24% (118) did not report on randomization. Acceptance and publication were both more likely for positive outcomes (P=.03). Funnel plots showed the classic distribution of positive-outcome ("publication") bias at each of the submission, acceptance, and publication phases. Meeting acceptance predicted publication with a sensitivity of only 51%, specificity of 71%, positive predictive value of 57%, and negative predictive value of 66%.
                Positive-outcome bias was evident when studies were submitted for consideration and was amplified in the selection of abstracts for both presentation and publication, neither of which was strongly related to study design or quality. Incidence rates for many types of cancer are higher among African American men than in the general population, yet African American men are less likely to participate in cancer screening trials. This paper describes the outcomes of a randomized trial (the AAMEN Project) designed to recruit African American men aged 55-74 years to a prostate, lung and colorectal cancer screening trial.
                The recruitment interventions address four types of barriers to clinical trial participation: sociocultural barriers, economic barriers, individual barriers and barriers inherent in study design. Subjects were randomized to a control group or one of three increasingly intensive intervention arms, which used different combinations of mail, phone and in person church-based recruitment.
                Of the 39,432 African American men residing in the geographically defined study population (southeastern Michigan and northern Ohio), 17,770 men (45%) could be contacted, and 12,400 (31% of 39,432) were found to be eligible to participate. No statistically significant differences in age, education or income level were found among participants in the four study arms. A significantly greater enrollment yield (3.9%) was seen in the most intensive, church-based intervention arm, compared to the enrollment yields in the other two intervention arms (2.5 and 2.8%) or the control group (2.9%) (P < 0.01).
                The intervention that involved the highest rate of face-to-face contact with the study participants produced the highest enrollment yield, but several strategies that were thought could improve yield had no effect. These findings, which are consistent with current literature on population-based recruitment, should facilitate the development of future recruitment efforts involving older African American men. Postal questionnaires are widely used to collect outcome data on participants. However, a poor response to questionnaires will reduce the statistical power of the study and may introduce bias. A meta analysis of ten trials offering study results, largely in the fields of education and marketing, was shown to be ineffective, with the odds ratio for response with offering research findings is 0.92 (95% CI 0.75 to 1.11). However uncertainty still exists as it is uncertain whether results from such trials can be extrapolated to that of a health care setting. The aim of this study was to assess whether offering participants study results increases the response rates to postal questionnaires.
                1038 women aged over 70 years were remotely randomised by computer in a 3:1 ratio. 250 participants did not receive the offer of knowing the results of the trial and 788 participants were offered the results of the trial in a postal questionnaire. The main outcome measure was response rate. Chi square test was used to evaluate the overall differences in response rate between the two groups. An adjusted analysis, adjusting for whether the participant was taking calcium and age was also undertaken.
                The response rates were not significantly different Odds Ratio 0.88 (95% confidence intervals 0.48 to 1.63) p = 0.69.
                Offering study results to women living in the community aged over 70 does not increase response rates to postal questionnaires. Although researchers have an ethical obligation to offer participants study results, since 10% of women did not wish to receive the results, investigators should give participants the option to opt out of receiving the study's results. The Multiphasic Health Checkup Evaluation Study, a long-term clinical trial, has been completed. A study group of 5156 men and women age 35-54 at entry was urged to have annual multiphasic health checkups (MHCs) for 16 years. A control group of 5557 comparable subjects was not so urged but was followed up in a comparable fashion. The mean and median number of MHCs per person were 6.8 and 6, respectively, in the study group and 2.8 and 1, respectively, in the control group. During 16 years the study group experienced a 30% reduction (p less than 0.05) in deaths from pre-specified "potentially postponable" causes, largely associated with lower death rates from colorectal cancer and hypertension. This reduction was most pronounced in the early years of the study. The two groups did not differ to a statistically significant degree in mortality from all other causes (84% of total mortality) or in total mortality. There was no difference in self-reported disability in the overall groups. In the setting of our prepaid health care plan where MHCs were already available on a voluntary basis, a program of urging middle-aged persons to undergo regular MHCs brought about a substantial reduction in mortality from preselected diseases. To explore whether reported methodologic quality affects estimated intervention effects in randomized trials and contributes to discrepancies between the results of large randomized trials and small randomized trials in meta-analyses.
                Meta-analyses of randomized trials that included at least one large trial (>/=1000 participants) were included, regardless of the therapeutic area. Eligible meta-analyses were identified through electronic searches and bibliographies of relevant articles.
                Full-length randomized trials.
                Methodologic quality was assessed according to reported randomization, double blinding, and follow-up as separate components and by using the Jadad composite scale.
                Fourteen meta-analyses involving 190 randomized trials from eight therapeutic areas were included. Compared with large trials, intervention effects were exaggerated in small trials with inadequate allocation sequence generation (ratio of odds ratios, 0.46 [95% CI, 0.25 to 0.83]; P = 0.011), inadequate allocation concealment (ratio of odds ratios, 0.49 [CI, 0.27 to 0.86]; P = 0.014), and no double blinding (ratio of odds ratios, 0.52 [CI, 0.28 to 0.96]; P = 0.01). Large trials did not differ significantly from small trials with adequate generation of the allocation sequence, adequate allocation concealment, or adequate double blinding. No association was seen between reported follow-up and intervention effects. The Jadad scale provided no additional information because the scale and the quality components overlapped substantially.
                Inadequate generation of the allocation sequence, allocation concealment, and double blinding lead to exaggerated estimates of intervention benefit and may contribute to discrepancies between the results of large randomized trials and small randomized trials in meta-analyses.
It is not possible to draw any conclusions from this review. The number of people included in the studies is less than 50 and the reported results are inadequate.