A comparison of handsearching versus MEDLINE searching to identify reports of randomized controlled trials. Full publication of trials initially reported as abstracts in the Australian and New Zealand Journal of Medicine 1980-2000. Identification of randomized controlled trials from the emergency medicine literature: comparison of hand searching versus MEDLINE searching. Abstracts of trials presented at the Vth World Congress of Psychiatry (Mexico, 1971): a cohort study. Handsearching the Journal of the Royal Naval Medical Service for Trials. Handsearching the Journal of the Royal Army Medical Corps for trials. Prevalence study of the randomized controlled trials in the Journal of Intellectual Disability Research: 1957-1994. Quality assessment of reports on clinical trials in the Journal of Hepatology. Publication bias and meta-analyses: a practical example.
This study aims to compare handsearching to a basic MEDLINE search for the identification of reports of randomized trials in specialized health care journals. Twenty-two specialized health care journals, published in the U.K., were handsearched for all reports of controlled trials (as defined by the Cochrane Collaboration). The reports of trials, which were judged to be definitely randomized, were identified from a random sample of three years per journal and form one element of this study. A MEDLINE search using the publication type terms 'randomized controlled trial' and 'controlled clinical trial' was also performed for the same journal years. The reports of trials retrieved by handsearching were then compared against those retrieved from the MEDLINE search, to identify differences in retrieval between the two techniques. Reports of randomized trials identified by the MEDLINE search but not found by handsearching were individually assessed to see if they met the Cochrane eligibility criteria for a report of a randomized trial. A total of 714 reports of randomized trials were found by using a combination of both handsearching and MEDLINE searching. Of these, 369 (52 per cent) were identified only by handsearching and 32 (4 per cent) were identified only by MEDLINE searching. Of the reports identified only by handsearching, 252 had no MEDLINE record, of which 232 (92 per cent) were meeting abstracts or published in supplements; 117 (25 per cent) of the 462 reports of randomized trials which had a MEDLINE record were missed by the electronic search because they did not have either of the publication type terms 'randomized controlled trial' or 'controlled clinical trial'. This proportion varied depending on when the reports of randomized trials were published (that is, before or after the introduction of the MEDLINE publication type terms above). The highest additional yield from handsearching compared to MEDLINE searching was for reports of randomized trials published prior to 1991 and from handsearching the non-MEDLINE indexed parts of a journal. The results of this study suggest that a combination of MEDLINE and handsearching is required to identify adequately reports of randomized trials.
                Copyright Crown copyright 2002. Reproduced with the permission of Her Majesty's Stationery Office. Published by John Wiley & Sons, Ltd. Abstract Failure to identify all relevant reports of controlled trials is a potential source of bias in systematic reviews of health-care interventions. The present study aims to identify how many reports of trials -initially published as conference abstracts in the Australian and New Zealand Journal of Medicine were subsequently published in full. We identified trial reports by handsearching conference abstracts published in the Journal from 1980 to 2000. We then searched the Cochrane Controlled Trials Register and PubMed to determine how many of these had been subsequently published in full. A total of 962 reports of controlled trials was identified from the conference proceedings of 17 medical societies. Of these, 589 (61%) reports of trials were subsequently published in full, and on average within 1-2 years. Handsearching conference abstracts identified a large number of reports of controlled trials, over one-third of which were unpublished and therefore not easily accessible. As part of an ongoing project to identify all the randomized controlled trials (RCTs) in the emergency medicine literature, in association with the Cochrane Collaboration, 2 discrete studies were undertaken; the first, to compare motives for active participation in hand searching of the literature by emergency medicine professionals, and the second, to compare hand searching with MEDLINE searching of a number of emergency medicine journals.
                All listed members of the British Association for Emergency Medicine (BAEM) and the Society for Academic Emergency Medicine (SAEM) received a standard letter outlining the objectives of the project, with 1 of 3 headings assigned on an alternate basis. Recruited volunteers hand searched journals prioritized from the emergency medicine literature. Each issue of each journal was hand searched for RCTs. In addition, a comprehensive MEDLINE search was conducted for each journal. The yields of RCTs from the 2 searching methods were compared for all journals and for each journal individually.
                No clear motivation for participation in this work could be ascertained because of the low response rates from BAEM and SAEM (10.1% and 1.8%, respectively). Only 18 (29.0%) of the 62 journals identified were indexed by MEDLINE. In the 14 journals indexed by MEDLINE for which hand searching was completed, a total of 710 RCTs were identified by a combination of the 2 approaches; of these, 592 (83.4%) were identified by hand searching alone and 483 (68.0%) by MEDLINE searching alone. Both methods identified 365 (51. 4%) RCTs; hand searching revealed an additional 227 (32.0%) that were not identified by MEDLINE searching, and MEDLINE searching found 118 (16.6%) that were not identified by hand searching. The difference between the proportions identified by hand searching and by MEDLINE searching (15.4%; 95% confidence interval [CI], 12.7% to 17.9%) was statistically significant (McNemar's chi2 test, 1 df, 33. 8; P <.0001). This difference was not significant for 8 of the journals.
                The response rates from mailing to members of the relevant professional organizations letters requesting participation in this work were very low and suggested that such an approach was not cost-effective. However, no formal costing exercise was undertaken. Searching results showed that, in the 14 emergency medicine journals indexed by MEDLINE for which hand searching was completed, hand searching led to identification of additional RCTs (primary articles) not found through MEDLINE searching. However, hand searching, although statistically significantly better than MEDLINE searching, failed to identify some of the RCTs found by MEDLINE searching, suggesting that hand searching is not a "gold standard" method and that the dual approach, promoted by the Cochrane Collaboration, may be the optimal approach for journals indexed by MEDLINE. Systematic reviews should identify all relevant trials in order to minimize the potential for bias and the play of chance in their results. Other specialities have shown that conference proceedings are a rich source of trials, but many of these trials are never fully published.
                All clinical trials presented at a single conference (Vth World Congress of Psychiatry, Mexico, 1971) were identified by hand searching. Full publications of these abstracts were then sought on five databases by searching for the authors or relevant key words.
                Full publications were found for 46% of the abstracts. The odds of publication decreased for abstracts that were from a non-Anglophone country or that failed to mention randomization.
                Anyone wishing to undertake a systematic review of a mental-health care topic should search relevant conference proceedings for trials. As part of the Cochrane Collaboration's international research endeavour, the authors carried out a handsearch of the Journal of the Royal Naval Medical Service from 1948 to 1998, searching for randomised controlled trials (RCTs) and controlled clinical trials (CCTs). Five trials were identified, of which three were RCTs and two were CCTs. The first trial was published in 1960. The identified trials were in the fields of dentistry (two trials), gastroenterology, occupational medicine and orthopaedic surgery. Of the five trials, only two had been located previously through a rigorous interrogation of Medline. The three newly identified trials were reported to the UK Cochrane Centre, and details of these three trials were entered into Medline for use by clinicians and investigators in the future. As part of the Cochrane Collaboration's international research endeavour, the authors carried out a handsearch of the RAMC Journal from 1948 to 1998, searching for randomised controlled trials (RCTs) and controlled clinical trials (CCTs). Ten trials were identified, of which 6 were RCTs and 4 were CCTs. The first trial was published in 1967. Four of the 10 identified trials were in the field of respiratory medicine, and 2 were in obstetrics and gynaecology. Of the 10 trials, only 3 had been found through a rigorous interrogation of Medline. The 7 newly identified trials were reported to the UK Cochrane Centre, and summary details of these trials were entered into Medline for use by clinicians and investigators in the future. Systematic reviews of care are increasingly potent guides to clinical practice, and it is important that all relevant randomized controlled trials (RCTs) are identified by those within the speciality of learning disability who produce such works. All RCTs in the Journal of Intellectual Disability Research and its predecessor were identified by hand-searching (1957-1994), and the frequency, origin, intervention and quality of reporting of randomization were described. Electronic searches for the trials were undertaken for the years 1974-1994, and the quality of indexing was inspected and tested. These electronic searches were then compared to a 'gold standard' search. Fifty-six RCTs were identified. None contained the world 'randomized' in the title and only nine mentioned it in the abstract. Out of the 37 RCTs published between 1974 and 1994, 36 are in PsycLIT and 37 in MEDLINE. One MEDLINE record contained the wrong abstract. The methodological phrases used in the electronic records were poor, and thus, the precision of electronic searches, using both databases, was low. This international journal contains many relevant RCTs from around the world, involving several types of interventions. Unfortunately, these trials cannot be readily accessed electronically using methodological phrases designed to find RCTs. Improved quality of indexing would facilitate identification of RCTs and their dissemination. Electronic searches on databases for randomised clinical trials and controlled clinical trials do not identify as many trials as handsearches, and trial reporting may be flawed. The aims were to identify all fully reported randomised clinical trials in the Journal of Hepatology and to make a qualitative assessment of the reporting.
                The publications were identified by systematically handsearching the full text of the journal and searching MEDLINE. Central dimensions of trial quality were used to assess the reporting quality of the trials.
                Randomised clinical trials represented 8.4% of the original articles (171/2028). Ten original articles (0.5%) could not be classified. A search on MEDLINE identified 81.3% of the randomised clinical trials, i.e., 139 out of the 171 identified by the handsearch. A total of 166 randomised clinical trials could be quality assessed. Forty-seven (28.3%) of them reported adequate generation of allocation sequence; 22 (13.3%) adequate allocation concealment; 95 (57.2%) allowed intention-to-treat analysis with only a few losses to follow-up; 50 (30.1%) were double-blind; 33 (19.9%) reported sample-size calculations; 13 trials (7.8%) employed the crossover design; and the median number of subjects per intervention arm in parallel group trials was 19 subjects (interquartile range: 11-31; range: 5-519). The quality of reporting was significantly better in regular issue articles than in supplement articles.
                Many important randomised clinical trials are published in the Journal of Hepatology, but there seems to be ample room for improvement of quality of reporting. Publication bias is widely appreciated, but considerable time and effort are needed to locate and obtain data from unpublished randomized controlled trials (RCTs), those published in non-English language journals or those reported in the gray literature; for this publication, we will call this collection of trials the "gray+literature." However, excluding such trials from systematic reviews could introduce bias and give rise to misleading conclusions.
                We aimed to explore and quantify the impact of inclusion of gray+ literature on the results of all completed individual patient data (IPD) reviews coordinated by our group (13 meta-analyses). For each IPD review, results were calculated for RCTs fully published in English language journals and RCTs fully published in English language journals and the gray+literature.
                The IPD meta-analyses based only on RCTs that were fully published in English language journals tended to give more favorable results than those that included RCTs from the gray+literature. Although in most cases the addition of gray+data gave less encouraging results, moving the estimated treatment effect toward a null result, the direction of effect was not always predictable.
                We recommend that all systematic reviews should at least attempt to identify trials reported in the gray+literature and, where possible, obtain data from them.
Handsearching still has a valuable role to play in identifying reports of randomized trials for inclusion in systematic reviews of health care interventions, particularly in identifying trials reported as abstracts, letters and those published in languages other than English, together with all reports published in journals not indexed in electronic databases. However, where time and resources are limited, searching an electronic database using a complex search (or the Cochrane HSSS) will identify the majority of trials published as full reports in English language journals, provided, of course, that the relevant journals have been indexed in the database.