Readers' evaluation of effect of peer review and editing on quality of articles in the Nederlands Tijdschrift voor Geneeskunde. Effects of peer review and editing on the readability of articles published in Annals of Internal Medicine. Manuscript quality before and after peer review and editing at Annals of Internal Medicine. A comparison of authors publishing in two groups of U.S. medical journals. Accuracy of references in five biomedical informatics journals. [Effect of statistical review on manuscript quality in Medicina Clínica (Barcelona): a randomized study]. Effect of including a clinical example on the ability of physical therapists to apply information in a technical research report. Comparison of review articles published in peer-reviewed and throwaway journals. Effects of training on quality of peer review: randomised controlled trial.
Academic biomedical journals use peer review and editing to help to select and improve the quality of articles. We have investigated whether articles accepted by the Nederlands Tijdschrift voor Geneeskunde, the Dutch Journal of Medicine, were improved after peer review and editing (post-acceptance scientific and copy editing).
                400 readers of the journal (100 each of medical students, recent medical graduates, general practitioners, and specialists) were invited to participate in a questionnaire survey. The first 25 from each group who agreed to participate were included. We posted a pack containing a set of identically appearing typescripts (ie, blinding) of the submitted, accepted, and published versions of 50 articles that had been published in Ned Tijdschr Geneeskd. Each evaluator received two of the sets of versions, and each set was evaluated by one person from each group. The package also included two questionnaires: the first was used to compare the submitted with the accepted version (25 questions), the second compared the accepted with the published version (17 questions). The questions were answered on five-point scales, and were about the quality of the articles or were general/overall scores. We analysed the data as scores of 3-5 (ie, improvement) versus 1-2.
                After peer review, the quality in 14 of 23 questions (61%) was significantly improved (p = 0.03 or smaller). In particular, the overall score and general medical value were significantly improved (p = 0.00001 for each). Editing led to significant improvement in 11 of 16 questions (69%, p = 0.017 or smaller), and especially in style and readability (p = 0.001 and p = 0.004). Generally, we found no differences between the scores of the four categories of evaluators. 72% of the evaluators correctly identified which version was which.
                Evaluations by readers of the Ned Tijdschr Geneeskd indicated significant improvement of published articles after both peer review and editing. We think that peer review and editing are worthwhile tasks. We also think that possible biases would have had a negligible effect on our results (including the fact that we selected the first 25 evaluators who responded, that some evaluators may have read the published version, and that one questionnaire may have looked more scientific than the other, more editorial one). To measure the effect of the peer review and editorial processes on the readability of original articles.
                Comparison of manuscripts before and after the peer review and editorial processes.
                Annals of Internal Medicine between March 1 and November 30, 1992. MANUSCRIPTS: One hundred one consecutive manuscripts reporting original research.
                Assessment of readability by means of two previously validated indexes: the Gunning fog index (units of readability in the fog index roughly correlate to years of education) and the Flesch reading ease score. Each manuscript was analyzed for readability and length on receipt and after it had passed through the peer review and editorial processes. Text and abstracts were analyzed similarly but separately. Mean readability scores were compared by two-tailed t tests for paired observations.
                Mean (+/- SD) initial readability scores of manuscripts and abstracts by the Gunning fog index were 17.16 +/- 1.55 and 16.65 +/- 2.80, respectively. At publication, scores were 16.85 +/- 1.42 and 15.64 +/- 2.42 (P = .0005 and P < .0001 for before-after differences, respectively). By comparison, studies of other print media showed scores of about 11 for the New York Times editorial page and about 18 for a typical legal contract. Similar changes were found for the Flesch scores. The median length of the manuscripts increased by 2.6% and that of the abstracts by 4.2% during the processes.
                The peer review and editorial processes slightly improved the readability of original articles and their abstracts, but both remained difficult to read at publication. Better readability scores may improve readership. To evaluate the effects of peer review and editing on manuscript quality.
                Editorial offices of Annals of Internal Medicine.
                Masked before-after study. MANUSCRIPTS: 111 consecutive original research manuscripts accepted for publication at Annals between March 1992 and March 1993.
                We used a manuscript quality assessment tool of 34 items to evaluate the quality of the research report, not the quality of the research itself. Each item was scored on a 1 to 5 scale. Forty-four expert assessors unaware of the design or aims of the study evaluated the manuscripts, with different persons evaluating the two versions of each manuscript (before and after the editorial process).
                33 of the 34 items changed in the direction of improvement, with the largest improvements seen in the discussion of study limitations, generalizations, use of confidence intervals, and the tone of conclusions. Overall, the percentage of items scored three or more increased by an absolute 7.3% (95% CI, 3.3% to 11.3%) from a baseline of 75%. The average item score improved by 0.23 points (CI, 0.07 to 0.39) from a baseline mean of 3.5. Manuscripts rated in the bottom 50% showed two- to threefold larger improvements than those in the top 50%, after correction for regression to the mean.
                Peer review and editing improve the quality of medical research reporting, particularly in those areas that readers rely on most heavily to decide on the importance and generalizability of the findings. This study compared the editorial peer review experiences of authors who published in two groups of indexed U.S. medical journals. The study tested the hypothesis that after one journal rejects a manuscript an author selects a less well-known journal for submission. Group One journals were defined as those indexed in 1992 MEDLINE that satisfied several additional qualitative measures; Group Two journals were indexed in the 1992 MEDLINE only. Surveys were sent to the first authors of 616 randomly selected articles, and 479 surveys were returned, for a response rate of 78.1%. A total of 20.8% of Group One and 15.7% of Group Two articles previously had been rejected. Group One authors were more likely to select a journal for its prestige and article quality, while Group Two authors were more likely to have been invited to submit the manuscript. More than 60% of both groups felt the peer review had offered constructive suggestions, but that it had changed article conclusions less than 3% of the time. Both groups thought the review process only marginally improved content, organization, or statistical analysis, or clarified conclusions. Between 3% and 15% of all authors received considerable conflicting advice from different reviewers. Authors from both groups differed as to their reasons for journal selection, their connections with the publishing journal, and patterns of resubmission after rejection. To determine the rate and type of errors in biomedical informatics journal article references.
                References in articles from the first 2004 issues of five biomedical informatics journals, Journal of the American Medical Informatics Association, Journal of Biomedical Informatics, International Journal of Medical Informatics, Methods of Information in Medicine, and Artificial Intelligence in Medicine were compared with MEDLINE for journal, authors, title, year, volume, and page number accuracy. If discrepancies were identified, the reference was compared with the original publication. Two reviewers independently evaluated each reference.
                The five journal issues contained 37 articles. Among the 656 eligible references, 225 (34.3%) included at least one error. Among the 225 references, 311 errors were identified. One or more errors were found in the bibliography of 31 (84%) of the 37 articles. The reference error rates by journal ranged from 22.1% to 40.7%. Most errors (39.0%) occurred in the author element, followed by the journal (31.2%), title (17.7%), page (7.4%), year (3.5%), and volume (1.3%) information.
                The study identified a considerable error rate in the references of five biomedical informatics journals. Authors are responsible for the accuracy of references and should more carefully check them, possibly using informatics-based assistance. The statistical review of biomedical articles should result in an improved quality. The objective of this study was to compare the effects of clinical review and joint clinical and statistical review on manuscript quality, in articles submitted to Medicina Clínica (Barcelona), a Spanish weekly journal of internal medicine.
                Original papers arriving between May 2000 and February 2001 were randomized either to a clinical review group or a clinical and statistical review group. Two evaluators, blinded to the paper's group, assessed the quality improvement in both groups, from submission to publication using a modified version ot the Goodman et al. scale. The protocol required that final versions arrived before the end of May 2001.
                Final sample size was 43 manuscripts, evaluated before and after peer review. On the intention to treat analysis, the estimated effect of statistical review was 1.35 (95% CI: -0.45 to 3.16) positive, but not statistically significant. The analysis of the reviewers' comments revealed some protocol deviations. Taking into account the spontaneous inclusion of statistical experts in the clinical group, the estimated effect was statistically significant, with a confidence interval of 0.3 to 3.7.
                The inclusion of a statistical expert in the peer review process improves manuscript quality, although in the intention to treat analysis the improvement was not statistically significant. The purpose of this study was to determine whether an example in a technical research report would affect application of content to hypothetical patient problems and to future patients.
                Subjects were 69 physical therapists who routinely used isokinetic equipment in the treatment of patients with knee joint pathologies.
                Thirty-five subjects (group 1) read a research report that described mathematical models for predicting preinjury quadriceps femoris and hamstring muscle performance. The report included an example of applying the information. The remaining 34 subjects (group 2) read the same research report with the example omitted. Subjects were asked to select appropriate prediction models and determine preinjury knee torques for two hypothetical patients. Subjects were also contacted 6 to 12 weeks after initial data collection to determine whether they had applied research report results in their treatments of patients.
                Chi-square analyses indicated manuscript type was not related to the selection of correct models, but group 1 subjects computed correct torque values more frequently than did group 2 subjects. Insufficient data were available regarding application of research report content to patients.
                The results indicate application of technically oriented research reports may be enhanced by including examples of applications. (Gross MT, Sekerak DK, Allen DD. Effect of including a clinical example on the ability of physical therapists to apply information in a technical research report. To compare the quality, presentation, readability, and clinical relevance of review articles published in peer-reviewed and "throwaway" journals.
                We reviewed articles that focused on the diagnosis or treatment of a medical condition published between January 1 and December 31, 1998, in the 5 leading peer-reviewed general medical journals and high-circulation throwaway journals. Reviewers independently assessed the methodologic and reporting quality, and evaluated each article's presentation and readability. Clinical relevance was evaluated independently by 6 physicians.
                Of the 394 articles in our sample, 16 (4.1%) were peer-reviewed systematic reviews, 135 (34.3%) were peer-reviewed nonsystematic reviews, and 243 (61.7%) were nonsystematic reviews published in throwaway journals. The mean (SD) quality scores were highest for peer-reviewed articles (0.94 [0.09] for systematic reviews and 0.30 [0.19] for nonsystematic reviews) compared with throwaway journal articles (0.23 [0.03], F(2,391) = 280.8, P<.001). Throwaway journal articles used more tables (P =.02), figures (P =.01), photographs (P<.001), color (P<.001), and larger font sizes (P<.001) compared with peer-reviewed articles. Readability scores were more often in the college or higher range for peer-reviewed journals compared with the throwaway journal articles (104 [77.0%] vs 156 [64.2%]; P =.01). Peer-reviewed article titles were judged less relevant to clinical practice than throwaway journal article titles (P<.001).
                Although lower in methodologic and reporting quality, review articles published in throwaway journals have characteristics that appeal to physician readers. To determine the effects of training on the quality of peer review.
                Single blind randomised controlled trial with two intervention groups receiving different types of training plus a control group.
                Reviewers at a general medical journal. Interventions Attendance at a training workshop or reception of a self taught training package focusing on what editors want from reviewers and how to critically appraise randomised controlled trials.
                Quality of reviews of three manuscripts sent to reviewers at four to six monthly intervals, evaluated using the validated review quality instrument; number of deliberate major errors identified; time taken to review the manuscripts; proportion recommending rejection of the manuscripts.
                Reviewers in the self taught group scored higher in review quality after training than did the control group (score 2.85 v 2.56; difference 0.29, 95% confidence interval 0.14 to 0.44; P = 0.001), but the difference was not of editorial significance and was not maintained in the long term. Both intervention groups identified significantly more major errors after training than did the control group (3.14 and 2.96 v 2.13; P < 0.001), and this remained significant after the reviewers' performance at baseline assessment was taken into account. The evidence for benefit of training was no longer apparent on further testing six months after the interventions. Training had no impact on the time taken to review the papers but was associated with an increased likelihood of recommending rejection (92% and 84% v 76%; P = 0.002).
                Short training packages have only a slight impact on the quality of peer review. The value of longer interventions needs to be assessed.
Surprisingly few studies have evaluated the effects of technical editing rigorously. However there is some evidence that the 'package' of technical editing used by biomedical journals does improve papers. A substantial number of references in biomedical articles are cited or quoted inaccurately.