Scientific impact of studies published in temporarily available radiation oncology journals: a citation analysis

The purpose of this study was to review all articles published in two temporarily available radiation oncology journals (Radiation Oncology Investigations, Journal of Radiosurgery) in order to evaluate their scientific impact. From several potential measures of impact and relevance of research, we selected article citation rate because landmark or practice-changing research is likely to be cited frequently. The citation database Scopus was used to analyse number of citations. During the time period 1996-1999 the journal Radiation Oncology Investigations published 205 articles, which achieved a median number of 6 citations (range 0-116). However, the most frequently cited article in the first 4 volumes achieved only 23 citations. The Journal of Radiosurgery published only 31 articles, all in the year 1999, which achieved a median number of 1 citation (range 0-11). No prospective randomized studies or phase I-II collaborative group trials were published in these journals. Apparently, the Journal of Radiosurgery acquired relatively few manuscripts that were interesting and important enough to impact clinical practice. Radiation Oncology Investigations’ citation pattern was better and closer related to that reported in several previous studies focusing on the field of radiation oncology. The vast majority of articles published in temporarily available radiation oncology journals had limited clinical impact and achieved few citations. Highly influential research was unlikely to be submitted during the initial phase of establishing new radiation oncology journals.


Background
Scientifc publishing is an important task for radiation oncologists pursuing an academic career (Holliday et al. 2014). Many of these physicians or physician-scientists undergo yearly rating, and are more or less forced to produce a certain number of publications or surrogate achievements such as impact factor (Holliday et al. 2013). Collaborative groups competing for research funding also feel the pressure of succeeding with clinical trials and publishing their results in prestigious journals (Nieder 2012). Many well-established journals with high impact factor have low acceptance rates of submitted manuscripts. Continuous increases in number of submissions and competition for space result in a need for additional publication channels. Publishing companies launching new journals face the challenges of attracting a sufficient number of scientifically sound manuscripts, achieving high visibility in search engines, and achieving indexing in databases such as PubMed. A rapidly increasing impact factor is also helpful for the continued success of new journals. During the second half of the 1990s, when the field was dominated by the "International Journal of Radiation Oncology, Biology and Physics" and "Radiotherapy and Oncology", two attempts were made to establish additional specialty journals, "Radiation Oncology Investigations" and "Journal of Radiosurgery". However, these traditional print journals without page charges did not succeed and were taken off the market after few volumes. We hypothesized that failure was caused by insufficient number of publications that were interesting and important enough to attract readers and impact clinical practice, and that this lack of appeal would be reflected by low number of citations for most published studies. In order to evaluate this hypothesis, patterns of citation for these two journals were analysed.

Materials and methods
A systematic search of the citation database Scopus (Elsevier B.V., http://www.elsevier.com/online-tools/scopus) by use of the function 'document search' was performed on 3 rd of July 2014. All articles published in "Radiation Oncology Investigations" and the "Journal of Radiosurgery" were selected irrespective of subject area, document and article type (review, clinical study, experimental study, case report etc.). Then, these articles were ranked by number of citations (field 'times cited' in the Scopus citation database). As requested during manuscript review, we also performed comparisons to the "International Journal of Radiation Oncology, Biology and Physics". The respective Scopus search was performed on 6 th of February 2015.
The "Journal of Radiosurgery" published 31 articles, all in the year 1999, which achieved a median number of 1 citation (range 0-11, Figure 1). References (Garell et al. 1999;Maire et al. 1999;Gibon et al. 1999;Solberg et al. 1999;Sanghavi et al. 1999) include the 5 most frequently cited articles (all had at least 5 citations and were published in volume 2). Both clinical and physics research as well as reviews achieved any citations. No randomized studies or phase I-II collaborative group trials were published.
The "International Journal of Radiation Oncology, Biology and Physics" published almost 2200 articles during the time period 1996-1999. These achieved a median number of 22 citations (range 0-1135). Overall, 9% of articles achieved at least 100 citations (p = 0.0002 in Chi-Square test comparing the three journals).

Discussion
The objective of this review was to identify pattern of scientific publication in two radiation oncology journals which were only temporarily available. While Radiation Oncology Investigations covered the whole field of radiation oncology, the Journal of Radiosurgery had a much narrower focus. After arbitrary decisions about which database to search (only those providing citation numbers could be considered for the purpose of this review), we performed a comprehensive evaluation of published research and number of citations accumulated during more than a decade after publication of the final journal volume. These citation numbers can be considered mature because previous analyses demonstrated that citation rate is gradually increasing for the first years after publication, followed by rapid decline (except for landmark randomized studies, which change clinical practice) (Stringer et al. 2010;Kondziolka 2011). Citation rate of published articles was chosen to define the most important contributions. Articles with high numbers of citations are likely those that impressed other clinicians/scientists and had impact on clinical practice or future developments in the field (Shao et al. 2013). It should be noticed that searches in different databases will result in more or less variable citation counts and that the present results therefore provide only a snapshot. This has been illustrated, e.g., in a cohort study of 328 articles published in JAMA, Lancet, or the New England Journal of Medicine between October 1, 1999, and March 31, 2000 (Kulkarni et al. 2009). Total citation counts for each article up to June 2008 were retrieved from Web of Science, Scopus, and Google Scholar. Google Scholar and Scopus retrieved more citations per article with a median of 160 and 149, respectively, than Web of Science (median 122, p < 0.001 for both comparisons). Importantly, Web of Science, Scopus and Google Scholar produced quantitatively and qualitatively different citation counts.
In a previous study of radiosurgery for various conditions, 1.5% of all articles (time period 1951-2010) achieved more than 100 citations (Kondziolka 2011). This figure corresponds well to that of articles in Radiation Oncology Investigations (3 out of 205, 1.5%), while the "International Journal of Radiation Oncology, Biology and Physics" achieved significantly higher proportions during the same time interval (1996-1999; 9%). The Top 100 radiation oncology articles from the time period 1999-2001 achieved a median of 208 citations (range 121-1149) (Nieder et al. 2013a), i.e. more than any of the publications reviewed here. When comparing articles published during different time periods, sources of bias must be acknowledged (variable temporal patterns of citation, increasing use of online databases in recent years, better access to published articles).
An analysis restricted to German radiation oncology publications revealed that most citations per year since publication were recorded for meta-analyses and randomized phase III trials (Nieder 2012). Lower figures were recorded for review articles, non-phase III prospective clinical trials, and retrospective clinical studies. Another analysis demonstrated that pattern of publication of the most influential radiation oncology studies was dominated by only two scientific journals: the Journal of Clinical Oncology and the International Journal of Radiation Oncology Biology and Physics (Nieder et al. 2013a). Several newly launched journals (first issue after 1999) managed to attract highly cited articles (Lancet Oncology, Nature Reviews Clinical Oncology or Nature Reviews Cancer, which had rapidly increasing impact factors). Apparently, despite controversy around impact factors and optimal evaluation of research productivity and quality (Kanaan et al. 2011;Durieux & Gevenois 2010), researchers find it attractive and desirable to publish their most important radiation oncology related work in the top journals of the field. The present findings are in line with these considerations.
Highly cited work is unlikely to be published in the first volumes of new journals. Prospective randomized trials and non-randomized trials performed by influential collaborative groups are prefentially submitted to well-established journals. It has also been shown that five newly established oncology journals, in this case open access publications without print issues, all published less than 50 articles in their first annual volume (Nieder et al. 2013b). First after 4-5 years number of articles per volume increased sharply. At that time, the two journals analysed here were already taken off the market. During the time period in question (1996)(1997)(1998)(1999), no other new radiation oncology journals entered the market. However, afterwards two successful journals were established (in 2006 "Radiation Oncology" (open access, no print issues); in 2011 "Practical Radiation Oncology" (print issues, sister journal to the "International Journal of Radiation Oncology, Biology and Physics")). Regarding "Radiation Oncology", the annual volumes 2006-2008 all consisted of less than 50 articles. In 2009, contents expanded to 71 articles, while the most recent volume (2014) featured approximately 300. On the one hand, one could hypothesize that open access fees might prevent authors from submitting to journals like "Radiation Oncology". On the other hand, advantages such as rapid publication and unlimited distribution and access appear to outweigh financial considerations. Institutional membership or publication funds and waiver of fees for authors from countries with limited resources might also encourage researchers to publish in open access journals.

Conclusions
Highly cited research typically appears in relatively few well-established journals. The vast majority of articles published in temporarily available radiation oncology journals had limited clinical impact and achieved few citations, especially those accepted in the initial phase of marketing.