Outcomes research resources in India: current status, need and way forward

Background Despite their importance, the number of outcomes research studies conducted in India are lesser than other countries. Information about the distribution of existing outcomes research resources and relevant expertise can benefit researchers and research groups interested in conducting outcomes research studies and policy makers interested in funding outcomes research studies in India. We have reviewed the literature to identify and map resources described in outcomes research studies conducted in India. Methods We reviewed the following online biomedical databases: Pubmed, SCIRUS, CINAHL, and Google scholar and selected articles that met the following criteria: published in English, conducted on Indian population, providing information about outcomes research resources (databases/registries/electronic medical records/electronic healthcare records/hospital information systems) in India and articles describing outcomes research studies or epidemiological studies based on outcomes research resources. After shortlisting articles, we abstracted data into three datasets viz. 1. Resource dataset, 2. Bibliometric dataset and 3. Researcher dataset and carried out descriptive analysis. Results Of the 126 articles retrieved, 119 articles were selected for inclusion in the study. The tally increased to 133 articles after a secondary search. Based on the information available in the articles, we identified a total of 91 unique research resources. We observed that most of the resources were Registries (62/91) and Databases ( 23/91) and were primarily located in Maharashtra (19/91) followed by Tamil Nadu (11/91), Chandigarh (8/91) and Kerala (7/91) States. These resources primarily collected data on Cancer (44/91), Stroke (5/91) and Diabetes (4/91). Most of these resources were Institutional (38/91) and Regional resources (35/91) located in Government owned and managed Academic Institutes/Hospitals (57/91) or Privately owned and managed non – Academic Institutes/Hospitals (14/91). Data from the Population based Cancer Registry, Mumbai was used in 41 peer reviewed publications followed by Population based Cancer Registry, Chennai (17) and Rural Cancer Registry Barshi (14). Most of the articles were published in International journals (139/193) that had an impact factor of 0–1.99 (43/91) and received an average of 0–20 citations (55/91). We identified 193 researchers who are mainly located in Maharashtra (37/193) and Tamil Nadu (24/193) states and Southern (76/193) and Western zones (47/193). They were mainly affiliated to Government owned & managed Academic Institutes /Hospitals (96/193) or privately owned and managed Academic Institutes/ Hospitals (35/193). Conclusions Given the importance of Outcomes research, relevant resources should be supported and encouraged which would help in the generation of important healthcare data that can guide health and research policy. Clarity about the distribution of outcomes research resources can facilitate future resource and funding allocation decisions for policy makers as well as help them measure research performance over time. Electronic supplementary material The online version of this article (doi:10.1186/2193-1801-2-518) contains supplementary material, which is available to authorized users.


Introduction
Outcomes research is concerned with determining the end results and in turn the effectiveness of healthcare practices, interventions and systems. Outcomes research focuses on topics ranging from effectiveness, appropriateness, access, quality of care, quality of life, health status, disease prevention, screening, drug treatment, medical procedures, medical practices, diagnostic tests, guidelines and healthcare policy (Jefford et al. 2003).
Despite the role of outcomes research studies in discerning practice variation (Pilote and Tager 2002), quality of care and determining "What actually works", a quick literature search in Gopubmed (GoPubMed® 2013) conducted by the authors revealed that the number of outcomes research studies in India have been lower in comparison to countries like USA, UK and Germany. (9362, 195663, 46028, 36693 respectively). (Search strategy: "Outcome Assessment (Health Care)" [mesh], Gopubmed,) Although there is a lack of studies evaluating the reason behind this trend, paucity of funding, non availability of infrastructure, lack of relevant expertise and trained staff could be factors responsible for this trend. Even when these factors are available, there exists a significant disconnect amongst them as data about existing resources is not widely accessible. If made available, information about existing resources can help 1. Policy makers to plan efficient strategies that can build up on existing outcomes research resources thus ensuring economies of scale as well as predict resource use and improve efficiency in the allocation of resources (Liu et al. 2008). 2. Researchers and research groups become aware of existing research resources thus avoiding duplication and facilitating higher productivity at a lower cost. Despite the importance of this information, till date no previous study has worked on compiling and sharing this information in a systematic manner.
India currently faces a mixed burden of both communicable and non-communicable diseases, the latter responsible for two-third of the total morbidity burden and more than half (53%) of total mortality in India. (WHO 2013) This dual burden poses significant public health challenges before India like safeguarding public health, expanding health care coverage and improving quality of care while controlling costs. Given the current economic downturn, Indian policy makers need to take cue from Australia, Japan, South Korea and China where Outcomes Research data are used for setting national policy, designing drug formulary and drafting pharmaceutical economics guidelines. (Garman 2013) In order to encourage and facilitate the conduct of outcomes research studies, knowledge, access and sustained support of pre-existing resources is essential. Few studies have mapped outcomes research resources in India.
In order to bridge this gap, we carried out a review of the literature to identify and map resources described in previous outcomes research studies conducted in India.

Ethics
We carried out a review of published literature and hence did not seek ethics approval for this review.

Definitions
Outcomes research resources include outcomes researchers, infrastructure, trained staff/manpower and electronic data sources like databases, registries, electronic health records, electronic medical records and hospital information systems. Data on researcher and electronic resources are reported in published literature but data on infrastructure and manpower is rarely published. Accordingly, we decided to focus on the former. For the purpose of this study, we used the following definitions of outcomes research resources: 1. Databases and Registries that collect data as a part of clinical practice or for research purposes. We used the following operational definition for a biomedical registry: A system for the registration, record keeping and referral of biomedical data, material or resources. (Dict.md, Medical dictionary, 2013) 2. (i) Electronic medical records defined as computerized systems that collect, manage and deliver healthcare data and information in electronic format as a part of routine practice (Luo 2006) (Rustagi and Singh 2012) and (ii) Electronic Healthcare records defined as a "comprehensive, cross-institutional, longitudinal collection of a patient's health and healthcare data". (Hoerbst and Ammenwerth 2010) 3. Experts -Details about researchers who carried out outcomes research studies using prospective or retrospective study designs.

Search strategy
Two reviewers (JS and AP) having previous experience in the conduct of reviews carried out an independent search in the following online biomedical databases: Pubmed, 1985

Eligibility criteria
We defined criteria that would help us filter through the initial list of search results and identify articles that would provide us the required data. We used the following inclusion criteria: 1. Articles published in English language, 2. Articles reporting outcomes research studies, 3. Articles reporting studies conducted using Indian data, 4. Articles providing information about outcomes research or epidemiological resources (databases/ registries/electronic medical records/electronic healthcare records/hospital information systems) in India, 5. Articles describing epidemiological studies based on outcomes research resources (databases/registries/ electronic medical records/electronic healthcare records/hospital information systems).
In case when a full text version of the article was unavailable, we included the abstract if it provided detailed information about the study. Articles retrieved by applying the search strategy were screened first by title, then by abstract and later by reviewing their full text version. At each step, articles dissatisfying the selection criteria were excluded. The shortlisted articles retrieved by each reviewer were compared and disagreements were resolved by discussion and mutual consent. Based on data present in the shortlisted articles, we compiled a list of outcomes research resources (Additional file 1). In order to cross-check our search results, we searched for all outcomes research articles published using these resources. We applied the selection criteria to the results of this secondary search and included articles that our search strategy missed during the primary search.

Data collection and data items
Two reviewers (JS and AP) independently reviewed each shortlisted article and captured information about the variables of interest in separate spreadsheets. The resulting data abstraction files populated by each reviewer were compared and disagreements were resolved by discussion and mutual consent.

Datasets
After removing duplicate entries, we identified a list of resources, articles published based on the resources, bibliometric data for the articles and bibliometric data for researchers who published the articles.

Resource dataset
We reviewed the methods section of each article to extract information about each resource. We also reviewed and extracted information available on their individual websites (if present) and internet in general (using Google search). We categorized the data for each resource using the following categories: 3. Researcher dataset From the 133 articles, we extracted the names of first and last authors, their institutional affiliations, location details (city, state, country) and email addresses.
Finally we carried out descriptive analysis of the 3 datasets described above.

Search results
We identified a total of 4911 articles based on keyword search. After removing duplicate entries (31) we were left with 4846 articles for review. After reviewing titles and abstracts of articles and applying selection criteria, we shortlisted 126 articles. Next, we reviewed the full text of 126 articles and excluded 7 articles dissatisfying our selection criteria thus yielding a total of 119 articles which were selected for inclusion in the study. After analyzing these 119 articles, we were able to identify a total of 91 unique resources. 14 more articles were retrieved through a secondary search carried out with an aim of identifying additional publications related to the resources reported in the articles. This increased the tally to 133 articles. We were able to identify a total of 91 unique resources and 193 researchers who had published outcomes research articles using these resources. (Figure 1: Flowchart describing review and article retrieval process)  Table 8]. b. Affiliation of outcomes researchers Analysis of the affiliation data for the outcomes researchers reveal that more than half of them  Table 9].

Discussion
To the best of our knowledge, this is the first study that carried out a systematic analysis of outcomes research resources in India as reported in published literature. We collected information relevant to 91 outcomes research resources in India and report details about each resource, bibliometric data of publications derived from these resources and researchers that conducted research studies using data derived from these resources.
We observed a predominance of registries and databases in India. Research registries collect long term clinical, health services and epidemiological data for a given population. They are essential to understand clinical and epidemiological trends as well as useful for policy analyses, planning and management of health care resources. (Roos and Nicole, Roos and Nicol 1999) (Broemeling et al. 2009). Databases are usually study specific or project specific. They are usually designed to collect data to answer a specific research question. The low number of EMR, EHR and HIS in India might be because of the fact that India has been slow in the adoption of biomedical and research informatics tools. Although having a wide range of advantages (Fraser et al., 2005) (Lobach and Detmer, 2007) (Mildon and Cohen, 2001) (Rustagi and Singh, 2012) concerns about privacy, reduction in clinical productivity, being resource intensive, (Rustagi and Singh, 2012) (Kluger, 2009) high purchase and maintenance costs make their adoption slower (Jha et al., 2009) (Hillestad et al., 2005).
We observed a geographical predominance of resources and researchers in southern and western zones indicating an imbalance. This imbalance may be further complicated by the fact that researchers from one zone may not have access to data from a resource located in another zone. This may significantly influence policy and funding decisions further resulting in a vicious cycle of resource duplication, under utilization of resources, and wastage of funding.
We also observed a predominance of Institutional and regional initiatives in spearheading/managing the resources. Although this trend is noteworthy and beneficial, it reflects small scale and medium scale research projects. National registries have their own importance    in nationwide policy decisions as data cannot always be extrapolated from regional data. There are numerous examples of large scale nationwide initiatives like Nationwide inpatient sample (HCUP-US NIS, 2013), National Health Insurance Research Database (NHRI, Taiwan, 2013), Disease registries maintained by National Registry of Diseases Office (NRDO, Singapore, 2013) that have and continue to significantly contribute to national healthcare decision making and planning as well as in the improvement of quality of healthcare. Thus a balanced distribution of regional and national resources is essential. We also noted that most of the resources and researchers were located in Government or Privately owned academic organizations. Although a good trend, these organizations usually serve the urban population and provide tertiary care. Given the fact that India is largely an agrarian country, equitable distribution of resources into urban and rural areas would facilitate the collection of data that is truly representative of the Indian population. Policies derived from such a representative sample will be more effective than those based on extrapolated data that do not represent real life scenarios.
Most of the resources collected data on Cancer, Stroke and Diabetes. Given the significant rise in cancer, cardiovascular and metabolic disorders in India, (Takiar et al., 2010) (Young et al., 2009) this distribution appears to be moving in the right direction. Yet, it should be noted    that there exists a vast difference amongst number of resources in each of these groups indicating a predominance of cancer resources. Accordingly, it prepares the case for the need of similar outcomes data resources for nationally prevalent diseases like Malaria, Tuberculosis. This can be implemented by incorporating relevant outcomes data variables in surveillance and national programs.
A predominance of publications derived from the short listed resources in international journals is a good trend as it helps disseminate results to a global audience. Yet the Journal impact factor (JIF) and citation index of these publications may be indicative of the quality and impact of results published. Training programs to help clinicians and researchers collect data using global accepted data standards and report them using standard reporting guidelines may make future publications reach a larger audience and gain higher impact. In this regard, a workshop on imparting outcomes research skills to medical faculty members was recently conducted with the aid of Indian Council of Medical Research. (Savardekar L, Shah J, Bacchav S, Kshirsagar N, Translating Ideas into Research Projects and Manuscripts in Outcomes Research: Experiences of An ICMR Workshop. unpublished observations).
Most of the registries and databases identified through this study have not been explored to their true potential. In most cases, data from these registries have resulted in one to three publications. Further, most of them do not have their own websites or web pages within their organization. Sharing of data dictionaries or actual dataa norm of current times is hardly applicable to these resources. This demonstrates that detailed information about these resources is not easily accessible. The Department of Science and Technology (India) conducted a National Survey on Resources Devoted to Science &Technology Activities (National Science & Technology Management Information System, India, 2013) but the resultant data is not publicly available. Secondly, the survey questionnaire does not capture granular information about research resources. Finally, we are not aware about its utility and effectiveness in facilitating collaborations and guiding policy decisions at a state and national level. It is thus evident that awareness of existing outcomes research resources in India is low thus impairing the ability of 1. Researchers and research groups to optimally utilize existing outcomes data for carrying out outcomes research studies and 2. Research policy makers to utilize resource availability and resource performance data while making resource allocation decisions.

Limitations
All efforts were made to do an exhaustive review of the literature but given the nature of research question and limitations in terms of keywords and filters, we may have missed relevant publications reporting information about outcomes research resources. Secondly, data resources and researchers are not the only factors that contribute to outcomes research. Factors like skilled manpower, training opportunities, availability of funding, institutional policy have a role to play. Since this information is not readily available on the web or in publications, we interpreted based on the data that was available to us. There is a need for national level initiatives to collect data about the location, capabilities and performance of outcomes research resources. Thirdly, we did not include keywords related to surveillance data in our search strategy as surveillance in itself is a huge area and beyond the scope of this project. We intend to pursue this in a subsequent study. Finally, although semi automated methods like natural language processing and computational ontologies could have been utilized to carry out data extraction and reasoning of data extracted from published articles (Lin et al., 2010) (Ceci et al., 2012), we preferred the manual method as 1. The number of relevant articles identified through an initial review was low and 2. To ensure higher quality of data abstraction.

Conclusion
Given the importance of Outcomes research, relevant resources should be supported and encouraged which would help in the generation of important healthcare data that can guide health and research policy. Clarity about the distribution of outcomes research resources can facilitate future resource and funding allocation decisions for policy makers as well as help them measure research performance over time.

Additional file
Additional file 1: Outcomes research resources identified.