Assessment of technological level of stem cell research using principal component analysis
SpringerPlus volume 5, Article number: 857 (2016)
In general, technological levels have been assessed based on specialist’s opinion through the methods such as Delphi. But in such cases, results could be significantly biased per study design and individual expert.
In this study, therefore scientific literatures and patents were selected by means of analytic indexes for statistic approach and technical assessment of stem cell fields. The analytic indexes, numbers and impact indexes of scientific literatures and patents, were weighted based on principal component analysis, and then, were summated into the single value. Technological obsolescence was calculated through the cited half-life of patents issued by the United States Patents and Trademark Office and was reflected in technological level assessment. As results, ranks of each nation’s in reference to the technology level were rated by the proposed method. Furthermore we were able to evaluate strengthens and weaknesses thereof.
Although our empirical research presents faithful results, in the further study, there is a need to compare the existing methods and the suggested method.
Over the last 15 years, our knowledge of stem cell biology has exponentially increased. The result of that there is an ever-increasing array of stem cells such as adult stem cell (ASC), embryonic stem cell (ESC), and induced pluripotent stem cell (iPSC). In the early stage of stem cell researches, ASC and ESC were extensively investigated but iPSC has become one of the major stem cell research fields since the development of iPSC by the Shinya Yamanaka group in 2007 (Litterman and Ekins 2015). In 2012, even its infancy, the key scientists of iPSC shared the award of Nobel Prize in Physiology and Medicine, emphasizing the significant of the advent of reprogramming and cell fate conversion. Further, scientists have become actively involved in stem cell biology and translation, participating in new fundamental discoveries and leading efforts into applications in biotechnology and medicine. Thus, the technical assessment of stem cells including this study might be timely and warranted.
Various studies are underway on establishing methodologies for strategic research and development (R&D) planning, and there has been an increase in application of technological level assessment. The technological level assessment is intended to compare scientific or industrial technologies of countries, industries and/or companies (Han et al. 2010). And, a technological level, an object of assessment, can be defined as performance of a specific technology measured at a specific time point. In this, the assessment needs a comparison between either two technologies or two different time points in reference to technological capability.
In the present study, we assessed the technology level of stem cell research based on published scientific literatures and patent analysis at national levels. Specifically, we analyzed ASC, ESC, and iPSC for eleven nations: Canada (CA), China (CN), France (FR), Germany (DE), Israel (IL), Japan (JP), Singapore (SG), South Korea (KR), Taiwan (TW), United Kingdom (UK), and United States (US).
Scientific literatures and patents for assessment of technological levels
Technological levels have been often assessed through specialized agencies. For example, in the US, the Office of Science and Technology Policy (OSTP) has been taking a primary role in assessment of nation’s technological levels. In JP, they conduct the technological level assessment prior to laying a science and technology master plan, wherewith it quantitatively and qualitatively assesses the levels of core technologies. In addition, the Japan Science and Technology Agency–Center for Research and Development Strategy (JST–CRDS) has been conducting international comparative assessment of technology level to support effective R&D policy-making since 2008. In KR, the levels of technologies (e.g., green technologies, information technologies, and industrial technologies) are often assessed by national agencies such as Korea Institute of Science and Technology Evaluation and Planning (KISTEP). The Table 1 depicts the summary of the recent cases of technology level assessment in different countries.
Patents are often regarded as one of the most representative technological outputs of R&D activities hence patents statistics might be a valuable source of information for technological strategy planning. It seems obvious therefore that patents statistics has been receiving more attention for implementing successful R&D as well as activities in high-tech industries such as biotechnology and information and communication technology. The Table 2 summarizes an important set of statistics used to analyze technological strategy in previous studies. The statistics were proposed for the evaluation of competitive positions in: number of patent, patent impact index, and cited-patent life time.
On the other hand, published research literatures could be also utilized as an excellent indicator for technology level assessment (Garfield and Welljams-Dorof 1992). Even though, numbers of scientific literatures have traditionally been regarded as an indicator of the productivity, quality of these works should not be overlooked. Such qualitative analysis however, is mostly performed by experts in the field thus is an often time-consuming and expensive process. In order to overcome these limitations, advent of citation databases, which track how often articles are referenced in subsequent publications, and by whom, have created new tools for indicating the impact of primary research literatures. By aggregating citation data, it is then possible to indicate the relative impact of individuals, journals, departments, institutions, and nations. In addition, the citation data can be used to identify emerging specialties, new technologies and even the structure of various research disciplines, fields, and science as a whole.
A technology becomes obsolescent over time. It might have been an up-to-date technology at one time, yet it may be regarded as an obsolete technology later. Bosworth (1978) analyzed statistical data on knowledge lie, from the creation of new technologies to their obsoleteness, by using the residual materials (Bosworth 1978). The usage rate of new technologies rose gradually and reached the peak on the 6th year after the creation of new technology, and then it showed a slow decline and reached nearly zero on the 16th year (Bosworth 1978). In the case of the manufacturing industry, for instance in UK, the obsolescence rate of technological knowledge estimated with renewed patents, reached approximately 10 %. In JP, the obsolescence rate was calculated from the reciprocal of average life of each industry, on the premise that the technological knowledge stock would decrease equally every year (KISTEP 2010). In other study, the obsolescence rate of R&D stock was estimated at 20 % because just 20 % of initial R&D expenses were developed into R&D stock (Pattel and Soete 1988). In this study, the obsolescence rate was estimated by the use of cited half-life (CHL). The CHL is intended to show how long specific journals or patents have been cited; more precisely, the period during which cumulative citation frequency reaches 50 %. In the study, therefore, the CHL was calculated using formula used in the Journal Citation Reports.
Approaches for technological level assessment: current status
Technological level assessment has been conducted by different assessors for various technologies, yet through similar methods, whereupon it has the following limitations. First, technological level was assessed via qualitative methods such as Delphi or the focus group interview. Although the qualitative methods are easy to conduct, often these methods are susceptible to methodological approaches, per se (KISTEP 2011). Consequently, results may be impacted per how an experimental group is composed. Second, the existing quantitative assessments, using scientific literatures and patent statistics, did not compute technological obsolescence caused by lapse of time. In practice, however, a technology has been obsolescent over time and become less valuable in the end, warranting the novel definition to account for this. In the current study, therefore this technological obsolescence was quantitatively computed utilizing the CHL (i.e., the period during which cumulative citation frequency reaches 50 %), and then, reflected for the assessment. Further, to compensate erstwhile defects, this study focused on technological scientific literatures and patents to which bibliometrics can be applied. The composite index of technological levels was calculated with literatures and patent indexes, weighted through principal component analysis (PCA) whilst technological obsolescence was assessed through the term, CHL.
In the present study, the Focust database was utilized for patents, respectively. For the database for scientific literature, we retrieved the articles using the database, Scopus, was utilized since it lists the largest numbers of journals (i.e., 21,568 journals) compared to other databases (e.g., Science Citation Index: 3746 journals; Arts and Humanities Citation Index: 1774 journals). Specific searching conditions were described in the Table 3. In addition, Table 4 shows specific indicators utilized for the analysis. Quantitative indicators were measured with the numbers of patents issued by the United States Patents and Trademark Office (USPTO) and scientific literatures listed in the database (i.e., Scopus). Moreover, qualitative indicators were measured using ‘Patent Impact Index’ and ‘Paper Impact Index.’ Such indicators were calculated in relation to every country and year. Every indicator was divided by the maximum value of every year, and the indicator values were re-scaled between 0 and 1 (Fig. 1).
The PCA was utilized to reduce correlated multivariate data to low-dimensional data in order to prevent the loss of information, meaning simplifying complex data sets for better interpretation (Raychaudhuri et al. 2000). PCs were generated via the summation of weighted variables. In this process, the weighting was determined via the optimization of information contained in initial variables to the end which may represent the widest dispersion. In case the initial variable contains a large number of principal components, every component should not be correlated, in other words, the inter-variable covariance should be 0. The relational expression was expressed in the Eq. 1 as follow:
In the formula, PC1 to PCm represent principal components while x 1 to x n indicate initial variables; a 11 to a mn mean weightings. The PCs were selected when its eigenvalue is 1 or higher (according to the Kaiser’s rule). Weightings, regarding respective variables, could be calculated by the reverse use of the factor matrix between PCs and variables. In this study, the composite technological level was analyzed through the weightings figured out with respect to each variable.
Lastly, in order to assess the technical level of stem cell technology, four weightings obtained from PCA were summated into a single value, from which technological level index was calculated. Then, CHL was applied to the technological obsolescence rate. With the technological level index, technological levels were analyzed in relation to years and countries. The formula to assess the technology level is described as below:
Results and discussion
Analysis of scientific literatures and patents
Stem cell-related scientific literatures and patents (e.g., ASC, ESC, and iPS cells) were retrieved, and then, the NP and PII were analyzed in relation to years and countries (Table 5). The indexes of different units were divided by the maximum value of each year, and all of them re-scaled between 0 and 1 (Table 6).
In order to determine the weightings of indexes, PCA was conducted with respect to the indexes of published literatures and patents registered between 1993 and 2012. A total of 470 sets of data were used, exclusive of data of which values were 0 for four variables (Table 7). Then, calculations were made of four components, their eigenvalues and cumulative contribution rates. The eigenvalues of the PC1 and the PC2 were 2.145 and 1.028, respectively, and their contribution rates were 53.6 and 25.7 % respectively. These two PCs comprised of 79.3 % of the total data set (Table 8). Eigenvalues of the two PCs (Table 8) and component matrix (Table 9) were used to calculate the weightings of indexes (Table 10).
The CHL was calculated using the citation frequency of USPTO issued patents in order to determine the technological obsolescence rate. In order to analyze the citation frequency, numbers of citations per each year were analyzed based on the year of registration of patents and expressed as percentile as shown in the Table 11. It indicates that the patent was citied 19.4 times on its 4th year from the point where it was registered if it was cited 100 times on the year of registration. In this, a calculation was made of the annual citation frequency of patents registered between 1993 and 2002. The Fig. 2 depicts the cumulative citation frequency of every year. In the result, CHL was 9.286 (Table 12; Fig. 2). Specifically, the patents averagely reached CHL 9.286 years after the registration. The Table 13 and Fig. 3 show the annual obsolescence rate on the premise that a technology obsolesces 50 % every 9.286 years.
Assessment of technological level
The technological level, assessed at a specific point of time, means the technological stock calculated up to then, and thus the cumulative value was calculated for the assessment of annual technological levels. Prior to the calculation of the cumulative value, a time-series analysis was conducted with respect to the technological level, by the use of the annual obsolescence rate. The obsolescence rates shown in the Table 13 were reflected in the re-scaled indicators of scientific literatures and patents, and the levels of technologies presented every year were assessed at the present point in time (Fig. 4). As results, the US has the highest technological level, followed by JP, IL, CA, UK, DE, FR, SG, CN, KR and TW, respectively; the technological levels of Asian countries, except for JP, were relatively lower compared to those of European countries. In case of the US ranked first, 4 scientific literature and patent indicators showed high percentages without significant differences. In CN, however scientific literatures accounted for a high percentage of the technological level, whereas patents or patent impact indexes made up a very small part. IL and CA had high levels of NP but low levels of PII; overall, however their technological levels were relatively high.
When it comes to the ASC, the US has the highest technological level, followed by CA, IL, JP and DE (Table 14; Fig. 5). In case of CA, the technological level has been rapidly improved, in comparison with other countries, since the late 1990s. Likewise, the US was rated first in the field of ESC and SG was the second, followed by IL and DE. When it comes to the ASC, countries did not show any significant difference, except for the US. In the field of ESC, SG, IL, DE, UK, JP, CA and FR constituted a group, while KR, CN and TW represented another group. There was remarkable difference between the two groups (Table 14; Fig. 5).
SG was found to be inferior to North American and European countries in the field of ASC as well as iPSC, yet in the ESC, it was the second highest country after the US. Of note, the technological level of human embryonic stem cell (hESC) was particularly high, hence, in a way to support the feasibility of the analysis result, we further reviewed the current hESC banks of each country (Korea Centers for Disease Control and Prevention, 2013). As expected, Singapore Stem Cell Consortium (SSCC) particularly focused on the hESC over other stem cell types; in fact it appeared that the SSCC developed clinical grade hESC in 2007 via their good manufacturing practice facilities. The Singapore Stem Cell Bank which is operated by the SSCC is currently selling four clinical grade hESC stocks as well (Korea Centers for Disease Control and Prevention 2013). It has been widely accepted that the development of clinical grade hESC requires cutting-edge technology which results only two countries have been successful so far (i.e., US and SG). To be specific, the US has developed two clinical grade cell stocks, while SG developed six clinical grade cell stocks (Crook et al. 2007). Given that, it seems reasonable to believe that the technological level of SG in the field of hESC stands out. The US stands unchallenged in the fields of ASC and ESC, while JP was the best country in the field of iPSC, over the US, which also makes sense given that (1) the Kyoto University was successful to generate the first iPSC in conjunction with the University of Wisconsin-Madison in 2007 and (2) JP produced a Nobel Prize winner in physiology or medicine, for iPSC in 2012.
The Table 14 summarizes rankings of technological levels of each stem cell field. First, not surprisingly, the US whose total technological level was the highest represented the highest values in four detail indicators (i.e., NP_Patent, PII_Patent, NP_Paper and PII_Paper) in the field of ASC. CA and IL whose total technological level was the second and third highest were relatively low in this field. Even though CN showed the second highest in number of scientific literatures (i.e., NP_Paper) after the US, it showed the lowest values in other indicators among the countries in comparison which partially explains their low ranking in the total technological level (8th out of 11 countries in the study; Table 14).
In the field of ESC, again, the US showed the highest values in all indicators except for PII_Paper, and its total technological level was highest. The scores of SG and IL in NP Patent and NP_Paper were low, but their values in PII_Patent and PII_Paper were relatively high which lead them to be placed in higher ranking in the field (2nd and 3rd out of 11 countries; Table 14). The technological level of CN in the field of ESC was the third highest in NP_Paper (alike in the field of ASC) after the US and JP, yet other indicators were (i.e., NP_Patent, PII_Patent, and PII_Paper) were the lowest amongst the countries.
On the other hand, due to the infancy of the field of the iPSC, the first patent was issued by the USPTO in 2011. Moreover, only three countries (US, JP, and TW) are holding the iPSC related patents issued by the USPTO. In this field, JP whose total score was the highest obtained a high value in patent-related indicators (NP_Patent and PII_Patent). The US showed the highest value in scientific literatures-related indicators (NP_Paper and PII_Paper), but they were ranked the second in the total technology level after the JP. If the new technology evaluation method presented in this study is applied, it is expected that not only the total technological level evaluation but also the comparison of individual indicators which compose the technological level will be possible.
This study assessed technological level utilizing a new quantitative method with a view to compensating limitations of the existing methods, i.e., qualitative assessment based on specialists’ opinion and quantitative one based on patent indicators. In this study, not only patents yet also scientific literatures were applied into the model as analytical indicators, the number of papers (NP_Paper), the paper impact index (PII_Paper), the number of patents (NP_Patent) and the patent impact index (PII_Patent) hence provide important insight for future R&D planning and establishment of related policy. There are two strengths of the assessment model we proposed herein compared to previous quantitative methods. First, a number of indexes for both scientific literatures and patents were summated into the single value based on the PCA. Although previous studies have attempted to summate a single value by analytic hierarchy process or specialists’ opinion (Cho and Park 2015), there was a limitation of multi-collinearity between indexes which may doubly affect overall assessment of technology levels; thus, the authors attempted to exclude multi-collinearity by using the PCA. Second, the existing quantitative assessments may not be appropriate to take account of technological obsolescence, because both recently registered and former patents were calculated as simple numeric values without consideration for technological obsolescence which made us to reflect it, caused by the lapse of time, in the technological level assessment. To achieve this, annual technological obsolescence was computed through the CHL. Nonetheless, in future, it is warranted to make a direct comparison between existing qualitative assessment methods against the present results as well as the methodological approach in terms of validity and reliability. Further, application of additional quantifiable indicators may result more accurate assessment as well. Lastly, it is required to analyze various methodologies to further find the optimum quantitative assessment given objective data. To conclude, one cannot stress enough the fact that the assessment of technological level is one of the most important steps with regards to establishment of R&D plans and policies. Considering limitations of current assessments approaches (e.g., Delphi), our novel quantitative approach would provide a good alternative mean. In addition, it will be also interesting to combine both qualitative and quantitative assessments thereby providing unbiased results in future.
There are several limitations of the study. First, since the technology level is a type of intangible indicator thus one should not solely rely on assessment utilizing quantitative data. In addition to scientific literature and patents, both quantitative and qualitative data (e.g., human resources, infrastructure, and clinical trials) should be comprehensively employed for the assessment, if applicable. Second, in the present study, the ranking of technology levels were made based on relative values determined through the equation. To take an example, in case of the ACS, the technology level of US was found to be 849.3 whereas South Korea was 146.7 (Table 14). These values can be used to rank nations yet should not be interpreted as the US technology level is 5.8 fold higher than South Korea because these values are relative values. In addition, even though we deliberately selected the largest database (i.e., Scopus) to retrieve scientific publications, there are other databases (e.g., Science Citation Index) and available that may list journals other than we were able to obtain from the Scopus. Last, in the present study, three major stem research fields were subjected to the analysis thus, it would be interesting and informative to further investigate subspecialized stem cells (e.g., cancer stem cells, foetal stem cells, and dental stem cells) and their technological field through the proposed tool in future studies. Utilization of technology diffusion models (e.g., Bass model or Logistic model) for the prediction of stem cell research might be also interesting to examine in conjunction with current trends and outcomes herein (Rao and Kishore 2010).
adult stem cell
embryonic stem cell
human embryonic stem cell
induced pluripotent stem cell
Japan Science and Technology Agency–Center for Research and Development Strategy
Korea Institute of Science and Technology Evaluation and Planning
Office of Science and Technology Policy
principal component analysis
research and development
Singapore Stem Cell Consortium
United States Patents and Trademark Office
Bosworth DL (1978) The rate of obsolescence of technical knowledge—a note. J Ind Econ 26:273–279
CHI Research Inc. (2005) The links between hotspot patents and publicly funded scientific research. CHI Research Inc., New Jersey
Chia KC (2004) Measuring the patent effectiveness in biotechnology industry. Dissertation, National Central University
Cho IG, Park MC (2015) Technological-level evaluation using patent statistics: model and application in mobile communications. Clust Comput 18:259–268
Cho SD, Lee CM, Hyun BH (2013) Analysis of technological competitiveness and technology-industry linkage structure of Korea, China and Japan utilizing the patent information in the field of biotechnology. J Technol Innov 21:141–163 (in Korean)
Crook JM, Peura TT, Kravets L, Bosman AG, Buzzard JJ, Horne R, Hentze H, Dunn NR, Zweigerdt R, Chua F, Upshall A, Colman A (2007) The generation of six clinical-grade human embryonic stem cell lines. Cell Stem Cell 1:490–494
Ernst H (2003) Patent information for strategic technology management. World Pat Inf 25:233–242
Garfield E, Welljams-Dorof A (1992) Citation data: their use as quantitative indicators for science and technology evaluation and policy-making. Sci Public Policy 19:321–327
Han MK, Kim BS, Ryu JY, Byeon SC (2010) Technology level evaluation based on technology growth curve model and its implication—in case of ‘biochip and biosensor technology. J Korea Technol Innov Soc 13:252–281 (in Korean)
Huang Z, Chen H, Yip A, Ng G, Guo F, Chen ZK, Roco MC (2003) Longitudinal patent analysis for nanoscale science and engineering: Country, institution and technology field. J Nanopart Res 5:333–363
Japan Science and Technology Agency—Center for Research and Development Strategy (2013) Japanese science and technology indicators 2013, Tokyo
Korea Centers for Disease Control and Prevention (2013) Trend of overseas stem cell bank, Chungcheongbuk-do (Text in Korean)
Korea Institute of Science & Technology Evaluation and Planning (2010) A study on the dynamic method of estimating technology levels based on the technology growth model, Seoul (Text in Korean)
Korea Institute of Science & Technology Evaluation and Planning (2011) Issue paper 2011–2014, Seoul (Text in Korean)
Korea Institute of Science & Technology Evaluation and Planning (2013) 2012 Technology level evaluation, Seoul (Text in Korean)
Litterman NK, Ekins S (2015) Database and collaboration require standards for human stem cell research. Drug Discov Today 20:247–254
Office of Science and Technology Policy (1995) National critical technologies report, Washington
Pattel P, Soete L (1988) Measuring the economic effects of technology. OECD STI Rev 4:121–166
Rao KU, Kishore VVN (2010) A review of technology diffusion models with special reference to renewable energy technologies. Renew Sustain Energy Rev 14:1070–1078
Raychaudhuri S, Stuart JM, Altman RB (2000) Principal component analysis to summarize microarray experiments: application to sporulation time series. In: Paper presented at the 5th annual conference, The Pacific Symposium on Biocomputing, Hawaii, 4–9 January 2000
World Technology Evaluation Center (2013) Systems engineering for clean and renewable energy manufacturing. World Technology Evaluation Center, Baltimore
World Technology Evaluation Center (2014) Assessment of physical sciences and engineering advances in life sciences and oncology. World Technology Evaluation Center, Baltimore
Yoo SH, Lee YH, Won DK (2006) A study on estimation of technology life span using analysis of patent citation. J Korean Oper Res Manag Sci Soc 31:1–11
SDC, BHH, and JKK designed study and performed data analysis. BHH assisted to draft the manuscript. SDC and JKK wrote the manuscript. BHH and JJK had primary responsibility for the final content, as a corresponding author. All authors read and approved the final manuscript.
The authors declare that they have no competing interests.
Byung Hwan Hyun and Jae Kyeom Kim equally contributed to this work as corresponding authors
About this article
Cite this article
Do Cho, S., Hwan Hyun, B. & Kim, J.K. Assessment of technological level of stem cell research using principal component analysis. SpringerPlus 5, 857 (2016). https://doi.org/10.1186/s40064-016-2494-9