Assessment of technological level of stem cell research using principal component analysis

Abstract

Background

In general, technological levels have been assessed based on specialist’s opinion through the methods such as Delphi. But in such cases, results could be significantly biased per study design and individual expert.

Findings

In this study, therefore scientific literatures and patents were selected by means of analytic indexes for statistic approach and technical assessment of stem cell fields. The analytic indexes, numbers and impact indexes of scientific literatures and patents, were weighted based on principal component analysis, and then, were summated into the single value. Technological obsolescence was calculated through the cited half-life of patents issued by the United States Patents and Trademark Office and was reflected in technological level assessment. As results, ranks of each nation’s in reference to the technology level were rated by the proposed method. Furthermore we were able to evaluate strengthens and weaknesses thereof.

Conclusions

Although our empirical research presents faithful results, in the further study, there is a need to compare the existing methods and the suggested method.

Background

Over the last 15 years, our knowledge of stem cell biology has exponentially increased. The result of that there is an ever-increasing array of stem cells such as adult stem cell (ASC), embryonic stem cell (ESC), and induced pluripotent stem cell (iPSC). In the early stage of stem cell researches, ASC and ESC were extensively investigated but iPSC has become one of the major stem cell research fields since the development of iPSC by the Shinya Yamanaka group in 2007 (Litterman and Ekins 2015). In 2012, even its infancy, the key scientists of iPSC shared the award of Nobel Prize in Physiology and Medicine, emphasizing the significant of the advent of reprogramming and cell fate conversion. Further, scientists have become actively involved in stem cell biology and translation, participating in new fundamental discoveries and leading efforts into applications in biotechnology and medicine. Thus, the technical assessment of stem cells including this study might be timely and warranted.

Various studies are underway on establishing methodologies for strategic research and development (R&D) planning, and there has been an increase in application of technological level assessment. The technological level assessment is intended to compare scientific or industrial technologies of countries, industries and/or companies (Han et al. 2010). And, a technological level, an object of assessment, can be defined as performance of a specific technology measured at a specific time point. In this, the assessment needs a comparison between either two technologies or two different time points in reference to technological capability.

In the present study, we assessed the technology level of stem cell research based on published scientific literatures and patent analysis at national levels. Specifically, we analyzed ASC, ESC, and iPSC for eleven nations: Canada (CA), China (CN), France (FR), Germany (DE), Israel (IL), Japan (JP), Singapore (SG), South Korea (KR), Taiwan (TW), United Kingdom (UK), and United States (US).

Data collection

Scientific literatures and patents for assessment of technological levels

Technological levels have been often assessed through specialized agencies. For example, in the US, the Office of Science and Technology Policy (OSTP) has been taking a primary role in assessment of nation’s technological levels. In JP, they conduct the technological level assessment prior to laying a science and technology master plan, wherewith it quantitatively and qualitatively assesses the levels of core technologies. In addition, the Japan Science and Technology Agency–Center for Research and Development Strategy (JST–CRDS) has been conducting international comparative assessment of technology level to support effective R&D policy-making since 2008. In KR, the levels of technologies (e.g., green technologies, information technologies, and industrial technologies) are often assessed by national agencies such as Korea Institute of Science and Technology Evaluation and Planning (KISTEP). The Table 1 depicts the summary of the recent cases of technology level assessment in different countries.

Table 1 Technology level assessments implemented in the US, Japan, and Korea

Patents are often regarded as one of the most representative technological outputs of R&D activities hence patents statistics might be a valuable source of information for technological strategy planning. It seems obvious therefore that patents statistics has been receiving more attention for implementing successful R&D as well as activities in high-tech industries such as biotechnology and information and communication technology. The Table 2 summarizes an important set of statistics used to analyze technological strategy in previous studies. The statistics were proposed for the evaluation of competitive positions in: number of patent, patent impact index, and cited-patent life time.

Table 2 Patent statistics for technology assessment in previous studies

On the other hand, published research literatures could be also utilized as an excellent indicator for technology level assessment (Garfield and Welljams-Dorof 1992). Even though, numbers of scientific literatures have traditionally been regarded as an indicator of the productivity, quality of these works should not be overlooked. Such qualitative analysis however, is mostly performed by experts in the field thus is an often time-consuming and expensive process. In order to overcome these limitations, advent of citation databases, which track how often articles are referenced in subsequent publications, and by whom, have created new tools for indicating the impact of primary research literatures. By aggregating citation data, it is then possible to indicate the relative impact of individuals, journals, departments, institutions, and nations. In addition, the citation data can be used to identify emerging specialties, new technologies and even the structure of various research disciplines, fields, and science as a whole.

Obsolescence rate

A technology becomes obsolescent over time. It might have been an up-to-date technology at one time, yet it may be regarded as an obsolete technology later. Bosworth (1978) analyzed statistical data on knowledge lie, from the creation of new technologies to their obsoleteness, by using the residual materials (Bosworth 1978). The usage rate of new technologies rose gradually and reached the peak on the 6th year after the creation of new technology, and then it showed a slow decline and reached nearly zero on the 16th year (Bosworth 1978). In the case of the manufacturing industry, for instance in UK, the obsolescence rate of technological knowledge estimated with renewed patents, reached approximately 10 %. In JP, the obsolescence rate was calculated from the reciprocal of average life of each industry, on the premise that the technological knowledge stock would decrease equally every year (KISTEP 2010). In other study, the obsolescence rate of R&D stock was estimated at 20 % because just 20 % of initial R&D expenses were developed into R&D stock (Pattel and Soete 1988). In this study, the obsolescence rate was estimated by the use of cited half-life (CHL). The CHL is intended to show how long specific journals or patents have been cited; more precisely, the period during which cumulative citation frequency reaches 50 %. In the study, therefore, the CHL was calculated using formula used in the Journal Citation Reports.

Approaches for technological level assessment: current status

Technological level assessment has been conducted by different assessors for various technologies, yet through similar methods, whereupon it has the following limitations. First, technological level was assessed via qualitative methods such as Delphi or the focus group interview. Although the qualitative methods are easy to conduct, often these methods are susceptible to methodological approaches, per se (KISTEP 2011). Consequently, results may be impacted per how an experimental group is composed. Second, the existing quantitative assessments, using scientific literatures and patent statistics, did not compute technological obsolescence caused by lapse of time. In practice, however, a technology has been obsolescent over time and become less valuable in the end, warranting the novel definition to account for this. In the current study, therefore this technological obsolescence was quantitatively computed utilizing the CHL (i.e., the period during which cumulative citation frequency reaches 50 %), and then, reflected for the assessment. Further, to compensate erstwhile defects, this study focused on technological scientific literatures and patents to which bibliometrics can be applied. The composite index of technological levels was calculated with literatures and patent indexes, weighted through principal component analysis (PCA) whilst technological obsolescence was assessed through the term, CHL.

Data collection

In the present study, the Focust database was utilized for patents, respectively. For the database for scientific literature, we retrieved the articles using the database, Scopus, was utilized since it lists the largest numbers of journals (i.e., 21,568 journals) compared to other databases (e.g., Science Citation Index: 3746 journals; Arts and Humanities Citation Index: 1774 journals). Specific searching conditions were described in the Table 3. In addition, Table 4 shows specific indicators utilized for the analysis. Quantitative indicators were measured with the numbers of patents issued by the United States Patents and Trademark Office (USPTO) and scientific literatures listed in the database (i.e., Scopus). Moreover, qualitative indicators were measured using ‘Patent Impact Index’ and ‘Paper Impact Index.’ Such indicators were calculated in relation to every country and year. Every indicator was divided by the maximum value of every year, and the indicator values were re-scaled between 0 and 1 (Fig. 1).

Table 3 Variables used for scientific literature and patent searching

Table 4 Indicators for scientific literatures and patent analysis

Fig. 1

Assessment of technological level using quantifiable outputs, scientific literature and patent: a research model

Statistical analysis

The PCA was utilized to reduce correlated multivariate data to low-dimensional data in order to prevent the loss of information, meaning simplifying complex data sets for better interpretation (Raychaudhuri et al. 2000). PCs were generated via the summation of weighted variables. In this process, the weighting was determined via the optimization of information contained in initial variables to the end which may represent the widest dispersion. In case the initial variable contains a large number of principal components, every component should not be correlated, in other words, the inter-variable covariance should be 0. The relational expression was expressed in the Eq. 1 as follow:

$$\begin{aligned} PC_{1}& = a_{11} x_{1} + a_{12} x_{2} + \cdots + a_{1n} x_{n} \hfill \\ PC_{2} &= a_{21} x_{1} + a_{22} x_{2} + \cdots + a_{2n} x_{n} \hfill \\& \ldots \hfill \\ PC_{m}& = a_{m1} x_{1} + a_{m2} x_{2} + \cdots + a_{mn} x_{n} \hfill \\ \end{aligned}$$

(1)

In the formula, PC₁ to PC_m represent principal components while x ₁ to x _n indicate initial variables; a ₁₁ to a _mn mean weightings. The PCs were selected when its eigenvalue is 1 or higher (according to the Kaiser’s rule). Weightings, regarding respective variables, could be calculated by the reverse use of the factor matrix between PCs and variables. In this study, the composite technological level was analyzed through the weightings figured out with respect to each variable.

Lastly, in order to assess the technical level of stem cell technology, four weightings obtained from PCA were summated into a single value, from which technological level index was calculated. Then, CHL was applied to the technological obsolescence rate. With the technological level index, technological levels were analyzed in relation to years and countries. The formula to assess the technology level is described as below:

$$\begin{aligned} TL_{t} & = \mathop \sum \limits_{{t^{\prime}}}^{t} \left( {\left( {NT_{t} \times NT_{\omega } } \right) + \left( {PT_{t} \times PT_{\omega } } \right) \times \left( {NR_{t} \times NR_{\omega } } \right) \times \left( {PR_{t} \times PR_{\omega } } \right)} \right) \times \delta_{t} \\ TL_{t} & = Technology \, level \, in \, t \, year, \, t^{\prime} = \, initial \, year, \\ NT \, & = NP\_Patent \, \left( {the \, number \, of \, patent} \right), \, NT_{\omega } = \, weight \, of \, NT \\ PT & = PII\_Patent \, \left( {patent \, impact \, index} \right), \, PT_{\omega } = \, weight \, of \, PT \\ NR & = NP\_Paper \, \left( {the \, number \, of \, paper} \right), \, NR_{\omega } = \, weight \, of \, NR \\ PR & = PII\_Paper \, \left( {paper \, impact \, index} \right), \, PR_{\omega } = \, weight \, of \, PR \\ \delta & = Obsolescence \, rate \\ \end{aligned}$$

(2)

Results and discussion

Analysis of scientific literatures and patents

Stem cell-related scientific literatures and patents (e.g., ASC, ESC, and iPS cells) were retrieved, and then, the NP and PII were analyzed in relation to years and countries (Table 5). The indexes of different units were divided by the maximum value of each year, and all of them re-scaled between 0 and 1 (Table 6).

Table 5 Results of scientific literatures and patent analysis

Table 6 Re-scaled indexes

PCA

In order to determine the weightings of indexes, PCA was conducted with respect to the indexes of published literatures and patents registered between 1993 and 2012. A total of 470 sets of data were used, exclusive of data of which values were 0 for four variables (Table 7). Then, calculations were made of four components, their eigenvalues and cumulative contribution rates. The eigenvalues of the PC1 and the PC2 were 2.145 and 1.028, respectively, and their contribution rates were 53.6 and 25.7 % respectively. These two PCs comprised of 79.3 % of the total data set (Table 8). Eigenvalues of the two PCs (Table 8) and component matrix (Table 9) were used to calculate the weightings of indexes (Table 10).

Table 7 Simple statistics of variables and their correlation coefficients

Table 8 Eigenvalue and cumulative variance of principal components (PCs)

Table 9 Component matrix

Table 10 Weights for statistics used in assessment of technological level

Obsolescence rate

The CHL was calculated using the citation frequency of USPTO issued patents in order to determine the technological obsolescence rate. In order to analyze the citation frequency, numbers of citations per each year were analyzed based on the year of registration of patents and expressed as percentile as shown in the Table 11. It indicates that the patent was citied 19.4 times on its 4th year from the point where it was registered if it was cited 100 times on the year of registration. In this, a calculation was made of the annual citation frequency of patents registered between 1993 and 2002. The Fig. 2 depicts the cumulative citation frequency of every year. In the result, CHL was 9.286 (Table 12; Fig. 2). Specifically, the patents averagely reached CHL 9.286 years after the registration. The Table 13 and Fig. 3 show the annual obsolescence rate on the premise that a technology obsolesces 50 % every 9.286 years.

Table 11 Percentage of citation frequency after patents issued

Fig. 2

Cumulative citation frequency after patent issued for the fields of stem cell research

Table 12 Calculation of cited half-life (CHL)

Table 13 Obsolescence rate

Fig. 3

The obsolescence rate curve. The curve of obsolescence rate was created given the calculated CHL (9.286 years) which indicates that 50 % of the residual value would be decreased by every 9.286 years. The obsolescence rate at certain time point can be calculated via subtracting residual value out of 100 % (see Table 13 for the residual value)

Assessment of technological level

The technological level, assessed at a specific point of time, means the technological stock calculated up to then, and thus the cumulative value was calculated for the assessment of annual technological levels. Prior to the calculation of the cumulative value, a time-series analysis was conducted with respect to the technological level, by the use of the annual obsolescence rate. The obsolescence rates shown in the Table 13 were reflected in the re-scaled indicators of scientific literatures and patents, and the levels of technologies presented every year were assessed at the present point in time (Fig. 4). As results, the US has the highest technological level, followed by JP, IL, CA, UK, DE, FR, SG, CN, KR and TW, respectively; the technological levels of Asian countries, except for JP, were relatively lower compared to those of European countries. In case of the US ranked first, 4 scientific literature and patent indicators showed high percentages without significant differences. In CN, however scientific literatures accounted for a high percentage of the technological level, whereas patents or patent impact indexes made up a very small part. IL and CA had high levels of NP but low levels of PII; overall, however their technological levels were relatively high.

Fig. 4

Technology level of stem cell (total)

When it comes to the ASC, the US has the highest technological level, followed by CA, IL, JP and DE (Table 14; Fig. 5). In case of CA, the technological level has been rapidly improved, in comparison with other countries, since the late 1990s. Likewise, the US was rated first in the field of ESC and SG was the second, followed by IL and DE. When it comes to the ASC, countries did not show any significant difference, except for the US. In the field of ESC, SG, IL, DE, UK, JP, CA and FR constituted a group, while KR, CN and TW represented another group. There was remarkable difference between the two groups (Table 14; Fig. 5).

Table 14 Technological levels of countries in stem cell field

Fig. 5

Technology level of stem cell (technological classification)

SG was found to be inferior to North American and European countries in the field of ASC as well as iPSC, yet in the ESC, it was the second highest country after the US. Of note, the technological level of human embryonic stem cell (hESC) was particularly high, hence, in a way to support the feasibility of the analysis result, we further reviewed the current hESC banks of each country (Korea Centers for Disease Control and Prevention, 2013). As expected, Singapore Stem Cell Consortium (SSCC) particularly focused on the hESC over other stem cell types; in fact it appeared that the SSCC developed clinical grade hESC in 2007 via their good manufacturing practice facilities. The Singapore Stem Cell Bank which is operated by the SSCC is currently selling four clinical grade hESC stocks as well (Korea Centers for Disease Control and Prevention 2013). It has been widely accepted that the development of clinical grade hESC requires cutting-edge technology which results only two countries have been successful so far (i.e., US and SG). To be specific, the US has developed two clinical grade cell stocks, while SG developed six clinical grade cell stocks (Crook et al. 2007). Given that, it seems reasonable to believe that the technological level of SG in the field of hESC stands out. The US stands unchallenged in the fields of ASC and ESC, while JP was the best country in the field of iPSC, over the US, which also makes sense given that (1) the Kyoto University was successful to generate the first iPSC in conjunction with the University of Wisconsin-Madison in 2007 and (2) JP produced a Nobel Prize winner in physiology or medicine, for iPSC in 2012.

The Table 14 summarizes rankings of technological levels of each stem cell field. First, not surprisingly, the US whose total technological level was the highest represented the highest values in four detail indicators (i.e., NP_Patent, PII_Patent, NP_Paper and PII_Paper) in the field of ASC. CA and IL whose total technological level was the second and third highest were relatively low in this field. Even though CN showed the second highest in number of scientific literatures (i.e., NP_Paper) after the US, it showed the lowest values in other indicators among the countries in comparison which partially explains their low ranking in the total technological level (8th out of 11 countries in the study; Table 14).

In the field of ESC, again, the US showed the highest values in all indicators except for PII_Paper, and its total technological level was highest. The scores of SG and IL in NP Patent and NP_Paper were low, but their values in PII_Patent and PII_Paper were relatively high which lead them to be placed in higher ranking in the field (2nd and 3rd out of 11 countries; Table 14). The technological level of CN in the field of ESC was the third highest in NP_Paper (alike in the field of ASC) after the US and JP, yet other indicators were (i.e., NP_Patent, PII_Patent, and PII_Paper) were the lowest amongst the countries.

On the other hand, due to the infancy of the field of the iPSC, the first patent was issued by the USPTO in 2011. Moreover, only three countries (US, JP, and TW) are holding the iPSC related patents issued by the USPTO. In this field, JP whose total score was the highest obtained a high value in patent-related indicators (NP_Patent and PII_Patent). The US showed the highest value in scientific literatures-related indicators (NP_Paper and PII_Paper), but they were ranked the second in the total technology level after the JP. If the new technology evaluation method presented in this study is applied, it is expected that not only the total technological level evaluation but also the comparison of individual indicators which compose the technological level will be possible.

This study assessed technological level utilizing a new quantitative method with a view to compensating limitations of the existing methods, i.e., qualitative assessment based on specialists’ opinion and quantitative one based on patent indicators. In this study, not only patents yet also scientific literatures were applied into the model as analytical indicators, the number of papers (NP_Paper), the paper impact index (PII_Paper), the number of patents (NP_Patent) and the patent impact index (PII_Patent) hence provide important insight for future R&D planning and establishment of related policy. There are two strengths of the assessment model we proposed herein compared to previous quantitative methods. First, a number of indexes for both scientific literatures and patents were summated into the single value based on the PCA. Although previous studies have attempted to summate a single value by analytic hierarchy process or specialists’ opinion (Cho and Park 2015), there was a limitation of multi-collinearity between indexes which may doubly affect overall assessment of technology levels; thus, the authors attempted to exclude multi-collinearity by using the PCA. Second, the existing quantitative assessments may not be appropriate to take account of technological obsolescence, because both recently registered and former patents were calculated as simple numeric values without consideration for technological obsolescence which made us to reflect it, caused by the lapse of time, in the technological level assessment. To achieve this, annual technological obsolescence was computed through the CHL. Nonetheless, in future, it is warranted to make a direct comparison between existing qualitative assessment methods against the present results as well as the methodological approach in terms of validity and reliability. Further, application of additional quantifiable indicators may result more accurate assessment as well. Lastly, it is required to analyze various methodologies to further find the optimum quantitative assessment given objective data. To conclude, one cannot stress enough the fact that the assessment of technological level is one of the most important steps with regards to establishment of R&D plans and policies. Considering limitations of current assessments approaches (e.g., Delphi), our novel quantitative approach would provide a good alternative mean. In addition, it will be also interesting to combine both qualitative and quantitative assessments thereby providing unbiased results in future.

There are several limitations of the study. First, since the technology level is a type of intangible indicator thus one should not solely rely on assessment utilizing quantitative data. In addition to scientific literature and patents, both quantitative and qualitative data (e.g., human resources, infrastructure, and clinical trials) should be comprehensively employed for the assessment, if applicable. Second, in the present study, the ranking of technology levels were made based on relative values determined through the equation. To take an example, in case of the ACS, the technology level of US was found to be 849.3 whereas South Korea was 146.7 (Table 14). These values can be used to rank nations yet should not be interpreted as the US technology level is 5.8 fold higher than South Korea because these values are relative values. In addition, even though we deliberately selected the largest database (i.e., Scopus) to retrieve scientific publications, there are other databases (e.g., Science Citation Index) and available that may list journals other than we were able to obtain from the Scopus. Last, in the present study, three major stem research fields were subjected to the analysis thus, it would be interesting and informative to further investigate subspecialized stem cells (e.g., cancer stem cells, foetal stem cells, and dental stem cells) and their technological field through the proposed tool in future studies. Utilization of technology diffusion models (e.g., Bass model or Logistic model) for the prediction of stem cell research might be also interesting to examine in conjunction with current trends and outcomes herein (Rao and Kishore 2010).