Table 9 Classification of selected studies, publication year and detailed quality assessment score

Sinha et al. (2006)

Solution proposal

Experiment

Yes

Yes

Yes

Q1

Ramasubbu and Balan (2012)

Solution proposal

Case study

Yes

Yes

Yes

CORE A*

Treude and Storey (2012)

Evaluation research

Case study

Yes

Yes

Yes

Q1

Niinimaki et al. (2010)

Evaluation research

Case study

Yes

Yes

Yes

Q2

Gorton et al. (1997b)

Solution proposal

Experiment

Yes

Yes

Yes

CORE A

Palacio et al. (2011)

Solution proposal

Experiment

Yes

Yes

Yes

Q3

Lanubile et al. (2010)

Review

Other

Yes

Yes

No

Q1

Portillo-Rodríguez et al. (2012)

Review

Other

Yes

Yes

No

Q1

Persson et al. (2009)

Solution proposal

Experiment

Yes

Yes

Yes

Q3

Bowen and Maurer (2002)

Solution proposal

Theory

Yes

Yes

No

CORE A*

Al-Ani et al. (2014)

Evaluation research

Survey

Yes

No

Yes

Q1

Samoilenko and Nahar (2012a)

Evaluation research

Theory

Yes

Yes

No

CORE A

Samoilenko and Nahar (2012b)

Evaluation research

Theory

Yes

Yes

No

CORE A

Scharff et al. (2010)

Evaluation research

Experiment

Yes

Yes

Yes

CORE C

Costa et al. (2010)

Review

Other

Yes

Yes

No

CORE A

Spanjers et al. (2006)

Evaluation research

Case study

Yes

Yes

Yes

CORE C

Goedicke et al. (2000)

Solution proposal

Theory

Yes

Yes

No

CORE A

Herring and Rees (2001)

Evaluation research

Case study

Yes

Yes

Yes

CORE C

Sakthivel (2005)

Other

Other

Yes

Yes

No

Q2

Dullemond and van Gameren (2012)

Solution proposal

Case study

Yes

Yes

Yes

CORE C

Prause et al. (2010)

Evaluation research

Survey

Yes

Yes

Yes

CORE C

Dullemond et al. (2010)

Solution proposal

Experiment

Yes

Yes

Yes

CORE C

Mullick et al. (2006)

Evaluation research

Experiment

Yes

Yes

Yes

CORE C

Niinimaki and Lassenius (2008)

Evaluation research

Case study

Yes

Yes

Yes

CORE C

Tell and Ali Babar (2011a)

Solution proposal

Theory

Yes

Yes

No

CORE A

Liukkunen et al. (2010)

Evaluation research

Case study

Yes

Yes

Yes

CORE C

Jaanu et al. (2012)

Evaluation research

Survey

Yes

Yes

Yes

CORE C

Gupta and Fernandez (2011)

Solution proposal

Survey

Yes

Yes

Yes

CORE C

Geisser et al. (2007)

Solution proposal

Experiment

Yes

Yes

Yes

CORE C

Winkler et al. (2010)

Evaluation research

Experiment

Yes

No

Yes

CORE A

Dullemond and van Gameren (2013)

Solution proposal

Survey

Yes

Yes

Yes

CORE C

Miyamoto et al. (2012)

Solution proposal

Experiment

Yes

Yes

Yes

No

Thissen et al. (2007)

Evaluation research

Case study

Yes

No

Yes

CORE B

Gorton et al. (1997a)

Solution proposal

Experiment

Yes

Yes

Yes

No

Lam and Maheshwari (2001)

Solution proposal

Theory

Yes

Yes

No

CORE B

Surjaputra and Maheshwari (1999)

Solution proposal

Theory

Yes

Yes

No

CORE B

Simmons and Ma (2006)

Solution proposal

Theory

Yes

Yes

No

CORE B

Prikladnicki et al. (2012)

Review

Other

Yes

No

No

Q1

Giuffrida and Dittrich (2011)

Evaluation research

Case study

Yes

Yes

Yes

No

Beecham et al. (2012)

Solution proposal

Experiment

Yes

Yes

Yes

No

Lanubile et al. (2013)

Evaluation research

Theory

Yes

No

No

Q1

Portillo-Rodríguez et al. (2014)

Review

Other

Partially

Yes

No

Q1

Eskeli et al. (2011)

Solution proposal

Theory

Yes

Yes

No

CORE C

Wesslin et al. (2011)

Other

Other

Partially

Yes

No

CORE A

Portillo-Rodríguez et al. (2010)

Review

Other

Yes

Yes

No

CORE C

Monasor et al. (2010b)

Solution proposal

Theory

Yes

Yes

No

CORE C

Lamersdorf and Munch (2009)

Solution proposal

Theory

Yes

Yes

No

CORE C

Martignoni (2009)

Review

Other

Yes

Yes

No

CORE C

Sinha et al. (2007)

Solution proposal

Theory

Yes

Yes

No

CORE C

Cook et al. (2005)

Solution proposal

Experiment

Yes

No

Yes

CORE C

Giuffrida and Dittrich (2013)

Review

Other

Yes

No

No

Q2

Cataldo et al. (2009)

Solution proposal

Theory

Yes

Yes

No

CORE C

Mak and Kruchten (2006)

Solution proposal

Theory

Yes

Yes

No

CORE C

Simmons (2003)

Solution proposal

Theory

Yes

Yes

No

CORE C

Clear (2009)

Solution proposal

Experiment

Yes

No

Yes

CORE C

Vathsavayi et al. (2014)

Solution proposal

Theory

Yes

Yes

No

CORE C

Costa and Murta (2013)

Review

Other

Yes

Yes

No

CORE C

Chubov and Droujkov (2007)

Other

Other

Partially

No

Yes

CORE B

Monasor et al. (2010a)

Solution proposal

Theory

Yes

Yes

No

No

de Souza and Fonseca (2007)

Solution proposal

Theory

Yes

Yes

No

No

Garrido et al. (2012)

Solution proposal

Theory

Yes

Yes

No

No

Pesola et al. (2011)

Solution proposal

Theory

Yes

Yes

No

No

Aranda et al. (2011)

Solution proposal

Theory

Yes

Yes

No

No

Wu (2012)

Evaluation research

Theory

Yes

Yes

No

No

Jiménez et al. (2009)

Review

Other

Partially

No

No

Q2

Dullemond et al. (2009)

Other

Other

Yes

No

No

CORE C

Aranda et al. (2008)

Other

Other

Yes

No

No

CORE C

da Silva et al. (2010)

Review

Other

Yes

No

No

CORE C

Aranda et al. (2006)

Other

Other

Yes

No

No

CORE C

Murdoch and Astley (2004)

Evaluation research

Theory

Partially

No

No

CORE B

Salger et al. (2010)

Other

Other

Partially

No

No

CORE C

Paulish (2006)

Other

Other

Partially

No

No

CORE C

Ali et al. (2010)

Other

Other

Partially

No

No

CORE C

van Hillegersberg and Herrera (2007)

Other

Other

Partially

No

No

No

Wang et al. (2012)

Review

Other

Partially

No

No

No

Tell and Ali Babar (2011b)

Solution proposal

Theory

Partially

No

No

No