Table 1 Related works on customization and evaluation of NLIDBs
From: Comparative study on the customization of natural language interfaces to databases
NLIDB | Customized by | Complexity of the DB | Complexity of the query corpora | Comparison versus other NLIDBs | Performance |
|---|---|---|---|---|---|
Masque/sql | – | – | – | – | – |
Precise | The implementers | ATIS: high; Mooney‘s dataset Geobase, Restbase and Jobdata DBs: low | ATIS: high; Mooney‘s dataset: moderate | AT&T, CMU, MIT, SRI, BBN, UNISYS, MITRE, HEY (on ATIS). EQ, Mooney (on Mooney’s dataset) | Accuracy: 93.8 % (on ATIS). Recall: 80 %, accuracy: 100 %, (on Mooney’s dataset) |
NLPQC | Presumably the DBA | Library of the Concordia | Low | – | – |
CLEF | The implementers | University: low moderate | High | – | Recall: 100 % |
DaNaLIX | The implementers | Geobase, Jobdata: low | Geoquery880, Jobquery640: moderate | COCKTAIL, GENLEX, NaLIX | Recall: ≈ 81 % |
C-PHRASE | Undergraduate students | Geobase: low | Geoquery880: moderate | Precise, WASP, SCISSOR, Z&C | Recall: ≈ 75 %, accuracy: ≈ 86 % |
Giordani and Moschitti (2012) | The implementers | Geobase: low | Geoquery500, Geoquery700: moderate | Precise, Krisp Model III+R, SemResp, UBL, DCS/DCS+ | F1*: 87 % |
ELF, Conlon et al. (2004) | An expert | High | Unknown | – | Recall: 70-80 % |