Table 5 Performance measures (%) for different feature sets and different classification techniques
Feature set | Reputation s | Base learners | |||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
C4.5 | NN | SVM | NB | ||||||||||
Precision | Recall | F-measure | Precision | Recall | F-measure | Precision | Recall | F-measure | Precision | Recall | F-measure | ||
(a) Segmenting type s = like | |||||||||||||
 F1 | Good | 63.89 | 46.00 | 53.49 | 56.76 | 63.00 | 59.72 | 64.38 | 47.00 | 54.34 | 64.38 | 47.00 | 54.34 |
Bad | 57.81 | 74.00 | 64.91 | 58.43 | 52.00 | 55.03 | 58.27 | 74.00 | 65.20 | 58.27 | 74.00 | 65.20 | |
 F1 + F2 | Good | 60.94 | 39.00 | 47.56 | 54.55 | 54.00 | 54.27 | 63.08 | 41.00 | 49.70 | 59.38 | 19.00 | 28.79 |
Bad | 55.15 | 75.00 | 63.56 | 54.46 | 55.00 | 54.73 | 56.30 | 76.00 | 64.68 | 51.79 | 87.00 | 64.93 | |
 F1 + F2 + F3 | Good | 60.94 | 39.00 | 47.56 | 54.64 | 53.00 | 53.81 | 63.08 | 41.00 | 49.70 | 58.06 | 18.00 | 27.48 |
Bad | 55.15 | 75.00 | 63.56 | 54.37 | 56.00 | 55.17 | 56.30 | 76.00 | 64.68 | 51.48 | 87.00 | 64.68 | |
 F1 + F2 + F3 + F4 | Good | 65.38 | 51.00 | 57.30 | 69.05 | 58.00 | 63.04 | 77.94 | 53.00 | 63.10 | 83.33 | 50.00 | 62.50 |
Bad | 59.84 | 73.00 | 65.77 | 63.79 | 74.00 | 68.52 | 64.39 | 85.00 | 73.28 | 64.29 | 81.00 | 71.68 | |
(b) Segmenting type s = dislike | |||||||||||||
 F1 | Good | 58.62 | 34.00 | 43.04 | 67.50 | 27.00 | 38.57 | 49.33 | 37.00 | 42.29 | 63.41 | 26.00 | 36.88 |
Bad | 53.52 | 76.00 | 62.81 | 54.38 | 87.00 | 66.92 | 49.60 | 62.00 | 55.11 | 53.97 | 68.00 | 60.18 | |
 F1 + F2 | Good | 59.34 | 54.00 | 56.54 | 64.06 | 41.00 | 50.00 | 59.74 | 46.00 | 51.98 | 64.58 | 31.00 | 41.89 |
Bad | 57.80 | 63.00 | 60.29 | 56.62 | 77.00 | 65.25 | 56.10 | 69.00 | 61.88 | 56.67 | 17.00 | 26.15 | |
 F1 + F2 + F3 | Good | 59.34 | 54.00 | 56.54 | 64.06 | 41.00 | 50.00 | 59.74 | 46.00 | 51.98 | 64.58 | 31.00 | 41.89 |
Bad | 57.80 | 63.00 | 60.29 | 56.62 | 77.00 | 65.25 | 56.10 | 69.00 | 61.88 | 56.67 | 17.00 | 26.15 | |
 F1 + F2 + F3 + F4 | Good | 70.00 | 63.00 | 66.32 | 73.63 | 67.00 | 70.16 | 77.91 | 67.00 | 72.04 | 73.63 | 67.00 | 70.16 |
Bad | 66.36 | 73.00 | 69.52 | 69.72 | 76.00 | 72.73 | 71.05 | 81.00 | 75.70 | 77.11 | 64.00 | 69.95 | |
(c) Segmenting type s = sum | |||||||||||||
 F1 | Good | 65.33 | 49.00 | 56.00 | 57.61 | 53.00 | 55.21 | 61.64 | 45.00 | 52.02 | 95.24 | 20.00 | 33.06 |
Bad | 59.20 | 74.00 | 65.78 | 56.48 | 61.00 | 58.65 | 56.69 | 72.00 | 63.44 | 55.06 | 98.00 | 70.50 | |
 F1 + F2 | Good | 64.71 | 55.00 | 59.46 | 58.95 | 56.00 | 57.44 | 69.14 | 56.00 | 61.88 | 75.47 | 40.00 | 52.29 |
Bad | 60.87 | 70.00 | 65.12 | 58.10 | 61.00 | 59.51 | 63.03 | 75.00 | 68.49 | 59.18 | 87.00 | 70.45 | |
 F1 + F2 + F3 | Good | 64.71 | 55.00 | 59.46 | 60.64 | 57.00 | 58.76 | 69.14 | 56.00 | 61.88 | 75.47 | 40.00 | 52.29 |
Bad | 60.87 | 70.00 | 65.12 | 59.43 | 63.00 | 61.17 | 63.03 | 75.00 | 68.49 | 59.18 | 87.00 | 70.45 | |
 F1 + F2 + F3 + F4 | Good | 69.12 | 47.00 | 55.95 | 58.62 | 51.00 | 54.55 | 70.11 | 61.00 | 65.24 | 75.68 | 56.00 | 64.37 |
Bad | 59.85 | 79.00 | 68.10 | 56.64 | 64.00 | 60.09 | 65.49 | 74.00 | 69.48 | 65.08 | 82.00 | 72.57 | |
(d) Segmenting type s = portfolio | |||||||||||||
 F1 | Good | 60.00 | 57.00 | 58.46 | 59.62 | 62.00 | 60.78 | 61.11 | 55.00 | 57.89 | 54.93 | 78.00 | 64.46 |
Bad | 59.05 | 62.00 | 60.49 | 60.42 | 58.00 | 59.18 | 59.09 | 65.00 | 61.90 | 62.07 | 36.00 | 45.57 | |
 F1 + F2 | Good | 65.66 | 65.00 | 65.33 | 62.63 | 62.00 | 62.31 | 63.37 | 64.00 | 63.68 | 56.94 | 82.00 | 67.21 |
Bad | 65.35 | 66.00 | 65.67 | 62.38 | 63.00 | 62.69 | 63.64 | 63.00 | 63.32 | 67.86 | 38.00 | 48.72 | |
 F1 + F2 + F3 | Good | 65.66 | 65.00 | 65.33 | 56.03 | 65.00 | 60.19 | 63.37 | 64.00 | 63.68 | 56.94 | 82.00 | 67.21 |
Bad | 65.35 | 66.00 | 65.67 | 58.33 | 49.00 | 53.26 | 63.64 | 63.00 | 63.32 | 67.86 | 38.00 | 48.72 | |
 F1 + F2 + F3 + F4 | Good | 62.83 | 71.00 | 66.67 | 69.16 | 74.00 | 71.50 | 69.91 | 79.00 | 74.18 | 76.77 | 76.00 | 76.38 |
Bad | 66.67 | 58.00 | 62.03 | 72.04 | 67.00 | 69.43 | 75.86 | 66.00 | 70.59 | 76.24 | 77.00 | 76.62 | |
Feature set | Reputation s | Ensemble learning methods | |||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
RS-C4.5 | RS-NN | RS-SVM | RS-NB | ||||||||||
Precision | Recall | F-measure | Precision | Recall | F-measure | Precision | Recall | F-measure | Precision | Recall | F-measure | ||
(a) Segmenting type s = like | |||||||||||||
 F1 | Good | 66.28 | 57.00 | 61.29 | 64.36 | 65.00 | 64.68 | 67.42 | 60.00 | 63.49 | 66.67 | 58.00 | 62.03 |
Bad | 62.28 | 71.00 | 66.36 | 64.65 | 64.00 | 64.32 | 63.96 | 71.00 | 67.30 | 62.83 | 71.00 | 66.67 | |
 F1 + F2 | Good | 70.45 | 62.00 | 65.96 | 71.76 | 61.00 | 65.95 | 71.26 | 62.00 | 66.31 | 88.00 | 22.00 | 35.20 |
Bad | 66.07 | 74.00 | 69.81 | 66.09 | 76.00 | 70.70 | 66.37 | 75.00 | 70.42 | 55.43 | 97.00 | 70.55 | |
 F1 + F2 + F3 | Good | 70.45 | 62.00 | 65.96 | 71.43 | 65.00 | 68.06 | 71.91 | 64.00 | 67.72 | 86.96 | 20.00 | 32.52 |
Bad | 66.07 | 74.00 | 69.81 | 67.89 | 74.00 | 70.81 | 67.57 | 75.00 | 71.09 | 54.80 | 97.00 | 70.04 | |
 F1 + F2 + F3 + F4 | Good | 81.40 | 70.00 | 75.27 | 83.72 | 72.00 | 77.42 | 96.74 | 89.00 | 92.71 | 96.34 | 79.00 | 86.81 |
Bad | 73.68 | 84.00 | 78.50 | 75.44 | 86.00 | 80.37 | 89.81 | 97.00 | 93.27 | 82.57 | 90.00 | 86.12 | |
(b) Segmenting type s = dislike | |||||||||||||
 F1 | Good | 54.76 | 69.00 | 61.06 | 52.26 | 81.00 | 63.53 | 52.25 | 93.00 | 66.91 | 71.88 | 23.00 | 34.85 |
Bad | 58.11 | 43.00 | 49.43 | 57.78 | 26.00 | 35.86 | 68.18 | 15.00 | 24.59 | 60.71 | 34.00 | 43.59 | |
 F1 + F2 | Good | 62.96 | 68.00 | 65.38 | 63.93 | 78.00 | 70.27 | 61.83 | 81.00 | 70.13 | 71.62 | 53.00 | 60.92 |
Bad | 65.22 | 60.00 | 62.50 | 71.79 | 56.00 | 62.92 | 72.46 | 50.00 | 59.17 | 87.80 | 36.00 | 51.06 | |
 F1 + F2 + F3 | Good | 63.11 | 65.00 | 64.04 | 66.09 | 76.00 | 70.70 | 61.54 | 80.00 | 69.57 | 71.62 | 53.00 | 60.92 |
Bad | 63.92 | 62.00 | 62.94 | 71.76 | 61.00 | 65.95 | 71.43 | 50.00 | 58.82 | 87.80 | 36.00 | 51.06 | |
 F1 + F2 + F3 + F4 | Good | 88.30 | 83.00 | 85.57 | 98.88 | 88.00 | 93.12 | 94.90 | 93.00 | 93.94 | 86.79 | 92.00 | 89.32 |
Bad | 83.96 | 89.00 | 86.41 | 89.19 | 99.00 | 93.84 | 93.14 | 95.00 | 94.06 | 100.00 | 79.00 | 88.27 | |
(c) Segmenting type s = sum | |||||||||||||
 F1 | Good | 73.68 | 70.00 | 71.79 | 69.15 | 65.00 | 67.01 | 66.67 | 52.00 | 58.43 | 82.93 | 34.00 | 48.23 |
Bad | 71.43 | 75.00 | 73.17 | 66.98 | 71.00 | 68.93 | 60.66 | 74.00 | 66.67 | 58.49 | 93.00 | 71.81 | |
 F1 + F2 | Good | 78.16 | 68.00 | 72.73 | 63.77 | 44.00 | 52.07 | 73.91 | 68.00 | 70.83 | 77.22 | 61.00 | 68.16 |
Bad | 71.68 | 81.00 | 76.06 | 57.25 | 75.00 | 64.94 | 70.37 | 76.00 | 73.08 | 67.77 | 82.00 | 74.21 | |
 F1 + F2 + F3 | Good | 75.58 | 65.00 | 69.89 | 64.06 | 41.00 | 50.00 | 73.03 | 65.00 | 68.78 | 77.22 | 61.00 | 68.16 |
Bad | 69.30 | 79.00 | 73.83 | 56.62 | 77.00 | 65.25 | 68.47 | 76.00 | 72.04 | 67.77 | 82.00 | 74.21 | |
 F1 + F2 + F3 + F4 | Good | 82.72 | 67.00 | 74.03 | 50.00 | 100.00 | 66.67 | 81.55 | 84.00 | 82.76 | 83.87 | 78.00 | 80.83 |
Bad | 72.27 | 86.00 | 78.54 | 0.00 | 0.00 | 0.00 | 83.51 | 81.00 | 82.23 | 80.20 | 81.00 | 80.60 | |
(d) Segmenting type s = portfolio | |||||||||||||
 F1 | Good | 58.82 | 60.00 | 59.41 | 57.29 | 55.00 | 56.12 | 60.95 | 64.00 | 62.44 | 65.09 | 69.00 | 66.99 |
Bad | 59.18 | 58.00 | 58.59 | 56.73 | 59.00 | 57.84 | 62.11 | 59.00 | 60.51 | 67.02 | 63.00 | 64.95 | |
 F1 + F2 | Good | 70.09 | 75.00 | 72.46 | 64.15 | 68.00 | 66.02 | 76.19 | 80.00 | 78.05 | 71.43 | 80.00 | 75.47 |
Bad | 73.12 | 68.00 | 70.47 | 65.96 | 62.00 | 63.92 | 78.95 | 75.00 | 76.92 | 77.27 | 68.00 | 72.34 | |
 F1 + F2 + F3 | Good | 71.56 | 78.00 | 74.64 | 69.70 | 69.00 | 69.35 | 71.13 | 69.00 | 70.05 | 71.93 | 82.00 | 76.64 |
Bad | 75.82 | 69.00 | 72.25 | 69.31 | 70.00 | 69.65 | 69.90 | 72.00 | 70.94 | 79.07 | 68.00 | 73.12 | |
 F1 + F2 + F3 + F4 | Good | 77.78 | 84.00 | 80.77 | 84.26 | 91.00 | 87.50 | 94.06 | 95.00 | 94.53 | 85.98 | 92.00 | 88.89 |
Bad | 82.61 | 76.00 | 79.17 | 90.22 | 83.00 | 86.46 | 94.95 | 94.00 | 94.47 | 91.40 | 85.00 | 88.08 | |