Page 140 - JOURNAL OF LIBRARY SCIENCE IN CHINA 2018 Vol. 42

OU Shiyan, TANG Zhengui & SU Feifei / Construction and usage of terminology services for information retrieval 139

Terminology
Goal A Service Usage
Mode(A)

Effectiveness Efficiency User satisfaction Learnability
Criteria B
(B1) (B2) (B3) (B4)
Representation Improvement Freedom of Operation Clicking
Sub-Criteria Bi degree of retrieval degree of retr- term selec-
requirement(B11) ieval results(B12) tion(B13) time(B21) times(B22)

Mode 1˖Multiple Mode 2˖Single Mode 3˖Hybrid of Mode 4˖Automa-
Alternatives C choice for query choice for query query expansion and tic query expansion
expansion (C1) replacement (C2) replacement(C3) (C4)
Figure 13. AHP hierarchy for terminology service usage mode selecting decision.

results have no much difference (Gao, 2007). Here we used the latent root method to calculate
the weight of each evaluation criterion (shown from Table 4 to Table 6). During calculating the
weights, a consistency ratio, which expresses the internal consistency of the judgments, was also
checked. In the above three decision matrixes, all the consistency ratios are less than 0.1 and thus
acceptable. According to the calculated weights, we obtained the usability evaluation index system
as shown in Table 7. Among these seven criteria, except for operation time and clicking times, the
other five criteria were judged subjectively based on 5-point Likert Scale. While calculating the
total usability score, to keep the measures of all the criteria consistent, we discretized the observed
values of operation time and clicking times to convert them into 5-point scales.

Table 3. The fundamental importance for pairwise comparisons
Intensity of 1 3 5 7 9 2、4、6、8
importance
Equal Moderate Strong Very strong Extreme Intermediate
Definition
importance importance importance importance importance values
Note：If the importance ratio between the element i and j is aij, then the importance ratio between the element j and i is
aji=1/aij

Table 4. Judgements in the decision matrix of the top criteria
A B1 B2 B3 B4 Weight (%)

B1 1 2 1/2 2 26.3
B2 1/2 1 1/3 1 14.1
B3 2 3 1 3 45.5

B4 1/2 1 1/3 1 14.1
Note：Consistency Ratio (CR) =0.4 %< 0.1