^{1}

^{2}

^{1}

^{3}

^{4}

^{1}

^{2}

^{3}

^{4}

The complementary judgment matrix (CJM) method is an MCDA (multicriteria decision aiding) method based on pairwise comparisons. As in AHP, the decision-maker (DM) can specify his/her preferences using pairwise comparisons, both between different criteria and between different alternatives with respect to each criterion. The DM specifies his/her preferences by allocating two nonnegative comparison values so that their sum is 1. We measure and pinpoint possible inconsistency by

The complementary judgment matrix (CJM) method is an MCDA (multicriteria decision aiding) method based on hierarchical decomposition of the decision criteria into subcriteria, evaluation of preferences using pairwise comparisons, and aggregating the results into an overall evaluation of the alternatives. The earliest publications about the CJM method are by Lin and Xu [

Hierarchy structure of a school selection problem.

At each node of the hierarchy, the decision-maker (DM) performs pairwise comparisons between each pair of criteria or subcriteria. At the bottom level, the DM is asked to compare each pair of alternatives with respect to each criterion. Thus, the DM evaluates through pairwise comparisons both the relative importance of different criteria, and the performance of each alternative with respect to these criteria.

The CJM method differs from AHP mainly in five aspects:

Subjective comparison values are always more or less uncertain or imprecise. In particular in group decision-making, it may be difficult to represent the preferences of multiple DMs by precise comparison values. Fuzzy set theory has been employed to cope with the uncertainty and vagueness involved in conducting the comparisons between components of a decision model [

This paper is organized as follows. Section

At each node of the criteria hierarchy (see Figure

Any number of preference levels can be used in CJM, but to allow comparison with AHP, we use nine levels as in AHP. Table

Verbal preference statements and comparison values in CJM (

CJM | AHP | AHP→CJM | CJM→AHP | |
---|---|---|---|---|

Verbal statement | | | (%) | |

| 50 | 1 | 50.0 | 1 |

55 | 2 | 66.7 | 1.22 | |

| 60 | 3 | 75.0 | 1.5 |

65 | 4 | 80.0 | 1.86 | |

| 70 | 5 | 83.3 | 2.33 |

75 | 6 | 85.7 | 3 | |

| 80 | 7 | 87.5 | 4 |

85 | 8 | 88.9 | 5.67 | |

| 90 | 9 | 90.0 | 9 |

At each node of the hierarchy, the comparison values are organized into a

For comparison, the AHP comparison values

The complementary judgement matrix

Different techniques to solve the weights have been presented in literature. Here we present a technique that is a little simpler and computationally more efficient than the eigenvalue method of AHP. The eigenvalue method requires iterative calculation of the eigenvector while the LSQ solution is obtained in closed form. First, we solve _{11}) is trivially found. When

Observe that this method of solving the weights does not require a complete set of comparisons. A sufficient requirement is that the graph formed by pairwise comparisons between entities is connected. This gives great flexibility for the DM in large problems, where comparing every pair of entities would be too laborious.

LSQ solution of the weights is also applicable with multiple DMs who provide their (precise) comparisons independently. All comparisons are then collected into a common linear equation system (

After the weights have been computed at each node of the criteria hierarchy, a score

The redundant information provided by pairwise comparisons serves two purposes in the CJM method. Firstly, the weights solved from the overdetermined system (

Xu [

We suggest here a different technique for the CJM method. We simply compute the

Some restrictions of the basic CJM method are that it cannot treat imprecise information, and it does not explicitly support combining the preferences of multiple DMs. Some extensions of the CJM method exist for treating imprecise information as intervals [

The DMs can give their pairwise comparisons either as precise values or as intervals. The inconsistency errors are computed for each DM, and if they are too large, the DMs are allowed to revise their comparisons. In case of intervals, we suggest computing the inconsistency errors based on the midpoints of the intervals. We next combine the individual DMs’ pairwise comparisons into intervals

After representing the aggregated pairwise comparisons by suitable distributions, the performance of each alternative is analysed through stochastic simulation by drawing simultaneously pairwise comparisons from their corresponding distributions and computing the score for each alternative as in the CJM method. A sufficient number of simulation rounds is between 10 000 and 100 000 [

The

The first rank

The

The

The

To illustrate the SMAA-CJM method, we consider the AHP problem for evaluating 3 high schools (A, B, C) in terms of 6 criteria (One,…, Six) [

In the original AHP problem, the preferences were expressed verbally and mapped on the AHP scale (1, 2, …, 9). For CJM comparisons we use the uniform scale (50%, 55%, …, 90%) presented in Table

Pairwise CJM comparisons (%) between criteria (One, …, Six).

Criterion | Two | Three | Four | Five | Six |
---|---|---|---|---|---|

One | 70 | 80 | 70 | 60 | 50 |

Two | 60 | 30 | 25 | 25 | |

Three | 35 | 30 | 30 | ||

Four | 30 | 25 | |||

Five | 50 |

Pairwise CJM comparisons between alternatives with respect to different criteria.

One | B | C | Two | B | C |
---|---|---|---|---|---|

A | 40 | 45 | A | 50 | 50 |

B | 60 | B | 50 | ||

| |||||

Three | B | C | Four | B | C |

| |||||

A | 70 | 50 | A | 90 | 80 |

B | 30 | B | 30 | ||

| |||||

Five | B | C | Six | B | C |

| |||||

A | 45 | 50 | A | 75 | 65 |

B | 55 | B | 40 |

Solving the weights from the precise CJM comparisons gives the criterion scores, average weights and overall scores for alternatives shown in Table

CJM criterion scores, criterion weights, and overall scores for alternatives

Alt. | One | Two | Three | Four | Five | Six | Score |
---|---|---|---|---|---|---|---|

A | 0.27 | 0.33 | 0.41 | 0.74 | 0.31 | 0.53 | 0.41 |

B | 0.43 | 0.33 | 0.18 | 0.08 | 0.38 | 0.18 | 0.29 |

C | 0.30 | 0.33 | 0.41 | 0.18 | 0.31 | 0.28 | 0.30 |

Weights (%) | 26.9 | 8.3 | 7.8 | 10.7 | 21.6 | 24.7 |

AHP criterion scores, criterion weights, and overall scores for alternatives

Alt. | One | Two | Three | Four | Five | Six | Score |
---|---|---|---|---|---|---|---|

A | 0.16 | 0.33 | 0.45 | 0.77 | 0.25 | 0.69 | 0.40 |

B | 0.59 | 0.33 | 0.09 | 0.05 | 0.50 | 0.09 | 0.36 |

C | 0.25 | 0.33 | 0.45 | 0.17 | 0.25 | 0.23 | 0.24 |

Weights (%) | 33.0 | 4.7 | 3.3 | 9.2 | 22.6 | 27.3 |

Table

Next we evaluate the consistency of the CJM comparisons in terms of the inconsistency errors (

The IR for the comparisons between criteria is 0.02 (CI = 0.03, RI = 1.25) which is clearly below the suggested threshold 0.1 for sufficient consistency. Because IR is a kind of average measure for inconsistency, it is insensitive to a single inconsistent comparison and fails to detect the clearly inconsistent comparison. For related discussion, see Bana e Costa and Vansnick [

For the comparisons between alternatives (Table

The disadvantage with performing the full set of pairwise comparisons between each pair of entities (alternatives or criteria) is that the number of comparisons increases quadratically by the number of compared entities. With

The LSQ method for solving the weights in CJM works also with a subset of pairwise comparisons, provided that the graph formed by pairwise comparisons between entities is connected. This means that for each pair of entities A, B, they are either compared directly, or there exists a path of comparisons connecting A and B via other entities. The minimal sufficient number of comparisons is

As a compromise between the maximal and minimal number of comparisons, we suggest (for problems with many entities) comparing each entity systematically only with a small number of other entities. In the following, we suggest two methods for reducing the number of comparisons.

Before making comparisons, the DM should first order the entities according to their importance or preference. Saaty [

After ordering the entities, the DM compares each entity only with the two following entities. The necessary number of comparisons is then 2

After ordering the entities, the DM compares each entity only with the first and last entity. With 6 entities this method results in comparisons: 1&6, 1&2, 2&6, 1&3, 3&6, 1&4, 4&6, 1&5, 5&6. This method has the advantage that it reduces the DMs cognitive load in the comparisons because during the process he/she becomes ‘more familiar’ with the first and last entities, and at least one of them appears in every comparison.

We should point out that we are not suggesting any particular order in which the subset of comparisons are made. Bozóki et al. [

We illustrate smaller sets of comparisons using the school selection problem. After ordering the criteria into importance order, the full set of comparisons is shown in Figure

Pairwise CJM comparisons (%) between criteria in importance order and subsets 1 & 2.

The comparisons according to the first method appear on the bottom two diagonals and for the second method on the first row and last column of Figure

Criterion weights (%) and overall scores for alternatives using full set of comparisons, and subsets of comparisons.

Criterion weights | Overall scores | ||||||||
---|---|---|---|---|---|---|---|---|---|

Comparisons | One | Two | Three | Four | Five | Six | A | B | C |

Full set | 26.9 | 8.3 | 7.8 | 10.7 | 21.6 | 24.7 | 0.41 | 0.29 | 0.30 |

Subset 1 | 29.8 | 6.3 | 4.9 | 9.7 | 22.5 | 26.7 | 0.40 | 0.30 | 0.29 |

Subset 2 | 26.9 | 8.3 | 7.8 | 10.7 | 21.6 | 24.7 | 0.42 | 0.29 | 0.29 |

Next we introduce imprecision to the problem and analyse it using SMAA-CJM. We assume that the uncertainty of each comparison

Rank acceptability indices and pairwise winning indices with imprecise comparisons.

Rank acceptability indices

Pairwise winning indices

Alternative A with 99.97% first rank acceptability is in practice the only candidate for the first rank while alternatives B and C obtain only 0.06% and 0.01% acceptability for the first rank. However, the second rank acceptability of B and C is now 42% and 58%, which shows clearly that we cannot be sure about which alternative is the second best one. The same conclusions can be made from the pairwise winning indices

We demonstrate next how the SMAA-CJM method can be used when no comparison information among criteria (Table

The resulting criterion scores are identical with the previous analysis, because we have the same comparison and uncertainty information between alternatives. The uniform weight distribution results into average weights for each criterion equal to 1/6 ≈ 16.7%. The average overall scores for the alternatives A, B, C are 0.43, 0.26, and 0.30, correspondingly.

Figure

Rank acceptability indices and pairwise winning indices with missing comparisons between criteria.

Rank acceptability indices

Pairwise winning indices: School 1 versus 2

Table

Central weights and confidence factors for the alternatives (%).

Alt. | One | Two | Three | Four | Five | Six | |
---|---|---|---|---|---|---|---|

A | 15 | 16 | 17 | 18 | 16 | 18 | 100 |

B | 39 | 16 | 8 | 4 | 26 | 7 | 55 |

C | 19 | 23 | 29 | 4 | 19 | 7 | 19 |

The most significant differences between CJM and AHP results stem from the different scales used to represent verbal preference statements. Because the verbal preference statements carry only ordinal information and DMs have different interpretation of the intensities of the preference statements, no fixed numerical scale can properly represent the ordinal verbal comparisons. However, the integer scale of AHP from 1 to 9 is particularly problematic, because in many cases it is impossible to express consistent comparisons between three or more entities. For example, if criterion 1 is moderately more important than criterion 2 (_{12}=a_{23}=70% which correspond to _{13}=84% which is very close to scale value 85%.

To compare the two scales, we generated a large number of random

If A is preferred to B by intensity

Random ordinally consistent comparison matrices between

Table

Inconsistency ratios and inconsistency errors with AHP and CJM scales.

Inconsistency ratio (IR) | Inconsistency error | |||||
---|---|---|---|---|---|---|

Number of criteria | AHP | CJM | AHP/CJM ratio | AHP | CJM | AHP/CJM ratio |

3 | 0.18 | 0.062 | 2.9 | 0.090 | 0.049 | 1.8 |

4 | 0.16 | 0.059 | 2.8 | 0.17 | 0.098 | 1.7 |

5 | 0.17 | 0.064 | 2.6 | 0.22 | 0.13 | 1.6 |

6 | 0.17 | 0.063 | 2.7 | 0.25 | 0.16 | 1.5 |

7 | 0.17 | 0.061 | 2.8 | 0.27 | 0.19 | 1.5 |

8 | 0.17 | 0.063 | 2.7 | 0.30 | 0.21 | 1.4 |

In terms of the average inconsistency error the results are similar. The inconsistency error for each matrix is the maximal

We conclude that although not perfect, the more balanced CJM scale is clearly better than the AHP scale in its ability to represent the cardinal preferences of an ordinally consistent DM. Similar results were previously obtained by Pöyhönen et al. [

The comparison values of CJM have a natural interpretation. Considering only two criteria at a time, the DM can interpret the comparison values as trade-off weights that he/she assigns to the criteria. The DM can express these weights either as a normalized complementary pair (e.g., 0.8, 0.2) or as a pair of nonnegative numbers (e.g., 4, 1) that are normalized to satisfy the complementarity condition. Similarly, when comparing two alternatives with respect to a criterion, the DM is in effect distributing partial value between the two alternatives. Of course, it is also possible to evaluate the performance of alternatives through other techniques and to use pairwise comparisons only for assessing criteria weights. For example, criteria measured on natural scales can be normalized to partial values in range

No fixed cardinal scale can represent precisely the verbal preference statements of different DMs. Instead, each DM could define their own cardinal scale that represents his/her verbal statements. Alternatively, DMs could express their preferences cardinally in the first place. In practice this may be difficult for many DMs.

Another approach for cardinalizing ordinal preference statements is based on ordinal regression, as in UTA [

We have introduced the SMAA-CJM method for representing uncertain or imprecise information through stochastic distributions in the Complementary Judgment Matrix method and a simulation approach for analysing the resulting model. A particular strength of the method is that it allows flexible modelling of different kinds of imprecision, uncertainty, or even partially missing preference information. This is useful in decision processes, where the information is gradually refined during the process. The method is also suitable for group decision-making problems, where it is difficult for DMs to agree on precise pairwise comparisons. The method allows using distributions that include each DM’s preferences. Alternatively, the weight solution method of CJM can find weights that match different DMs’ preferences as well as possible in the LSQ sense.

We also introduced the

We conducted simulation experiments using a large number of different sized (3,…,8 criteria) ordinally consistent comparison matrices. The results showed that the balanced comparison scale of CJM results in more consistent results than the standard AHP scale. The consistency was better in terms of both the inconsistency ratio (IR) of AHP and the inconsistency error of SMAA-CJM. An earlier empirical study with students gave similar results.

The data used to support the findings of this study are available from the corresponding author upon request.

The authors declare that there are no conflicts of interest regarding the publication of this paper. The funds received did not lead to any conflicts of interest.

This work was supported by the China National Key Research and Development Program-China-Finland Intergovernmental Cooperation in Science and Technology Innovation (Funding no. 2016YFE0114500) and Academy of Finland Funding (Grant no. 299186). The authors would also like to acknowledge the ‘Xinghai’ Talent Project of Dalian University of Technology.