Paired Comparison Analysis

A paired comparison analysis is used to rate or rank options where evaluation criteria are subjective. The analysis is particularly useful when there is a lack of clear priorities or objective data on which to base decisions. A paired comparison analysis evaluates a range of options by comparing them against each other.[1]

Paired comparison is often used to choose the most compelling problem to solve or to select the alternative that will be the most effective. It is helpful in many applications, from selecting the concept design for a new product before it goes into production to deciding the skills and qualifications when hiring people for a new position. Decisions like these are more difficult than comparing which investment to take or which vendor to select (we will select the vendor who will give us the highest quality with the best price and the fastest delivery).

Paired comparison analysis is often performed with the aid of a matrix. This matrix should be made in a way that avoids comparing an option with itself or duplicating any comparison. Two additional rows can be added to the end of the table to represent:

  • The number of times each option has been selected.
  • The ranking of all options based on their count.[2]

Steps in Paired Comparison Analysis

To apply Paired Comparison Method, it’s wise to use a large sheet of paper or a flip chart. Follow the steps below one by one for the analysis to work best.

  • Step 1: Creating table: Make a table with rows and columns and fill out the options that will be compared to one another in the first row and the first column (the headers of the rows and columns). The empty cells will stay empty for now. If there are 4 options, there are 4 rows and 4 columns and 16 cells; when there are three options, you get three rows and three columns and 9 cells, etcetera.
  • Step 2: Assigning letters: Every option is assigned a letter (A, B, C, etc.). The options are mentioned in the headers of the rows and columns, and each now has a letter so the options can be properly compared to each other.
  • Step 3: Blocking cells: It’s important to block out the cells in the table in which the same options overlap. Cells that contain a comparison that has been displayed earlier in the table also have to be blocked out. Every comparison should only be made once.
  • Step 4: Comparing options: The cells that are left will now compare the options in the rows to the options in the columns. The letter of the most important option will be noted. For example, when A is compared to C and C is a more important option, a C will be written down in that cell.
  • Step 5: Rating options: The difference in importance will now get a rating that will range, for example, from 0 (no difference) to 3 (an important difference).
  • Step 6: Listing results: The results are now consolidated by adding all values for each option in question. If necessary, these totals can be converted to percentages.[3]

Advantages of Paired Comparison Analysis

Paired Comparison analysis has some specific advantages. The advantages are,

  • It is easy to calculate.
  • Also, It can be used when the priorities are not clear.
  • It is especially found helpful when you have no objective data to depend upon.

Applying this tool, we can easily identify the best option.[4]

Limitations to the paired comparison analysis

While the paired comparison analysis removes subjectivity from decision-making, it does not incorporate statistical inferences. In other words, teams have no means of discovering whether the differences between option pairs are statistically significant.

Inevitably, there will also be disagreement on which of the compared options is more important. Since there are only two choices, the paired comparison analysis leaves little room for interpretation in the degree of importance for team members. This has the potential to stifle collaboration and compromise the integrity of the results.[5]

See Also