Multi-attribute decision making: A simulation comparison of select methods

Several methods have been proposed for solving multi-attribute decision making problems (MADM). A major criticism of MADM is that different techniques may yield different results when applied to the same problem. The problem considered in this study consists of a decision matrix input of N criteria weights and ratings of L alternatives on each criterion. The comparative performance of some methods has been investigated in a few, mostly field, studies. In this simulation experiment we investigate the performance of eight methods: ELECTRE, TOPSIS, Multiplicative Exponential Weighting (MEW), Simple Additive Weighting (SAW), and four versions of AHP (original vs. geometric scale and right eigenvector vs. mean transformation solution). Simulation parameters are the number of alternatives, criteria and their distribution. The solutions are analyzed using twelve measures of similarity of performance. Similarities and differences in the behavior of these methods are investigated. Dissimilarities in weights produced by these methods become stronger in problems with few alternatives; however, the corresponding final rankings of the alternatives vary across methods more in problems with many alternatives. Although less significant, the distribution of criterion weights affects the methods differently. In general, all AHP versions behave similarly and closer to SAW than the other methods. ELECTRE is the least similar to SAW (except for closer matching the top-ranked alternative), followed by MEW. TOPSIS behaves closer to AHP and differently from ELECTRE and MEW, except for problems with few criteria. A similar rank-reversal experiment produced the following performance order of methods: SAW and MEW (best), followed by TOPSIS, AHPs and ELECTRE. It should be noted that the ELECTRE version used was adapted to the common MADM problem and therefore it did not take advantage of the method's capabilities in handling problems with ordinal or imprecise information.