The myth of scalable high performance

ral. Sparse matrix problems are parameterized by the dimensions of the matrix as well as by the number of nonzeros. Sorting problems may be parameterized by the range (and distribution) of the keys as well as by the number of items to be sorted. Thus, performance landscapes may sit in spaces with three or more independent variables. Agreeing on the cuts through such spaces that give appropriate scalability graphs may not be worth the effort. Assuming that the methodological problems could be overcome, the main advantage of scalability analysis is that it allows comparison of algorithms without regard to the ratio of communication cost to computation cost. If one algorithm is scalable and a second isn't then, for any fixed (or increasing) ratio, there is some number of processors beyond which the scalable algorithm is always better. However, from the point of view of the performance programmer [AC94], this information gives no insight into which algorithm one should use on any specific