ABSTRACT Motivated by the fractional order multilevel decompositions of finite element spaces developed previously, we exploit additive representations of popular multigrid (MG) cycles to design fractional order MG decompositions. The additive representations enable us to scale the individual hierarchical components, thus ending up with fractional order hierarchical decompositions that are based on the readily available MG components. This results in a highly efficient and scalable (in terms of high‐performance) fractional order hierarchical MG decompositions that we tested in the setting of finite element white noise sampling as an alternative to PDE‐based white noise sampling using fractional order shifted Laplacians.