Testing One Hypothesis Multiple Times

The identification of new rare signals in data, the detection of a sudden change in a trend, and the selection between competing models, are among the most challenging problems in statistical practice. In these settings, standard regularity conditions (e.g., those required by Wilks theorem) fail to hold and thus classical inferential tools, such as the generalized likelihood ratio test, are not applicable. In this thesis, we show how these challenges can be tackled using a test of hypothesis where a nuisance parameter is present only under the alternative. Several solutions have been proposed in the statistical literature and their practical implementation often reduces the problem into one of Testing One Hypothesis Multiple times (TOHM). Specifically, a fine discretization of the space of the non-identifiable parameter is specified, and an ensemble of sub-test statistics is obtained to test the null hypothesis against a set of sub-alternative hypothesis where the value of the non-identifiable parameter is fixed at each point of the discretization. The goal is to provide a global p-value as the standard of evidence for comparing the null hypothesis and the alternative hypothesis. In this thesis, we combine elements of extreme value theory, differential geometry, graph theory and simulations methods to provide an easy to compute, computationally efficient and highly generalizable inferential tool to perform TOHM under stringent significance requirements, such as those typically required in the physical sciences. The methods proposed in this thesis formalize and extend recent results presented in physics literature. Several applications are discussed in the context of indirect searches for dark matter.