Who is an expert? A robust coefficient for expertise quantification adopting web scraping and weighting methods

The Decision-Making (DM) process involves the continuous evaluation of multiple interconnected variables, including objectives, available information, resources, alternatives, risks, and impacts, to determine the optimal decision. Within this framework, expert judgments serve as a crucial source of information, significantly influencing final decisions. However, the selection and engagement of experts are typically based on subjective criteria, such as facilitators’ evaluations or self-assessment, both of which introduce cognitive biases that can distort the assessment of expertise. A key challenge, therefore, lies in establishing a reliable and objective method to quantify expertise. To address this limitation, this paper introduces a novel approach to expertise quantification utilizing web scraping and developing a robust coefficient, defined as the Expertise Reliability Coefficient (ERC). The ERC is designed to minimize subjectivity by integrating bibliometric indicators, including h-index, citation counts, and the frequency of relevant publications, reports, and policies. This method enables a more rigorous evaluation of experts, enhancing the reliability of their selection and assessment within the decision-making process. Statistical models, including the Mann–Whitney and Kendall tests, are employed to analyze discrepancies between the self-assessed expertise and the ERC. The findings reveal systematic cognitive biases, demonstrating that experts often overestimate their abilities, while less competent individuals exhibit an even greater degree of overestimation compared to their more knowledgeable counterparts, thus confirming cognitive biases. By providing a structured and quantifiable approach to expertise evaluation, the ERC enhances the integrity of expert-driven DM, offering a more objective alternative to conventional self-assessment methods.