Nature has been linked to various psychophysiological benefits for human restoration from long- and short-term stress. However, progressing urbanization and noise pollution threaten the restorative potential of recreational spaces. The question arises which characteristics of recreational spaces are particularly important for restoration. In this experimental VR study, which utilized 360° video and third-order ambisonics, 37 participants were immersed in 18 restoration environments with varying audio-visual characteristics in terms of visual setting (forest, lake, urban built space), soundscape (natural, anthropogenic), and sound pressure level (low, medium, high). Psychological ratings (acoustic quality, overall restorative potential) and physiological responses (changes in skin conductance level as a biomarker for stress) were assessed. The results simultaneously highlight the beneficial links of natural land- and soundscapes, and the adverse links of built environments and anthropogenic noise, with restoration: (1) The highest restorative potential was observed in scenarios where both auditory and visual features were natural (e.g., a forest or lake featuring birdsong and wind); (2) Introducing a built environment and/or an anthropogenic soundscape decreased restorative potential; (3) The lowest psychological and physiological restoration was observed in built spaces with anthropogenic soundscapes; and (4) Increasing sound pressure level was associated with lower restorative potential only for anthropogenic soundscapes, but not for natural ones. These findings underscore the detrimental associations of noise pollution, particularly with the restorative qualities of natural environments, and highlight the importance to provide access to recreational spaces with natural land- and soundscape features to promote population health.