Enhancing Optomechanical Entanglement and Mechanical Squeezing by the Synergistic Effect of Quadratic Optomechanical Coupling and Coherent Feedback

ABSTRACT In this paper, we investigate how to achieve strong optomechanical entanglement and mechanical squeezing in a membrane‐embedded cavity optomechanical system incorporating a coherent feedback loop, where the membrane interacts with the cavity mode through both linear and quadratic optomechanical couplings. This hybrid optomechanical architecture offers a flexible tunability of intrinsic system parameters, thereby enabling controlled stiffening or softening of the mechanical mode through adjusting quadratic optomechanical coupling, as well as effective modulation of the cavity decay rate via feedback control. More importantly, the synergistic interplay effect allows for a strategic reconfiguration of the system's stability regime, which in turn permits the presence of significantly enhanced effective optomechanical coupling strengths before entering the unstable regime. Exploiting these unique features, we showcase that optomechanical entanglement can be substantially enhanced with positive coupling sign and suitable feedback parameters, while strong mechanical squeezing beyond the 3dB limit is simultaneously achieved over a broad parameter range with negative coupling sign, reaching squeezing degree above 10dB under optimized conditions. Our proposal, establishing an all‐optical method for generating highly entangled or squeezed states in cavity optomechanical systems, opens up a new route to explore macroscopic quantum effects and to advance quantum information processing.