Amid growing threats of image data leakage and misuse, image encryption has become a critical safeguard for protecting visual information. However, many recent image encryption algorithms remain constrained by trade-offs between security, efficiency, and practicability. To address these challenges, this paper first proposes a novel two-dimensional variable fractional-order coupled quadratic hyperchaotic map (2D-VFCQHM), which incorporates a state-dependent dynamic memory effect, wherein the fractional-order is adaptively determined at each iteration by the mean of the system’s current state. This mechanism substantially enhances the complexity and unpredictability of the underlying chaotic dynamics. Building upon the superior hyperchaotic properties of the 2D-VFCQHM, we further develop a high-performance image encryption algorithm that integrates a novel fusion strategy within a dynamic vector-level diffusion-scrambling framework (IEA-VMFD). Comprehensive security analyses and experimental results demonstrate that the proposed algorithm achieves robust cryptographic performance, including a key space of 2298, inter-pixel correlation coefficients below 0.0018, ciphertext entropy greater than 7.999, and near-ideal plaintext sensitivity. Crucially, the algorithm attains an encryption speed of up to 126.2963 Mbps. The exceptional balance between security strength and computational efficiency underscores the practical viability of our algorithm, rendering it well-suited for modern applications such as telemedicine, instant messaging, and cloud computing.