This paper introduces a third-order magnetic flux-controlled memristor into a four-wing hyperchaotic system, constructing a novel five-dimensional four-wing memristive hyperchaotic system. Systematic study of its dynamic characteristics via equilibrium point analysis, Lyapunov exponent spectra, bifurcation diagrams, phase portraits, 0–1 test, multistability analysis, Poincaré maps, and two-parameter bifurcation diagrams shows that the system exhibits periodic, quasi-periodic, chaotic, and hyperchaotic states, with a four-wing chaotic attractor and rich dynamical behaviors. Owing to three unstable equilibrium points, it has significant multistability and generates coexisting chaotic and periodic attractors under different initial conditions. The secure communication scheme based on this system features more complex keys, better encryption effects, and greater difficulty in tracing the original signal. Using the Lorenz system’s time series as the original signal, encrypted transmission and decryption via chaos masking technology achieve undistorted signals with stable transmission performance. Additionally, FPGA-based hardware implementation of the system, synchronization control, and secure communication system is completed. State observer synchronization control realizes drive-response system synchronization, verifying encrypted transmission feasibility. Random sequence tests indicate the system’s sequences pass all 15 SP800-22 tests, with better randomness than Lorenz and Chen systems, confirming its application advantages in secure communication.