The discovery of lead compounds and novel mechanisms of action has always been two key issues in the field of pesticides research, which provide an effective way to addressing pest control. Isochamaejasmin (ICM) is a biflavonoid compound of plant origin with insecticidal activity, possessing high potential for development of new pest control agents. Although ICM has demonstrated strong insecticidal activity, its mechanism of action is still unclear. This study explored the effects of ICM on Spodoptera frugiperda (Sf9) cells to elucidate its mode of action at the cellular level. The results demonstrated that ICM has potential toxicity against S. frugiperda both in vivo and in vitro via a concentration- and time-dependent manner. RNA-Seq analysis revealed substantial differential gene expression that indicating ICM disrupts mitochondrial structure and function and activates mitochondria-mediated programmed cell death pathways. Subsequent biochemical assays at the cellular level confirmed these results, demonstrating that ICM exposure significantly reduced the activities of antioxidant enzymes (SOD and CAT), increased MDA levels, and stimulated ROS generation. Further analysis indicated that excessive ROS impaired mitochondrial function, which was accompanied by alterations in organelle structure and activity. Additionally, ICM induced apoptosis in Sf9 cells, characterized by cyt-c release, upregulated Bax and caspase-9/3 expression, and downregulated Bcl-2 levels. Western blot and morphological analyses further revealed that ICM activated autophagy due to mitochondrial damage, as evidenced by elevated expression of autophagy-related proteins (LC3 I/II, Beclin-1, and P62) and typical characteristics (autophagic vesicles and autolysosomes). These findings collectively indicate that ICM induces oxidative stress, resulting in mitochondria damage, which subsequently activates a co-regulation pathway mediated by mitochondria, leading to apoptosis and autophagy, and ultimately inhibiting the proliferation of S. frugiperda cells.