Peach firmness is a critical quality attribute, yet conventional destructive measurement methods are unsuitable for batch detection in industrial settings. This study investigated a noncontact method for firmness assessment across multiple peach cultivars based on acoustic vibration technology. Three peach cultivars were mechanically excited via a controlled air jet, and the resulting acoustic vibration responses were captured noninvasively using a laser Doppler vibrometer. The frequency-domain acoustic vibration spectra were used as input for firmness prediction models developed using partial least squares regression (PLSR), support vector regression (SVR), and a one-dimensional convolutional neural network (ISNet-1D) that incorporated Inception and squeeze-and-excitation modules. Comparative analysis demonstrated that the ISNet-1D substantially outperformed the conventional linear and nonlinear methods on an independent test set, achieving superior predictive accuracy, with a coefficient of determination ( RP2) of 0.8069, a root mean square error (RMSEP) of 0.9206 N/mm, and a residual prediction deviation ( RPDP) of 2.2879. The good performance of the ISNet-1D can be attributed to the integration of multi-scale convolutional filters with a channel-wise attention mechanism. This integration allows the network to adaptively prioritize discriminative spectral features, thereby enhancing its prediction accuracy. A hierarchical transfer learning strategy was proposed to improve model generalizability, offering a practical and cost-effective means to adapt to diverse cultivars. In summary, the combination of noncontact acoustic vibration and deep learning presents a robust, accurate, and nondestructive methodology for assessing peach firmness, demonstrating considerable potential for cross-cultivar application in industrial sorting and quality control.