In recent years, machine learning (ML) has been increasingly applied to the processing of optical signals, but traditional ML algorithms that rely solely on data processing require large datasets due to their inability to understand optical signals. An optical fiber sensing system based on the prior knowledge of physics is proposed to demodulate complex spectrum with a limited sample size. By extracting the projection of the domain of the optical path difference (OPD) by introducing both real and imaginary parts, the model realizes the mapping establishment of the interference principle and overcomes the disadvantage of the traditional model that the required number of datasets is too enormous. In addition, the performance of the model can be further improved by refactoring it to the parallel neural network (PNN) with synthesized information. A torsion sensor with a complex spectrum is proposed to verify the demodulation ability of the model for multimode complex spectra. The model can realize the physics-based data augmentation and compression of the demodulation model. It enables the demodulation for the complex spectrum of fiber sensors with nonlinear and nonmonotonic responses by fewer training samples. Furthermore, the simultaneous demodulation of magnitude and orientation is achieved in combination with structural improvements based on spatial misalignment. The proposed model provides a new idea for the ML model employed in the fiber sensing system and has a high application prospect in the field of optical sensing and the deployment of the Internet of Things (IoT).