Wide-angle X-ray diffraction is a crucial technique for probing the nanoscale texture and strain gradient of nanofiber-based composite materials, particularly in determining the 3D orientation distribution of crystalline nanofiber networks. However, extracting 3D orientation information of nanofibers from diffraction patterns remains a significant challenge, especially when dealing with diffraction patterns resulting from multiple fiber sets. Here we introduce Restrfcn, an end-to-end framework which integrates a transformer encoder with a fully connected network through residual connection. We demonstrate its capability in extracting fiber orientation parameters even when the number of nanofiber sets is a variable. To eliminate ineffective neurons in the network, which can simplify the architecture and enhance the model's fitting performance, the Restrfcn model is optimized by using a statistical hypothesis testing method. The deployment of Restrfcn has significant potential for providing real-time data analysis in high-throughput and multi-dimensional synchrotron diffraction experiments.