While SERS-based detection can bring some advantages, it is far away from being established as a routine method in clinical diagnostics. In this study, a SERS-labeled immunochromatographic test paper was prepared. The rapid detection of SARS-CoV-2 was realized by the machine learning algorithm of the Raman probe; the whole testing process takes less than 25 min, and the rapid detection of SARS-CoV-2 can be realized. After experimental evaluation, the sensitivity of the test strip for SARS-CoV-2 N protein detection can be 1 pg/mL, which is 3 orders of magnitude higher than that of the colloidal gold antigen detection strip on the market. In the detection of clinical samples, nucleic acid detection was used as the gold standard, and the accuracy was 84.21%.