Concerning the problem of modulation identification in the Rayleigh channel,a new algorithm based on cumulants was proposed.The method was efficient and could easily classify seven kinds of signals of BPSK(Binary Phase Shift Keying),QPSK(Quadrature Phase Shift Keying),4ASK(4-ary Amplitude Shift Keying),16QAM(16-ary Quadrature Amplitude Modulation),32QAM(32-ary Quadrature Amplitude Modulation),64QAM(64-ary Quadrature Amplitude Modulation) and OFDM(Orthogonal Frequency Division Multiplexing) by using the decision tree classifier and the feature parameters that were extracted from combination of four-order cumulant and six-order cumulant.Through theoretical derivation and analysis,the algorithm is insensitive to Rayleigh fading and AWGN(Additive White Gaussian Noise).The computer simulation results show that the successful rates are over 90% when SNR(Signal-to-Noise Ratio) is higher than 4 dB in Rayleigh channel,which demonstrates the feasibility and effectiveness of the proposed algorithm.