The non-Markovian effect on entropy production rate is an open question in quantum thermodynamics. By means of the change rate of the trace distance as the witness, we explore the non-Markovianities for their origin, i.e., from the initial correlation between the system and its heat reservoir and/or from the spectral density of the system-reservoir interaction. We show the possibility of positive entropy production rate regarding the initial correlation, which challenges the existing viewpoint of definitely negative entropy production rate under the non-Markovian condition. Our result is justified by an experimentally feasible model.