Influence of water matrix on the removal of compounds of emerging concern (CECs) by nanofiltration (NF) has not been clearly delineated. In this study, rejection of eight CECs in different water matrices by NF is investigated. Two additives (humic acid and alginate) were added as natural organic matters (NOMs) to Milli-Q water to emulate raw river water and lake water. The results showed that an intermediate NOM concentration had better rejections of CECs than lower or higher concentrations in the levels of concentrations studied here. When a water matrix contained no NOM, CECs that carried negative charges gained better rejection than those that carried neutral or positive charge at pH of 7, which implies that the major mechanism of rejection is electrostatic repulsion. As the NOM concentrations in the water matrix got relatively higher, adsorption became a predominant mechanism for rejection; under these circumstances, CECs which had higher octanol–water distribution coefficient (kow) could be adsorbed on NOMs and removed with them. The influence of NOM concentration on CEC rejection was considered and used to modify a quantitative structure–activity relationship (QSAR) model. The predicted values of the modified QSAR model were closer to the actual experimental data than those of the original model.