In this paper, we present a new canonical replica exchange molecular dynamics (REMD) simulation method with normal pressure for all replicas (REMD-NV(p) T). This method is suitable for systems for which conventional constant NPT-setups are difficult to implement. In this implementation, each replica has an individual volume, with normal pressure maintained for each replica in the simulation. We derive a novel exchange term and validate this method on the structural properties of SPC/E water and dialanine (Ala2) in the bulk and in the presence of a graphene layer. Compared to conventional constant NPT-REMD and NVT-REMD simulations, we find that the structural properties of our new method are in good agreement with simulations in the NPT-ensemble at all temperatures. The structural properties of the systems considered are affected by high pressures at elevated temperatures in the constant NVT-ensemble, an effect that our method corrects for. Unprojected distributions reveal that essential motions of the peptide are affected by the presence of the barostat in the NPT implementation but that the dynamical eigenmodes of the NV(p)T method are in close quantitative agreement with the NVT-ensemble.