Reducing simulation time is critical for applications such as closed loop control or iterativedesign op- timisation. In this context, model reduction techniques have become agrowing area of research in the last decades. While research efforts have mainly beencentered around feature based approaches like POD, BPOD or DMD, direct approachesleveraging Deep Neural Networks have been proposed in recent years with great success.Despite these promising results, neural network architectures provide little to no physicalguarantees, and have limited interpretability. On the other hand, feature based methodsoften reconstruct the final solution through a linear combination of modes embedded withphysical constraints. However, this often comes at the cost of loss of information andincreased error rates.POD-Galerkin models are a perfect example of this trade-off between physical guaranteesand performance loss. These models have been shown to be very efficient for the reductionof linear systems, but they are extremely limited when applied to nonlinear systems suchas the Navier-Stokes equations. For example, Noack et al. have shown that a simple 3equations model was able to capture the oscillatory dynamics of a flow over a cylinder,but failed to correctly predict the transition time and trajectory from a steady point ofthe system to its oscillatory regime.To address these shortcomings, we propose to add a closure term to POD-Galerkin modelsto correct their dynamics. Observing that the information lost during the projection onthe POD basis can be retrieved by considering the past states of the system, we usesimple neural networks in combination with delay differential equations to reconstructthe required correction. We show that a satisfactory model can be trained through theNeural ODE framework to learn a memory based correction from simulation data. Thefinal architecture can be compared to a time-continuous recurrent neural network.With this approach, we preserve the simple structure and low computational cost ofGalerkin models while improving their performance. Using the 3 modes model examplefrom [4], we show that the corrected ROM reproduces perfectly the original transitiontrajectory, and generalises well to unseen initial conditions. On-going work is concernedwith validating and improving the proposed approach by applying it on the more challengingchaotic pinball case.