The effect of particle emission of the intermediate giant dipole resonance on the excitation of the double giant dipole resonance in heavy ion reactions is calculated. General assessment of the time scales involved in such excitation and subsequent decay is given. Both the direct as well as the nondirect, Brink-Axel, paths are considered.