We use first-principles density functional theory calculations to investigate the dielectric response of BaZrO$_3$ perovskite. A previous study [Arkbarzadeh {\em et al.} Phys. Rev. B {\bf 72}, 205104 (2005)] reported a disagreement between experimental and theoretical low temperature dielectric constant $\epsilon$ for the high symmetry BaZrO$_3$ structure. We show that a fully relaxed 40-atom BaZrO$_3$ structure exhibits O$_6$ octahedral tilting, and $\epsilon$ that agrees with experiment. The change in $\epsilon$ from high-symmetry to low-symmetry structure is due to increased phonon frequencies as well as decreased mode effective charges.