The quadrupole-quadrupole interaction between a pair of gravitationally polarizable objects induced by vacuum fluctuations of the quantum linearized gravitational field is first obtained with a relatively simple method, which is then used to investigate the contribution of thermal fluctuations of a bath of gravitons to the interaction at temperature $T$. Our result shows that, in the high-temperature limit, the contribution of thermal fluctuations dominates over that of vacuum fluctuations and the interaction potential behaves like $T/{r}^{10}$, where $r$ is the separation between the objects, and in the low-temperature limit, the contribution of thermal fluctuations is proportional to ${T}^{10}/r$, which only provides a small correction to the interaction induced by zero-point fluctuations.