The Rayleigh scattering intensity in fluorine-doped silica glass was found to decrease very effectively with lowering frozen-in temperature of the glass, so-called the fictive temperature, while such reduction becomes less in samples with higher fluorine concentrations. The present result implies that the Rayleigh scattering in fluorine-doped silica glass is affected by concentration fluctuation besides density fluctuation. From the result, we established a simulated annealing effect of the glass on reduction of the Rayleigh scattering intensity.