This paper presents a design method for diffractive optical system based on particle swarm optimization (PSO). This method uses the radius of the diffuse spot formed by light rays with different apertures and wavelengths on the image plane as the objective function. Using multiple iterations, the optimal initial structural parameters satisfying the optical system parameters can be quickly calculated. In order to verify the design method, we designed a refractive-diffractive hybrid optical system and a reflective-diffractive hybrid optical system. Simulations and experiments demonstrate the feasibility and potential of the method in initial structural design of diffractive optical systems.