A fast calculation method for computer-generated holograms for hidden surface removal is proposed. In this method, a three-dimensional object is considered as a set of point light sources emitting light rays. To achieve the hidden surface removal, only appropriate light rays are selected according to their geometrical position, which are then converted into a Fourier spectrum of the wavefront. After the Fourier spectrum on the spherical surface is obtained, diffraction in arbitrary directions is calculated. Numerical simulation of a series of diffracted wavefronts onto arbitrary observation planes has been demonstrated to verify the effectiveness of our proposal.