The paper addresses the solution of non-adhesive and adhesive normal contact problems between a rigid smooth indenter of arbitrary shape and a linear viscoelastic material with the boundary element method. After summarizing existing non-adhesive methods in a general form, a new method is presented based on the convolution of the second derivative in time of pressure for the computation of the induced displacements, which yields more accurate results. Any configuration of the viscoelastic solid where a relationship between applied pressure and resulting displacements can be formulated in the Fourier and Laplace space (half-space, layered material, graded material, etc., all with arbitrary relaxation functions) can be solved with the method presented. For the treatment of the adhesive contact problem detachment criteria are formulated which apply to the limiting cases of fast indentation and retraction.