We propose improving the privacy properties of a dataset by publishing only a\nstrategically chosen "core-set" of the data containing a subset of the\ninstances. The core-set allows strong performance on primary tasks, but forces\npoor performance on unwanted tasks. We give methods for both linear models and\nneural networks and demonstrate their efficacy on data.\n