Nonradiative decay of surface plasmons (SPs) is usually considered an unwanted process. However, recent studies have proven that hot carriers generated from nonradiative SP decay can be used for photodetection that circumvents the bandgap limitation in semiconductors. The major problem plaguing the plasmonic hot carrier photodetectors stems from the low quantum efficiency. In this review, we discuss recent progress of engineering plasmonic hot carrier dynamics and describe a host of plasmon-enhanced photodetectors, including optical antenna-based photodetectors, planar photodetectors, photodetectors coupled with 2D materials, functionalized photodetectors, photodetectors for integrated nanophotonics, and hot-hole photodetectors. Finally, we herein highlight some new directions in the plasmonic photodetection.