Rapid developments of integrated circuits in electronics, aerospace, and aeronautics are always accompanied with the issue of heat diffusion, which impacts service efficiency and instrument life. Considering the weight, corrosion resistance, processing, and cost, carbon-based polymer composites are the most promising candidates for conductive materials; they have been widely investigated during the past few decades. The foremost objective of this article is to introduce the thermal conduction mechanism in the bulk polymer, crystalline particles, and carbon-based polymer composites, and review recent studies of carbon nanotube- or graphene-based interfacial thermal materials. The relative results and progress are summarized, and some thermal conductive parameters, such as filler structure, filler orientation, filler dispersion, and interfacial thermal resistance are comprehensively reviewed. An outlook of the future challenges of the thermal conductive materials is proposed at the end. This article provides highly accessible and comprehensive thermal conductive knowledge to researchers who are working on thermal conduction area.