Iron (Fe) is the most important ferromagnetic element, not only for its high magnetic moment and high Curie temperature but for its abundance as well. Fe-based magnetic materials are therefore widely applied in technologies and industries, with most of the applications for soft magnetic materials, because of the low magnetocrystalline anisotropy (MCA) of Fe. However, it is possible to realize magnetic hardening in Fe-based materials as we have learned from the early carbon steel permanent magnets although their coercivity was modest. Recent efforts to search for rare-earth-free hard magnetic materials have shown more promising evidences for achieving high MCA in Fe-based materials. In this paper, we review the history and the recent developments of Fe-based hard and semi-hard magnetic materials with a focus on mechanisms of high MCA in Fe-based phases and the related crystal and electronic structures. We have tabulated and discussed the structures and the magnetic properties of the Fe-based binary or ternary systems containing p-block and d-block elements, with many of them showing considerable MCA. Furthermore, it is important to know and to understand that the MCA in Fe-based magnetic materials can be tailored/enhanced through chemical and/or structural modifications that will lead to “artificially engineered” hard and semi-hard magnetic materials for advanced permanent magnets in the future.