The structure of HPC is changing since the growing applications of GPGPU,this change points out a new direction for HPC development.CUDA provided by NIVIDIA is a programming environment using C language for developing parallel computing applications.The efficiency of compute system can be improved by using CUDA to speed large scale and extensive parallel computing on specified graphic cards.This essay mainly introduces the situation of the development of GPU and how to using CUDA to developing parallel computing applications.