FALCON: FPGA Accelerated Real-Time Intelligent Controller for Autonomous Systems

The growing complexity and stringent real-time demands of autonomous systems, such as self-driving cars and drones, have driven the adoption of intelligent control methods based on deep neural networks (DNNs). While these methods offer improved control performance over traditional modelbased approaches, they also pose significant computational challenges, particularly for resource-constrained platforms. FieldProgrammable Gate Arrays (FPGAs) offer an attractive solution due to their energy efficiency and customizable architecture. In this work, we propose FALCON, an innovative approach for designing real-time intelligent controllers for autonomous systems using FPGA accelerators. Our approach begins with designing DNN-based intelligent controllers with varying levels of complexity and accuracy on the FPGA platform. Then, a performance function is proposed to capture the interplay among controller complexity, computational behavior, physical system characteristics, and overall control performance. Based on this performance function, we develop an algorithm-hardware codesign framework to determine the optimal control complexity, hardware configuration, and resource allocation. Finally, a case study on the co-design of intelligent controllers and FPGAbased overlay processors, together with a hardware-in-the-loop simulator, is conducted to demonstrate the advantages of the proposed methods. Compared to benchmarking controllers on other platforms, FALCON's optimized intelligent controller using FPGA accelerators shows competitive control performance with superior real-time capability and power efficiency. FALCON's optimization reduces the worst-case response time (WCRT) by up to 46.52%, improves the control performance by $1.93 \times$ compared to the default setup. For performance per power efficiency, FALCON achieves a $3.67 \times$ improvement compared to the DNN intelligent controller on TX2 and a remarkable $30.78 \times$ improvement compared to traditional MPC on CPU.