With the rapid development of intelligent manufacturing, human–robot collaboration (HRC) has emerged as a crucial work paradigm that combines human flexibility with robotic efficiency. In complex assembly tasks, traditional single-role sequence generation methods and predefined task orders are often inadequate for dynamic human–robot interaction environments due to process complexity and inherent uncertainty. Task-sequence planning for HRC therefore remains a critical challenge. To overcome these limitations, this study presents a HRC assembly task-sequence planning approach that integrates a dynamic assembly knowledge graph with multi-agent reinforcement learning (MARL). The proposed framework employs scene-graph generation to construct a temporal human–robot assembly knowledge graph that captures the spatial and temporal relationships within the workspace. Based on this graph, the MARL algorithm performs task allocation and sequence optimization by considering role states and task attributes. Experiments conducted on an electric gripper assembly validate the feasibility and effectiveness of the proposed approach. Compared with conventional graph search methods, the proposed method demonstrates improved adaptability and problem-solving efficiency in dynamic assembly scenarios.