Selecting targets to attack and assigning weapons are among the most critical decisions on the battlefield. The decision problem is represented as a dynamic weapon-target assignment (DWTA) problem. While deep reinforcement learning (DRL) is the state-of-the-art approach for DWTA, previous studies have limitations in three key aspects: 1) representing topological relationships on the battlefield; 2) scalability to increased problem sizes; and 3) performance metric relevance. To overcome these limitations, this study aims to solve the DWTA problem by leveraging DRL and graph neural networks (GNNs), with a novel partially observable Markov decision process (POMDP) design, including graph-based action representation, observation features, and reward design. Experiments are conducted across multiple military domains, including naval and ground combat, comparing the proposed approach with existing heuristic and meta-heuristic methodologies. The effectiveness of the GNN and decision-making pattern is extensively analyzed through comprehensive experimental validation.