Accurate prediction of protein-ligand binding affinity is vital for accelerating drug discovery, yet challenges persist in efficiently capturing critical structural and chemical interactions efficiently. This study introduces amino acid pair positional encoding (AAPPE), a deep learning framework that integrates spatial relationships between amino acids in protein pockets and ligand molecular fingerprints. By encoding pairwise distances of position-determining atoms (selected based on biological relevance) and discretizing them into fixed positional ranges, AAPPE constructs a 3124-dimensional feature set that avoids dependency on ligand binding poses. Evaluated on the CASF-2016 benchmark, the model achieved robust predictive performance (MAE = 0.99, RMSE = 1.28, and R = 0.82), with ablation studies confirming the importance of biologically informed atom selection. The method's computational efficiency and pose-free design provide a practical tool for prioritizing essential interactions in protein-ligand complexes, offering a generalizable and interpretable approach to advance structure-based drug design.