In the marine environment, the use of visible light communication to achieve accurate positioning of marine surveillance equipment is becoming increasingly common. For the problem of photodetector (PD) selection in marine visible light localization based on time difference of arrival (TDOA), this study proposed the joint angle and range information of PD selection (JARS) algorithm. First, using the equivalence of the Cramer–Rao lower bound (CRLB) of the TDOA and time of arrival (TOA) measurement models, the PD selection problem is transformed into an optimization problem aiming to minimize CRLB. In order to solve the problem of local convergence and high computational complexity in traditional methods, the JARS algorithm combines angle and distance information to avoid semi-definite relaxation (SDR) ambiguity, and reduces the error effect by weight optimization. The simulation results show that the performance of the JARS algorithm is close to that of the exhaustive search (ES) algorithm with low computational complexity, and it shows good robustness in practical applications. In addition, combined with Chan-Taylor (CT) localization algorithm, the JARS algorithm can further improve the localization accuracy and verify its effectiveness in practical applications, including oceanographic research and underwater navigation.