Partial differential equations (PDEs) are usually used to model complex electromagnetic systems, but solving them can be computationally expensive. Data-driven techniques have emerged as a promising solution due to their speed advantages in online tests, but they still face challenges related to training data quality and quantity, as well as generalization issues. To address these challenges, we present a Born Fourier neural operator (B-FNO) to solve generalized Poisson's equation. First, we utilize the Born approximation method to rapidly compute a rough solver. Second, we construct a Fourier neural operator (FNO) that approximates the mapping between the input Born approximation solver and the true solution, achieved by leveraging a residual learning structure. Importantly, we introduce conformal mapping tools to learning-based methods, allowing the trained B-FNO to be transferred across solving PDEs defined in different regions without requiring re-training. Extensive numerical examples based on benchmark datasets demonstrate that B-FNO learning scheme is effective in reducing the size of the training dataset and enhancing the generalizability to unseen scenarios compared to traditional methods. It is expected that the proposed B-FNO learning scheme will find its applications in data-driven electromagnetic solvers with the requirements of less training data and high generalization abilities.