Algorithmic Currency Trading Based on Reinforcement Learning Combining Action Shaping and Advantage Function Shaping

This paper investigates high frequency currency trading with neural networks trained via Reinforcement Learning. A neural network-based agent is proposed to learn the temporal pattern in data and automatically trades according to the current market condition and the historical data. We propose two techniques: action shaping and advantage function shaping, to improve the total profit. The action shaping is used to avoid the agent outputting illegal actions since we assume that the agent trades fixed position sizes in a single security. The advantage function shaping is proposed to increase the probability of actions that lead to more profit. The proposed system has been back-tested on the currency market. The results demonstrate that our method performs well in most conditions.