Discrete Time Hedging in Frictional Markets: An Enhanced Rainbow Deep-Q Learning Approach

This paper explores how Rainbow DQN can be adapted to derive optimal hedging strategies for European options in discrete time and under market frictions. While classical theory has developed a rich framework to hedge options in complete markets and in arbitrage free environments, we consider environments in which none of these assumptions hold. We adopt a mean-variance utility framework which in turn involves optimizing a trade-off between mitigating transaction costs and hedging error and show how this optimization can be formulated under the reinforcement learning paradigm. We propose modifications to the conventional Rainbow DQN algorithm, such as the addition of a multi-armed bandit to identify the best trained agents. We then compare the performance of our trained agent to a benchmark strategy based off the Black-Scholes model. This comparison is conducted on simulated asset price data derived from a geometric Brownian motion environment as well as a stochastic volatility environment, with options of different maturities and levels of transaction costs. We find that our agent outperforms the benchmark under both simulation environments. This outperformance is strongest under the stochastic volatility setting for longer dated options and under high transaction costs wherein our agent reduces both the hedging costs by 30% and the variance of profit and loss by 8% compared to the benchmark. We also test the robustness of our agent on environments that different substantially from that used for training. In doing so we continue to notice outperformance, albeit at a slightly lower level. Our results show the promise of our enhanced Rainbow Deep-Q approach to generate successful hedging strategies for a broader range of financial derivatives.