标题： 基于Transformer的深度强化学习与Black-Litterman模型结合的组合优化方法 显示英文标题

标题： Combining Transformer based Deep Reinforcement Learning with Black-Litterman Model for Portfolio Optimization

摘要： 作为一种无模型算法，深度强化学习（DRL）智能体通过与环境进行无监督交互来学习和做出决策。 近年来，学者们已将DRL算法广泛应用于连续交易时期的组合优化，因为DRL智能体可以动态适应市场变化，并且不依赖于资产间联合动态的规范。 然而，典型的用于组合优化的DRL智能体无法学习到一种能够感知组合资产收益之间动态相关性的策略。 由于组合资产之间的动态相关性在优化组合中至关重要，缺乏这种知识使得DRL智能体难以在单位风险下最大化收益，尤其是在允许做空的市场（即美国股票市场）中。 在本研究中，我们提出了一种混合组合优化模型，结合DRL智能体和Black-Litterman（BL）模型，使DRL智能体能够学习组合资产收益之间的动态相关性，并基于相关性实施有效的多空策略。 本质上，DRL智能体被训练以学习应用BL模型来确定目标组合权重的策略。 为了测试我们的DRL智能体，我们基于道琼斯工业平均指数的所有构成股票构建了组合。 在真实世界美国股票市场数据上进行的实验结果表明，我们的DRL智能体在累计收益方面至少比各种比较组合选择策略和其他DRL框架高出42%。 在单位风险收益方面，我们的DRL智能体显著优于各种比较组合选择策略和其他基于其他机器学习框架的替代策略。 显示英文摘要

摘要： As a model-free algorithm, deep reinforcement learning (DRL) agent learns and makes decisions by interacting with the environment in an unsupervised way. In recent years, DRL algorithms have been widely applied by scholars for portfolio optimization in consecutive trading periods, since the DRL agent can dynamically adapt to market changes and does not rely on the specification of the joint dynamics across the assets. However, typical DRL agents for portfolio optimization cannot learn a policy that is aware of the dynamic correlation between portfolio asset returns. Since the dynamic correlations among portfolio assets are crucial in optimizing the portfolio, the lack of such knowledge makes it difficult for the DRL agent to maximize the return per unit of risk, especially when the target market permits short selling (i.e., the US stock market). In this research, we propose a hybrid portfolio optimization model combining the DRL agent and the Black-Litterman (BL) model to enable the DRL agent to learn the dynamic correlation between the portfolio asset returns and implement an efficacious long/short strategy based on the correlation. Essentially, the DRL agent is trained to learn the policy to apply the BL model to determine the target portfolio weights. To test our DRL agent, we construct the portfolio based on all the Dow Jones Industrial Average constitute stocks. Empirical results of the experiments conducted on real-world United States stock market data demonstrate that our DRL agent significantly outperforms various comparison portfolio choice strategies and alternative DRL frameworks by at least 42% in terms of accumulated return. In terms of the return per unit of risk, our DRL agent significantly outperforms various comparative portfolio choice strategies and alternative strategies based on other machine learning frameworks.