论文标题
在数字双网络中卸载随机计算的深度强化学习
Deep Reinforcement Learning for Stochastic Computation Offloading in Digital Twin Networks
论文作者
论文摘要
工业互联网（IIT）的快速发展需要工业生产，以提高网络效率。 Digital Twin是一项有前途的技术，可以通过创建物理对象的虚拟模型来增强IIOT的数字转换。但是，由于资源受限的设备，随机任务和资源异质性，IIOT的网络效率非常具有挑战性。 IIT网络中的分布式资源可以通过计算卸载有效利用，以减少能源消耗，同时提高数据处理效率。在本文中，我们首先提出了一个新的范式数字双网络（DTN），以在IIOT系统中构建网络拓扑和随机任务到达模型。然后，我们制定随机计算的卸载和资源分配问题，以最大程度地降低长期能源效率。由于公式的问题是一个随机编程问题，因此我们利用Lyapunov优化技术将原始问题转换为确定性的每次插槽问题。最后，我们提出异步参与者 - 批评（AAC）算法，以找到最佳的随机计算卸载策略。说明性结果表明，我们提出的方案能够显着优于基准。
The rapid development of Industrial Internet of Things (IIoT) requires industrial production towards digitalization to improve network efficiency. Digital Twin is a promising technology to empower the digital transformation of IIoT by creating virtual models of physical objects. However, the provision of network efficiency in IIoT is very challenging due to resource-constrained devices, stochastic tasks, and resources heterogeneity. Distributed resources in IIoT networks can be efficiently exploited through computation offloading to reduce energy consumption while enhancing data processing efficiency. In this paper, we first propose a new paradigm Digital Twin Networks (DTN) to build network topology and the stochastic task arrival model in IIoT systems. Then, we formulate the stochastic computation offloading and resource allocation problem to minimize the long-term energy efficiency. As the formulated problem is a stochastic programming problem, we leverage Lyapunov optimization technique to transform the original problem into a deterministic per-time slot problem. Finally, we present Asynchronous Actor-Critic (AAC) algorithm to find the optimal stochastic computation offloading policy. Illustrative results demonstrate that our proposed scheme is able to significantly outperforms the benchmarks.
