Network function virtualization (NFV) and software-defined network (SDN) have become emerging network paradigms, allowing virtualized network function (VNF) deployment at a low cost. Even though VNF deployment can be flexible, it is still challenging to optimize VNF deployment due to its high complexity. Several studies have approached the task as dynamic programming, e.g., integer linear programming (ILP). However, optimizing VNF deployment for highly complex networks remains a challenge. Alternatively, reinforcement learning (RL) based approaches have been proposed to optimize this task, especially to employ a scaling action-based method which can deploy VNFs within less computational time. However, the model architecture can be improved further to generalize to the different networking settings. In this paper, we propose an enhanced model which can be adapted to more general network settings. We adopt the improved GNN architecture and a few techniques to obtain a better node representation for the VNF deployment task. Furthermore, we apply a recently proposed RL method, phasic policy gradient (PPG), to leverage the shared representation of the service function chain (SFC) generation model from the value function. We evaluate the proposed method in various scenarios, achieving a better QoS with minimum resource utilization compared to the previous methods. Finally, as a qualitative evaluation, we analyze our proposed encoder's representation for the nodes, which shows a more disentangled representation.

翻译：网络功能虚拟化(NFV)和软件定义网络(SDN)已经成为新出现的网络模式,使得虚拟化网络功能(VNF)能够以低成本部署。尽管VNF的部署可以灵活,但由于其高度复杂,优化VNF的部署仍具有挑战性。一些研究将这项任务作为动态编程,例如整线编程(ILP)处理。然而,为高度复杂的网络优化VNF的部署仍是一个挑战。或者,为了优化这项工作,提出了基于强化学习(RL)的方法,特别是采用基于行动的方法,在较少计算时间内部署VNFS。然而,模型结构可以进一步改进,以便推广到不同的网络设置。在本文件中,我们提出了一个可以适应更一般网络设置的强化模式。我们采用了改进的GNNFS架构和一些技术,以便为VNFF的部署任务获得更好的节点代表。此外,我们最近提出的RL方法,即基于政策梯度(PPGGG),以利用服务链的共享代表制式(SC)模式,从以前的价值函数中实现更好的生成模式。我们最后评估了一种最起码的方法。我们用的方法来比较评估。