论文标题
混合回归的联邦学习的全球融合
Global Convergence of Federated Learning for Mixed Regression
论文作者
论文摘要
本文研究了客户表现出聚类结构时联合学习下模型培训的问题。我们将这个问题与混合回归中的情况相关化，在混合回归中，每个客户端的本地数据限制了从$ k $未知回归模型之一生成的本地数据。我们设计了一种从任何初始化中实现全局融合的算法，即使本地数据量量高度不平衡，也可能存在包含$ o（1）$数据点的客户端。我们的算法首先在一些锚点客户端（每个都有$ \tildeΩ（k）$ data Points）上运行Moment下降，以获得粗糙的模型估计。然后，每个客户端交替估计其群集标签，并根据FedAvg或FedProx来完善模型估计。我们分析中的一个关键创新是对聚类误差的统一估计，我们通过基于代数几何理论来界定一般多项式概念类别的VC维度。
This paper studies the problem of model training under Federated Learning when clients exhibit cluster structure. We contextualize this problem in mixed regression, where each client has limited local data generated from one of $k$ unknown regression models. We design an algorithm that achieves global convergence from any initialization, and works even when local data volume is highly unbalanced -- there could exist clients that contain $O(1)$ data points only. Our algorithm first runs moment descent on a few anchor clients (each with $\tildeΩ(k)$ data points) to obtain coarse model estimates. Then each client alternately estimates its cluster labels and refines the model estimates based on FedAvg or FedProx. A key innovation in our analysis is a uniform estimate on the clustering errors, which we prove by bounding the VC dimension of general polynomial concept classes based on the theory of algebraic geometry.
