Mitigating Disparate Influence of Differential Privateness in Federated Studying via Sturdy Clustering
Authors: Saber Malekmohammadi, Afaf Taik, Golnoosh Farnadi
Summary: Federated Studying (FL) is a decentralized machine studying (ML) strategy that retains information localized and sometimes incorporates Differential Privateness (DP) to boost privateness ensures. Just like earlier work on DP in ML, we noticed that differentially personal federated studying (DPFL) introduces efficiency disparities, significantly affecting minority teams. Current work has tried to handle efficiency equity in vanilla FL via clustering, however this technique stays delicate and vulnerable to errors, that are additional exacerbated by the DP noise in DPFL. To fill this hole, on this paper, we suggest a novel clustered DPFL algorithm designed to successfully determine shoppers’ clusters in extremely heterogeneous settings whereas sustaining excessive accuracy with DP ensures. To this finish, we suggest to cluster shoppers primarily based on each their mannequin updates and coaching loss values. Our proposed strategy additionally addresses the server’s uncertainties in clustering shoppers’ mannequin updates by using bigger batch sizes together with Gaussian Combination Mannequin (GMM) to alleviate the influence of noise and potential clustering errors, particularly in privacy-sensitive eventualities. We offer theoretical evaluation of the effectiveness of our proposed strategy. We additionally extensively consider our strategy throughout various information distributions and privateness budgets and present its effectiveness in mitigating the disparate influence of DP in FL settings with a small computational price
