MEAT: Median-Ensemble Adversarial Coaching for Bettering Robustness and Generalization
Authors: Zhaozhe Hu, Jia-Li Yin, Bin Chen, Luojun Lin, Bo-Hao Chen, Ximeng Liu
Summary: Self-ensemble adversarial coaching strategies enhance mannequin robustness by ensembling fashions at totally different coaching epochs, akin to mannequin weight averaging (WA). Nevertheless, earlier analysis has proven that self-ensemble protection strategies in adversarial coaching (AT) nonetheless endure from sturdy overfitting, which severely impacts the generalization efficiency. Empirically, within the late phases of coaching, the AT turns into extra overfitting to the extent that the people for weight averaging additionally endure from overfitting and produce anomalous weight values, which causes the self-ensemble mannequin to proceed to bear sturdy overfitting because of the failure in eradicating the burden anomalies. To unravel this drawback, we intention to sort out the affect of outliers within the weight area on this work and suggest an easy-to-operate and efficient Median-Ensemble Adversarial Coaching (MEAT) methodology to resolve the sturdy overfitting phenomenon current in self-ensemble protection from the supply by looking for the median of the historic mannequin weights. Experimental outcomes present that MEAT achieves one of the best robustness in opposition to the highly effective AutoAttack and may successfully allievate the sturdy overfitting. We additional reveal that almost all protection strategies can enhance sturdy generalization and robustness by combining with MEAT
