Publications

2026

1. ICML

   Generalization Bounds for Out-of-distribution Generalization

   Xin Zou, Xiuwen Gong, and Weiwei Liu

   In ICML, 2026

   Abs HTML

   Out-of-distribution (OOD) generalization has attracted increasing research attention in recent years, owing to its promising empirical results in real-world applications. However, theoretical studies on OOD generalization remain limited, particularly with respect to lower bounds on the generalization error. To better understand how source data contributes to improved OOD generalization performance, we take an initial step toward establishing a lower bound on the OOD generalization error, and subsequently investigate upper bounds from the perspective of statistical learning theory. Interestingly, we find that under some conditions, simply minimizing the average empirical risk over the source domains can yield a nearly optimal error rate (up to a logarithmic factor) without requiring knowledge or estimation of distributional parameters or the discrepancy between source and target domains. This finding offers an explanation for the surprising phenomenon observed in DomainBed, where carefully designed OOD generalization algorithms fail to outperform the simple empirical risk minimization (ERM) algorithm. Our results also imply a no-free-lunch theorem and provide an optimistic bound for OOD generalization.

2. ICML

   On Learnability and Disambiguation of Multiclass Partial Concept Classes

   Jingyuan Xu, Xin Zou, Xiuwen Gong, and Weiwei Liu

   In ICML, 2026

   Abs HTML

   We study the Probably Approximately Correct (PAC) learnability of partial concept classes in the multiclass setting, where the label space can be infinite. While the Natarajan dimension characterizes learnability for finite label spaces, we show it fails when the label space is unbounded. Instead, we prove that the Daniely-Shalev (DS) dimension provides a characterization of learnability for partial concept classes in the general multiclass setting. Furthermore, our analysis reveals a surprising phenomenon we call the “Disambiguation Paradox”: disambiguation schemes with simple label space can destroy learnability, while richer labeling may preserves it. We further characterize how the number and structure of disambiguation labels control the induced DS dimension, yielding a trade-off between label complexity and sample complexity.

3. ICLR

   Neural Collapse in Multi-Task Learning

   Youjun Wang, Boqi Li, Xin Zou, and Weiwei Liu

   In ICLR, 2026

   Abs HTML PDF

   Neural collapse (NC) plays a key role in understanding deep neural networks. However, existing empirical and theoretical studies of NC focus on one single task. This paper studies neural collapse in multi-task learning. We consider two standard feature-based multi-task learning scenarios: Single-Source Multi-Task Classification (SSMTC) and Multi-Source Multi-Task Classification (MSMTC). Interestingly, we find that the task-specific linear classifier and features converge to the Simplex Equiangular Tight Frame (ETF) in the setting of MSMTC. In the setting of SSMTC, task-specific linear classifier converges to the task-specific ETF and these task-specific ETFs are mutually orthogonal. Moreover, the shared features across tasks converge to the scaled sum of the weight vectors associated with the task-specific labels in each task’s classifier. We also provide the theoretical guarantee for our empirical findings. Through detailed analysis, we uncover the mechanism of MTL where each task learns task-specific latent features that together form the shared features. Moreover, we reveal an inductive bias in MTL that task correlation reconfigures the geometry of task-specific classifiers and promotes alignment among the features learned by each task.

2025

1. TPAMI

   A Unified Decision Rule for Generalized Out-of-Distribution Detection

   Xinsong Ma, Jie Wu, Xin Zou, and Weiwei Liu

   IEEE Transactions on Pattern Analysis and Machine Intelligence, 2025

   Abs HTML PDF

   Generalized Out-of-distribution (OOD) detection task plays the key role in reliable and safety-critical applications. Existing researches mainly devote to designing or training the powerful score function but overlook investigating the decision rule based on the proposed score function. Different from previous work, this paper aims to design a decision rule with rigorous theoretical guarantee and well empirical performance. Specifically, we provide a new insight for the OOD detection task from a hypothesis testing perspective and propose a novel generalized Benjamini Hochberg (g-BH) procedure to solve the testing problem. Teoretically, the g-BH procedure controls false discovery rate (FDR) under pre-specified level without the consideration of dependence for the p-values. Furthermore, we derive an upper bound and a lower bound of the expectation of false positive rate (FPR) for the g-BH procedure based on the tailed generalized Gaussian distribution family, indicating that the FPR of g-BH procedure converges to zero in probability. Finally, the extensive experimental results verify the superiority of g-BH procedure over the traditional threshold-based decision rule on several generalized OOD detection benchmarks.

2. ICML

   An Online Statistical Framework for Out-of-Distribution Detection

   Xinsong Ma, Xin Zou, and Weiwei Liu

   In ICML, 2025

   Abs HTML PDF

   Out-of-distribution (OOD) detection task is significant in reliable and safety-critical applications. Existing approaches primarily focus on developing the powerful score function, but overlook the design of decision-making rules based on these score function. In contrast to prior studies, we rethink the OOD detection task from an perspective of online multiple hypothesis testing. We then propose a novel generalized LOND (g-LOND) algorithm to solve the above problem. Theoretically, the g-LOND algorithm controls false discovery rate (FDR) at pre-specified level without the consideration for the dependence between the p-values. Furthermore, we prove that the false positive rate (FPR) of the g-LOND algorithm converges to zero in probability based on the generalized Gaussian-like distribution family. Finally, the extensive experimental results verify the effectiveness of g-LOND algorithm for OOD detection.

2024

1. NeurIPS

   A Boosting-Type Convergence Result for AdaBoost.MH with Factorized Multi-Class Classifiers

   Solve a COLT 2014 open problem

   Xin Zou*, Zhengyu Zhou*, Jingyuan Xu, and Weiwei Liu

   In NeurIPS, 2024

   Abs HTML PDF

   AdaBoost is a well-known algorithm in boosting. Schapire and Singer propose, an extension of AdaBoost, named AdaBoost.MH, for multi-class classification problems. Kégl shows empirically that AdaBoost.MH works better when the classical one-against-all base classifiers are replaced by factorized base classifiers containing a binary classifier and a vote (or code) vector. However, the factorization makes it much more difficult to provide a convergence result for the factorized version of AdaBoost.MH. Then, Kégl raises an open problem in COLT 2014 to look for a convergence result for the factorized AdaBoost.MH. In this work, we resolve this open problem by presenting a convergence result for AdaBoost.MH with factorized multi-class classifiers.

2. PR

   Residual network with self-adaptive time step size

   Xiyuan Li, Xin Zou, and Weiwei Liu

   Pattern Recognition, 2024

   Abs HTML PDF

   Residual Networks (ResNet) are pivotal in machine learning. The connection between ResNets and ordinary differential equations (ODEs) has inspired enhancements of ResNets using sophisticated numerical methods for ODE systems. Recent advancements in numerical self-adaptive schemes, which adjust time step sizes based on the feature maps or parameters of residual blocks, have demonstrated promising results in enhancing ResNet performance, surpassing those achieved with fixed time step methods. However, these self-adaptive time step constructions lack theoretical support and can limit performance improvements since the self-adaptive time steps should theoretically depend on both the feature maps and the parameters of the residual blocks. In this study, we conduct a rigorous theoretical analysis of the residual functions associated with fixed or self-adaptive time step methods to demonstrate the advantages and rational designs of the self-adaptive approach. Subsequently, we introduce a novel self-adaptive ResNet, AdaTS-ResNet, which effectively incorporates both feature maps and parameters in its time step adjustments. Experimental results on the ImageNet and CIFAR datasets reveal that AdaTS-ResNet surpasses TSCLSTM-ResNet (Yang et al., 2020) in prediction accuracy and computational efficiency, where TSCLSTM represents the latest advancements of the methods designed based on the self-adaptive scheme of ODE. Our findings highlight the potential of improving ResNet architectures through adaptive techniques of dynamical systems, offering insights for future enhancements in deep learning models.

3. ICML

   A Provable Decision Rule for Out-of-Distribution Detection

   Xinsong Ma, Xin Zou, and Weiwei Liu

   In ICML, 2024

   Abs HTML PDF

   Out-of-distribution (OOD) detection task plays the key role in reliable and safety-critical applications. Existing researches mainly devote to designing or training the powerful score function but overlook investigating the decision rule based on the proposed score function. Different from previous work, this paper aims to design a decision rule with rigorous theoretical guarantee and well empirical performance. Specifically, we provide a new insight for the OOD detection task from a hypothesis testing perspective and propose a novel generalized Benjamini Hochberg (g-BH) procedure to solve the testing problem. Theoretically, the g-BH procedure controls false discovery rate (FDR) at pre-specified level. Furthermore, we derive an upper bound of the expectation of false positive rate (FPR) for the g-BH procedure based on the tailed generalized Gaussian distribution family, indicating that the FPR of g-BH procedure converges to zero in probability. Finally, the extensive experimental results verify the superiority of g-BH procedure over the traditional threshold-based decision rule on several OOD detection benchmarks. Particularly, combining SHE with the g-BH procedure, the FPR95 is reduced by 13.65% on average compared with the vanilla SHE.

4. AAAI

   Coverage-Guaranteed Prediction Sets for Out-of-Distribution Data

   Xin Zou, and Weiwei Liu

   In AAAI, 2024

   Abs HTML PDF

   Out-of-distribution (OOD) generalization has attracted increasing research attention in recent years, due to its promising experimental results in real-world applications. In this paper, we study the confidence set prediction problem in the OOD generalization setting. Split conformal prediction (SCP) is an efficient framework for handling the confidence set prediction problem. However, the validity of SCP requires the examples to be exchangeable, which is violated in the OOD setting. Empirically, we show that trivially applying SCP results in a failure to maintain the marginal coverage when the unseen target domain is different from the source domain. To address this issue, we develop a method for forming confident prediction sets in the OOD setting and theoretically prove the validity of our method. Finally, we conduct experiments on simulated data to empirically verify the correctness of our theory and the validity of our proposed method.

2023

1. NeurIPS

   On the Adversarial Robustness of Out-of-distribution Generalization Models

   Xin Zou, and Weiwei Liu

   In NeurIPS, 2023

   Abs HTML PDF

   Out-of-distribution (OOD) generalization has attracted increasing research attention in recent years, due to its promising experimental results in real-world applications. Interestingly, we find that existing OOD generalization methods are vulnerable to adversarial attacks. This motivates us to study OOD adversarial robustness. We first present theoretical analyses of OOD adversarial robustness in two different complementary settings. Motivated by the theoretical results, we design two algorithms to improve the OOD adversarial robustness. Finally, we conduct experiments to validate the effectiveness of our proposed algorithms. Our code is available at https://github.com/ZouXinn/OOD-Adv.

2. JMLR

   Generalization Bounds for Adversarial Contrastive Learning

   Xin Zou, and Weiwei Liu

   JMLR, 2023

   Abs HTML PDF

   Deep networks are well-known to be fragile to adversarial attacks, and adversarial training is one of the most popular methods used to train a robust model. To take advantage of unlabeled data, recent works have applied adversarial training to contrastive learning (Adversarial Contrastive Learning; ACL for short) and obtain promising robust performance. However, the theory of ACL is not well understood. To fill this gap, we leverage the Rademacher omplexity to analyze the generalization performance of ACL, with a particular focus on linear models and multi-layer neural networks under \ell_p attack (p≥1). Our theory shows that the average adversarial risk of the downstream tasks can be upper bounded by the adversarial unsupervised risk of the upstream task. The experimental results validate our theory.

2022

1. NeurIPS

   Defending Against Adversarial Attacks via Neural Dynamic System

   Xiyuan Li, Xin Zou, and Weiwei Liu

   In NeurIPS, 2022

   Abs HTML PDF

   Although deep neural networks (DNN) have achieved great success, their applications in safety-critical areas are hindered due to their vulnerability to adversarial attacks. Some recent works have accordingly proposed to enhance the robustness of DNN from a dynamic system perspective. Following this line of inquiry, and inspired by the asymptotic stability of the general nonautonomous dynamical system, we propose to make each clean instance be the asymptotically stable equilibrium points of a slowly time-varying system in order to defend against adversarial attacks. We present a theoretical guarantee that if a clean instance is an asymptotically stable equilibrium point and the adversarial instance is in the neighborhood of this point, the asymptotic stability will reduce the adversarial noise to bring the adversarial instance close to the clean instance. Motivated by our theoretical results, we go on to propose a nonautonomous neural ordinary differential equation (ASODE) and place constraints on its corresponding linear time-variant system to make all clean instances act as its asymptotically stable equilibrium points. Our analysis suggests that the constraints can be converted to regularizers in implementation. The experimental results show that ASODE improves robustness against adversarial attacks and outperforms state-of-the-art methods.