New research note on Your Reward Shaping Might Disappear at Normalization: GRPO within-group standardization cancels a positive per-prompt reward scale (KAPPO, Matrenok et al.) unless that scale is applied after normalization, with a companion reading of gradient interference in group-aware RLHF. Read the research note.
TRUSTWORTHY AI / FROM OPTIMIZATION TO SELF-IMPROVING AGENTS
Duo Zhou
Ph.D. Student · Computer Science · UIUC
I am a Computer Science Ph.D. student at the University of Illinois Urbana-Champaign. I study how to make increasingly capable AI systems reliable - from the optimization and post-training methods that shape model behavior to the agents and multi-agent systems that act, coordinate, and improve over time. My work develops scalable algorithms, formal guarantees, and system-level mechanisms for reinforcement learning, neural network verification, and recursive self-improvement.
02 / UPDATE STREAM
News
recent.log
Show 15 earlier updates
New research note on Geometric-Disentanglement Unlearning (EMNLP 2026 Main): why Euclidean orthogonality is not retain invariance under adaptive optimizers, and how projecting in the optimizer's metric separates forgetting from capability preservation. Read the research note.
New research note on DR-SAC (ICLR 2026): making worst-case soft Bellman backups tractable in offline continuous control via KL duality and functional optimization. Read the research note.
Cumulative Fisher Protection: during RLHF, what matters is which directions the policy moves, not how far. ProContract keeps a long-horizon agent's responsibilities outside the harness. Both under review. arXiv and code coming soon.
Gradient Interference in Group-Aware RLHF: single-reward RLHF can collapse for a minority group while the group rewards stay positively correlated. arXiv and code coming soon. Read the research note.
GEAR, our GPU-accelerated global solver for nonlinear programs via linear bound propagation, accepted to NeurIPS 2026. Read the research note.
Geometric-Disentanglement Language Model Unlearning accepted to EMNLP 2026 Main. Read the research note.
Won VNN-COMP 2026 on CAV 2026.
InvSTG-PLM accepted to KDD 2026.
Lookahead Branching accepted to IJCAI 2026.
AdaFuse accepted to ACL 2026 Main.
DR-SAC accepted to ICLR 2026. Read the research note.
ShortageSim accepted to AAAI 2026 as an oral presentation.
Clip-and-Verify accepted to NeurIPS 2025.
Won VNN-COMP 2025 on CAV 2025.
BICCOS accepted to NeurIPS 2024.
03 / SELECTED OUTPUT
Publications
* equal contribution · † corresponding · Complete publication list on Google Scholar ↗
Not All Reward Heterogeneity is Value Conflict: Gradient Interference in Group-Aware RLHF
Manuscript
Where the Policy Moves Matters: Capability Preservation via Cumulative Fisher Protection in RLHF
Under review
ProContract: Decoupling Responsibility and Progress from the Harness in Long-Horizon Agents
Under review
GEAR: A GPU-Accelerated Global Solver for Nonlinear Programs via Linear Bound Propagation
Advances in Neural Information Processing Systems · NeurIPS 2026
Geometric-Disentanglement Language Model Unlearning
Proceedings of the 2026 Conference on Empirical Methods in Natural Language Processing · Main Conference
Invariant Structure Learning with Pre-trained Language Models for Spatio-temporal Graph
Proceedings of the 32nd ACM SIGKDD Conference on Knowledge Discovery and Data Mining, pp. 5045-5056
AdaFuse: Adaptive Ensemble Decoding for Large Language Models
Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics · Main Conference, pp. 42644-42657
Lookahead Branching for Neural Network Verification
Proceedings of the 35th International Joint Conference on Artificial Intelligence
DR-SAC: Distributionally Robust Soft Actor-Critic for Reinforcement Learning under Uncertainty
The Fourteenth International Conference on Learning Representations Vol. 2026, pp. 92720-92750
E-Globe: Scalable ε-Global Verification of Neural Networks via Tight Upper Bounds and Pattern-Aware Branching
arXiv preprint arXiv:2602.05068
Agentic Reasoning for Large Language Models
Transactions on Machine Learning Research
ShortageSim: Simulating Drug Shortages under Information Asymmetry
Proceedings of the AAAI Conference on Artificial Intelligence Vol. 40, No. 45, pp. 38321-38330
Clip-and-Verify: Linear Constraint-Driven Domain Clipping for Accelerating Neural Network Verification
Advances in Neural Information Processing Systems 38, 174849-174895
Scalable Neural Network Verification with Branch-and-bound Inferred Cutting Planes
Advances in Neural Information Processing Systems 37, 29324-29353
Distributionally Robust Path Integral Control
American Control Conference, pp. 1164-1171
An Adaptive and Dynamical Neural Network for Machine Remaining Useful Life Prediction
IEEE Transactions on Industrial Informatics 20(2), 1093-1102
04A / MILESTONES
Honors
- 01
Winner of both the Regular and Extended tracks at VNN-COMP 2026.
- 02
Winner of both the Regular and Extended tracks at VNN-COMP 2025.
- 03
Winner of both the Regular and Extended tracks at VNN-COMP 2024.
- 04
First-Class Scholarship for the Graduating Class, 2021.
- 05
Meritorious Winner, Mathematical Contest in Modeling and Interdisciplinary Contest in Modeling, 2020.
- 06
Provincial First Prize, Contemporary Undergraduate Mathematical Contest in Modeling, 2019.
04B / COMMUNITY
Service
- 01
Session Chair: Informs, EMNLP.
- 02
Reviewer: JMLR, NeurIPS, ICLR, ICML, AAAI, IJCAI, KDD, and ARR.