UCLA
I am a Ph.D. Candidate at UCLA Computer Science Department, advised by Prof. Cho-Jui Hsieh. Before that, I received my B.Eng. degree from the CST department at Tsinghua University where I worked with Prof. Minlie Huang.
My research interest is machine learning, and my primary research focus is on trustworthy machine learning and the robustness of machine learning models. My works include:
General and scalable formal verification for neural networks: It is about formally bounding and verifying the output of a neural network (NN) given uncertain inputs from a region. I developed formal verification methods for general neural network-based models, from Transformers to general computational graphs and higher-order computational graphs. I am a main developer of the auto_LiRPA software (originally proposed in our NeurIPS 2020 paper) for perturbation analysis and verified bound computation on general computational graphs (including NNs with arbitrary architectures). Based on auto_LiRPA and complete verification algorithm (including branch-and-bound for general models) we also developed a complete verification toolbox named alpha-beta-CROWN which is the winner at the International Verification of Neural Networks Competitions (VNN-COMP) for three consecutive years from 2021 to 2023. While the properties to be verified are often robustness under small perturbtions, we have also extended formal verification for NNs to properties beyond robustness, such as monotonicity/fairness, Lyapunov stability of controllers in dynamic systems, constraints of power systems, etc.
Adversarial robustness in NLP: I proposed methods for generating adversarial examples for NLP by leveraging modern language models. To generate word-substitution attacks with high-quality synonyms that are compatible with context, I proposed to leverage a masked BERT model, use a instructional-following model such as ChatGPT, or use a text-completion model such as LLaMA. I proposed to red team large language model detectors by LLM-generated word substitution attacks or instructional prompts. We also recently proposed to defend against LLM jailbreaking attacks by backtranslation.
Out-of-distribution robustness: I proposed a new understanding and evaluation on the effective robustness of multimodal pre-trained models especially CLIP models. I found that pre-training data in CLIP can interfere the previous evaluation of OOD effective robustness rather than improve effective robustness, and suggested that CLIP is not effectively more robust than traditional models.
Efficiently training robust neural networks: I developed methods for faster certified robust training by an improved interval bound propagation-based training.
TA at UCLA: