Education

  • Ph.D.: 2008-2011, Department of Computer Science, City University of Hong Kong, Hong Kong SAR
  • Master of Science: 2002-2005, School of Mathematical Science, Peking University, P. R. China
  • Bachelor of Science: 1998-2002, School of Mathematics and Statistics, Wuhan University, P. R. China

Work Experience

  • Full Professor: 2019-present, Gaoling School of Artificial Intelligence, Renmin University of China, P. R. China
  • Associate Professor: 2013-2019, School of Information, Renmin University of China, P. R. China
  • Assistant Professor: 2011-2013, Institute of Computer Science and Technology, Peking University, P. R. China

RESEARCH INTERESTS

Computer Vision: large multimodal model, generative diffusion model, text-to-image generation, text-to-video generation

Machine Learning: continual learning, self-supervised learning, few-shot learning, meta learning

PUBLICATIONS

2025
MMRole: A Comprehensive Framework for Developing and Evaluating Multimodal Role-Playing Agents
Yanqi Dai, Huanran Hu, Lei Wang, Shengjie Jin, Xu Chen*, and Zhiwu Lu*
International Conference on Learning Representations (ICLR), 2025.
Enhancing Data-Free Class-Incremental Learning via Image-Centric Dual Distillation
Feifei Fu and Zhiwu Lu*
IEEE International Conference on Acoustics, Speech and SP (ICASSP), 2025. (CCF B)
Leveraging Large Vision-Language Model as User Intent-aware Encoder for Composed Image Retrieval
Zelong Sun, Dong Jing, Guoxing Yang, Nanyi Fei, and Zhiwu Lu*
AAAI Conference on Artificial Intelligence (AAAI), 2025. (CCF A)
2024
FineCLIP: Self-distilled Region-based CLIP for Better Fine-Grained Understanding
Dong Jing, Xiaolong He, Yutian Luo, Nanyi Fei, Guoxing Yang, Wei Wei, Huiwen Zhao, Zhiwu Lu*
Annual Conference on Neural Information Processing Systems (NeurIPS), 2024. (CCF A)
Enhancing Class-Incremental Learning for Image Classification via Bidirectional Transport and Selective Momentum
Feifei Fu, Yizhao Gao, Zhiwu Lu*
International Conference on Multimedia Retrieval (ICMR), 2024. (CCF B)
Dual-Enhanced Coreset Selection with Class-wise Collaboration for Online Blurry Class Incremental Learning
Yutian Luo, Shiqi Zhao, Haoran Wu, and Zhiwu Lu*
IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2024. (CCF A)
UniAdapter: Unified Parameter-Efficient Transfer Learning for Cross-modal Modeling
Haoyu Lu, Yuqi Huo, Guoxing Yang, Zhiwu Lu*, Wei Zhan, Masayoshi Tomizuka, and Mingyu Ding
International Conference on Learning Representations (ICLR), 2024.
VDT: General-purpose Video Diffusion Transformers via Mask Modeling
Haoyu Lu, Guoxing Yang, Nanyi Fei, Yuqi Huo, Zhiwu Lu*, Ping Luo, and Mingyu Ding
International Conference on Learning Representations (ICLR), 2024.
VEMO: A Versatile Elastic Multi-Modal Model for Search-Oriented Multi-Task Learning
Nanyi Fei, Hao Jiang, Haoyu Lu, Jinqiang Long, Yanqi Dai, Tuo Fan, Zhao Cao, and Zhiwu Lu*
European Conference on Information Retrieval (ECIR), 2024.
Unsupervised Continual Learning of Image Representation via Rememory-Based SimSiam
Feifei Fu, Yizhao Gao, Zhiwu Lu*, Haoran Wu, and Shiqi Zhao
IEEE International Conference on Acoustics, Speech and SP (ICASSP), 2024. (CCF B)
Image Retrieval with Composed Query by Multi-Scale Multi-Modal Fusion
Zelong Sun, Guoxing Yang, Zhiwu Lu*, Hao Jiang, Guojie Zhu, and Zhao Cao
IEEE International Conference on Acoustics, Speech and SP (ICASSP), 2024. (CCF B)
Multi-level Contrastive Learning for Hybrid Cross-modal Retrieval
Yiming Zhao, Haoyu Lu, Shiqi Zhao, Haoran Wu, and Zhiwu Lu*
IEEE International Conference on Acoustics, Speech and SP (ICASSP), 2024. (CCF B)
Progressive Image Synthesis from Semantics to Details with Denoising Diffusion GAN
Guoxing Yang, Haoyu Lu, Chongxuan Li, Guang Zhou, Haoran Wu, and Zhiwu Lu*
IEEE International Conference on Acoustics, Speech and SP (ICASSP), 2024. (CCF B)
2023
Mixup-Inspired Video Class-Incremental Learning
Jinqiang Long, Yizhao Gao, and Zhiwu Lu*
IEEE International Conference on Data Mining (ICDM), 2023. (CCF B)
R-STAR: Robust Self-Taught Task-Wise Reweighting for Rehearsal-Based Class Incremental Learning
Yutian Luo, Yizhao Gao, Haoran Wu, Ruitao Ma, and Zhiwu Lu*
European Conference on Artificial Intelligence (ECAI), 2023. (CCF B)
Improvable Gap Balancing for Multi-Task Learning
Yanqi Dai, Nanyi Fei, and Zhiwu Lu*
International Conference on Uncertainty in Artificial Intelligence (UAI), 2023. (CCF B)
CMMT: Cross-Modal Meta-Transformer for Video-Text Retrieval
Yizhao Gao and Zhiwu Lu*
ACM SIGMM International Conference on Multimedia Retrieval (ICMR), 2023. (CCF B)
Shot Retrieval and Assembly with Text Script for Video Montage Generation
Guoxing Yang, Haoyu Lu, Zelong Sun, and Zhiwu Lu*
ACM SIGMM International Conference on Multimedia Retrieval (ICMR), 2023. (CCF B)
Learning with Adaptive Knowledge for Continual Image-Text Modeling
Yutian Luo, Yizhao Gao, and Zhiwu Lu*
ACM SIGMM International Conference on Multimedia Retrieval (ICMR), 2023. (CCF B)
DiST-GAN: Distillation-based Semantic Transfer for Text-Guided Face Generation
Guoxing Yang, Feifei Fu, Nanyi Fei, Haoran Wu, Ruitao Ma, and Zhiwu Lu*
IEEE International Conference on Multimedia & Expo (ICME), 2023. (CCF B)
2022
BMU-MoCo: Bidirectional Momentum Update for Continual Video-Language Modeling
Yizhao Gao, Nanyi Fei, Haoyu Lu, Zhiwu Lu*, Hao Jiang, Yijie Li, and Zhao Cao
Annual Conference on Neural Information Processing Systems (NeurIPS), 2022. (Spotlight, CCF A)
LGDN: Language-Guided Denoising Network for Video-Language Modeling
Haoyu Lu, Mingyu Ding, Nanyi Fei, Yuqi Huo, and Zhiwu Lu*
Annual Conference on Neural Information Processing Systems (NeurIPS), 2022. (Spotlight, CCF A)
Fine-Grained Analysis of Stability and Generalization for Modern Meta Learning Algorithms
Jiechao Guan, Yong Liu, and Zhiwu Lu*
Annual Conference on Neural Information Processing Systems (NeurIPS), 2022. (Spotlight, CCF A)
Supervised Contrastive Learning for Few-Shot Action Classification
Hongfeng Han, Nanyi Fei, Zhiwu Lu*, and Ji-Rong Wen
European Conference on Machine Learning and Principles and Practice of Knowledge Discovery in Databases (ECML PKDD), 2022. (CCF B)
Visual Prompt Tuning for Few-Shot Text Classification
Jingyuan Wen, Yutian Luo, Nanyi Fei, Guoxing Yang, Zhiwu Lu*, Hao Jiang, Jie Jiang and Zhao Cao
International Conference on Computational Linguistics (COLING), 2022. (CCF B)
Towards artificial general intelligence via a multimodal foundation model
Nanyi Fei, Zhiwu Lu*, Yizhao Gao, Guoxing Yang, Yuqi Huo, Jingyuan Wen, Haoyu Lu, Ruihua Song, Xin Gao, Tao Xiang, Hao Sun* and Ji-Rong Wen*
Nature Communications (Nat Commun), vol. 13, article no. 3094, 2022.
Fast-Rate PAC-Bayesian Generalization Bounds for Meta-Learning
Jiechao Guan and Zhiwu Lu*
International Conference on Machine Learning (ICML), 2022. (CCF A)
COTS: Collaborative Two-Stream Vision-Language Pre-Training Model for Cross-Modal Retrieval
Haoyu Lu , Nanyi Fei, Yuqi Huo, Yizhao Gao, Zhiwu Lu*, and Ji-Rong Wen
IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2022. (CCF A)
Learning Versatile Neural Architectures by Propagating Network Codes
Mingyu Ding, Yuqi Huo, Haoyu Lu, Linjie Yang, Zhe Wang, Zhiwu Lu, Jingdong Wang, and Ping Luo
International Conference on Learning Representations (ICLR), 2022.
Task Relatedness-Based Generalization Bounds for Meta Learning
Jiechao Guan and Zhiwu Lu*
International Conference on Learning Representations (ICLR), 2022. (Spotlight)

Services

  • Reviewer for IEEE TPAMI, IJCV, IEEE TIP, IEEE TMM, IEEE TNNLS, IEEE TKDE, TOIS
  • SPC/PC Member of ICML, NeurIPS, CVPR, ICCV, ECCV, AAAI, IJCAI, UAI
  • Member of CCF, ACM, IEEE

Contact

Tel：+86-10-62515670

Email：luzhiwu@ruc.edu.cn

Website：https://gsai.ruc.edu.cn/english/luzhiwu

Address：