Education

  • Ph.D.: 2008-2011, Department of Computer Science, City University of Hong Kong, Hong Kong SAR
  • Master of Science: 2002-2005, School of Mathematical Science, Peking University, P. R. China
  • Bachelor of Science: 1998-2002, School of Mathematics and Statistics, Wuhan University, P. R. China

Work Experience

  • Full Professor: 2019-present, Gaoling School of Artificial Intelligence, Renmin University of China, P. R. China
  • Associate Professor: 2013-2019, School of Information, Renmin University of China, P. R. China
  • Assistant Professor: 2011-2013, Institute of Computer Science and Technology, Peking University, P. R. China

RESEARCH INTERESTS

Computer Vision: large multimodal model, generative diffusion model, text-to-image generation, text-to-video generation

Machine Learning: continual learning, self-supervised learning, few-shot learning, meta learning

PUBLICATIONS

2024
FineCLIP: Self-distilled Region-based CLIP for Better Fine-Grained Understanding
Dong Jing, Xiaolong He, Yutian Luo, Nanyi Fei, Guoxing Yang, Wei Wei, Huiwen Zhao, Zhiwu Lu*
Annual Conference on Neural Information Processing Systems (NeurIPS), 2024. (CCF A)
Enhancing Class-Incremental Learning for Image Classification via Bidirectional Transport and Selective Momentum
Feifei Fu, Yizhao Gao, Zhiwu Lu*
International Conference on Multimedia Retrieval (ICMR), 2024. (CCF B)
Dual-Enhanced Coreset Selection with Class-wise Collaboration for Online Blurry Class Incremental Learning
Yutian Luo, Shiqi Zhao, Haoran Wu, and Zhiwu Lu*
IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2024. (CCF A)
UniAdapter: Unified Parameter-Efficient Transfer Learning for Cross-modal Modeling
Haoyu Lu, Yuqi Huo, Guoxing Yang, Zhiwu Lu*, Wei Zhan, Masayoshi Tomizuka, and Mingyu Ding
International Conference on Learning Representations (ICLR), 2024.
VDT: General-purpose Video Diffusion Transformers via Mask Modeling
Haoyu Lu, Guoxing Yang, Nanyi Fei, Yuqi Huo, Zhiwu Lu*, Ping Luo, and Mingyu Ding
International Conference on Learning Representations (ICLR), 2024.
VEMO: A Versatile Elastic Multi-Modal Model for Search-Oriented Multi-Task Learning
Nanyi Fei, Hao Jiang, Haoyu Lu, Jinqiang Long, Yanqi Dai, Tuo Fan, Zhao Cao, and Zhiwu Lu*
European Conference on Information Retrieval (ECIR), 2024.
Unsupervised Continual Learning of Image Representation via Rememory-Based SimSiam
Feifei Fu, Yizhao Gao, Zhiwu Lu*, Haoran Wu, and Shiqi Zhao
IEEE International Conference on Acoustics, Speech and SP (ICASSP), 2024. (CCF B)
Image Retrieval with Composed Query by Multi-Scale Multi-Modal Fusion
Zelong Sun, Guoxing Yang, Zhiwu Lu*, Hao Jiang, Guojie Zhu, and Zhao Cao
IEEE International Conference on Acoustics, Speech and SP (ICASSP), 2024. (CCF B)
Multi-level Contrastive Learning for Hybrid Cross-modal Retrieval
Yiming Zhao, Haoyu Lu, Shiqi Zhao, Haoran Wu, and Zhiwu Lu*
IEEE International Conference on Acoustics, Speech and SP (ICASSP), 2024. (CCF B)
Progressive Image Synthesis from Semantics to Details with Denoising Diffusion GAN
Guoxing Yang, Haoyu Lu, Chongxuan Li, Guang Zhou, Haoran Wu, and Zhiwu Lu*
IEEE International Conference on Acoustics, Speech and SP (ICASSP), 2024. (CCF B)
2023
Mixup-Inspired Video Class-Incremental Learning
Jinqiang Long, Yizhao Gao, and Zhiwu Lu*
IEEE International Conference on Data Mining (ICDM), 2023. (CCF B)
R-STAR: Robust Self-Taught Task-Wise Reweighting for Rehearsal-Based Class Incremental Learning
Yutian Luo, Yizhao Gao, Haoran Wu, Ruitao Ma, and Zhiwu Lu*
European Conference on Artificial Intelligence (ECAI), 2023. (CCF B)
Improvable Gap Balancing for Multi-Task Learning
Yanqi Dai, Nanyi Fei, and Zhiwu Lu*
International Conference on Uncertainty in Artificial Intelligence (UAI), 2023. (CCF B)
CMMT: Cross-Modal Meta-Transformer for Video-Text Retrieval
Yizhao Gao and Zhiwu Lu*
ACM SIGMM International Conference on Multimedia Retrieval (ICMR), 2023. (CCF B)
Shot Retrieval and Assembly with Text Script for Video Montage Generation
Guoxing Yang, Haoyu Lu, Zelong Sun, and Zhiwu Lu*
ACM SIGMM International Conference on Multimedia Retrieval (ICMR), 2023. (CCF B)
Learning with Adaptive Knowledge for Continual Image-Text Modeling
Yutian Luo, Yizhao Gao, and Zhiwu Lu*
ACM SIGMM International Conference on Multimedia Retrieval (ICMR), 2023. (CCF B)
DiST-GAN: Distillation-based Semantic Transfer for Text-Guided Face Generation
Guoxing Yang, Feifei Fu, Nanyi Fei, Haoran Wu, Ruitao Ma, and Zhiwu Lu*
IEEE International Conference on Multimedia & Expo (ICME), 2023. (CCF B)
2022
BMU-MoCo: Bidirectional Momentum Update for Continual Video-Language Modeling
Yizhao Gao, Nanyi Fei, Haoyu Lu, Zhiwu Lu*, Hao Jiang, Yijie Li, and Zhao Cao
Annual Conference on Neural Information Processing Systems (NeurIPS), 2022. (Spotlight, CCF A)
LGDN: Language-Guided Denoising Network for Video-Language Modeling
Haoyu Lu, Mingyu Ding, Nanyi Fei, Yuqi Huo, and Zhiwu Lu*
Annual Conference on Neural Information Processing Systems (NeurIPS), 2022. (Spotlight, CCF A)
Fine-Grained Analysis of Stability and Generalization for Modern Meta Learning Algorithms
Jiechao Guan, Yong Liu, and Zhiwu Lu*
Annual Conference on Neural Information Processing Systems (NeurIPS), 2022. (Spotlight, CCF A)
Supervised Contrastive Learning for Few-Shot Action Classification
Hongfeng Han, Nanyi Fei, Zhiwu Lu*, and Ji-Rong Wen
European Conference on Machine Learning and Principles and Practice of Knowledge Discovery in Databases (ECML PKDD), 2022. (CCF B)
Visual Prompt Tuning for Few-Shot Text Classification
Jingyuan Wen, Yutian Luo, Nanyi Fei, Guoxing Yang, Zhiwu Lu*, Hao Jiang, Jie Jiang and Zhao Cao
International Conference on Computational Linguistics (COLING), 2022. (CCF B)
Towards artificial general intelligence via a multimodal foundation model
Nanyi Fei, Zhiwu Lu*, Yizhao Gao, Guoxing Yang, Yuqi Huo, Jingyuan Wen, Haoyu Lu, Ruihua Song, Xin Gao, Tao Xiang, Hao Sun* and Ji-Rong Wen*
Nature Communications (Nat Commun), vol. 13, article no. 3094, 2022.
Fast-Rate PAC-Bayesian Generalization Bounds for Meta-Learning
Jiechao Guan and Zhiwu Lu*
International Conference on Machine Learning (ICML), 2022. (CCF A)
COTS: Collaborative Two-Stream Vision-Language Pre-Training Model for Cross-Modal Retrieval
Haoyu Lu , Nanyi Fei, Yuqi Huo, Yizhao Gao, Zhiwu Lu*, and Ji-Rong Wen
IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2022. (CCF A)
Learning Versatile Neural Architectures by Propagating Network Codes
Mingyu Ding, Yuqi Huo, Haoyu Lu, Linjie Yang, Zhe Wang, Zhiwu Lu, Jingdong Wang, and Ping Luo
International Conference on Learning Representations (ICLR), 2022.
Task Relatedness-Based Generalization Bounds for Meta Learning
Jiechao Guan and Zhiwu Lu*
International Conference on Learning Representations (ICLR), 2022. (Spotlight)
2021
Compressed Video Contrastive Learning
Yuqi Huo, Mingyu Ding, Haoyu Lu, Nanyi Fei, Zhiwu Lu*, Ji-Rong Wen, and Ping Luo
Thirty-Fifth Conference on Neural Information Processing Systems (NeurIPS), 2021. (CCF A)
Z-Score Normalization, Hubness, and Few-Shot Learning
Nanyi Fei, Yizhao Gao, Zhiwu Lu*, and Tao Xiang
International Conference on Computer Vision (ICCV), 2021. (CCF A)
Contrastive Prototype Learning with Augmented Embeddings for Few-Shot Learning
Yizhao Gao, Nanyi Fei, Guangzhen Liu, Zhiwu Lu*, and Tao Xiang
International onference on Uncertainty in Artificial Intelligence (UAI), 2021. (CCF B)
Self-Supervised Video Representation Learning with Constrained Spatiotemporal Jigsaw
Yuqi Huo, Mingyu Ding, Haoyu Lu, Ziyuan Huang, Mingqian Tang, Zhiwu Lu*, and Tao Xiang
International Joint Conference on Artificial Intelligence (IJCAI), 2021. (CCF A)
A Global Occlusion-Aware Approach to Self-Supervised Monocular Visual Odometry
Yao Lu, Xiaoli Xu, Mingyu Ding, Zhiwu Lu*, and Tao Xiang
AAAI Conference on Artificial Intelligence (AAAI), 2021. (CCF A)
IEPT: Instance-Level and Episode-Level Pretext Tasks for Few-Shot Learning
Manli Zhang, Jianhong Zhang, Zhiwu Lu*, Tao Xiang, Mingyu Ding, and Songfang Huang
International Conference on Learning Representations (ICLR), 2021.
MELR: Meta-Learning via Modeling Episode-Level Relationships for Few-Shot Learning
Nanyi Fei, Zhiwu Lu*, Tao Xiang, and Songfang Huang
International Conference on Learning Representations (ICLR), 2021.
HR-NAS: Searching Efficient High-Resolution Neural Architectures with Lightweight Transformers
Mingyu Ding, Xiaochen Lian, Linjie Yang, Peng Wang, Xiaojie Jin, Zhiwu Lu, and Ping Luo
IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2021. (Oral, CCF A)
Counterfactual VQA: A Cause-Effect Look at Language Bias
Yulei Niu, Kaihua Tang, Hanwang Zhang, Zhiwu Lu, Xian-Sheng Hua, and Ji-Rong Wen
IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2021. (CCF A)
L2M-GAN: Learning to Manipulate Latent Space Semantics for Facial Attribute Editing
Guoxing Yang, Nanyi Fei, Mingyu Ding, Guangzhen Liu, Zhiwu Lu*, and Tao Xiang
IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2021. (Oral, CCF A)
Variational Context: Exploiting Visual and Textual Context for Grounding Referring Expressions
Yulei Niu, Hanwang Zhang, Zhiwu Lu, and Shih-Fu Chang
IEEE Transactions on Pattern Analysis and Machine Intelligence (TPAMI), vol. 43, no. 1, pp. 347-359, 2021. (CCF A)
Zero and Few Shot Learning with Semantic Feature Synthesis and Competitive Learning
Jiechao Guan, Zhiwu Lu*, Tao Xiang, Aoxue Li, An Zhao, and Ji-Rong Wen
IEEE Transactions on Pattern Analysis and Machine Intelligence (TPAMI), vol. 43, no. 7, pp. 2510-2523, 2021. (CCF A)

Services

  • Reviewer for IEEE TPAMI, IJCV, IEEE TIP, IEEE TMM, IEEE TNNLS, IEEE TKDE, TOIS
  • SPC/PC Member of ICML, NeurIPS, CVPR, ICCV, ECCV, AAAI, IJCAI, UAI
  • Member of CCF, ACM, IEEE

Contact

Tel：+86-10-62515670

Email：luzhiwu@ruc.edu.cn

Website：https://gsai.ruc.edu.cn/english/luzhiwu

Address：