视频简介

工作经历

  • 2019年8月—现在，中国人民大学高瓴人工智能学院，教授
  • 2013年9月—2019年7月，中国人民大学信息学院，副教授
  • 2011年5月—2013年7月，北京大学计算机所，助理研究员

研究方向

计算机视觉：多模态大模型，扩散生成模型，文生图，文生视频
机器学习：连续学习，自监督学习，元学习，小样本学习
量子位报道：https://mp.weixin.qq.com/s/oXAx-b0ci7R8WHOYsf10gg
机器之心报道：https://mp.weixin.qq.com/s/qpg_UPGO_T4kpeZAm_pniw

学生要求

对研究有浓厚兴趣，自我驱动力强，有较好的动手能力（指导的学生获得2021年CCF优博）

教授课程

  • 研究生课：《高级机器学习》，《小样本学习》
  • 本科生课：《迁移学习》，《分布式系统与云计算》，《统计学习》

科研项目

  • 国家自然科学基金重点项目（课题），面向智慧教育的多模态模型构建方法，2025.01-2029.12，主持
  • 国家自然科学基金面上项目，大规模多模态预训练的关键问题研究，2024.01-2027.12，主持
  • 华为-人大联合项目，基于多任务自监督与稀疏激活的多模态语义模型，2022.02-2023.02，主持
  • 联通-人大联合项目，面向联通业务场景的大规模预训练模型，2022.07-2023.11，主持
  • 阿里达摩院合作项目，面向视觉语言的多模态特征学习，2020.09-2021.09，主持
  • 国家自然科学基金面上项目，小样本学习关键问题研究，2020.01-2023.12，主持
  • 国家自然科学基金面上项目，噪声环境下的弱监督图像语义分割研究，2016.01-2019.12，主持

学术论文

2024
FineCLIP: Self-distilled Region-based CLIP for Better Fine-Grained Understanding
Dong Jing, Xiaolong He, Yutian Luo, Nanyi Fei, Guoxing Yang, Wei Wei, Huiwen Zhao, Zhiwu Lu*
Annual Conference on Neural Information Processing Systems (NeurIPS), 2024. (CCF A)
下载：
链接： 链接
Enhancing Class-Incremental Learning for Image Classification via Bidirectional Transport and Selective Momentum
Feifei Fu, Yizhao Gao, Zhiwu Lu*
International Conference on Multimedia Retrieval (ICMR), 2024. (CCF B)
下载： 论文文档
Dual-Enhanced Coreset Selection with Class-wise Collaboration for Online Blurry Class Incremental Learning
Yutian Luo, Shiqi Zhao, Haoran Wu, and Zhiwu Lu*
IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2024. (CCF A)
下载： 论文文档
UniAdapter: Unified Parameter-Efficient Transfer Learning for Cross-modal Modeling
Haoyu Lu, Yuqi Huo, Guoxing Yang, Zhiwu Lu*, Wei Zhan, Masayoshi Tomizuka, and Mingyu Ding
International Conference on Learning Representations (ICLR), 2024.
下载： 论文文档
VDT: General-purpose Video Diffusion Transformers via Mask Modeling
Haoyu Lu, Guoxing Yang, Nanyi Fei, Yuqi Huo, Zhiwu Lu*, Ping Luo, and Mingyu Ding
International Conference on Learning Representations (ICLR), 2024.
下载： 论文文档
VEMO: A Versatile Elastic Multi-Modal Model for Search-Oriented Multi-Task Learning
Nanyi Fei, Hao Jiang, Haoyu Lu, Jinqiang Long, Yanqi Dai, Tuo Fan, Zhao Cao, and Zhiwu Lu*
European Conference on Information Retrieval (ECIR), 2024.
下载： 论文文档
Unsupervised Continual Learning of Image Representation via Rememory-Based SimSiam
Feifei Fu, Yizhao Gao, Zhiwu Lu*, Haoran Wu, and Shiqi Zhao
IEEE International Conference on Acoustics, Speech and SP (ICASSP), 2024. (CCF B)
下载： 论文文档
Image Retrieval with Composed Query by Multi-Scale Multi-Modal Fusion
Zelong Sun, Guoxing Yang, Zhiwu Lu*, Hao Jiang, Guojie Zhu, and Zhao Cao
IEEE International Conference on Acoustics, Speech and SP (ICASSP), 2024. (CCF B)
下载： 论文文档
Multi-level Contrastive Learning for Hybrid Cross-modal Retrieval
Yiming Zhao, Haoyu Lu, Shiqi Zhao, Haoran Wu, and Zhiwu Lu*
IEEE International Conference on Acoustics, Speech and SP (ICASSP), 2024. (CCF B)
下载： 论文文档
Progressive Image Synthesis from Semantics to Details with Denoising Diffusion GAN
Guoxing Yang, Haoyu Lu, Chongxuan Li, Guang Zhou, Haoran Wu, and Zhiwu Lu*
IEEE International Conference on Acoustics, Speech and SP (ICASSP), 2024. (CCF B)
下载： 论文文档
2023
Mixup-Inspired Video Class-Incremental Learning
Jinqiang Long, Yizhao Gao, and Zhiwu Lu*
IEEE International Conference on Data Mining (ICDM), 2023. (CCF B)
下载： 论文文档
R-STAR: Robust Self-Taught Task-Wise Reweighting for Rehearsal-Based Class Incremental Learning
Yutian Luo, Yizhao Gao, Haoran Wu, Ruitao Ma, and Zhiwu Lu*
European Conference on Artificial Intelligence (ECAI), 2023. (CCF B)
下载： 论文文档
Improvable Gap Balancing for Multi-Task Learning
Yanqi Dai, Nanyi Fei, and Zhiwu Lu*
International Conference on Uncertainty in Artificial Intelligence (UAI), 2023. (CCF B)
下载： 论文文档
CMMT: Cross-Modal Meta-Transformer for Video-Text Retrieval
Yizhao Gao and Zhiwu Lu*
ACM SIGMM International Conference on Multimedia Retrieval (ICMR), 2023. (CCF B)
下载： 论文文档
Shot Retrieval and Assembly with Text Script for Video Montage Generation
Guoxing Yang, Haoyu Lu, Zelong Sun, and Zhiwu Lu*
ACM SIGMM International Conference on Multimedia Retrieval (ICMR), 2023. (CCF B)
下载： 论文文档
Learning with Adaptive Knowledge for Continual Image-Text Modeling
Yutian Luo, Yizhao Gao, and Zhiwu Lu*
ACM SIGMM International Conference on Multimedia Retrieval (ICMR), 2023. (CCF B)
下载： 论文文档
DiST-GAN: Distillation-based Semantic Transfer for Text-Guided Face Generation
Guoxing Yang, Feifei Fu, Nanyi Fei, Haoran Wu, Ruitao Ma, and Zhiwu Lu*
IEEE International Conference on Multimedia & Expo (ICME), 2023. (CCF B)
下载： 论文文档
2022
BMU-MoCo: Bidirectional Momentum Update for Continual Video-Language Modeling
Yizhao Gao, Nanyi Fei, Haoyu Lu, Zhiwu Lu*, Hao Jiang, Yijie Li, and Zhao Cao
Annual Conference on Neural Information Processing Systems (NeurIPS), 2022. (Spotlight, CCF A)
下载： 论文文档
LGDN: Language-Guided Denoising Network for Video-Language Modeling
Haoyu Lu, Mingyu Ding, Nanyi Fei, Yuqi Huo, and Zhiwu Lu*
Annual Conference on Neural Information Processing Systems (NeurIPS), 2022. (Spotlight, CCF A)
下载： 论文文档
Fine-Grained Analysis of Stability and Generalization for Modern Meta Learning Algorithms
Jiechao Guan, Yong Liu, and Zhiwu Lu*
Annual Conference on Neural Information Processing Systems (NeurIPS), 2022. (Spotlight, CCF A)
下载： 论文文档
Supervised Contrastive Learning for Few-Shot Action Classification
Hongfeng Han, Nanyi Fei, Zhiwu Lu*, and Ji-Rong Wen
European Conference on Machine Learning and Principles and Practice of Knowledge Discovery in Databases (ECML PKDD), 2022. (CCF B)
下载： 论文文档
Visual Prompt Tuning for Few-Shot Text Classification
Jingyuan Wen, Yutian Luo, Nanyi Fei, Guoxing Yang, Zhiwu Lu*, Hao Jiang, Jie Jiang and Zhao Cao
International Conference on Computational Linguistics (COLING), 2022. (CCF B)
下载： 论文文档
Towards artificial general intelligence via a multimodal foundation model
Nanyi Fei, Zhiwu Lu*, Yizhao Gao, Guoxing Yang, Yuqi Huo, Jingyuan Wen, Haoyu Lu, Ruihua Song, Xin Gao, Tao Xiang, Hao Sun* and Ji-Rong Wen*
Nature Communications (Nat Commun), vol. 13, article no. 3094, 2022.
下载： 论文文档 海报
Fast-Rate PAC-Bayesian Generalization Bounds for Meta-Learning
Jiechao Guan and Zhiwu Lu*
International Conference on Machine Learning (ICML), 2022. (CCF A)
下载： 论文文档
COTS: Collaborative Two-Stream Vision-Language Pre-Training Model for Cross-Modal Retrieval
Haoyu Lu , Nanyi Fei, Yuqi Huo, Yizhao Gao, Zhiwu Lu*, and Ji-Rong Wen
IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2022. (CCF A)
下载： 论文文档
Learning Versatile Neural Architectures by Propagating Network Codes
Mingyu Ding, Yuqi Huo, Haoyu Lu, Linjie Yang, Zhe Wang, Zhiwu Lu, Jingdong Wang, and Ping Luo
International Conference on Learning Representations (ICLR), 2022.
下载： 论文文档
Task Relatedness-Based Generalization Bounds for Meta Learning
Jiechao Guan and Zhiwu Lu*
International Conference on Learning Representations (ICLR), 2022. (Spotlight)
下载： 论文文档
2021
Compressed Video Contrastive Learning
Yuqi Huo, Mingyu Ding, Haoyu Lu, Nanyi Fei, Zhiwu Lu*, Ji-Rong Wen, and Ping Luo
Thirty-Fifth Conference on Neural Information Processing Systems (NeurIPS), 2021. (CCF A)
下载： 论文文档
Z-Score Normalization, Hubness, and Few-Shot Learning
Nanyi Fei, Yizhao Gao, Zhiwu Lu*, and Tao Xiang
International Conference on Computer Vision (ICCV), 2021. (CCF A)
下载： 论文文档
Contrastive Prototype Learning with Augmented Embeddings for Few-Shot Learning
Yizhao Gao, Nanyi Fei, Guangzhen Liu, Zhiwu Lu*, and Tao Xiang
International onference on Uncertainty in Artificial Intelligence (UAI), 2021. (CCF B)
下载： 论文文档
Self-Supervised Video Representation Learning with Constrained Spatiotemporal Jigsaw
Yuqi Huo, Mingyu Ding, Haoyu Lu, Ziyuan Huang, Mingqian Tang, Zhiwu Lu*, and Tao Xiang
International Joint Conference on Artificial Intelligence (IJCAI), 2021. (CCF A)
下载： 论文文档
A Global Occlusion-Aware Approach to Self-Supervised Monocular Visual Odometry
Yao Lu, Xiaoli Xu, Mingyu Ding, Zhiwu Lu*, and Tao Xiang
AAAI Conference on Artificial Intelligence (AAAI), 2021. (CCF A)
下载： 论文文档
IEPT: Instance-Level and Episode-Level Pretext Tasks for Few-Shot Learning
Manli Zhang, Jianhong Zhang, Zhiwu Lu*, Tao Xiang, Mingyu Ding, and Songfang Huang
International Conference on Learning Representations (ICLR), 2021.
下载： 论文文档
MELR: Meta-Learning via Modeling Episode-Level Relationships for Few-Shot Learning
Nanyi Fei, Zhiwu Lu*, Tao Xiang, and Songfang Huang
International Conference on Learning Representations (ICLR), 2021.
下载： 论文文档
HR-NAS: Searching Efficient High-Resolution Neural Architectures with Lightweight Transformers
Mingyu Ding, Xiaochen Lian, Linjie Yang, Peng Wang, Xiaojie Jin, Zhiwu Lu, and Ping Luo
IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2021. (Oral, CCF A)
下载： 论文文档
Counterfactual VQA: A Cause-Effect Look at Language Bias
Yulei Niu, Kaihua Tang, Hanwang Zhang, Zhiwu Lu, Xian-Sheng Hua, and Ji-Rong Wen
IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2021. (CCF A)
下载： 论文文档
L2M-GAN: Learning to Manipulate Latent Space Semantics for Facial Attribute Editing
Guoxing Yang, Nanyi Fei, Mingyu Ding, Guangzhen Liu, Zhiwu Lu*, and Tao Xiang
IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2021. (Oral, CCF A)
下载： 论文文档
Variational Context: Exploiting Visual and Textual Context for Grounding Referring Expressions
Yulei Niu, Hanwang Zhang, Zhiwu Lu, and Shih-Fu Chang
IEEE Transactions on Pattern Analysis and Machine Intelligence (TPAMI), vol. 43, no. 1, pp. 347-359, 2021. (CCF A)
Zero and Few Shot Learning with Semantic Feature Synthesis and Competitive Learning
Jiechao Guan, Zhiwu Lu*, Tao Xiang, Aoxue Li, An Zhao, and Ji-Rong Wen
IEEE Transactions on Pattern Analysis and Machine Intelligence (TPAMI), vol. 43, no. 7, pp. 2510-2523, 2021. (CCF A)

荣誉奖励

  • 昇腾AI创新大赛2023全国总决赛应用赛道银奖
  • 中国人民大学2022年度优秀科研成果奖特等奖
  • Top 25 Nature Communications social science and human behavior articles of 2022
  • AISTATS 2022 Top reviewer
  • NeurIPS 2021 Outstanding Reviewer Award
  • 指导的学生获得2021年CCF优博
  • 指导的学生获得2021年百度奖学金
  • 2021年设计首个公开的中文通用图文预训练模型文澜BriVL
  • ICONIP 2018最佳学生论文奖亚军
  • 2015年IBM SUR Award
  • 中国人民大学2015届优秀学士学位论文指导老师
  • 计算机图形学国际会议CGI 2014最佳论文奖

社会兼职

  • Reviewer for IEEE TPAMI, IJCV, IEEE TIP, IEEE TMM, IEEE TNNLS, IEEE TKDE, TOIS
  • SPC/PC Member of ICML, NeurIPS, CVPR, ICCV, ECCV, AAAI, IJCAI, UAI
  • Member of CCF, ACM, IEEE

联系

电话：010-62515670

邮箱：luzhiwu(@)ruc.edu.cn

个人网页：https://gsai.ruc.edu.cn/luzhiwu

办公地址：--