视频简介

Current Time 0:00
Duration -:-
Loaded: 0%
Stream Type LIVE
Remaining Time -:-
 
1x

工作经历

  • 2019年8月—现在，中国人民大学高瓴人工智能学院，教授
  • 2013年9月—2019年7月，中国人民大学信息学院，副教授
  • 2011年5月—2013年7月，北京大学计算机所，助理研究员

研究方向

计算机视觉：多模态大模型，大模型后训练，图像可控生成，视频生成，具身智能
机器学习：连续学习，强化学习，自监督学习，元学习，小样本学习
南风窗报道：https://mp.weixin.qq.com/s/pV6xZ4wTIJH0UoBfMft8Sg
量子位报道：https://mp.weixin.qq.com/s/oXAx-b0ci7R8WHOYsf10gg
机器之心报道：https://mp.weixin.qq.com/s/qpg_UPGO_T4kpeZAm_pniw

学生要求

对研究有浓厚兴趣，自我驱动力强，有较好的动手能力（指导学生获得2021年CCF优博）

教授课程

  • 研究生课：《高级机器学习》，《小样本学习》，《自然语言处理》
  • 本科生课：《迁移学习》，《统计学习》

科研项目

  • 国家自然科学基金重点项目（课题），面向智慧教育的多模态模型构建方法，2025.01-2029.12，主持
  • 国家自然科学基金面上项目，大规模多模态预训练的关键问题研究，2024.01-2027.12，主持
  • 华为-人大联合项目，基于多任务自监督与稀疏激活的多模态语义模型，2022.02-2023.02，主持
  • 联通-人大联合项目，面向联通业务场景的大规模预训练模型，2022.07-2023.11，主持
  • 阿里达摩院合作项目，面向视觉语言的多模态特征学习，2020.09-2021.09，主持
  • 国家自然科学基金面上项目，小样本学习关键问题研究，2020.01-2023.12，主持
  • 国家自然科学基金面上项目，噪声环境下的弱监督图像语义分割研究，2016.01-2019.12，主持

学术论文

2025
MMRole: A Comprehensive Framework for Developing and Evaluating Multimodal Role-Playing Agents
Yanqi Dai, Huanran Hu, Lei Wang, Shengjie Jin, Xu Chen*, and Zhiwu Lu*
International Conference on Learning Representations (ICLR), 2025.
下载： 论文文档
Enhancing Data-Free Class-Incremental Learning via Image-Centric Dual Distillation
Feifei Fu and Zhiwu Lu*
IEEE International Conference on Acoustics, Speech and SP (ICASSP), 2025. (CCF B)
下载： 论文文档
Leveraging Large Vision-Language Model as User Intent-aware Encoder for Composed Image Retrieval
Zelong Sun, Dong Jing, Guoxing Yang, Nanyi Fei, and Zhiwu Lu*
AAAI Conference on Artificial Intelligence (AAAI), 2025. (CCF A)
2024
FineCLIP: Self-distilled Region-based CLIP for Better Fine-Grained Understanding
Dong Jing, Xiaolong He, Yutian Luo, Nanyi Fei, Guoxing Yang, Wei Wei, Huiwen Zhao, Zhiwu Lu*
Annual Conference on Neural Information Processing Systems (NeurIPS), 2024. (CCF A)
下载： 论文文档
Enhancing Class-Incremental Learning for Image Classification via Bidirectional Transport and Selective Momentum
Feifei Fu, Yizhao Gao, Zhiwu Lu*
International Conference on Multimedia Retrieval (ICMR), 2024. (CCF B)
下载： 论文文档
Dual-Enhanced Coreset Selection with Class-wise Collaboration for Online Blurry Class Incremental Learning
Yutian Luo, Shiqi Zhao, Haoran Wu, and Zhiwu Lu*
IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2024. (CCF A)
下载： 论文文档
UniAdapter: Unified Parameter-Efficient Transfer Learning for Cross-modal Modeling
Haoyu Lu, Yuqi Huo, Guoxing Yang, Zhiwu Lu*, Wei Zhan, Masayoshi Tomizuka, and Mingyu Ding
International Conference on Learning Representations (ICLR), 2024.
下载： 论文文档
VDT: General-purpose Video Diffusion Transformers via Mask Modeling
Haoyu Lu, Guoxing Yang, Nanyi Fei, Yuqi Huo, Zhiwu Lu*, Ping Luo, and Mingyu Ding
International Conference on Learning Representations (ICLR), 2024.
下载： 论文文档
VEMO: A Versatile Elastic Multi-Modal Model for Search-Oriented Multi-Task Learning
Nanyi Fei, Hao Jiang, Haoyu Lu, Jinqiang Long, Yanqi Dai, Tuo Fan, Zhao Cao, and Zhiwu Lu*
European Conference on Information Retrieval (ECIR), 2024.
下载： 论文文档
Unsupervised Continual Learning of Image Representation via Rememory-Based SimSiam
Feifei Fu, Yizhao Gao, Zhiwu Lu*, Haoran Wu, and Shiqi Zhao
IEEE International Conference on Acoustics, Speech and SP (ICASSP), 2024. (CCF B)
下载： 论文文档
Image Retrieval with Composed Query by Multi-Scale Multi-Modal Fusion
Zelong Sun, Guoxing Yang, Zhiwu Lu*, Hao Jiang, Guojie Zhu, and Zhao Cao
IEEE International Conference on Acoustics, Speech and SP (ICASSP), 2024. (CCF B)
下载： 论文文档
Multi-level Contrastive Learning for Hybrid Cross-modal Retrieval
Yiming Zhao, Haoyu Lu, Shiqi Zhao, Haoran Wu, and Zhiwu Lu*
IEEE International Conference on Acoustics, Speech and SP (ICASSP), 2024. (CCF B)
下载： 论文文档
Progressive Image Synthesis from Semantics to Details with Denoising Diffusion GAN
Guoxing Yang, Haoyu Lu, Chongxuan Li, Guang Zhou, Haoran Wu, and Zhiwu Lu*
IEEE International Conference on Acoustics, Speech and SP (ICASSP), 2024. (CCF B)
下载： 论文文档
2023
Mixup-Inspired Video Class-Incremental Learning
Jinqiang Long, Yizhao Gao, and Zhiwu Lu*
IEEE International Conference on Data Mining (ICDM), 2023. (CCF B)
下载： 论文文档
R-STAR: Robust Self-Taught Task-Wise Reweighting for Rehearsal-Based Class Incremental Learning
Yutian Luo, Yizhao Gao, Haoran Wu, Ruitao Ma, and Zhiwu Lu*
European Conference on Artificial Intelligence (ECAI), 2023. (CCF B)
下载： 论文文档
Improvable Gap Balancing for Multi-Task Learning
Yanqi Dai, Nanyi Fei, and Zhiwu Lu*
International Conference on Uncertainty in Artificial Intelligence (UAI), 2023. (CCF B)
下载： 论文文档
CMMT: Cross-Modal Meta-Transformer for Video-Text Retrieval
Yizhao Gao and Zhiwu Lu*
ACM SIGMM International Conference on Multimedia Retrieval (ICMR), 2023. (CCF B)
下载： 论文文档
Shot Retrieval and Assembly with Text Script for Video Montage Generation
Guoxing Yang, Haoyu Lu, Zelong Sun, and Zhiwu Lu*
ACM SIGMM International Conference on Multimedia Retrieval (ICMR), 2023. (CCF B)
下载： 论文文档
Learning with Adaptive Knowledge for Continual Image-Text Modeling
Yutian Luo, Yizhao Gao, and Zhiwu Lu*
ACM SIGMM International Conference on Multimedia Retrieval (ICMR), 2023. (CCF B)
下载： 论文文档
DiST-GAN: Distillation-based Semantic Transfer for Text-Guided Face Generation
Guoxing Yang, Feifei Fu, Nanyi Fei, Haoran Wu, Ruitao Ma, and Zhiwu Lu*
IEEE International Conference on Multimedia & Expo (ICME), 2023. (CCF B)
下载： 论文文档
2022
BMU-MoCo: Bidirectional Momentum Update for Continual Video-Language Modeling
Yizhao Gao, Nanyi Fei, Haoyu Lu, Zhiwu Lu*, Hao Jiang, Yijie Li, and Zhao Cao
Annual Conference on Neural Information Processing Systems (NeurIPS), 2022. (Spotlight, CCF A)
下载： 论文文档
LGDN: Language-Guided Denoising Network for Video-Language Modeling
Haoyu Lu, Mingyu Ding, Nanyi Fei, Yuqi Huo, and Zhiwu Lu*
Annual Conference on Neural Information Processing Systems (NeurIPS), 2022. (Spotlight, CCF A)
下载： 论文文档
Fine-Grained Analysis of Stability and Generalization for Modern Meta Learning Algorithms
Jiechao Guan, Yong Liu, and Zhiwu Lu*
Annual Conference on Neural Information Processing Systems (NeurIPS), 2022. (Spotlight, CCF A)
下载： 论文文档
Supervised Contrastive Learning for Few-Shot Action Classification
Hongfeng Han, Nanyi Fei, Zhiwu Lu*, and Ji-Rong Wen
European Conference on Machine Learning and Principles and Practice of Knowledge Discovery in Databases (ECML PKDD), 2022. (CCF B)
下载： 论文文档
Visual Prompt Tuning for Few-Shot Text Classification
Jingyuan Wen, Yutian Luo, Nanyi Fei, Guoxing Yang, Zhiwu Lu*, Hao Jiang, Jie Jiang and Zhao Cao
International Conference on Computational Linguistics (COLING), 2022. (CCF B)
下载： 论文文档
Towards artificial general intelligence via a multimodal foundation model
Nanyi Fei, Zhiwu Lu*, Yizhao Gao, Guoxing Yang, Yuqi Huo, Jingyuan Wen, Haoyu Lu, Ruihua Song, Xin Gao, Tao Xiang, Hao Sun* and Ji-Rong Wen*
Nature Communications (Nat Commun), vol. 13, article no. 3094, 2022.
下载： 论文文档 海报
Fast-Rate PAC-Bayesian Generalization Bounds for Meta-Learning
Jiechao Guan and Zhiwu Lu*
International Conference on Machine Learning (ICML), 2022. (CCF A)
下载： 论文文档
COTS: Collaborative Two-Stream Vision-Language Pre-Training Model for Cross-Modal Retrieval
Haoyu Lu , Nanyi Fei, Yuqi Huo, Yizhao Gao, Zhiwu Lu*, and Ji-Rong Wen
IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2022. (CCF A)
下载： 论文文档
Learning Versatile Neural Architectures by Propagating Network Codes
Mingyu Ding, Yuqi Huo, Haoyu Lu, Linjie Yang, Zhe Wang, Zhiwu Lu, Jingdong Wang, and Ping Luo
International Conference on Learning Representations (ICLR), 2022.
下载： 论文文档
Task Relatedness-Based Generalization Bounds for Meta Learning
Jiechao Guan and Zhiwu Lu*
International Conference on Learning Representations (ICLR), 2022. (Spotlight)
下载： 论文文档

荣誉奖励

  • 南风窗2024年度科学家
  • 昇腾AI创新大赛2023全国总决赛应用赛道银奖
  • 中国人民大学2022年度优秀科研成果奖特等奖
  • Top 25 Nature Communications social science and human behavior articles of 2022
  • AISTATS 2022 Top reviewer
  • NeurIPS 2021 Outstanding Reviewer Award
  • 指导学生获得2021年CCF优博
  • 指导学生获得2021年百度奖学金
  • 2021年设计首个公开的中文通用图文预训练模型文澜BriVL
  • ICONIP 2018最佳学生论文奖亚军
  • 2015年IBM SUR Award
  • 中国人民大学2015届优秀学士学位论文指导老师
  • 计算机图形学国际会议CGI 2014最佳论文奖

社会兼职

  • Reviewer for IEEE TPAMI, IJCV, IEEE TIP, IEEE TMM, IEEE TNNLS, IEEE TKDE, TOIS
  • SPC/PC Member of ICML, NeurIPS, CVPR, ICCV, ECCV, AAAI, IJCAI, UAI
  • Member of CCF, ACM, IEEE

联系

电话：--

邮箱：luzhiwu(@)ruc.edu.cn

个人网页：https://gsai.ruc.edu.cn/luzhiwu

办公地址：立德楼17层