Education
- 2006.9 -2010.12, Dept. of Computer Science, Shanghai Jiao Tong University, Ph.D.; PhD thesis: Research on Web Information Retrieval Evaluation; Co-Advisors: Prof. YU Yong and Dr. HON Hsiao-Wuen
- 2000.9 - 2003.1, Dept. of Computer Science and Technology, Tsinghua University, M.E.; Master thesis: Information Retrieval Based on Structural and Semantic Information; Advisor: Prof. MA Shaoping
- 1996.9 - 2000.7, Dept. of Computer Science and Technology, Tsinghua University, B.E.
Work Experience
- 2020.9 – now, Gaoling School of ArtificialIntelligence, Renmin University of China; As a tenured Associate Professor; Research Interests: multimodal understanding and generation.
- 2017.7–2020.8, Microsoft Software Center Asia; As a PrincipleData & Applied Science Lead; Research Interests: AI based creation, session-oriented chatbots, and multi-modality
- 2003.4–2017.7, Microsoft Research Asia; As a Lead Researcher; Research Interests: Social and Mobile Computing, Web Information Retrieval, and Information Extraction
RESEARCH INTERESTS
Social and Mobile Computing
Web Information Retrieval
Web Information Extraction
Publications
Multimodal Understanding and Generation
? Wenhui Tan, Bei Liu, Junbo Zhang, Ruihua Song, Jianlong
Fu. RoLD: Robot Latent
Diffusion for Multi-task Policy Modeling. In
Proceedings of the 31st International Conference on Multimedia Modeling,
340-353, 2025. (Best Paper Award)
? Yuchong Sun, Bei Liu, Xu Chen, Ruihua Song, Jianlong Fu.
ViCo: Engaging
Video Comment Generation with Human Preference Rewards. In Proceedings of the 6th ACM International
Conference on Multimedia in Asia, 2024. (Best Student Paper
Runners-up Award)
? Xihua Wang, Yuyue Wang, Yihan Wu, Ruihua Song, Xu Tan,
Zehua Chen, Hongteng Xu, Guodong Sui. TiVA: Time-aligned
Video-to-audio Generation. In Proceedings of the 32nd ACM International Conference
on Multimedia, 573-582, 2024. (Oral)
? Qian Cao, Xu Chen, Ruihua Song, Xiting Wang, Xinting
Huang, Yuchen Ren. See or Guess: Counterfactually
Regularized Image Captioning. In Proceedings of the 32nd ACM International Conference
on Multimedia, 965-974, 2024
? Chuhao Jin, Kening Ren, Lingzhen Kong, Xiting Wang,
Ruihua Song, Huan Chen. Persuading across Diverse
Domains: a Dataset and Persuasion Large Language Model. In Proceedings of the 62nd Annual Meeting of
the Association for Computational Linguistics, 1678-1706, 2024
? Yuchong Sun, Che Liu, Kun Zhou, Jinwen Huang, Ruihua
Song, Wayne Xin Zhao, Fuzheng Zhang, Di Zhang, Kun Gai. Parrot: Enhancing
Multi-turn Instruction Following for Large Language Models. In Proceedings
of the 62nd Annual Meeting of the Association for Computational Linguistics,
9729-9750, 2024
? Yan Zeng, Hanbo Zhang, Jiani Zheng, Jiangnan Xia,
Guoqiang Wei, Yang Wei, Yuchen Zhang, Tao Kong, Ruihua Song. What Matters in
Training a GPT4-Style Language Model with Multimodal Inputs?
In Proceedings of the 2024
Conference of the North American Chapter of the Association for Computational
Linguistics, 7930-7957, 2024
? Yihan Wu, Ruihua Song, Xu Chen, Hao Jiang, Zhao Cao, Jin
Yu. Understanding
Human Preferences: Towards More Personalized Video to Text Generation. In Proceedings of the ACM on Web Conference
2024, 3952-3963, 2024
? Hongpeng Lin, Ludan Ruan, Wenke Xia, Peiyu Liu, Jingyuan
Wen, Yixin Xu, Di Hu, Ruihua Song, Wayne Xin Zhao, Qin Jin, Zhiwu Lu. TikTalk: A
Video-Based Dialogue Dataset for Multi-Modal Chitchat in Real World. In
Proceedings of the 31st ACM International Conference on Multimedia, 1303-1313,
2023
? Xu Gu, Yuchong Sun, Feiyue Ni, Shizhe Chen, Xihua Wang,
Ruihua Song, Boyuan Li, Xiang Cao. TeViS: Translating
Text Synopses to Video Storyboards. In Proceedings of the 31st
ACM International Conference on Multimedia, 4968-4979, 2023
? Yihan Wu, Junliang Guo, Xu Tan, Chen Zhang, Bohan Li,
Ruihua Song, Lei He, Sheng Zhao, Arul Menezes, Jiang Bian. Videodubber:
Machine Translation with Speech-aware Length Control for Video Dubbing.
In Proceedings of the AAAI Conference on Artificial Intelligence 37 (11),
13772-13779, 2023
? Yuchong Sun, Hongwei Xue, Ruihua Song, Bei Liu, Huan
Yang, Jianlong Fu. Long-form
Video-language Pre-training with Multimodal Temporal Contrastive Learning. Advances
in neural information processing systems 35, 38032-38045, 2022
? Qian Cao, Xu Chen, Ruihua Song, Hao Jiang, Guang Yang,
Zhao Cao. Multi-modal
Experience Inspired AI Creation. In Proceedings of the 30th ACM
International Conference on Multimedia, 1445-1454, 2022
? Nanyi Fei, Zhiwu Lu, Yizhao Gao, Guoxing Yang, Yuqi Huo,
Jingyuan Wen, Haoyu Lu, Ruihua Song, Xin Gao, Tao Xiang, Hao Sun, Ji-Rong Wen. Towards Artificial
General Intelligence via a Multimodal Foundation Model. Nature
Communications 13 (1), 3094, 2022
? Yuqi Huo, Manli Zhang,
Guangzhen Liu, Haoyu Lu, Yizhao Gao, Guoxing Yang, Jingyuan Wen, Heng Zhang,
Baogui Xu, Weihao Zheng, Zongzheng Xi, Yueqian Yang, Anwen Hu, Jinming Zhao,
Ruichen Li, Yida Zhao, Liang Zhang, Yuqing Song, Xin Hong, Wanqing Cui, Danyang
Hou, Yingyan Li, Junyi Li, Peiyu Liu, Zheng Gong, Chuhao Jin, Yuchong Sun,
Shizhe Chen, Zhiwu Lu, Zhicheng Dou, Qin Jin, Yanyan Lan, Wayne Xin Zhao,
Ruihua Song, Ji-Rong Wen. WenLan: Bridging
Vision and Language by Large-scale Multi-modal Pre-training.
arXiv preprint arXiv:2103.06561, 2021
? Xu Han, Zhengyan Zhang, Ning Ding, Yuxian Gu, Xiao Liu,
Yuqi Huo, Jiezhong Qiu, Yuan Yao, Ao Zhang, Liang Zhang, Wentao Han, Minlie
Huang, Qin Jin, Yanyan Lan, Yang Liu, Zhiyuan Liu, Zhiwu Lu, Xipeng Qiu, Ruihua
Song, Jie Tang, Ji-Rong Wen, Jinhui Yuan, Wayne Xin Zhao, Jun Zhu. Pre-trained
Models: Past, Present and Future. AI Open 2, 225-250, 2021
? Shizhe
Chen, Bei Liu, Jianlong Fu, Ruihua Song, Qin Jin, Pingping Lin, Xiaoyu Qi,
Chunting Wang, and Jin Zhou. Neural Storyboard Artist: Visualizing Stories with Coherent Image
Sequences. In Proceedings of ACM Multimedia 2019, 2019
Social and Mobile Computing
? Yutao Zhu, Ruihua Song, Jian-Yun Nie, Pan Du, Zhicheng
Dou, Jin Zhou. Leveraging
Narrative to Generate Movie Script. ACM Transactions on
Information Systems (TOIS) 40 (4), 1-32, 2022
? Lisong Qiu, Ying Wai Shiu, Pingping Lin, Ruihua Song, Yue Liu,
Dongyan Zhao and Rui Yan. What If Bots Feel Moods? Towards Controllable Retrieval-based
Dialogue Systems with Emotion-Aware Transition Networks. In Proceedings of
ACM SIGIR 2020, 2020
? Yutao Zhu, Ruihua Song, Zhicheng Dou,
Jian-Yun NIE and Jin Zhou. ScriptWriter: Narrative-Guided Script Generation.
In Proceedings of ACL 2020, 2020
? Zhaohao Zeng,
Pingping Lin, Ruihua Song, and Tetsuya Sakai. Attitude Detection for
One-Round Conversation: jointly Extracting Target-Polarity Pairs. In
Proceedings of WSDM 2019, 2019
? Danning Zheng, Ruihua Song, Tianran
Hu, Hao Fu, and Jin Zhou. “Love is as Complex as Math”:
Metaphor Generation System for Social Chatbot. In Proceedings of the 20th Chinese Lexical
Semantic Workshop (CLSW 2019), 2019. (Outstanding Paper Award)
? Wen-Feng Cheng, Chao-Chung Wu,
Ruihua Song, Jianlong Fu, Xing Xie, and Jian-Yun Nie. Image
Inspired Poetry Generation in XiaoIce. arXiv preprint arXiv:1808.03090. 2018
? Kazutoshi Umemoto, Ruihua
Song, Jian-Yun Nie, Xing Xie, Katsumi Tanaka, Yong Rui:
Search by Screenshots for Universal Article Clipping in Mobile Apps. ACM Transaction
on Information Systems (TOIS). 35(4): 34:1-34:29, 2017.
? Tianran Hu, Ruihua Song, Maya Abtahian, Philip Ding, Xing Xie, Jiebo Luo:
A World of Difference: Divergent Word Interpretations Among People. ICWSM 2017:
548-551, 2017
? Chenyi Zhuang,
Nicholas Jing Yuan,
Ruihua Song, Xing Xie,
Qiang Ma:
Understanding People
Lifestyles: Construction of Urban Movement Knowledge Graph from GPS Trajectory. In
Proceedings of IJCAI 2017: 3616-3623, 2017.
? Tianran Hu, Ruihua Song, Yingzi
Wang, Xing Xie, and Jiebo Luo. Mining Shopping Patterns for
Divergent Urban Regions by Incorporating Mobility Data. In Proceedings of CIKM'16, 2016.
? Zhou Zhao, Ruihua Song, Xing Xie,
Xiaofei He, Yueting Zhuang. Mobile Query Recommendation via
Tensor Function Learning. In Proceedings of the 24th International Conference on
Artificial Intelligence (AAAI'15), 2015
Web Information Retrieval
? Ji-Rong Wen, Zhicheng Dou, Ruihua Song:
Personalized Web Search. Encyclopedia of Database
Systems (2nd edition), 2018.
? Ruihua Song, Dingquan Wang,
Jian-Yun Nie, Ji-Rong Wen, and Yong Yu. Enhancing
Web Search with Queries of Equivalent Intents. Information Retrieval Journal 19(6), 573-593, 2016.
? Zhicheng Dou, Zhengbao Jiang, Sha
Hu, Ji-Rong Wen, Ruihua Song. Automatically Mining Facets for
Queries from Their Search Results. IEEE Transactions on Knowledge and
Data Engineering 28 (2), 385-397, 2016
? Tetsuya Sakai and Ruihua Song. Diversified
Search Evaluation: Lessons from the NTCIR-9 INTENT Task. Information Retrieval Journal 16(4),
2013.
? Yang Liu, Ruihua Song, Yu Chen, Jian-Yun Nie, and Ji-Rong
Wen. Adaptive Query Suggestion for
Difficult Queries. In Proceedings of SIGIR2012, 2012.
? Tetsuya Sakai, Zhicheng Dou, Ruihua Song, and Noriko Kando. The Reusability of a Diversified Search
Test Collection. In Proceedings of AIRS2012, 2012. (Best Paper Award)
? Ruihua Song, Min Zhang, Tetsuya Skai, Makoto P. Kato, Yiqun
Liu, Miho Sugimoto, Qinglei Wang, and Naoki Orii. Overview of the NTCIR-9 INTENT Task. In Proceedings of NTCIR-9,
2011.
? Zhicheng Dou, Sha Hu, Yulong Luo, Ruihua Song, and Ji-Rong
Wen. Finding Dimensions for Queries. In
Proceedings of CIKM2011, 2011.
? Tetsuya Sakai and Ruihua Song. Evaluating Diversified Search Results Using Per-Intent Graded
Relevance. In Proceedings of ACM SIGIR 2011, 2011.
? Zhicheng Dou, Sha Hu, Kun Chen, Ruihua Song, and Ji-Rong
Wen. Multi-dimensional Search Result
Diversification. In Proceedings of WSDM'11, February 2011.
? Ruihua Song, Liqian Yu, Ji-Rong Wen, and Hsiao-Wuen. A Proximity
Probabilistic Model for Information Retrieval. No.
MSR-TR-2011-7, 2011.
? Ruihua Song, Qingwei Guo, Ruochi Zhang, Guomao Xin, Ji-Rong
Wen, Yong Yu, and Hsiao-Wuen. Select-the-Best-Ones:
A New Way to Judge Relative Relevance. Information Processing and
Management, Vol. 47, No. 1. (06 January 2011), pp. 37-52.
? Yi Liu, Liangjie Zhang, Ruihua Song,
Jian-Yun Nie, and Ji-Rong Wen. Clustering
Queries for Better Document Ranking. In Proceedings of CIKM'09 (short paper),
2009.
? Zhicheng Dou,
Ruihua Song, Jian-Yun Nie, and Ji-Rong Wen. Using Anchor Texts with Their
Hyperlink Structure for Web Search. In Proceedings of the 32nd Annual ACM
SIGIR Conference (SIGIR '09). 2009.
? Ruihua Song,
Zhenxiao Luo, Jian-Yun Nie, Yong Yu, and Hsiao-Wuen Hon. Identification of
Ambiguous Queries in Web Search. Information Processing and Management.
2008.
? Zhicheng Dou,
Ruihua Song, Xiaojie Yuan, and Ji-Rong Wen. Are Click-Through Data Adequate
for Learning Web Search Rankings? In Proceedings of The 17th ACM Conference
on Information and Knowledge Management. 2008.
? Ruihua Song,
Michael J. Taylor, Ji-Rong Wen, Hsiao-Wuen Hon, and Yongyu. Viewing Term
Proximity from a Different Perspective. In Proceedings of the 30th European
Conference on Information Retrieval 2008. 2008.
? Ruihua Song,
Zhenxiao Luo, Ji-Rong Wen, Yong Yu, and Hsiao-Wuen Hon. Identify Ambiguous
Queries in Web Search. In Proceedings of the 16th International Conference
on World Wide Web 2007 (poster). 2007.
? Zhicheng Dou,
Ruihua Song, and Ji-Rong Wen. A Large-scale Evaluation and Analysis of
Personalized Search Strategies. In Proceedings of the 16th International
Conference on World Wide Web 2007. 2007.
? Ruihua Song,
Guomao Xin, Shuming Shi, Ji-Rong Wen, and Wei-Ying Ma. Exploring URL Hit
Priors for Web Search. In Proceedings of the 28th European Conference on
Information Retrieval (ECIR'06). London, United Kingdom, April 10-12, 2006.
? Shuming Shi,
Ji-Rong Wen, Qing Yu, Ruihua Song, and Wei-Ying Ma. Gravitation-based Model
for Information Retrieval. In Proceedings of the 28th Annual International
ACM SIGIR Conference on Research and Development in Information Retrieval
(SIGIR'05). Salvador, Brazil, August 15-19, 2005.
? Yunhua Hu,
Guomao Xin, Ruihua Song, Guoping Hu, Shuming Shi, Yunbo Cao, and Hang Li. Title
Extraction from Bodies of HTML Documents and its Application to Web Page
Retrieval. In Proceedings of the 28th Annual International ACM SIGIR
Conference on Research and Development in Information Retrieval (SIGIR'05). Salvador,
Brazil, August 15-19, 2005.
? Ruihua Song,
Ji-Rong Wen, Shuming Shi, Guomao Xin, Tie-Yan Liu, et al. Microsoft Research
Asia at Web Track and Terabyte Track of TREC 2004. In 2004 Text REtrieval
Conference (TREC'04).
? Ji-Rong Wen,
Ruihua Song, Deng Cai, Kaihua Zhu, Shipeng Yu, Shaozhi Ye, and Wei-Ying Ma. Microsoft
Research Asia at the Web Track of TREC 2003. TREC 2003: 408-417
? Min Zhang,
Ruihua Song, Chuan Lin, Liang Ma, Zhe Jiang, Yijiang Jin, Yiqun Liu, Le Zhao,
and Shaoping Ma. THU TREC 2002: Novelty Track Experiments. In
Proceedings of Text Retrieval Conference 2002, 2002.
? Min Zhang,
Ruihua Song, Chuan Lin, Liang Ma, Zhe Jiang, Yijiang Jin, Yiqun Liu, Le Zhao,
and Shaoping Ma. THU TREC 2002: Web Track Experiments. In Proceedings of
Text Retrieval Conference 2002, 2002.
Web Information Extraction
? Shuyi Zheng, Ruihua Song, Ji-Rong Wen,
and C. Lee Giles. Efficient Record-Level
Wrapper Induction. In Proceedings of CIKM'09, 2009.
? Shuyi Zheng,
Matthew R. Scott, Ruihua Song, and Ji-Rong Wen. Pictor: an interactive
system for importing data from a website. In the Proceedings of SIGKDD 2008
(demo), 2007
? Shuyi Zheng,
Ruihua Song, and Ji-Rong Wen. Joint Optimization of Template Detection and
Wrapper Generation. In the Proceedings of SIGKDD 2007, 2007
? Shuyi Zheng,
Ruihua Song, and Ji-Rong Wen. Template-Independent News Extraction Based on
Visual Consistency. In the Proceedings of AAAI 2007, 2007
? Ruihua Song,
Haifeng Liu, Ji-Rong Wen, and Wei-Ying Ma. Learning Block Importance Models
for Web Pages. WWW 2004: 203-211, 2004. (Best Paper Award Runners-Up)
?Ruihua Song, Gang Chen, and Shaoping Ma. A
HTML Page Segmentation Approach for Enhancing Chinese Search Engine. In
Journal of Chinese Information, Volume 7, 2003
PUBLICATIONS
Honors and Awards
- • My paper entitle “RoLD: Robot Latent Diffusion for Multi-task Policy Modeling.” won Best Paper Award from ACM MMM 2025.
- • My paper entitled “ViCo: Engaging Video Comment Generation with Human Preference Rewards.” won Best Student Paper Runner-up Award from ACM MM Asia 2024.
- • My paper entitled “Love is as Complex as Math”: Metaphor Generation System for Social Chatbot won Outstanding Paper Award from the 20th Chinese Lexical Semantic Workshop (CLSW 2019).
- • My project of XiaoIce Writing Poems receives Intelligent Pioneers Award from CCTV-1 and Chinese Academy of Sciences in 2017.
- • My paper entitled The Reusability of a Diversified Search Test Collection won Best Paper Award from AIRS 2012.
- • My paper entitled Learning Block Importance Models for Web Pages won Best Paper Runner-up Award from WWW 2004.
Services
- Served the following journals:
- • Editorial board of Information Retrieval Journal
- • Editorial board of Foundations and Trends in Information Retrieval
- Served the following conferencesas a PC member or a Senior PC member, Area Chair,
- or a Co-Chairs:
- • ACM SIGIR Conference on Information Retrieval
- • Annual Meeting of the Association for Computational Linguistics
- • IEEE Conference on Computer Vision and Pattern Recognition
- • ACM International Conference on Multimedia
- • ACM International Conference on Web Search and Data Mining
- • VLDB: International Conference on Very Large Data Bases
- • IJCAI International Joint Conference on Artificial Intelligence
- • ACM SIGKDD Conference on Knowledge Discovery and Data Mining
- • Conference on Empirical Methods in Natural Language Processing
- • European Conference on Information Retrieval
- • ICDM: IEEE International Conference on Data Mining
- • NTCIR (NII Test Collection for IR Systems)
- • CAAI GAITC (Global Artificial Intelligence Technology Conference)
- Served as:
- • Senior Member of ACM
- • Senior Member of IEEE
Contact
Tel：
Email：rsong@ruc.edu.cn
Website：https://scholar.google.com/citations?user=v5LctN8AAAAJ
Address：BLD Lide, No. 59 Zhongguncun Da Street, Haidian District, Beijing 100872, China