刘鑫
长聘副教授
电子邮箱:xin.liu@sjtu.edu.cn
刘鑫,上海交通大学心理学院长聘副教授,入选上海市海外高层次人才“白玉兰人才计划创新项目”。 2019年博士毕业于芬兰奥卢大学,师从赵国英教授;2016年博士毕业于西安交通大学,师从齐春教授。拥有十余年海外科研与教学工作经历,曾任芬兰拉彭兰塔-拉赫蒂工业大学副教授,并先后在英国帝国理工学院、澳大利亚悉尼大学及芬兰奥卢大学等机构从事科研工作。长期致力于心理学与人工智能的交叉前沿研究,旨在通过计算机视觉与机器学习技术,为情感理解与心理健康评估提供客观的计算方法与工具。目前的核心研究方向为可信情感智能与人机共情,在情感多模态大模型、微姿态与微表情分析、远程生理信号检测等领域取得系列创新成果。先后主持国家自然科学基金、芬兰科学院、芬兰教育部、芬兰国家IT科学中心、澳大利亚教育部等国内外重要科研项目;并获得诺基亚、KAUTE等知名企业及基金会资助。在PAMI、IJCV、CVPR、ICLR等国际top期刊和会议上发表论文80余篇,获批专利20余项。目前担任包括中科院一区期刊IEEE Transactions on Multimedia在内的多个国际期刊编委。
个人主页: https://linuxsino.github.io/
人机共情
可信情感智能
情感多模态大模型
2026.01 – 至今,上海交通大学,长聘副教授
2021.08 – 2025.07,芬兰拉彭兰塔-拉赫蒂工业大学,副教授 (Tenure Track)
2020.08 – 2021.07,天津大学,副教授
2019.04 – 2020.07,奥卢大学(芬兰科学院职位),博士后
2019.03 – 2019.08,帝国理工学院,访问学者
2017.08 – 2018.02,悉尼大学,奋进研究学者
芬兰拉彭兰塔-拉赫蒂工业大学兼职博导
IEEE Transactions on Multimedia(中科院一区,CCF A)等三个国际期刊编委
CVPR, ICCV, ECCV, ICLR, NeurIPS, ICML, ACMMM, AAAI等国际会议领域主席/程序委员
IEEE Senior Member
Member of ACM, APA, ELLIS, SPIE
代表性期刊论文
1. R. Gao, X. Liu, Z. Hu, B. Xing, B. Xia, Z. Yu, and H. Kälviäinen. (2026). Identity-free Artificial Emotional Intelligence via Micro-Gesture Understanding. IEEE Transactions on Affective Computing.
2. K. Yuan, Z. Yu, X. Liu, B. Xing, Y. Zhang, W. Xie, L. Shen, and B. Schuller. (2026). Multi-granularity Facial Emotional Representation with Unlabeled Data and Textual Supervision. IEEE Transactions on Image Processing.
3. D. Li, J. Shao, B. Xing, R. Gao, B. Wen, H. Kälviäinen, and X. Liu. (2026). MSF-Mamba: Motion-aware State Fusion Mamba for Efficient Micro-Gesture Recognition. IEEE Transactions on Multimedia.
4. Q. Ye, Z. Yu, R. Shao, Y. Cui, X. Kang, X. Liu, P. Torr, and X. Cao. (2025). CAT+: Investigating and Enhancing Audio-visual Understanding in Large Language Models. IEEE Transactions on Pattern Analysis and Machine Intelligence.
5. R. Wu, W. Zhuo, J. Shi, X. Liu, L. Shen, Y. Gong, and G. Zhao. (2025). HemNet: Hemoglobin-Assistant Network for Video-based Remote Photoplethysmography Measurement. IEEE Transactions on Circuits and Systems for Video Technology.
6. Y. Zhang, H. Lu, X. Liu, Y. Chen, and K. Wu. (2025). Advancing Generalizable Remote Physiological Measurement through the Integration of Explicit and Implicit Prior Knowledge. IEEE Transactions on Image Processing.
7. C. Hao, Z. Yu, X. Liu, J. Xu, H. Yue, and J. Yang. (2025). A Simple yet Effective Network based on Vision Transformer for Camouflaged Object and Salient Object Detection. IEEE Transactions on Image Processing.
8. X. Liu, K. Yuan, X. Niu, J. Shi, Z. Yu, H. Yue, and J. Yang. (2024). Multi-scale Promoted Self-adjusting Correlation Learning for Facial Action Unit Detection. IEEE Transactions on Affective Computing.
9. Z. Yu, R. Cai, Y. Cui, X. Liu, Y. Hu, and A. Kot. (2024). Rethinking Vision Transformer and Masked Autoencoder in Multimodal Face Anti-Spoofing. International Journal of Computer Vision.
10. X. Liu, Y. Zhang, Z. Yu, H. Lu, H. Yue, and J. Yang. (2024). rPPG-MAE: Self-Supervised Pretraining with Masked Autoencoders for Remote Physiological Measurements. IEEE Transactions on Multimedia.
11. H. Chen, H. Shi, X. Liu, X. Li, and G. Zhao. (2023). SMG: A Micro-Gesture Dataset Towards Spontaneous Body Gestures for Emotional Stress State Analysis. International Journal of Computer Vision.
12. Z. Wang, Z. Yu, X. Wang, J. Li, C. Zhao, X. Liu, and Z. Lei. (2023). Consistency Regularization for Deep Face Anti-Spoofing. IEEE Transactions on Information Forensics and Security.
13. X. Liu and G. Zhao. (2021). 3D Skeletal Gesture Recognition using Sparse Coding of Time-Warping Invariant Riemannian Trajectories. IEEE Transactions on Multimedia.
14. Z. Yu, B. Zhou, J. Wan, P. Wang, H. Chen, X. Liu, S. Li, and G. Zhao. (2021). Searching Multi-Rate and Multi-Modal Temporal Enhanced Networks for Gesture Recognition. IEEE Transactions on Image Processing.
15. X. Liu, H. Shi, X. Hong, H. Chen, D. Tao, and G. Zhao. (2020). 3D Skeletal Gesture Recognition via Hidden States Exploration. IEEE Transactions on Image Processing.
16. H. Chen, X. Liu, J. Shi, and G. Zhao. (2020). Temporal hierarchical dictionary guided decoding for online gesture segmentation and recognition. IEEE Transactions on Image Processing.
代表性会议论文
1. B. Xing, X. Liu, G. Zhao, C. Liu, X. Fu, and H. Kälviäinen. (2026). EmotionHallucer: Evaluating Emotion Hallucinations in Multimodal Large Language Models. International Conference on Learning Representations (ICLR).
2. R. Gao, X. Liu, Z. Hu, B. Xing, B. Xia, Z. Yu, and H. Kälviäinen. (2025). FSBench: A Figure Skating Benchmark for Advancing Artistic Sports Understanding. IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).
3. Y. Zhang, H. Lu, Q. Hu, Y. Wang, K. Yuan, X. Liu, and K. Wu. (2025). Period-LLM: Extending the Periodic Capability of Multimodal Large Language Model. IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).
4. D. Li, B. Xing, X. Liu, B. Xia, B. Wen, and H. Kälviäinen. (2025). DEEMO: De-identity Multimodal Emotion Recognition and Reasoning. ACM International Conference on Multimedia (ACMMM).
5. Z. Hu, K. Yuan, X. Liu, Z. Yu, Y. Zong, J. Shi, H. Yue, and J. Yang. (2025). FEALLM: Advancing Facial Emotion Analysis in Multimodal Large Language Models with Emotional Synergy and Reasoning. ACM International Conference on Multimedia (ACMMM).
6. S. Chu, J. Shi, X. Cheng, H. Chen, X. Liu, and G. Zhao. (2025). To Remember, To Adapt, To Preempt: A Stable Continual Test-Time Adaptation Framework for Remote Physiological Measurement in Dynamic Domain Shifts. ACM International Conference on Multimedia (ACMMM).
7. J. Yang, X. Lin, Y. Cui, L. Zhang, X. Liu, H. Li, and Z. Yu. (2025). Dual Alignment of Domain and Modality for Face Anti-spoofing. International Conference on Computer Vision (ICCV).
8. C. Cao, H. Yue, X. Liu, and J. Yang. (2025). Zero-shot Video Restoration and Enhancement Using Pre-Trained Image Diffusion Model. AAAI Conference on Artificial Intelligence (AAAI).
9. K. Yuan, Z. Yu, X. Liu, W. Xie, H. Yue, and J. Yang. (2024). AUFormer: Vision Transformers are Parameter-Efficient Facial Action Unit Detectors. European Conference on Computer Vision (ECCV).
10. X. Ge, X. Liu, Z. Yu, J. Shi, C. Qi, J. Li, and H. Kälviäinen. (2024). DiffFAS: Face Anti-Spoofing via Generative Diffusion Models. European Conference on Computer Vision (ECCV).
11. H. Yue, Y. Cheng, X. Liu, and J. Yang. (2023). Recaptured Raw Screen Image and Video Demoiréing via Channel and Spatial Modulations. Neural Information Processing Systems (NeurIPS).
12. X. Liu, H. Shi, H. Chen, Z. Yu, X. Li, and G. Zhao. (2021). iMiGUE: An Identity-free Video Dataset for Micro-Gesture Understanding and Emotion Analysis. IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).
发明专利
1. 刘鑫; 胡卓昭; 陶文婷; 岳焕景; 杨敬钰. “基于视觉语言预训练和多模态协同融合的情感识别方法”,中国发明专利,202411001937.0.
2. 刘鑫; 张雨婷; 余梓彤; 岳焕景; 杨敬钰. “基于掩码自编码器进行远程生理测量的自监督预训练方法”,中国发明专利,202310445533.X.
3. 刘鑫; 苑凯深; 岳焕景; 杨敬钰. “基于动态关联信息嵌入的面部动作单元检测”,中国发明专利,202310452017.X.
4. 刘鑫; 高榕; 杨敬钰; 岳焕景. “一种基于片段驱动对比学习的骨骼动作识别方法”,中国发明专利,202210957982.8.
5. 刘鑫; 郝超; 岳焕景; 杨敬钰. “基于动作识别和序列推理的动作预测方法”,中国发明专利,202311833547.5.
6. 刘鑫; 葛新旭; 余梓彤; 岳焕景; 杨敬钰. “一种基于扩散模型的人脸反欺骗方法”,中国发明专利,202410402334.5.
1. 芬兰国家IT科学中心高性能计算项目,“面向人类情绪预测的多模态大模型”,主持
2. 国家自然科学基金面上项目,“基于自监督学习的人体微姿态识别和情感分析”,主持
3. 芬兰教育部博士联合培养基金,“基于无身份微姿态的情感识别和理解”,主持
4. 芬兰科学院博士后基金,“胜败之间:基于身份无关肢体语言识别和理解的情绪人工智能”,主持
5. 芬兰KAUTE基金会项目,“面向隐性情绪理解的人体微姿态分析”,主持
6. 芬兰技术促进基金会项目,“基于计算机视觉方法的交际手势分析”,主持
7. 澳大利亚教育部奋进研究学者,“基于时序动态与上下文理解的交际手势分析”,主持
课题组每年招收研究助理和客座/访问学生若干,有意者请邮件联系。详细招生信息请见:https://linuxsino.github.io/position.html