Derek F. Wong

Title

Understanding the Cross-Lingual Mechanism of Large Language Models

Abstract

Current large language models (LLMs) are pre-trained on multilingual corpora. However, their black-box nature limits our understanding of their internal multilingual processing and cross-lingual mechanisms. This talk explores the uncovering of cross-language alignments, neural activation mechanisms, and efficient fine-tuning methods of large language models, offering insights into their cross-language capabilities.

Bio

Derek F. Wong serves as an Associate Professor in the Department of Computer and Information Science at the University of Macau, where he leads the Natural Language Processing & Portuguese-Chinese Machine Translation Laboratory (NLP2CT Lab). He earned his Ph.D. from Tsinghua University and focuses his research primarily on neural machine translation and natural language processing. In 2012 and 2022, Dr. Wong was awarded the Second-class Prize in the Science and Technology Progress category and the Second-class Prize in the Technological Invention category of the Macao Science and Technology Awards, respectively. His research excellence was further recognized in 2022 when he received the FST’s Research Excellence Award and the Incentive Scheme for Outstanding Academic Staff at the University of Macau. In 2024, he was honored with the University of Macau’s Teaching Excellence Award for his exceptional teaching performance. Dr. Wong is a proactive member of the professional community, serving on the Board of Directors and the Machine Translation Steering Committee of the Chinese Information Processing Society of China (CIPS), and as a Member-at-Large of the Asian Federation of Natural Language Processing (AFNLP). He also holds editorial roles, including Associate Editor for IEEE/ACM TASLP and ACM TALLIP, Action Editor for TACL and ACL Rolling Review, and as a member of the Editorial Boards for the Springer Machine Translation journal and Elsevier's Natural Language Processing series. He has also made significant contributions as a Program Co-Chair, Senior Area Co-Chair, Area Co-Chair, and Program Committee Member at numerous important conferences, such as ACL, NeurIPS, ICML, IJCAI, AAAI, EMNLP, NAACL, COLING, and IJCNLP.

黄辉教授

标题

理解大语言模型的跨语言机制

摘要

大语言模型在预训练中涵盖了多种语言的知识。由于其黑箱特性，我们对其如何在内部处理不同语言以及跨语言转换的机制知之甚少。本次报告将通过探索大语言模型内部的跨语言对齐关系、神经激活机制和高效训练方法，为理解其跨语言能力提供一定启示。

报告人介绍

黄辉，澳门大学，副教授，博士生导师，澳门大学自然语言处理与中葡机器翻译实验室主任。自1998年一直在澳门大学及澳门科学技术发展基金委员会的强力支持下专注于自然语言处理和机器学习的基础与应用研究，尤其是具澳门特色的跨语言机器翻译研究。所承担的澳门科学技术发展基金项目《葡中机器翻译系统技术与应用》于2012年获得了澳门科技进步奖二等奖（评审未评一等奖）、于2022年获得了澳门科技发明奖二等奖（评审未评一等奖），获得2022年澳门大学科技学院优秀科研奖、及澳门大学优秀学术年度奖项、于2024年获得了澳门大学卓越教学奖。现任中国中文信息学会理事、中国中文信息学会机器翻译专委会委员、亚洲自然语言处理学会(AFNLP)全体会员代表、IEEE/ACM TASLP和ACM TALLIP期刊副主编、TACL和ACL Rolling Review Action Editor、Springer Machine Translation Journal编辑委员会委员、Elsevier Natural Language Processing编辑委员会委员、国家自然科学基金评审专家。曾担任MIT Computational Linguistics (CL)、IEEE/ACM TASLP、ACM TALLIP、Elsevier CSL、Elsevier DKE等杂志审稿人和国际自然语言处理和人工智能领域主要的学术会议ACL、NeurIPS、ICML、IJCAI、AAAI、EMNLP、NAACL、COLING、IJCNLP等会议的共同主席和程序委员会委员。