Towards AI Models That Can Visually Understand the World's Cultures

Neubig will discuss a new frontier in AI models, vision-language models that understand the world's cultures. First, he will talk through the training of multilingual, multimodal, multicultural models that understand images and text and have an increased ability to answer culture-specific questions about multimodal data. Then he will discuss work on "image transcreation", where models have been developed that can transform images to make them more relevant to a particular culture. This work has applications in several areas, such as cultural localization of educational materials (to accompany translated text). The talk will focus on examples from the African context and the challenges we currently face.

Graham Neubig is an associate professor at the Language Technologies Institute of Carnegie Mellon University. His research focuses on natural language processing, with a particular interest in fundamentals, applications, and understanding of large language models for tasks such as question answering, code generation, and multilingual applications. His final goal is that every person in the world should be able to communicate with each other, and with computers in their own language. He also contributes to making NLP research more accessible through open publishing of research papers, advanced NLP course materials and video lectures, and open-source software, all of which are available on his website.