Imagine a day when people focus their smartphone cameras on a sign language speaker and have their gestures instantly translated on their screens—that’s Xuan Zhang’s research goal.
A fifth-year doctoral candidate in the Department of Computer Science, Zhang originally came to Hopkins to dive deeper into her study of machine learning, natural language processing, and computer vision to bolster her sign language research, but she quickly discovered there was much more to learn.
“At first, I simply thought sign language was really beautiful,” she says. But when she took an introductory course in American Sign Language at the Krieger School, “it made me realize how ignorant I was and how much I took my privilege of hearing for granted. I learned about Deaf culture and the struggles the community has been through, and it made me consider how I could be more respectful in my research.
“The more I learned, the more I realized how important it is to reduce the communication gap between the Deaf and hearing worlds,” she says.
Xuan Zhang
Supported by a JHU + Amazon Initiative for Interactive AI Fellowship, Zhang is researching sign language recognition and translation tasks with the aim of developing a large vision-language model capable of translating signed words into spoken language.
At present, she is working on integrating valuable linguistic insights—such as fingerspelling, where words are articulated letter by letter, and classifiers, which are shapes and movements that represent entire classes of words—into modern sign language processing methods, which currently neglect these techniques despite their frequent use by the Deaf community.
“My goal is to build a more robust, linguistically nuanced sign language processing system that transcends the conventional limitations of technology,” she says.
Zhang plans to continue her research on real-time sign language translation after completing her PhD this summer, with the ultimate goal of turning her findings into practical tools to benefit the Deaf community. She stresses the necessity of interdisciplinary expertise and collaboration in advancing her mission.
“I’m convinced that when we bring our heads together, pooling our diverse expertise, we can create something truly amazing,” she says. “It’s this kind of teamwork that I believe will lead us to develop more inclusive and useful communication tools for the Deaf community.”