About Me

I am a Ph.D. candidate in Computer Science and Technology at Xinjiang University, supervised by Prof. Hao Huang. I am also a research intern at the Speech Processing and Machine Intelligence (SPMI) Lab at Tsinghua University, supervised by Prof. Zhijian Ou.

My research focuses on data-efficient multilingual and crosslingual automatic speech recognition. I am particularly interested in phoneme-based modeling and how shared speech representations can help build recognition systems for low-resource languages.

My work includes Whistle, which uses weak phonetic supervision, and JSA-SPG, a pronunciation-lexicon-free training approach. I am also a core contributor to the CAT speech recognition toolkit.

News

  • Our paper Latent Softmax was accepted to IEEE SLT 2026.
  • I was a speaker at the Interspeech 2026 tutorial on data-efficient multilingual and crosslingual ASR powered by LLMs.
  • Our paper on pronunciation-lexicon-free crosslingual ASR, JSA-SPG, was published online in IEEE TASLP.
  • Whistle was published online in IEEE TASLP.

Research Area

I study methods that make speech recognition more accessible across languages, particularly when transcribed data and pronunciation lexicons are limited.

  • Multilingual, crosslingual and low-resource speech recognition
  • Phoneme-based modeling and speech-language interfaces
  • Weakly supervised and lexicon-free learning
  • Speech foundation models and LLM-based speech recognition

Publications

See also my Google Scholar profile.

  • Latent Softmax for Data-Efficient Phoneme-Based Multilingual ASR Across Tonal and Non-Tonal Languages.
    Saierdaer Yusuyin, Nanling Jiang, Hao Huang, and Zhijian Ou.
    IEEE Spoken Language Technology Workshop (SLT), 2026, accepted. [arXiv]
  • Pronunciation-Lexicon Free Training for Phoneme-Based Crosslingual ASR via Joint Stochastic Approximation.
    Saierdaer Yusuyin, Te Ma, Hao Huang, and Zhijian Ou.
    IEEE Transactions on Audio, Speech and Language Processing, 2026. [Paper] [arXiv] [Code]
  • Whistle: Data-Efficient Multilingual and Crosslingual Speech Recognition via Weakly Phonetic Supervision.
    Saierdaer Yusuyin, Te Ma, Hao Huang, Wenbo Zhao, and Zhijian Ou.
    IEEE Transactions on Audio, Speech and Language Processing, 2025. [Paper] [arXiv] [Code]
  • LLM-based Phoneme-to-Grapheme for Phoneme-Based Speech Recognition.
    Te Ma, Min Bi, Saierdaer Yusuyin, Hao Huang, and Zhijian Ou.
    Interspeech, 2025. [Paper]
  • Investigation into Phone-Based Subword Units for Multilingual End-to-End Speech Recognition.
    Saierdaer Yusuyin, Hao Huang, Junhua Liu, and Cong Liu.
    ICASSP, 2023. [Paper]

Research Experience

  • Research Intern, Speech Processing and Machine Intelligence Lab, Tsinghua University Jan 2023 - present
    Supervisor: Prof. Zhijian Ou

Education

  • Ph.D. in Computer Science and Technology, Xinjiang University Sep 2021 - Dec 2026 (expected)
    Supervisor: Prof. Hao Huang
  • Master's in Software Engineering, Xinjiang University Sep 2019 - Jul 2021
    Supervisor: Prof. Liejun Wang, Transferred to the Ph.D. program.
  • B.S. in Software Engineering, Northwestern Polytechnical University Sep 2015 - Jul 2019

Talks & Tutorial

  • Advances in Data-Efficient Multilingual and Crosslingual Speech Recognition Powered by LLMs.
    Interspeech 2026 tutorial, Sydney, Australia, September 2026. [Tutorial Website]
  • Phoneme-based Speech Recognition Driven by Large Language Models and Sampling Marginalization.
    20th National Conference on Man-Machine Speech Communication, October 2025.
  • Phonemes as Speech-Language Interface for Data-Efficient Multilingual and Crosslingual Speech Recognition.
    Invited talk, Tsinghua University, May 2025.
  • Investigation into Phone-Based Subword Units for Multilingual End-to-End Speech Recognition.
    ICASSP, June 2023.