Yang Zhang

Yang Zhang is a speech researcher whose work includes DiffS4L, a method for augmenting limited unannotated speech with diffusion-generated audio. He coauthored the ICML 2024 paper with Heting Gao, Kaizhi Qian, Junrui Ni, Chuang Gan, Mark A. Hasegawa-Johnson, and Shiyu Chang, and introduced it in the recorded ICML coverage published by Latent Space, identifying himself as being from IBM Research.[4][1]

DiffS4L trains a diffusion model on the available unannotated speech, uses it to generate synthetic speech with different levels of variation, and pretrains a speech self-supervised learning model on both real and synthetic audio. The approach expands the training material without requiring speech annotations.[4]

The authors reported a 6.26-percentage-point reduction in word error rate for a pretrained HuBERT model in an English automatic speech recognition experiment. They also reported improvements from synthetic speech with new prosody, new speakers, and new content—even though that new content was “mostly babble.” This result concerns their reported experiment, rather than a general improvement across speech recognition systems.[4]

Zhang's earlier publications provide context for this speech-learning work: in 2019 and 2020, he collaborated on voice style transfer and the separation of speech into distinct components.[5][6]

In 2019, Zhang and Kaizhi Qian contributed equally to AutoVC, a method for zero-shot voice style transfer using only autoencoder loss. Their coauthors were Shiyu Chang, Xuesong Yang, and Mark Hasegawa-Johnson. Zhang was then affiliated with MIT-IBM Watson AI Lab and IBM Research in Cambridge.[5]

In 2020, Zhang coauthored SpeechSplit with Kaizhi Qian, Shiyu Chang, Mark Hasegawa-Johnson, and David Cox. The method addresses the difficulty of separating pitch, rhythm, and content when those properties remain mixed together. It uses three information bottlenecks to decompose speech into timbre, pitch, rhythm, and content, allowing separate style transfer on timbre, pitch, and rhythm without text labels.[6]

Zhang earned his PhD at the University of Illinois at Urbana-Champaign under Mark Hasegawa-Johnson, who also appears among his collaborators on AutoVC, SpeechSplit, and DiffS4L.[2][5][6][4]