法国国家科学研究中心LIRIS博士后职位: 由文本和音频驱动的视频合成
Post-Doctoral Position: Synthesis Of Videos Driven By Text And Audio
Description of the research project
In recent years, voice interaction with computers has made significant progress. Virtual agents offer a user-friendly human-machine interface while reducing maintenance costs. Speechbased interaction is already effective, as proved by Siri, Google Assistant or Alexa virtual agents, however, their visual counterpart is still far behind. The level of user engagement for audiovisual interactions is much higher than for purely audio interactions. Therefore, it is desirable to be able to associate face visual animations with the generated audio.
A notable advancement in video generation was made by a team at Stanford University in 2019 in partnership with Adobe 1. Their work is aimed at enabling a video editing technology of a person's face-to-face scene to revise its speech script and adapt the rendering automatically based simply on this revised text.
The latest advances in the field of audio-driven face video synthesis were presented in 2. The proposed approach generalizes across different people, to synthesize videos of a target actor with the voice of any actor from an unknown source or even synthetic voices that can be generated using standard text-to-speech approaches.
During this post-doctoral contract, we want to work on the development of a prototype of a Text to Speech to Video technology with a sufficient level of accuracy.
Duration, place to work and supervisors
The funding covers 18 months of post-doc, the desired start is April-May 2022. The postdoctoral fellow will be attached to the LIRIS (Laboratory of Computer Science in Image and Information Systems) on the campus of the University Lyon 2 in Bron. Some stages of work can be conducted in the office of Mon Petit Placement in Lyon.
The post-doc will be supervised by IuliiaTkachenko and Serge Miguet (LIRIS). The project manager on the side of Mon Petit Placement is the technical director of the startup Thibault Jaillon.
References
1 O. Fried, A. Tewari, M. Zollhöfer, A. Finkelstein, E. Shechtman, D. Goldman, K. Genova, Z. Jin, C. Theobalt, M. Agrawala, “Text-based editing of talking-head video”, ACM Transactions on Graphics (TOG), Vol.38, 2019.
2 J. Thies, M. Elgharib, A. Tewari, C. Theobalt, M. Niessner, “Neural Voice Puppetry: Audio-driven Facial Reenactment”, ECCV 2020 (https: // justusthies.github.io/posts/neural-voice-puppetry/ )
Offer Requirements Specific Requirements
The candidate must have a PhD in computer science, specializing in image and video processing
Programming languages: Python/C++
Neural network libraries: PyTorch/Keras/Tensorflow
Programming tools for image analysis: OpenCV
Scientific knowledge: machine learning and deep learning, video analysis and processing
Good writing skills and proficiency in written and spoken English (French is not a requirement)
Contact Information
Organisation/Company: CNRS LIRIS
Organisation Type: Small Medium Enterprise, Start-up
Website: https: // perso.liris.cnrs.fr/itkachenko/
Country: France
(责任编辑：shenqian)
更多信息，请扫描下方二维码关注博士研究生招聘网官方微信（微信号：job100zp）与官方微博。
官方微信
