We want to build an ASR model that jointly learns the transcription and punctuation/case restoration tasks. Solving such a task efficiently would be beneficial to produce more readable transcripts to humans (for instance automatic minuting) or to machines (for further NLP process such as machine translation).
Currently, punctuation and case are restored through a (pipeline) post-processing process that most of the time takes only the ASR transcript as input. Another approach consists in directly producing the cased and punctuated transcription from speech input but large amount of training data is needed for this. In addition to evaluating those baseline approaches, we aim at a better integration of both transcription and restoration tasks. For this we will investigate multi-task objectives and dual decoders. Such a work could be also extended to joint translation and repunctuation from speech. All the developpements related to this internship will be made within the SpeechBrain toolkit.
This internship is a co-supervision between NAVER LABS Europe and SpeechBrain.
https://arxiv.org/abs/2011.00747 Dual-decoder Transformer for Joint Automatic Speech Recognition and Multilingual Speech Translation, Hang Le, Juan Pino, Changhan Wang, Jiatao Gu, Didier Schwab, Laurent Besacier.
Recent Interspeech 2021 papers on punctuation restoration:
NAVER LABS is a world class team of self-motivated and highly engaged researchers, engineers and interface designers collaborating together to create next generation ambient intelligence technology and services that are rich with the organic understanding they have of users, their contexts and situations.
Since 2013 LABS has led NAVER’s innovation in technology through products such as the AI-based translation app ‘Papago’, the omni-tasking web browser ‘Whale’, the virtual AI assistant ‘WAVE’, in-vehicle information entertainment system ‘AWAY’ and M1, the 3D indoor mapping robot.
The team in Europe is multidisciplinary and extremely multicultural specializing in artificial intelligence, machine learning, computer vision, natural language processing, UX and ethnography. We collaborate with many partners in the European scientific community on R&D projects.
NAVER LABS Europe is located in the south east of France in Grenoble. The notoriety of Grenoble comes from its exceptional natural environment and scientific ecosystem with 21,000 jobs in public and private research. It is home to 1 of the 4 French national institutes in AI called MIAI (Multidisciplinary Innovation in Ai) It has a large student community (over 62,000 students) and is a lively and cosmopolitan place, offering a host of leisure opportunities. Grenoble is close to both the Swiss and Italian borders and is the ideal place for skiing, hiking, climbing, hang gliding and all types of mountain sports.
You may choose which kind of cookies you allow when visiting this website. Click on "Save cookie settings" to apply your choice.
FunctionalThis website uses functional cookies which are required for the search function to work and to apply for jobs and internships.
AnalyticalOur website uses analytical cookies to make it possible to analyse our website and optimize its usability.
Social mediaOur website places social media cookies to show YouTube and Vimeo videos. Cookies placed by these sites may track your personal data.
This content is currently blocked. To view the content please either 'Accept social media cookies' or 'Accept all cookies'.
For more information on cookies see our privacy notice.