Neural language generation models, with the advent of the Transformer architecture , have recently progressed to a point where they can produce highly fluent texts. However, they can be deficient on other important levels: semantic consistency, faithfulness to the facts, toxic or socially biased content.
In order to avoid such problems, we propose to augment these generators with explicit controls that operate at the level of global text sequences, complementing the local training regime of autoregressive models.
Energy-Based Models (EBM) [2,3] provide a natural framework for exploiting such controls and have started to be effective in this respect . However, certain challenges remain: (1) how to increase the ability of such models to efficiently generate texts that fully satisfy the desired controls; (2) how to quickly adapt to dynamic control specifications, without incurring a costly full re-training of the models.
We are looking for a motivated intern to help us develop techniques and algorithms addressing these challenges. Experiments will be conducted on selected text generation tasks, in particular in the context of open-text generation over pre-trained models such as GPT-2/3.
The successful candidate should be enrolled in a graduate program, at the Master or (preferably) PhD level.
Publication in major conferences/journals will be strongly encouraged.
 Vaswani et al. Attention is all you need, NIPS 2017
 LeCun et al. A tutorial on energy-based learning, 2006
 Bakhtin et al. Energy-based models for text, 2020
 Anonymous, A Distributional Approach to Controlled Text Generation, under submission to ICLR-2021
NAVER LABS Europe has full-time positions, PhD and PostDoc opportunities throughout the year which are advertised here and on international conference sites that we sponsor such as CVPR, ICCV, ICML, NeurIPS, EMNLP etc.
NAVER LABS Europe is an equal opportunity employer.
NAVER LABS are in Grenoble in the French Alps. We have a multi and interdisciplinary approach to research with scientists in machine learning, computer vision, artificial intelligence, natural language processing, ethnography and UX working together to create next generation ambient intelligence technology and services that deeply understand users and their contexts.