Abstract
Publications home
Distributional reinforcement learning for energy-based sequential models
at
| Tetiana Parshakova, Marc Dymetman, Jean-Marc Andreoli |
| Workshop on the Optimization Foundations of Reinforcement Learning (OPTRL) at the Conference on Neural Information Processing Systems (NeurIPS), Vancouver, British Columbia, Canada, 8-14 December, 2019 |
| Download |
@article{parshakova2019distributional,
title={Distributional Reinforcement Learning for Energy-Based Sequential Models},
author={Parshakova, Tetiana and Andreoli, Jean-Marc and Dymetman, Marc},
journal={arXiv preprint arXiv:1912.08517},
year={2019}
}Global Autoregressive Models (GAMs) are a recent proposal [Parshakova et al., CoNLL 2019] for exploiting global properties of sequences for data-efficient learning of seq2seq models. In the first phase of training, an Energy-Based model (EBM) over sequences is derived. This EBM has high representational power, but is unnormalized and cannot be directly exploited for sampling. To address this issue [Parshakova et al., CoNLL 2019] proposes a distillation technique, which can only be applied under limited conditions. By relating this problem to Policy Gradient techniques in RL, but in a \emph{distributional} rather than \emph{optimization} perspective, we propose a general approach applicable to any sequential EBM. Its effectiveness is illustrated on GAM-based experiments.
Related Content
This web site uses cookies for the site search, to display videos and for aggregate site analytics.
Learn more about these cookies in our privacy notice.
Cookie settings
You may choose which kind of cookies you allow when visiting this website. Click on "Save cookie settings" to apply your choice.
This content is currently blocked. To view the content please either 'Accept social media cookies' or 'Accept all cookies'.
For more information on cookies see our privacy notice.