Retrieval-augmented generation in multilingual settings
Retrieval-augmented generation (RAG) in the multilingual setting (mRAG). Our findings highlight that despite the availability of high-quality off-the-shelf multilingual retrievers and generators, task-specific prompt engineering is needed to enable generation in user languages. Moreover, current evaluation metrics need adjustments for multilingual setting, to account for variations in spelling named entities.
DEBiT (Dual Encoder Binocular Transformer)
An end-to-end trained agent for image goal navigation. Accompanies ICLR24 paper End-to-End (Instance)-Image Goal Navigation through Correspondence as an Emergent Phenomenon.
.
MASt3R
Based upon the breakthrough framework, DUSt3R, MASt3R provides metric 3D reconstruction and dense local feature maps capable of handling thousands of images.
Can be used for machine translation, speech translation, language modeling and dialogue supporting a number of popular pre-trained models.
mHuBERT-147
A promising compact model for speech processing pipelines, offering an unprecedented balance between high performance and parameter efficiency. Developed within the the EU UTTER project.
3D reconstruction and visual localization with no user intervention and no priors using only a few images.
A toolkit for controlling language models and other generative models.
Unsupervised representation learning task trained from pairs of images showing the same scene from different viewpoints.
These prompted datasets to benchmark the ability of a model to perform completely unseen tasks specified in natural language.
These prompted datasets to benchmark the ability of a model to perform completely unseen tasks specified in natural language.
A general framework for imposing constraints on samples of pretrained language models
