CODE & DATA

A unified encoder of different foundation models excelling in 2D vision, 3D understanding, and 3D human perception. Code accompanies the CVPR 2025 paper.

A principled approach to enforcing strict guarantees for LLMs without compromising their generative capabilities combining an autoregressive proposal distribution with rejection sampling.

A novel query-dependent online soft compression method for RAG that reduces computational overhead while preserving performance. Unlike traditional hard compression methods, which shorten retrieved texts, or soft compression approaches, which map documents to continuous embeddings offline, OSCAR dynamically compresses retrieved information at inference time, eliminating storage overhead and enabling higher compression rates.

Geo4D is a method to repurpose video diffusion models for monocular 3D reconstruction of dynamic scenes including that of long videos.

A transformer-based multi-modal alignment retrieval model  that processes 3D poses, person’s pictures and textual pose descriptions to produce an enhanced 3D-, visual- and semantic aware human pose representation that can sort out partial information (e.g. image with the lower body occluded). Code with ECCV 2024 paper.

Code to learn to solve 16 combinatorial optimization problems with strong transfer learning capacities.

Retrieval-augmented generation (RAG) in the multilingual setting (mRAG). Our findings highlight that despite the availability of high-quality off-the-shelf multilingual retrievers and generators, task-specific prompt engineering is needed to enable generation in user languages. Moreover, current evaluation metrics need adjustments for multilingual setting, to account for variations in spelling named entities.

Covers 12 languages from different families and inherits from the original MASSIVE dataset the annotations for the intent prediction and slot filling tasks. See also the Interspeech 2024 paper.

General encoder for classification. Accompanies ECCV’24 paper.

An end-to-end trained agent for image goal navigation. Accompanies ICLR24 paper End-to-End (Instance)-Image Goal Navigation through Correspondence as an Emergent Phenomenon.

.

Designed to ease the reproducibility and integration of new datasets and models and identify strong baselines.

The meeting data used in this benchmark originally comes from the ELITR dataset. This dataset and experiments are described in the paper and are an output of the EU UTTER project.

Can be used for machine translation, speech translation, language modeling and dialogue supporting a number of popular pre-trained models.

A novel classifier that uses semantic knowledge from class hierarchies. Can be seamlessly integrated with any off-the-shelf OvOD detector, with no additional computational overhead during inference.

Code that accompanies the CVPR 24 paper, Label Propagation for Zero-shot Classification with Vision-Language Models.

3D reconstruction and visual localization with no user intervention and no priors using only a few images.

Code for the paper “Placing Objects in Context via Inpainting for Out-of-distribution Segmentation”, ECCV 2024

Several releases: SPLADE V-2, SPLADE V-3, CoSPLADE etc.

A multitask and multilingual speech model covering 99 languages.

A simple yet effective single-shot method to detect multiple people in an image and estimate their pose, body shape and expression. Training and demo code.

Code to learn to solve 4 standard combinatorial optimization problems: TSPs, CVRP. OP and KP accompanying NeurIPS23 paper.

The SHOWMe dataset comprises 96 videos with their associated high-quality textured meshes of a hand holding an object.

Collaboration with INRIA.

SPLADE is sparse bi-encoder BERT-based model for effective and efficient first-stage ranking.