SPLADE
This repository contains the weights for NAVER LABS Europe SPLADE Models.
SPLADE is a first-stage ranker trained to predict sparse keyword representations, which are later used in a traditional inverted index. SPLADE obtained state of the art zero shot performance in the zero-shot BEIR benchmark and competitive results on TREC benchmarks.
Available Models
For instructions on the code visit SPLADE on Github.
We provide models with different regularization hyperparameters controlling their sparsity. In addition, we include models with distillation on different datasets (including the latest training data released by Nils Reimers (see here))
We provide results for various models (different init/training). MRR, R@1k and NDCG numbers are multiplied by 100 for simplicity of presentation.
| Models | MS MARCO | TREC-2019 | TREC-2020 | BEIR | DownLoad Link | ||||
|---|---|---|---|---|---|---|---|---|---|
| MRR@10 | R@1k | NDCG@10 | R@1k | NDCG@10 | R@1k | NDCG@10 | FLOPS | |
bert (sentence-transformers/msmarco-bert-base-dot-v5) | 38.1 | - | 71.9 | - | 72.3 | - | - | N/A | |
distilSPLADE v2 | 36.8 | 97.9 | 72.9 | 86.5 | 71 | 83.4 | 49.9 | 3.82 | |
SPLADE-max (train_max.py) | |||||||||
distilbert-base-uncased,λq=0.0008, λd=0.0006 | 36.8 | 97.7 | 72.4 | 82.7 | 70.6 | 78.1 | 48.7 | 1.14 | |
Luyu/co-condenser-marco,λq=0.0008, λd=0.0006 | 38.2 | 98.5 | 73.6 | 84.3 | 72.4 | 78.7 | 50.2 | 1.48 | |
DistilSPLADE-max (train_distill.py) | |||||||||
Luyu/co-condenser-marco,λq=0.01, λd=0.008 | 39.3 | 98.4 | 72.5 | 87.8 | 73 | 83.5 | 50.1 | 5.35 | |
Luyu/co-condenser-marco,λq=0.1, λd=0.08 | 39 | 98.2 | 74.2 | 87.5 | 71.8 | 83.3 | 50.3 | 1.96 | |
Luyu/co-condenser-marco,λq=1.0, λd=0.8 | 37.8 | 97.8 | 71 | 85.4 | 70 | 80.4 | 46.4 | 0.42 | |