amazon/Titan-text-embeddings-v2

--- license: other license_name: amazon-service-terms license_link: https://aws.amazon.com/service-terms/ language:

• en
• fr
• de
• es
• ja
• zh
• hi
• ar
• it
• pt
• sv
• ko
• he
• cs
• tr
• tl
• ru
• nl
• pl
• ta
• mr
• ml
• te
• kn
• vi
• id
• fa
• hu
• el
• ro
• da
• th
• fi
• sk
• uk
• 'no'
• bg
• ca
• sr
• hr
• lt
• sl
• et
• la
• bn
• lv
• ms
• bs
• sq
• az
• gl
• is
• ka
• mk
• eu
• hy
• ne
• ur
• kk
• mn
• be
• uz
• km
• nn
• gu
• my
• cy
• eo
• si
• tt
• sw
• af
• ga
• pa
• ku
• ky
• tg
• or
• lo
• fo
• mt
• so
• lb
• am
• oc
• jv
• ha
• ps
• sa
• fy
• mg
• as
• ba
• br
• tk
• co
• dv
• rw
• ht
• yi
• sd
• zu
• gd
• bo
• ug
• mi
• rm
• xh
• su
• yo

tags:

• feature-extraction
• sentence-similarity
• mteb

inference: false model-index:

• name: Titan-text-embeddings-v2

results:

• task:

type: Classification dataset: type: mteb/amazon_counterfactual name: MTEB AmazonCounterfactualClassification (en) config: en split: test revision: e8379541af4e31359cca9fbcf4b00f2671dba205 metrics:

• type: accuracy

value: 79.31343283582089

• type: ap

value: 43.9465851246623

• type: f1

value: 73.6131343594374

• task:

type: Classification dataset: type: mteb/amazon_counterfactual name: MTEB AmazonCounterfactualClassification (de) config: de split: test revision: e8379541af4e31359cca9fbcf4b00f2671dba205 metrics:

• type: accuracy

value: 70.94218415417559

• type: ap

value: 82.30115528468109

• type: f1

value: 69.37963699148699

• task:

type: Classification dataset: type: mteb/amazon_counterfactual name: MTEB AmazonCounterfactualClassification (en-ext) config: en-ext split: test revision: e8379541af4e31359cca9fbcf4b00f2671dba205 metrics:

• type: accuracy

value: 82.29385307346327

• type: ap

value: 29.956638709449372

• type: f1

value: 68.88158061498754

• task:

type: Classification dataset: type: mteb/amazon_counterfactual name: MTEB AmazonCounterfactualClassification (ja) config: ja split: test revision: e8379541af4e31359cca9fbcf4b00f2671dba205 metrics:

• type: accuracy

value: 80.06423982869379

• type: ap

value: 25.2439835379337

• type: f1

value: 65.53837311569734

• task:

type: Classification dataset: type: mteb/amazon_polarity name: MTEB AmazonPolarityClassification config: default split: test revision: e2d317d38cd51312af73b3d32a06d1a08b442046 metrics:

• type: accuracy

value: 76.66435

• type: ap

value: 70.76988138513991

• type: f1

value: 76.54117595647566

• task:

type: Classification dataset: type: mteb/amazon_reviews_multi name: MTEB AmazonReviewsClassification (en) config: en split: test revision: 1399c76144fd37290681b995c656ef9b2e06e26d metrics:

• type: accuracy

value: 35.276

• type: f1

value: 34.90637768461089

• task:

type: Classification dataset: type: mteb/amazon_reviews_multi name: MTEB AmazonReviewsClassification (de) config: de split: test revision: 1399c76144fd37290681b995c656ef9b2e06e26d metrics:

• type: accuracy

value: 38.826

• type: f1

value: 37.71339372044998

• task:

type: Classification dataset: type: mteb/amazon_reviews_multi name: MTEB AmazonReviewsClassification (es) config: es split: test revision: 1399c76144fd37290681b995c656ef9b2e06e26d metrics:

• type: accuracy

value: 39.385999999999996

• type: f1

value: 38.24347249789392

• task:

type: Classification dataset: type: mteb/amazon_reviews_multi name: MTEB AmazonReviewsClassification (fr) config: fr split: test revision: 1399c76144fd37290681b995c656ef9b2e06e26d metrics:

• type: accuracy

value: 39.472

• type: f1

value: 38.37157729490788

• task:

type: Classification dataset: type: mteb/amazon_reviews_multi name: MTEB AmazonReviewsClassification (ja) config: ja split: test revision: 1399c76144fd37290681b995c656ef9b2e06e26d metrics:

• type: accuracy

value: 35.897999999999996

• type: f1

value: 35.187204289589346

• task:

type: Classification dataset: type: mteb/amazon_reviews_multi name: MTEB AmazonReviewsClassification (zh) config: zh split: test revision: 1399c76144fd37290681b995c656ef9b2e06e26d metrics:

• type: accuracy

value: 36.068

• type: f1

value: 35.042441064207175

• task:

type: Retrieval dataset: type: arguana name: MTEB ArguAna config: default split: test revision: None metrics:

• type: map_at_1

value: 27.027

• type: map_at_10

value: 42.617

• type: map_at_100

value: 43.686

• type: map_at_1000

value: 43.695

• type: map_at_3

value: 37.684

• type: map_at_5

value: 40.532000000000004

• type: mrr_at_1

value: 27.667

• type: mrr_at_10

value: 42.88

• type: mrr_at_100

value: 43.929

• type: mrr_at_1000

value: 43.938

• type: mrr_at_3

value: 37.933

• type: mrr_at_5

value: 40.774

• type: ndcg_at_1

value: 27.027

• type: ndcg_at_10

value: 51.312000000000005

• type: ndcg_at_100

value: 55.696

• type: ndcg_at_1000

value: 55.896

• type: ndcg_at_3

value: 41.124

• type: ndcg_at_5

value: 46.283

• type: precision_at_1

value: 27.027

• type: precision_at_10

value: 7.9159999999999995

• type: precision_at_100

value: 0.979

• type: precision_at_1000

value: 0.099

• type: precision_at_3

value: 17.022000000000002

• type: precision_at_5

value: 12.731

• type: recall_at_1

value: 27.027

• type: recall_at_10

value: 79.161

• type: recall_at_100

value: 97.937

• type: recall_at_1000

value: 99.431

• type: recall_at_3

value: 51.06699999999999

• type: recall_at_5

value: 63.656

• task:

type: Clustering dataset: type: mteb/arxiv-clustering-p2p name: MTEB ArxivClusteringP2P config: default split: test revision: a122ad7f3f0291bf49cc6f4d32aa80929df69d5d metrics:

• type: v_measure

value: 41.775131599226874

• task:

type: Clustering dataset: type: mteb/arxiv-clustering-s2s name: MTEB ArxivClusteringS2S config: default split: test revision: f910caf1a6075f7329cdf8c1a6135696f37dbd53 metrics:

• type: v_measure

value: 34.134214263072494

• task:

type: Reranking dataset: type: mteb/askubuntudupquestions-reranking name: MTEB AskUbuntuDupQuestions config: default split: test revision: 2000358ca161889fa9c082cb41daa8dcfb161a54 metrics:

• type: map

value: 63.2885651257187

• type: mrr

value: 76.37712702809655

• task:

type: STS dataset: type: mteb/biosses-sts name: MTEB BIOSSES config: default split: test revision: d3fb88f8f02e40887cd149695127462bbcf29b4a metrics:

• type: cos_sim_pearson

value: 89.53738990667027

• type: cos_sim_spearman

value: 87.13210584606783

• type: euclidean_pearson

value: 87.33265405736388

• type: euclidean_spearman

value: 87.18632394893399

• type: manhattan_pearson

value: 87.33673166528312

• type: manhattan_spearman

value: 86.9736685010257

• task:

type: BitextMining dataset: type: mteb/bucc-bitext-mining name: MTEB BUCC (de-en) config: de-en split: test revision:…