I asked her, “Why Times Roman?” and she replied with a shrug… and a pout! Use the above mentioned Multi-QA models to achieve the optimal performance.Somehow Kate loves typing in MS Word using Times Roman font.
Note: The DPR models perform comparabily bad. They also trained models on the combination of Natural Questions, TriviaQA, WebQuestions, and CuratedTREC.įacebook-dpr-question_encoder-multiset-base trained models based on Google’s Natural Questions dataset:įacebook-dpr-question_encoder-single-nq-base In Dense Passage Retrieval for Open-Domain Question Answering Karpukhin et al. Use the above mentioned Multi-QA models to achieve the optimal performance. You can index the passages as shown here. cos_sim ( query_embedding, passage_embedding )) encode ( 'How many people live in London?' ) #The passages are encoded as,. As detailed here, LaBSE works less well for assessing the similarity of sentence pairs that are not translations of each other.Įxtending a model to new languages is easy by following the description here.įrom sentence_transformers import SentenceTransformer, util model = SentenceTransformer ( 'nq-distilbert-base-v1' ) query_embedding = model. Works well for finding translation pairs in multiple languages. If this is your use-case, the following model gives the best performance: Paraphrase-multilingual-mpnet-base-v2 - Multilingual version of paraphrase-mpnet-base-v2, trained on parallel data for 50+ languages.īitext mining describes the process of finding translated sentence pairs in two languages. Paraphrase-multilingual-MiniLM-L12-v2 - Multilingual version of paraphrase-MiniLM-L12-v2, trained on parallel data for 50+ languages.
This version supports 50+ languages, but performs a bit weaker than the v1 model. Supports 15 languages: Arabic, Chinese, Dutch, English, French, German, Italian, Korean, Polish, Portuguese, Russian, Spanish, Turkish.ĭistiluse-base-multilingual-cased-v2: Multilingual knowledge distilled version of multilingual Universal Sentence Encoder. These models find semantically similar sentences within one language or across languages:ĭistiluse-base-multilingual-cased-v1: Multilingual knowledge distilled version of multilingual Universal Sentence Encoder. We used the following 50+ languages: ar, bg, ca, cs, da, de, el, en, es, et, fa, fi, fr, fr-ca, gl, gu, he, hi, hr, hu, hy, id, it, ja, ka, ko, ku, lt, lv, mk, mn, mr, ms, my, nb, nl, pl, pt, pt-br, ro, ru, sk, sl, sq, sr, sv, th, tr, uk, ur, vi, zh-cn, zh-tw. Details are in our publication Making Monolingual Sentence Embeddings Multilingual using Knowledge Distillation. You do not need to specify the input language. The following models generate aligned vector spaces, i.e., similar inputs in different languages are mapped close in vector space.
0 kommentar(er)
