pretrained-language-model

A curated list of pretrained sentence and word embedding models

word2vec, sentence2vec, machine reading comprehension, dialog system, text classification, pretrained language model (i.e., XLNet, BERT, ELMo, GPT), sequence labeling, information retrieval, information extraction (i.e., entity, relation and event extraction), knowledge graph, text generation, network embedding

Code associated with the Don't Stop Pretraining ACL 2020 paper

TextBox is an open-source library for building text generation system.

Source code and data for ACL 2022 paper "P-tuning v2: Prompt Tuning Can Be Comparable to Fine-tuning Across Scales and Tasks"

A plug-and-play library for parameter-efficient-tuning (Delta Tuning)

Worth-reading papers and related resources on attention mechanism, Transformer and pretrained language model (PLM) such as BERT. 值得一读的注意力机制、Transformer和预训练语言模型论文与相关资源集合

MWPToolkit is an open-source framework for math word problem(MWP) solvers.

[NeurIPS 2021] COCO-LM: Correcting and Contrasting Text Sequences for Language Model Pretraining

ELECTRA기반 한국어 대화체 언어모델

AiSpace: Better practices for deep learning model development and deployment For Tensorflow 2.0

Implementation of ICLR 21 paper: Probing BERT in Hyperbolic Spaces

CODER: Knowledge infused cross-lingual medical term embedding for term normalization. [JBI]

[WWW 2022] Topic Discovery via Latent Space Clustering of Pretrained Language Model Representations

This repository contains the code for the paper in Findings of EMNLP 2021: "EfficientBERT: Progressively Searching Multilayer Perceptron via Warm-up Knowledge Distillation".

"Unsupervised Paraphrase Generation using Pre-trained Language Model."

code for our paper "Understanding by Understanding Not: Modeling Negation in Language Models"

Generating Training Data with Language Models: Towards Zero-Shot Language Understanding

Translate Natural Language Processing to SPARQL Query and vice versa

NLP Pretrained Language Models Implementation Study

CoBERTa is a pre-trained models are the pre-trained language models for Comment/ Social Vietnamese datasets.

Examples for pre-training retrieval-extraction based language model

DS-TOD: Efficient Domain Specialization for Task Oriented Dialog

Personal implementation of paper "Approximate Nearest Neighbor Negative Contrastive Learning for Dense Text Retrieval"

Pre-trained AWD-LSTM language model trained on Filipino text corpus using fastai v2. Instructions included.

The source code for #5 in the Logical Reasoning Reading Comprehension Leaderboard `ReClor`.

This repository contains the code for the paper in EMNLP 2021: "HRKD: Hierarchical Relational Knowledge Distillation for Cross-domain Language Model Compression".

Introductory workshop series on NLP and Pretrained Language Models

DS-TOD: Efficient Domain Specialization for Task Oriented Dialog

SynPL: a zero-shot prompt language model to process multiple-choice questions on synonyms