Skip to content
#

nlp-library

Here are 228 public repositories matching this topic...

transformers
stas00
stas00 commented Jun 22, 2021

huggingface/transformers#12276 introduced a new --log_level feature, which now allows users to set their desired log level via CLI or TrainingArguments.

run_translation.py was used as a "model" for other examples.

Now we need to replicate this to all other Trainer-based examples under examples/pytorch/, the 3 changes are

  1. importing datasets
  2. using `training

Ekphrasis is a text processing tool, geared towards text from social networks, such as Twitter or Facebook. Ekphrasis performs tokenization, word normalization, word segmentation (for splitting hashtags) and spell correction, using word statistics from 2 big corpora (english Wikipedia, twitter - 330mil english tweets).

  • Updated Feb 8, 2021
  • Python

PyNLPl, pronounced as 'pineapple', is a Python library for Natural Language Processing. It contains various modules useful for common, and less common, NLP tasks. PyNLPl can be used for basic tasks such as the extraction of n-grams and frequency lists, and to build simple language model. There are also more complex data types and algorithms. Moreover, there are parsers for file formats common in NLP (e.g. FoLiA/Giza/Moses/ARPA/Timbl/CQL). There are also clients to interface with various NLP specific servers. PyNLPl most notably features a very extensive library for working with FoLiA XML (Format for Linguistic Annotation).

  • Updated Mar 13, 2019
  • Python
neomatrix369
neomatrix369 commented Oct 25, 2020

Missing functionality

Currently, the release process (to GitHub and PyPi) is done manually, it's prone to errors, and the two scripts used work best in happy-path use-case scenarios while edge-case even though less to worry about are not taken care of, as well as they could have been.

The release to PyPi should be fail-safe as there is no way to revert if a mistake is made.

**Proposed

Improve this page

Add a description, image, and links to the nlp-library topic page so that developers can more easily learn about it.

Curate this topic

Add this topic to your repo

To associate your repository with the nlp-library topic, visit your repo's landing page and select "manage topics."

Learn more