Skip to content
#

tokenization

Here are 417 public repositories matching this topic...

LunaSec - Open Source AppSec platform that automatically notifies you the next time vulnerabilities like Log4Shell or node-ipc happen. Track your dependencies and builds in a centralized service. Get started in one-click via our GitHub App or host it yourself. https://github.com/apps/lunatrace-by-lunasec/

  • Updated Apr 30, 2022
  • TypeScript
soullesscomputerboy
soullesscomputerboy commented Jun 1, 2021

Currently different transaction types show different details. Asset transactions don't show "To" and "From". On RVN Transactions the "From" is almost always listed as "unknown". Maybe change this to list a sending address and if there is a label on one of the sending addresses, put the label after it in parenthesis like it has handled in the "To" section. See inconsistencies below:
![assettx](htt

Ekphrasis is a text processing tool, geared towards text from social networks, such as Twitter or Facebook. Ekphrasis performs tokenization, word normalization, word segmentation (for splitting hashtags) and spell correction, using word statistics from 2 big corpora (english Wikipedia, twitter - 330mil english tweets).

  • Updated Feb 8, 2021
  • Python

This repository consists of a complete guide on natural language processing (NLP) in Python where we'll learn various techniques for implementing NLP including parsing & text processing and understand how to use NLP for text feature engineering.

  • Updated Nov 9, 2021
  • Jupyter Notebook
firthmj
firthmj commented Jan 21, 2021

How easy would it be to change the library to have versions of the encode and decode functions where the payload JSON was provided / returned just as the JSON text?

There are other good JSON generation / parsing libraries available, and some people may wish to use them to generate or process the payload, rather than the built in claim processing.

enhancement good first issue

Improve this page

Add a description, image, and links to the tokenization topic page so that developers can more easily learn about it.

Curate this topic

Add this topic to your repo

To associate your repository with the tokenization topic, visit your repo's landing page and select "manage topics."

Learn more