Skip to content
#

Deep learning

Deep learning is an AI function and subset of machine learning, used for processing large amounts of complex data.

Here are 42,559 public repositories matching this topic...

transformers
NielsRogge
NielsRogge commented Jan 2, 2022

Related to #5142, AlbertTokenizer (which uses SentencePiece) doesn't decode special tokens (like [CLS], [MASK]) properly. This issue was discovered when adding the Nystromformer model (#14659), which uses this tokenizer.

To reproduce (Transformers v4.15 or below):

!pip install -q transformers sentencepiece

from transformers import AlbertTokenizer

tokenizer = AlbertTokenizer.from

Data science Python notebooks: Deep learning (TensorFlow, Theano, Caffe, Keras), scikit-learn, Kaggle, big data (Spark, Hadoop MapReduce, HDFS), matplotlib, pandas, NumPy, SciPy, Python essentials, AWS, and various command lines.

  • Updated Nov 4, 2021
  • Python
Wikipedia
Wikipedia