Deep learning

Current implementation of Go binding can not specify options.

GPUOptions struct is in internal package. And go generate doesn't work for protobuf directory. So we can't specify GPUOptions for NewSession.

Related to #5142, AlbertTokenizer (which uses SentencePiece) doesn't decode special tokens (like [CLS], [MASK]) properly. This issue was discovered when adding the Nystromformer model (#14659), which uses this tokenizer.

To reproduce (Transformers v4.15 or below):

!pip install -q transformers sentencepiece

from transformers import AlbertTokenizer

tokenizer = AlbertTokenizer.from

🐛 Describe the bug

Usage of RRefContext::handleException in torch/csrc/distributed/rpc/rref_context.cpp is wrong when the future has an error.

RRefContext::handleException uses TORCH_CHECK which throws.

Callers of RRefContext::handleException don't expect that and run code after it without any guarding.

Versions

master

cc @pietern @mrshenli @pritamdamania87

After training a model, we should expose utilities so that the user can load their best checkpoint in memory.

1. Trainer.load_checkpoint(path) to load an arbitrary checkpoint.
2. trainer.best_checkpoint to load trainer.best_checkpoint_path into memory.