Current implementation of Go binding can not specify options.

GPUOptions struct is in internal package. And go generate doesn't work for protobuf directory. So we can't specify GPUOptions for NewSession.

🚀 Add missing tokenizer test files

Several tokenizers currently have no associated tests. I think that adding the test file for one of these tokenizers could be a very good way to make a first contribution to transformers.

Tokenizers concerned

not yet claimed

none

claimed

• LED @nnlnr
• Flaubert @anmolsjoshi
• Longformer @tgadeliya
• Electra @rajathb

Followup to pytorch/pytorch#74955 (comment).

It turns out that that cmake version was just bad and we can now unpin cmake once again.

cc @seemethere @malfet @pytorch/pytorch-dev-infra

Description

Per https://discuss.ray.io/t/how-do-i-sample-from-a-ray-datasets/5308, we should add a random_sample(N) API that returns N records from a Dataset. This can be implemented via a map_batches() followed by a take().

cc @simon-mo @clarkzinzow

Use case

Random sample is useful for a variety of scenarios, including creating training batches, and downsampling the dataset for