Let's use this Issue to track performance issues and enhancement requests, so it's easier to prioritize the work.

This is for pytorch transformers

Also I will label it as a Good Difficult Issue in case someone is ready for a challenging but rewarding experience of figuring things out. If you do want to take the challenge comment in the corresponding Issue/PR that resonates with you s

Is there a way to train a bidirectional RNN (like LSTM or GRU) on trax nowadays?

From paper, it mentioned

Instead, the training data generator chooses 15% of tokens at random, e.g., in the sentence my
dog is hairy it chooses hairy.

It means that 15% of token will be choose for sure.

From https://github.com/codertimo/BERT-pytorch/blob/master/bert_pytorch/dataset/dataset.py#L68,
for every single token, it has 15% of chance that go though the followup procedure.

建议在readme里面添加预训练模型环境变量地址修改的提示。
我之前遇到过磁盘空间不够，原因就是这个问题
但不是所有人都会debug源码去查看框架读的是哪个系统变量

Till now, we have released the English model which is trained on
academic datasets. We are planning to support recognition models of more languages.

If you want to support a new language, please provide us with two files:

1. A char_list.txt file, which lists all the characters that used in the new language.
2. A dict_list.txt file, which lists the words in the new language as more as po

Hi, I am so interesting in your project, and wonder if you need contributor and how could I make my own contribution?