transformer

https://github.com/huggingface/transformers/blob/546dc24e0883e5e9f5eb06ec8060e3e6ccc5f6d7/src/transformers/models/gpt2/modeling_gpt2.py#L698

Assertions can't be relied upon for control flow because they can be disabled, as per the following:

$ python --help
usage: python [option] ... [-c cmd | -m mod | file | -] [arg] ...
...
-O     : remove assert and __debug__-dependent statem

Is there a way to train a bidirectional RNN (like LSTM or GRU) on trax nowadays?

From paper, it mentioned

Instead, the training data generator chooses 15% of tokens at random, e.g., in the sentence my
dog is hairy it chooses hairy.

It means that 15% of token will be choose for sure.

From https://github.com/codertimo/BERT-pytorch/blob/master/bert_pytorch/dataset/dataset.py#L68,
for every single token, it has 15% of chance that go though the followup procedure.

1.希望可以把底层的api文档再完善一些，比如encoder,decoder，以便于复现一些论文
2.希望可以维护一个pytorch和paddle的api对照一览表，尽量全一些
3.错误日志能否准确一些，有时候datalaoder出的一些错误信息不好定位
4.能否增加使用梯度累加特性，进一步提高batch size

Hi all,

We are asking for help from our community, hoping to provide Chinese documentation to all users. It's not difficult and here is the chance to be one of the contributors of MMOCR, an open-source rising star!

Here is a list of documentation needed to be translated:

• docs/getting_started.md
• [docs/ins

Hi, I am so interesting in your project, and wonder if you need contributor and how could I make my own contribution?