https://github.com/huggingface/transformers/blob/546dc24e0883e5e9f5eb06ec8060e3e6ccc5f6d7/src/transformers/models/gpt2/modeling_gpt2.py#L698

Assertions can't be relied upon for control flow because they can be disabled, as per the following:

$ python --help
usage: python [option] ... [-c cmd | -m mod | file | -] [arg] ...
...
-O     : remove assert and __debug__-dependent statem

Dear all,

I hope to find you well. The current codebase is totally closed and it is impossible to retrieve all the metrics computed. Allow all metrics to be returned inside dataset.evaluate. For instance, currently, we cannot get the map per class that is printed inside [eval_map](https://github.com/open-mmlab/mmdetection/blob/414c62cb12d1b0ebf6151a697a32b84adc269ca4/mmdet/core/evaluati

🐛 Bug

Tensorboard logging metrics from different GPUs when using DataParallel training.

To Reproduce

Use any arbitrary toy model with DP as an accelerator and in training_step(), include the below line of code:

self.log(mode + '_loss', sum(losses), on_epoch=False, on_step=True, prog_bar=True, logger=True, sync_dist=True)

In the trainer, specify tensorboard logger with

Change tensor.data to tensor.detach() due to
pytorch/pytorch#6990 (comment)
tensor.detach() is more robust than tensor.data.

Bug Report

Is the issue related to model conversion?

If the ONNX checker reports issues with this model then this is most probably related to the converter used to convert the original framework model to ONNX. Please create this bug in the appropriate converter's GitHub repo (pytorch, tensorflow-onnx, sklearn-onnx, keras-onnx, onnxmltools) to get the best help.

Describe the bug

T

Is your feature request related to a problem? Please describe.
I typically used compressed datasets (e.g. gzipped) to save disk space. This works fine with AllenNLP during training because I can write my dataset reader to load the compressed data. However, the predict command opens the file and reads lines for the Predictor. This fails when it tries to load data from my compressed files.

Discussed in microsoft/nni#4070

[2021-08-14 10:13:41] INFO (NNIDataStore) Datastore initialization done
[2021-08-14 10:13:41] INFO (RestServer) RestServer start
[2021-08-14 10:13:41] INFO (RestServer) RestServer base port is 8080
[2021-08-14 10:13:41] I