pandas

We discussed in the past about making pandas examples in the documentation runnable. The original idea was to use Binder for it, which requires a decent amount of hosting, besides setting up things in our end.

There is now a new alternative, based on webassembly, Jupyter Lite. The idea is that th

Describe the bug

Streaming Datasets can't be pickled, so any interaction between them and multiprocessing results in a crash.

Steps to reproduce the bug

import transformers
from transformers import Trainer, AutoModelForCausalLM, TrainingArguments
import datasets

ds = datasets.load_dataset('oscar', "unshuffled_deduplicated_en", split='train', streaming=True).with_format("

• Base README.md
• Quizzes
• Introduction base README
  • Defining Data Science README
  • Defining Data Science assignment
  • Ethics README
  • Ethics assignment
  • Defining Data README
  • Defining Data assignment
  • Stats and Probability README
  • Stats and Probability assignment
• Working with Data base README
  • Rel

I naively tried to do dd.merge(a, b, on="column_with_ten_values"), where a and b were both large DataFrames with thousands of partitions each.

Eventually the compute failed with:

[File /opt/conda/envs/coiled/lib/python3.9/site-packages/dask/dataframe/multi.py:275, in merge_chunk()

File /opt/conda/envs/coiled/lib/python3.9/site-packages/pandas/core/frame.py:9329, i

Recently in Morpheus we encountered a bug where get_current_device_resource was undefined in a place we were not explicitly using it. Most public-facing libcudf APIs provide a memory_resource* as a default argument by calling get_current_device_resource, defined in rmm/mr/per_device_resource.hpp, however in some places this header is not included which requires the caller of libcudf APIs t

Reading currencies, alphavantage returns a greeting note ("welcome") and this note raises an error in alphavantage.py line 363.

            elif "Note" in json_response and self.treat_info_as_error:
                raise ValueError(json_response["Note"])

For this reason, alphavantage does not work in home assistant.

I would like to convert a DataFrame to a JSON object the same way that Pandas does with to_dict().

toJSON() treats rows as elements in an array, and ignores the index labels. But to_dict() uses the index as keys.

Here is an example of what I have in mind:

function to_dict(df) {
    const rows = df.toJSON();
    const entries = df.index.map((e, i) => ({ [e]: rows[i] }));