21,963 questions
-1
votes
0
answers
37
views
High-performance asynchronous processing with dynamic Task prioritization and cancellation [closed]
I'm building a high-throughput data processing pipeline in C# using asynchronous programming with Task and async/await. The pipeline needs to handle a large volume of incoming requests; each ...
- 766
0
votes
2
answers
31
views
What is the difference between an MPI nonblocking collective write, iwrite_all vs a "nonblocking" noncollective iwrite combined with a file sync?
I'm setting up IO for a largescale CFD code using the MPI library and the file IO is starting to eat into computation time as my problems scale.
As far as I can find the "done" thing in the ...
0
votes
1
answer
140
views
How can I optimize my 'foreach' loop to read and process each line of a text file more efficiently in C#?
I have C# code that reads a .TXT file, processes each line, and subsequently inserts documents into a MongoDB database. Since the file contains many lines (approximately 70k), both reading all lines ...
- 19
0
votes
0
answers
19
views
Extracting WAIC using rjags::jags.samples() from single chain of bayesian model run in parallel with JagsUI
I am using JagsUI to run many single and multi-predictor models, using the autojags() function to run them until convergence. I would like to do model selection using WAIC, rather than DIC. The JagsUI ...
- 15
0
votes
0
answers
29
views
Filter collection asynchronously/in parallel?
Consider this:
enum State { OK, Maybe, No };
State CheckState(string s) { /*Some logic*/ }
void Work() {
HashSet<string> myStrings = GetStrings();
var result = myStrings.Where(s => ...
- 1,529
0
votes
1
answer
54
views
How to check the parallel degree of tables and indexes in OceanBase?
In OceanBase (MySQL Mode), I know that I can set the parallel degree of a table or index using SQL commands like:
alter table my_table parallel 4;
alter index my_index parallel 2;
However, for ...
0
votes
0
answers
28
views
Does the mirai package use shared memory parallelism?
Does the mirai R package use shared memory parallelism?
The parallel package uses shared memory parallelism with the mc* (e.g. mclapply) functions and fork clusters, in the sense that the input data ...
- 1,140
2
votes
1
answer
48
views
Solving for cost-optimal processes for bucket sort using Ts/pTp = 1
I am doing a class on parallelization. We're given an MPI program and told to determine the maximum big O for processes so that the program is cost-optimal.
Cost optimality is defined where big theta ...
- 23
-3
votes
1
answer
50
views
Crystal Report generates reports correctly in parallel on IIS but throws System.Runtime.InteropServices.COMException in NUnit test
When running NUnit test to check if multiple PDF reports have been generated correctly the following exception is being thrown System.Runtime.InteropServices.COMException : Invalid index. (Exception ...
- 127
2
votes
1
answer
89
views
Powershell script error when using parallel processing
I'm using PowerShell with -Parallel to speed up a script that checks file formats using ImageMagick. The script works without parallel processing, but it's too slow with a large number of files. When ...
- 45
-3
votes
1
answer
56
views
How to safely thread DBContext in EF Core using Parallel.ForEachAsync [closed]
I am attempting to do some parallel processing interacting with a DbContext in EF Core as follows:
var processingGroups = GenerateProcessingGroups(subscriptions);
await Parallel.ForEachAsync(...
- 943
2
votes
1
answer
53
views
Error in parallel processing in R when using the atakrig package
I am using the atakrig package. When setting the number of cores to 8 for parallel computation I am getting this error: 0%Error in serialize(data, node$con) : error writing to connection. If I set the ...
- 522
0
votes
0
answers
85
views
Pathos multiprocessing map not completing on Linux machine, works on Mac M2
I am using the pathos multiprocessing to parallelize a gradient calculation that is embarrassingly parallel, using finite difference.
Below is a high level example of how it is set up,
`
...
class ...
- 1
0
votes
0
answers
47
views
Use of SLURM scheduler leading to requested threads not being spawned by a program
I am working on a wrapper Rust program that launches many commands via the std::process interface. To parallelize the work, I use the threadpool library. The relevant part of the code is
pool = ...
- 11
2
votes
4
answers
205
views
Is it possible to create a task without starting it, so I can limit max concurrency?
I have a weird scenario. I have a list of groups. For each group, I need to start a task for all the contents, but I need to make sure the last value in each group is run last. Simplified function ...
- 168