Quantcast
Viewing all articles
Browse latest Browse all 14069

Running joblib over multiple nodes in a HPC cluster

Hello I am tryning to solve a simple optimization for a million times with different inputs. For that I have the following structure:

input = list of N arraysmyfun() = Takes one array solves and returns another list as answer

Now I want to parallelize this over three nodes, where each node has 48 CPUs.

Can anyone tell me how to proceed?

Currently, I can run a joblib parallel delayed function, but that can only work over a single node.

Currently, I can run a joblib parallel delayed function, but that can only work over a single node.


Viewing all articles
Browse latest Browse all 14069

Trending Articles



<script src="https://jsc.adskeeper.com/r/s/rssing.com.1596347.js" async> </script>