I'd like to know the best practice of running multiline shell commands in a ml notebook
Here is some pseudocode and how i would run IT in a notebook cell.
%%bash conda activate myenv &&torchrun --standalone --nnodes=1 --nproc-per-node=$NUM_TRAINERS YOUR_TRAINING_SCRIPT.py (--arg1 ... train script args...
I am activating the conda env because I noticed that the magic bash function alone doesn't use the correct environment.
Would it be better to use the command function from azure.ai.ml?
I find this way somehow error prone, even with activating conda and using the magic function.
Are there better alternatives?