Quantcast
Channel: Active questions tagged python - Stack Overflow
Viewing all articles
Browse latest Browse all 14126

Nextflow Execution Environment Differs Between Processes

$
0
0

I am defining two nextflow processes. The first one, scatter(), creates two files. Then, parallel() is spawned twice, once for each file.

Here is my setup.

// bug.nfnextflow.enable.dsl = 2workflow {    main:        scatter(params.config)        scatter.out.configs            | flatten            | parallel}process scatter {    container "python:3.11.8"    input:        path "config.txt"    output:        path "config*.txt", emit: configs    script:"""        echo $PWD        ls -hal /home/alex/my_cool_repo        touch config1.txt        touch config2.txt"""}process parallel {    container "python:3.11.8"    input:        path "config.txt"    script:"""        echo $PWD        ls -hal /home/alex/my_cool_repo"""}
// run commandnextflow run nextflow/bug.nf --config /home/alex/my_cool_repo/my_cool_repo/config/bla.txt

The ls output from all processes should look the same but it does not.

Output from scatter() (truncated):

/home/alex/my_cool_repototal 656Kdrwxrwxr-x 16 1035 1036 4.0K Feb 17 13:20 .drwxr-xr-x  3 root root 4.0K Feb 17 13:20 ..-rw-rw-r--  1 1035 1036 3.3K Feb 17 11:09 .dockerignore-rw-rw-r--  1 1035 1036 3.2K Feb  6 15:33 .gitignoredrwxrwxr-x  4 1035 1036 4.0K Feb 17 13:20 .nextflow-rw-rw-r--  1 1035 1036 5.4K Feb 17 13:20 .nextflow.log-rw-rw-r--  1 1035 1036    5 Jan 26 18:18 .python-versiondrwxrwxr-x  6 1035 1036 4.0K Feb  7 14:20 .venvdrwxrwxr-x  2 1035 1036 4.0K Feb  6 13:28 .vscode-rw-rw-r--  1 1035 1036  848 Feb 17 12:28 Dockerfile-rw-rw-r--  1 1035 1036  627 Feb  6 15:33 README.mddrwxrwxr-x  3 1035 1036 4.0K Feb 17 12:55 nextflow-rw-rw-r--  1 1035 1036 527K Feb 17 11:45 poetry.lock-rw-rw-r--  1 1035 1036   32 Jan 26 18:18 poetry.toml-rw-rw-r--  1 1035 1036 2.2K Feb 16 19:36 pyproject.tomldrwxrwxr-x  9 1035 1036 4.0K Feb  6 13:28 my_cool_repodrwxrwxr-x  3 1035 1036 4.0K Feb 17 13:20 work

Output from the two parallel() processes:

/home/alex/my_cool_repototal 12Kdrwxr-xr-x 3 root root 4.0K Feb 17 13:20 .drwxr-xr-x 3 root root 4.0K Feb 17 13:20 ..drwxrwxr-x 5 1035 1036 4.0K Feb 17 13:20 work

Why are the outputs not the same?

Context: Instead of ls is actually would like to run poetry run ... but poetry gives the following error message for the parallel() processes: Poetry could not find a pyproject.toml file in /home/alex/my_cool_repo/work/f3/766313fbc5d6aeeb39f19193956ffd or its parents.


Viewing all articles
Browse latest Browse all 14126

Trending Articles



<script src="https://jsc.adskeeper.com/r/s/rssing.com.1596347.js" async> </script>