We have a genotype imputation workflow on Dockstore and once we pull it into the WES engine with WESKit not all the GitHub repository files are available. Doing the same pull from WorkflowHub do however return the full set of directories and files from GitHub. It seems that Dockstore only pulls in files that are in a way mentioned in nextflow.config. Can you please point us in a direction of how we should configure things in Dockstore so that all directories and files are being pulled down?
Hi,
In general for nextflow, I believe that we do pull in files that are reachable from the nextflow.config (and from files that are reachable from it transitively). I think we also look in a few well-known directories such as bin, conf, modules, subworkflows, and workflows with an example at Dockstore
It is possible that we missed an edge case, do you have a link to the workflow?
A fork of the repository we have been testing is here: https://github.com/grbot/chipimputation and on Dockstore the workflow it is here: https://dockstore.org/workflows/github.com/grbot/chipimputation . The WorkflowHub workflow is here: https://workflowhub.eu/workflows/1871?version=1
Hi,
I took a look and saw the extra nextflow files in chipimputation/tools at main · grbot/chipimputation · GitHub
This is indeed a case that we have not experimented with in Nextflow though we have seen it in WDL (for example warp/.dockstore.yml at develop · broadinstitute/warp · GitHub ) . You can add a .dockstore.yml and some extra config files to add these as extra workflows coming from the same repository and it will look like chipimputation/.dockstore.yml at main · denis-yuen/chipimputation · GitHub
I’m not sure that this will be too useful since the launch instructions for Dockstore will probably not work. But this can be a way to get more content from the repository if certain parts of the workflow are not reachable or are considered a separate workflow.
Thank you Dyuen,
Is there a possibility to import the folders /assets, /tools, /templates and /conf?
Regards
Gerrit
Hmmm, the content of tools can be treated like separate workflows like above.
The content of conf can be treated like test parameters like this chipimputation/.dockstore.yml at main · denis-yuen/chipimputation · GitHub which shows up like Dockstore
Templates and assets don’t really fit in any category we have, but I can put in a feature request to add user-customizable directories. Please go to Add user-defineable extra content directories for workflows · Issue #6146 · dockstore/dockstore · GitHub and upvote for our tracking.