We are trying to better integrate the WILDS WDL Library with Dockstore. We’ve been able to upload workflows with relative ease (see this collection), but there is still a large part of that repository called “modules” that don’t really fit the Dockstore “workflow” paradigm. Specifically, they are collections of common tasks that can be imported and executed within other workflows, but don’t actually contain a workflow and aren’t executable on their own (see “modules/ww-bwa/ww-bwa.wdl” in the repo as an example).
This structure makes me think that modules fit more neatly under the Dockstore “tools” paradigm. Is that possible? I know there’s some guidance out there recommending not to add WDL’s as tools, but this seems like an exception. Curious to hear what you folks think, happy to provide more details if needed! Thanks for your help!
It’s worth a shot. The general recommendation is due to how WDL does not have a strong tools vs workflows separation as compared to CWL where they are explicitly different types of entities. You also will not be able to run them directly in Terra. However, if you feel that your modules are more like tools and listing them on Dockstore is more to make them visible to other workflow developers you can give it a shot.
Basically, you register them via the github app similar to workflows and they should appear in the separate”my tools” and “tools” section of the site.
Do give us a shout if you run into issues (or even if you don’t). This isn’t a heavily used corner of Dockstore so we are curious how it turns out and we would like to give a hand as well if you do run into issues.
A quick heads-up, I looked into this and it seems that there is a old forgotten validation decision that stands in the way here. We’ll need to do a quick tweak on our end to enable this use-case.
Gotcha, thanks for the heads up, and thanks for taking a look at our PR! We’ll keep an eye out for that tweak to be merged and add those modules to Dockstore once you’re ready.
I noticed that the PR you mentioned has been merged to main, do you think we’re good to try adding those WILDS WDL modules as tools in Dockstore? Or is there still more testing planned?
We’re planning on deploying Wednesday afternoon, along with a couple other fixes we’re testing. WDL tools seem to work. If you’re curious you could test on https://staging.dockstore.org/ with a fork (as to not confuse production). Otherwise, we would recommend merging on Wednesday.