Possible to use git submodule?

General
1

Hello,
I have a wdl workflow in a GitHub repository that uses a git submodule, i.e. a task in the workflow is in a different GitHub repository.
Running locally on a Linux machine works as long as I run a recursive git clone.
Is it possible to put such workflows into dockstore and have them run on terra, for example, or is it mandatory to have all the wdl scripts in one GitHub repository?

2

Hi there!

Thanks for a great question! I wasn’t aware of anyone having tried this and I can confirm it works on the Dockstore side! Unfortunately it seems like Terra itself may not support this.

Take a look at this simple example that contains two workflows
GitHub - david4096/wdl-bootcamp-jun24 defined in the .dockstore.yml with the second existing within a submodule.

You can see that after setting up the GitHub app connection on Dockstore the published versions showed up from the base repository and from the submodule.

However, I tried launching into Terra from there and it appears there is an issue on the Terra side. I encourage you reach out to them to see if they might work on supporting your use case https://support.terra.bio/hc/en-us .

Automatic Syncing with GitHub Apps and .dockstore.yml – Overview

Best,
David

3

Hi David,
thanks for your response. Good to know that dockstore supports this. I did run into an error when I tried to launch the workflow on terra and contacted terra support. Their response was that “it might be possible to turn on dockstore for the submodule as well, but I think the best solution would be to add it to the main WDL repository directly.” I think you are right, terra doesn’t support this, which is too bad, I find code reuse and not duplicating code is a basic best practice.

1 Like
4

Hi again,

Yes submodules are a great way to share code between repos! If you have any further contact with them about getting this supported let us know and we can help highlight it. We will also mention it to encourage them to support this use case.

And do let us know if you have any other questions about using Dockstore!

Thanks,
David

5

Hi David,
thanks for you support.
I submitted a feature request on terra.bio to support git submodules.
If you get a chance, please upvote:
request to support git submodules

6

Thanks for doing that. We’ll highlight your request as well!