Dockstore could not find a workflow in git using yml though it worked previously

I’m using .dockstore.yml to automatically update workflows in Dockstore. It was working with previous tags but recently stopped with the latest tag of our git repo. The error given is
**/.dockstore.yml**: Could not find the primary descriptor file
but the descriptor file (wdl workflow) hasn’t moved. Link to yml and example of workflow. This is happening for many other workflows in yml

Screen Shot 2021-04-05 at 16.49.11

Hi,

Can you confirm that 4.1.8.1 is the version of the workflow you are interested in? We can see gatk/.dockstore.yml at 4.2.0.0 · broadinstitute/gatk · GitHub but not https://github.com/broadinstitute/gatk/blob/4.1.8.1/.dockstore.yml as in the screenshot.

Otherwise, what is the URL of the workflow in Dockstore and do you see anything interesting in the “GitHub App Logs” from my-workflows ?

Ahh, wrong yml link, I’m interested in 4.2.0.0.

Here is the public URL link to the workflow in Dockstore : Dockstore

The logs doesn’t show anything odd, just several successful pushes and deletes for the past 3 days.

Hmmm, it might be worth trying a new tag if possible.
4.2.0.0 was created February 19, 2021.

I can see that newer branches of the workflow like Dockstore (created April 5 seem to have located the main descriptor) also tags on a different workflow Dockstore (April 5th).

We did have a minor release March 8th which deals with some issues we had with large repos.

Just in case I did a test with the same dockstore.yml and just the workflow directory in a separate miniature repo. The workflows were properly transferred over in the master version and also in a mock tag release. So its more likely the gatk repo size or branch number might be the problem and not a bad dockstore.yml file.

If the March update in Dockstore doesn’t help in the next GATK release, would adding a filter to the yml so that it only grabs the master branch and release tags for workflows help?

Hello,

We believe we have fixed the known problems in our 1.10.2 release, which was deployed March 18th. If you can push to that branch again it should be good. And should be good going forward.

That said, the GATK workflow is a bit of an outlier with its 700+ branches and ~15 workflows, and if you can reduce what needs to be updated via filters that would be great.

When is your next GATK release?

GATK4.2.0.0 was a large release, so it will be a while until the next release. The good news is after creating and pushing a branch with the master filter in the dockstore.yml, some of the gatk workflows 4.2.0.0 tags appeared in dockstore

Note, looking back at this issue in 2023 from Dockstore automatic sync stopped working for new tags - #2 by dyuen

Worth noting that the .dockstore.yml in the 4.2.0.0 tag

has no filters (i.e. the successful 4.2.0.0 release may have been successful due to the new (at the time) dockstore release and not the addition of filters.