I’m using .dockstore.yml to automatically update workflows in Dockstore. It was working with previous tags but recently stopped with the latest tag of our git repo. The error given is
**/.dockstore.yml**: Could not find the primary descriptor file
but the descriptor file (wdl workflow) hasn’t moved. Link to yml and example of workflow. This is happening for many other workflows in yml
Can you confirm that 188.8.131.52 is the version of the workflow you are interested in? We can see gatk/.dockstore.yml at 184.108.40.206 · broadinstitute/gatk · GitHub but not https://github.com/broadinstitute/gatk/blob/220.127.116.11/.dockstore.yml as in the screenshot.
Otherwise, what is the URL of the workflow in Dockstore and do you see anything interesting in the “GitHub App Logs” from my-workflows ?
Ahh, wrong yml link, I’m interested in 18.104.22.168.
Here is the public URL link to the workflow in Dockstore : Dockstore
The logs doesn’t show anything odd, just several successful pushes and deletes for the past 3 days.
Hmmm, it might be worth trying a new tag if possible.
22.214.171.124 was created February 19, 2021.
I can see that newer branches of the workflow like Dockstore (created April 5 seem to have located the main descriptor) also tags on a different workflow Dockstore (April 5th).
We did have a minor release March 8th which deals with some issues we had with large repos.
Just in case I did a test with the same dockstore.yml and just the workflow directory in a separate miniature repo. The workflows were properly transferred over in the master version and also in a mock tag release. So its more likely the gatk repo size or branch number might be the problem and not a bad dockstore.yml file.
If the March update in Dockstore doesn’t help in the next GATK release, would adding a filter to the yml so that it only grabs the master branch and release tags for workflows help?
We believe we have fixed the known problems in our 1.10.2 release, which was deployed March 18th. If you can push to that branch again it should be good. And should be good going forward.
That said, the GATK workflow is a bit of an outlier with its 700+ branches and ~15 workflows, and if you can reduce what needs to be updated via filters that would be great.
When is your next GATK release?
GATK126.96.36.199 was a large release, so it will be a while until the next release. The good news is after creating and pushing a branch with the master filter in the dockstore.yml, some of the gatk workflows 188.8.131.52 tags appeared in dockstore
Note, looking back at this issue in 2023 from Dockstore automatic sync stopped working for new tags - #2 by dyuen
Worth noting that the .dockstore.yml in the 184.108.40.206 tag
has no filters (i.e. the successful 220.127.116.11 release may have been successful due to the new (at the time) dockstore release and not the addition of filters.