Hi,
My team has recently started using dockstore to run workflows from a github repo in Terra. This has been immensely useful, but there is one pretty major pain point. The problem is that only the person who registered the workflow to dockstore seems to be able to refresh the workflow to pull new changes from github. Optimally this would be automatic, and it appears there is a solution for this which is using github apps. This sounds awesome, but as far as I can tell this is not yet implemented for workflows? Any idea on when this functionality will be added? It would be a truly massive improvement imo.
While waiting for automatic syncing, are there any work arounds for this problem? I found a post here (Share Permissions for Git linked Workflows) which seems to imply that workflows that I register from github should be visible and manageable by others in my github org, but that does not seem to be the case. Others on my team cannot see workflows that I have registered in their “My Workflows” page, and I cannot see workflows other have registered from the same repo in my “My Workflows” page. Am I misunderstanding that we should be able to? Or is there something special we need to do to enable that ability?
Thanks in advance for any help
Hey,
Thanks for posting. I’ll try to address all your questions.
GitHub apps will be coming in the 1.9.0 release, which is scheduled for the beginning of July.
For the other point, you are correct. You should be able to see the other workflows, assuming that you and your teammates all have access to the GitHub workflows (ex. through an organisation). The trick is that you must press the refresh all (or refresh organisation) button. These buttons do roughly three things
- Update all your existing workflows
- Add new workflows that have been added since the last refresh
- Add users to workflows they should have access to
The third point is what is important in your case. So I think if everyone does a refresh all (or refresh organisation if it is all within one organisation) then you will see each others workflows. Let me know if that works for you.
We are adding a sync button in the next release that will explicitly do this, though we are also looking at making this completely automatic in the near future.
-Andrew
That’s fantastic news about github apps, very excited to hear that! Do you know, will the functionality be easily added to workflows which already exist, or will they need to be removed and re-added?
I suspect you are correct that the refresh all will make things work, I hadn’t realized that we need to do that on both ends. However, currently when I try to use refresh all or refresh organization the page thinks for a long time and then eventually hits an error with the following banner
is it possible that there’s some sort of service issue causing that?
Thanks,
Chris
1 Like
Hey Chris,
Sorry for the delayed response, discourse is strange and sometimes notifications get lost.
Yes, the functionality will be quite easy to add to existing workflows. Simply install our GitHub app and add a .dockstore.yml! The existing workflow will be automatically converted. All existing versions of the workflow will persist on Dockstore. Any new version must be added via GitHub Apps.
If you need to update an existing version, there are two options.
- add a .dockstore.yml to the version, it will be converted
- refresh the specific version - Go to the my workflows page for the workflow -> versions tab -> actions -> refresh version. This is only available for versions that existed before the migration.
For the second issue, refresh all takes a very long time as you get more workflows added to your Dockstore account. This can reach a timeout on the server.
In your case, we added a new sync button “Discover Existing Dockstore Workflows”. This will add you to any workflows on Dockstore that you have access to with your GitHub account. If you or your colleagues press this button, you should be able to see and refresh workflows that they have added. You can find this button on the my workflows page on the left side.It is in the … menu beside the + button.
I can provide more information on using GitHub apps if you are interested.
I hope that helps!
-Andrew
Here are some useful links for working with GitHub apps and Dockstore.
This link will take you to a landing page for all of our GitHub app tutorials.
https://docs.dockstore.org/en/develop/getting-started/github-apps/github-apps-landing-page.html
The migration tutorial might be particularly useful for you.
https://docs.dockstore.org/en/develop/getting-started/github-apps/migrating-workflows-to-github-apps.html
