Workflow - Info - Descriptor Type: CWL is wrongly set instead of WDL

I am trying to create a workflow in Dockstore based on our GitHub repository with the .dockstore.yml below. I did set subclass: WDL.

$ cat .dockstore.yml 
version: 1.2
workflows:
  - name: bio-diversity-genomics-garg
    subclass: WDL
    primaryDescriptorPath: /hello.wdl
    testParameterFiles:
      - /hello.inputs.json
    authors:
      - name: XXXX (masked)
        email: XXXX (masked)

However, In the Info tab, the Descriptor Type is detected as “CWL”.

And in the Files tab, it seems that the Descriptor files - /hello.wdl is recognized as CWL file. I see the message below on the Files tab.

**/hello.wdl**: CWL file is malformed or missing, cannot extract metadata: expected '<document start>', but found '<scalar>' in 'string', line 16, column 6: File output_greeting = stdout() ^

As a reference, here is the .dockstore.yml, recognized at Dockstore. I am not sure how often the Docksrtore GitHub app synchronizes the info from GitHub. It’s old version unlike the one on the GitHub repository.

version: 1.2
workflows:
  - subclass: WDL
    primaryDescriptorPath: /hello.wdl
    testParameterFiles:
      - /hello.inputs.json
    authors:
      - name: XXXX (masked)
        email: XXXX (masked)

I see the workflow is “Last Modified 20 hours ago”.

However, the GitHub App logs are updated just after a few minutes when I updated the GitHub repository. So, it seems that the GitHub App logs are correctly updated. But the tabs info is not.

So, how to update the workflow tabs information? It is delayed?

Hello, Jun. I’m investigating your problem. It looks like somehow, the CWL descriptor type got associated with your workflow. At any point in the past, did you specify that your workflow was CWL?

Thanks for the info,
Steve

I found the cause of the not updating Files tab. When the workflows - name attribute was added in the .dockstore file and pushed, the Files tab was not updated. Then I reverted the commit, then the Files tab was updated. I don’t know why. This is surprising. Though I still see the Descriptor Type: CWL.

diff --git a/.dockstore.yml b/.dockstore.yml
index c71c701..98f831e 100644
--- a/.dockstore.yml
+++ b/.dockstore.yml
@@ -1,6 +1,7 @@
 version: 1.2
 workflows:
-  - subclass: WDL
+  - name: bio-diversity-genomics-garg
+    subclass: WDL
     primaryDescriptorPath: /bio-diversity-genomics-garg.wdl
     testParameterFiles:
       - /bio-diversity-genomics-garg.inputs.json

Here is the current .dockstore.

$ cat .dockstore.yml 
version: 1.2
workflows:
  - subclass: WDL
    primaryDescriptorPath: /bio-diversity-genomics-garg.wdl
    testParameterFiles:
      - /bio-diversity-genomics-garg.inputs.json
    authors:
      - name: XXXX (masked)
        email: XXXX  (masked)
      - name: XXXX (masked)
        email: XXXX (masked)

Thanks for your help! No. I didn’t specify that.

Our initial files were below. There was 1 mistake in the files. As I did set the hello.inputs.json as empty {} wrongly, it caused the error.

$ cat .dockstore.yml 
version: 1.2
workflows:
  - name: bio-diversity-genomics-garg
    subclass: WDL
    # primaryDescriptorPath: /bio-diversity-genomics-garg.wdl
    primaryDescriptorPath: /hello.wdl
    testParameterFiles:
      # -  /bio-diversity-genomics-garg.inputs.json
      -  /hello.inputs.json

$ cat hello.inputs.json 
{
}

Then our initial push for the workflow was the error below. Then we fixed it.

2022-09-04T19:15 	junaruga	BioDivGenomics	refs/heads/main	Failed	Push
Error reading .dockstore.yml: Cannot create property=testParameterFiles for JavaBean=io.dockstore.common.yaml.YamlWorkflow@67ef7d43 in 'string', line 3, column 5: - subclass: WDL ^ No single argument constructor found for interface java.util.List : null in 'string', line 5, column 25: testParameterFiles: /hello.inputs.json ^ 

Hi, Jun. Right now, on our site, I see two corresponding workflows: one named “BioDivGenomics” with descriptor type CWL, and the other named “BioDivGenomics/bio-diversity-genomics-garg” with descriptor type WDL. I haven’t conclusively determined why the first workflow was created with type CWL, it was possibly due to a bug in our system…

At the moment, for workflows registered via .dockstore.yml, the descriptor type is sticky, and the workflow cannot be deleted, but as a workaround, you can unpublish any workflows that you do not want visible to the public.

Apologies for the inconvenience,
Steve

@svonworl thanks for your investigation! I think the two workflows are related to Workflow - Info - Descriptor Type: CWL is wrongly set instead of WDL - #4 by junaruga .

At the moment, for workflows registered via .dockstore.yml, the descriptor type is sticky, and the workflow cannot be deleted, but as a workaround, you can unpublish any workflows that you do not want visible to the public.

OK. How to unpublish? I don’t see any published workflows by the command below.

$ dockstore workflow list
NAME   DESCRIPTION   GIT REPO   PUBLISHED

The dockstore workflow publish --unpub <something>?

It seems that the database table data is broken? I am ok that you delete the two workflows data on the database if it is possible. Then we can recreate the new workflow.

Hi, Jun. Yes, you can use the CLI to unpublish a workflow with the following invocation:

dockstore workflow publish --unpub --entry <full-entry-name>

Where <full-entry-name> is the complete name of the workflow i.e. github.com/org/repo/name.

OK. I executed the commands below. But is this your expected result? Perhaps, deleting the current Dockstore account, and recreating the new account is easy, isn’t it? I really want to fix this issue ASAP.

$ dockstore workflow publish --unpub --entry github.com/GargGroup/BioDivGenomics
The following workflow is already unpublished: github.com/GargGroup/BioDivGenomics

$ echo $?
0
$ dockstore workflow publish --unpub --entry github.com/GargGroup/BioDivGenomics/bio-diversity-genomics-garg
08:46:45.310 [main] ERROR io.dockstore.client.cli.ArgumentUtility - No workflow found with path github.com/GargGroup/BioDivGenomics/bio-diversity-genomics-garg

$ echo $?
12

Hi, Jun. In the latest version of your .dockstore.yml, there is a single workflow with no name. This is probably colliding with the CWL version that’s already in the database. The output you included from the CLI indicates that the workflow was unpublished when you ran the command.

As a quick fix, you can add a name to the workflow in the .dockstore.yml. This will cause Dockstore to register it as a different workflow with the specified name. Then, if you publish that workflow, it will appear to the public.

Please let me know if you have any other questions!

Thanks,
Steve

OK. I will try it. But 2 days ago when I added the name bio-diversity-genomics-garg, the Files tab was not updated. Should I set a different name?

-  - subclass: WDL
+  - name: bio-diversity-genomics-garg
+    subclass: WDL

I added the name bio-diversity-genomics-garg. But the Info and Files tabs were not updated. Then I renamed the bio-diversity-genomics-garg to bio-div-genomics-garg But the Info and Files tabs were not updated.

Do keep in mind that, regarding the name field in .dockstore.yml, to Dockstore, a workflow with no name and a workflow the name bio-diversity-genomics-garg are two different workflows. You can specify multiple workflows with different names in the same .dockstore.yml.

If you go to “My workflows”, do you see an entry for bio-diversity-genomics-garg, and if so, when you click on it, do you see the correct files?

OK. But I cannot find the entry at “My workflows” page. Where is it on the page?

After you log in, on the home page, click on the button labelled with your user name at upper right, then select “My Workflows” from the menu. On the left, you will see the “My workflows” sidebar, which lists your workflows, grouped by organization. You should see (or be able to open) the github.com/GargGroup subsection and see it there.

Alternately, you should be able to view the workflow at https://www.dockstore.org/my-workflows/github.com/GargGroup/BioDivGenomics/bio-diversity-genomics-garg

I know the the “My workflows” sidebar.

No, I can not see it. See the page’s screenshot. Where is the subsection?

The URL https://www.dockstore.org/my-workflows/github.com/GargGroup/BioDivGenomics/bio-diversity-genomics-garg and https://www.dockstore.org/my-workflows/github.com/GargGroup/BioDivGenomics/bio-div-genomics-garg redirected to the https://www.dockstore.org/my-workflows/github.com/GargGroup/BioDivGenomics.

Hmmm. To eliminate some possible failure modes from consideration, could you please refresh the dockstore page, and look for it in “My Workflows” again, and let me know if you see it?

Clicked my browser (Firefox on Fedora 36)'s reload button, and then clicked “My Workflows” on the right top side bar, and I don’t see the subsection.

I assume that the redirection above is controlled on the server side, right? Can you find the application server log, then can you find why the redirection happened? Or check this page’s template file, then find the criteria to show the subsection?

Good news! Now I can see the subsections on the left top side bar. And I can see the updated the Descriptor Type: WDL at https://dockstore.org/my-workflows/github.com/GargGroup/BioDivGenomics/bio-div-genomics-garg.

Below are the steps that we did.

  1. Deleted Dockstore Account: GarpGroup
  2. Created Dockstore Account: GarpGroup22 (difference account name)
  3. Deleted Dockstore GitHub app on the GitHub side.
  4. Clicked “My Workflows” - and clicked a link like “Discover existing workflows on your accessed GitHub” (I don’t remember the exact name).
  5. Installed Dockstore GitHub app again.

Here is the page’s screenshot. You see the 3 items on the left top side bar.

  • BioDivGenomics
  • BioDivGenomics/bio-div-genomics-garg
  • BioDivGenomics/bio-diversity-genomics-garg

Now I really want to delete “BioDivGenomics” and “BioDivGenomics/bio-div-genomics-garg”.