Unable to get Publish button to become active

I have been attempting to publish a workflow registered from GitHub using a process that I have used before to successfully Publish a workflow. I am unable this time around to get the Publish button to become active. I have checked the paths to all the files and checked the validity of the WDLs, etc. I have attempted to do this from different GitHub repositories and each time I cannot hit Publish.

What are some ideas to troubleshoot?

Hey schaluva,

Just brainstorming some ideas based on your post:

  1. On the versions tab, does it state whether the workflow has any valid versions? At least one valid version is required to publish.
  2. What version of WDL are you using? Currently the production version of Dockstore does not support 1.0 (though you should still be able to publish it with some temp workarounds we have).
  3. Could you link the the github repo and the what files in that repo are used for the test? I can take a look to see if anything stands out.
  4. Are you able to run the workflows locally with Cromwell? Does it say if there are any errors?

Thanks,
Andrew Duncan

Hi Andrew,

  1. It does not show any valid versions but I have another registered workflow that has been able to be Published successfully and has no valid versions.
  2. We are using WDL 1.0 but have been able to Publish other workflows that have a WDL 1.0.
  3. Here is the link to the repo: https://github.com/gevro/gatk4-exome-analysis-pipeline-flat
  4. I don’t believe it has been tested locally on Cromwell but it was exported to a Terra workspace via a non-Dockstore repository and it is running on the platform.

Hey schaluva,

  1. This sounds like a bug on the site. Do you have a link to the workflow with no valid versions that is published?
  2. So currently for WDL 1.0 workflows we have relaxed the validation since we our current parsing library only supports draft-1 and draft-2. The upcoming release will use a new library which will support wdl 1.0 and beyond. Hopefully this means more helpful validation messages in the future.
  3. Thanks, I’ve forked it and will try adding it to Dockstore to recreate the issue. I’ll also test it with our upcoming release to see if it is fixed in that. I’ll get back to you once I have done this.
1 Like
  1. Here is the GitHub link to the version with no valid versions that published: https://github.com/gatk-workflows/gatk4-exome-analysis-pipeline/tree/1.1.0
    and here is the Dockstore link to the registered workflow from above: https://dockstore.org/workflows/github.com/gatk-workflows/gatk4-exome-analysis-pipeline/ExomeGermlineSingleSample:1.1.0?tab=info
1 Like

That is weird, I’m not sure how that happened. I’ve created two issues regarding this post and you can find them below. The issue seems to be related to a bug in our parsing code. I have a fix for this that will go into the next release of Dockstore (September-ish). I am going to see if I can find a workaround for the current version of Dockstore and will post my findings here.

The two issues I made are:


1 Like

Hi,
I’ve manually set the publish flag for the workflows you have from the “gatk4-exome-analysis-pipeline-flat” repo. As Andrew noted, this isn’t really adding official support for WDL 1.0, but your workflows should be visible and reachable by the public.

However, there are some warning messages saying that the workflow doesn’t appear to be WDL draft-2.

1 Like

I created a tool on dockstore based on the container image in quay.io for a bio tool as a trial. And the publish bottom is not active with the message “Unable to publish: No valid version found.”. What is the definition of the valid version?

Hi, Jun. When you select the “Files” tab, what do you see? If you could please post a screenshot here, that would help us to diagnose the problem.

Thanks,
Steve

Thanks for your help! Here are the screenshots for sub tabs in the “Files” tab.

warning A Dockerfile associated with this Docker container could not be found.

warning A Descriptor File associated with this Docker container could not be found.

warning A Test Parameter File associated with this Docker container, descriptor type and version could not be found.

I noticed that I cannot read the manifest info of the “Quay” used for the tool page by docker manifest inspect. I created and pushed the docker container image by Podman originally. The unsupported manifest info might be the cause of the issue above.

$ docker --version
Docker version 20.10.17, build aa7e414

$ podman --version
podman version 4.2.0
$ docker manifest inspect quay.io/junaruga/garg-fastk
unsupported manifest media type and no default available: application/vnd.docker.distribution.manifest.v1+json
$ docker manifest inspect docker.io/ubuntu
{
   "schemaVersion": 2,
   "mediaType": "application/vnd.docker.distribution.manifest.list.v2+json",
   "manifests": [
      {
         "mediaType": "application/vnd.docker.distribution.manifest.v2+json",
         "size": 529,
         "digest": "sha256:2d7ecc9c5e08953d586a6e50c29b91479a48f69ac1ba1f9dc0420d18a728dfc5",
         "platform": {
            "architecture": "amd64",
            "os": "linux"
         }
      },
      {
         "mediaType": "application/vnd.docker.distribution.manifest.v2+json",
         "size": 529,
         "digest": "sha256:7723f8c211cbc089f836e13136b35157ba572b61e0419d8f978917fca049db68",
         "platform": {
            "architecture": "arm",
            "os": "linux",
            "variant": "v7"
         }
      },
      {
         "mediaType": "application/vnd.docker.distribution.manifest.v2+json",
         "size": 529,
         "digest": "sha256:1bc0bc3815bdcfafefa6b3ef1d8fd159564693d0f8fbb37b8151074651a11ffb",
         "platform": {
            "architecture": "arm64",
            "os": "linux",
            "variant": "v8"
         }
      },
      {
         "mediaType": "application/vnd.docker.distribution.manifest.v2+json",
         "size": 529,
         "digest": "sha256:7ecd20a7c308c281d7695ba41f4cdaa9544cbe118e71765ad48414023fa444a7",
         "platform": {
            "architecture": "ppc64le",
            "os": "linux"
         }
      },
      {
         "mediaType": "application/vnd.docker.distribution.manifest.v2+json",
         "size": 529,
         "digest": "sha256:9c374fef8ababbbb0ec1f3c452de291159e258abb01bea320207fd29ec850c94",
         "platform": {
            "architecture": "riscv64",
            "os": "linux"
         }
      },
      {
         "mediaType": "application/vnd.docker.distribution.manifest.v2+json",
         "size": 529,
         "digest": "sha256:be541d4b5a77aad6ee06262f67641213785c2be5aac180f86418455f711f9f06",
         "platform": {
            "architecture": "s390x",
            "os": "linux"
         }
      }
   ]
}

Maybe I found the reason. The docker manifest inspect can read the manifest schema version 2.2 (= "schemaVersion": 2 and vnd.docker.distribution.manifest.list.v2+json). But it cannot read the schema version 2.1 that was used for the quay.io/junaruga/garg-fastk container image I built below. It seems that the version 2.1 can happen on the combination podman + container registry quay.io.[1]

$ skopeo inspect --raw docker://quay.io/junaruga/garg-fastk:latest
{"schemaVersion":2,"mediaType":"application/vnd.oci.image.manifest.v1+json","config":{"mediaType":"application/vnd.oci.image.config.v1+json","digest":"sha256:f6311b6a015d832e6bfa0d22d4af149261a8a75a06e279b04fb54dac185d7cf0","size":2962},"layers":[{"mediaType":"application/vnd.oci.image.layer.v1.tar+gzip","digest":"sha256:a5ae1e707212700852a8ffccb6218577421efc37a97315bf16f462b42b7b73e3","size":29819107},{"mediaType":"application/vnd.oci.image.layer.v1.tar+gzip","digest":"sha256:167c814f36cac94e1f1512f7352ee06d99a3caae5e1664bff4eb6a55f31df359","size":28005099},{"mediaType":"application/vnd.oci.image.layer.v1.tar+gzip","digest":"sha256:2258b27e4e5847428617b77ba12aafa73ef1ddc8a7f2e38e9afb4a9ba2a927eb","size":88487936},{"mediaType":"application/vnd.oci.image.layer.v1.tar+gzip","digest":"sha256:291b909514bd8f0745dd71b4bed287e83b56f135a15ba445afbf576874f92111","size":14465374},{"mediaType":"application/vnd.oci.image.layer.v1.tar+gzip","digest":"sha256:a3d41363cd17c3f10ee968361862764a239cbc6968a809b9cc6969f96ee4dded","size":16998}

It is identified at [2][3]. And it seems that the schema version 2.2 is supported at [4]. That means Podman 5.11 or later versions support the schema version 2.2 while my current using Podman version is 4.2.0.

Now I built and pushed the container image by docker. Then I can see the manifest by docker manifest inspect.

$ docker manifest inspect quay.io/junaruga/garg-fastk
{
	"schemaVersion": 2,
	"mediaType": "application/vnd.docker.distribution.manifest.v2+json",
	"config": {
		"mediaType": "application/vnd.docker.container.image.v1+json",
		"size": 3621,
		"digest": "sha256:751d7add66986597796459a2e8a3a1e133ef69d7ad29f7cbc88b3305fca5c715"
	},
	"layers": [
		{
			"mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip",
			"size": 29819107,
			"digest": "sha256:a5ae1e707212700852a8ffccb6218577421efc37a97315bf16f462b42b7b73e3"
		},
		{
			"mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip",
			"size": 26806299,
			"digest": "sha256:68f85d448dda2b1fff91c0495ad978d7ad4046397210313a8da1770ba4a6d0cc"
		},
		{
			"mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip",
			"size": 85274696,
			"digest": "sha256:3d741ae03bd9e270119bb63d3191f6c16ef8644a3e7fa0897e57cda5d41802ec"
		},
		{
			"mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip",
			"size": 94,
			"digest": "sha256:a354fde7f1cc2c34f0ddd188ba06f9731ab9ad1f265e05fc68991242feb139c5"
		},
		{
			"mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip",
			"size": 14320263,
			"digest": "sha256:44b7a7553650ae4389d69e3cb3c2c84a6d53feb84b1bdfe2be88e09fc94056f1"
		},
		{
			"mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip",
			"size": 16229,
			"digest": "sha256:d67fb3ac213a6e452115717a5b472ab8c6a39d03744b9c0abd0079265003d921"
		},
		{
			"mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip",
			"size": 5995911,
			"digest": "sha256:a27dd37d0701628e43f66baf9dc878186903888262db9ccd4797831b45e74da0"
		},
		{
			"mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip",
			"size": 1512652,
			"digest": "sha256:7412d8d276fdaeb6cc9ea506fbbe8b38c5970e42f334bb91a789a80a88c34204"
		},
		{
			"mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip",
			"size": 93,
			"digest": "sha256:33bed01d99ea20992a52e1b88b4be4dd6c426865818c0e71f1980a4413247060"
		}
	]
}

References

When you register a tool, Dockstore attempts to retrieve the Dockerfile used to build the tool image and the associated CWL/WDL and test files. From the “Files” tab output you provided, it appears that the quay tool wasn’t linked back to the github repository from which it was built, so our system couldn’t read the Dockerfile etc.

For more information, please see:
https://docs.dockstore.org/en/stable/advanced-topics/legacy/tool-legacy-registration.html

Best regards,
Steve

OK. Thanks for the investigation. I didn’t set the link at the quey.io container page intentionally from the GitHub repository. Because the GitHub repository is to manage “some” docker containers temporarily for the bio-tools where the container images are officially not available.