AWS Batch

Amazon Web Services Batch has been created to provide a simple way of running containers and simple commands on AWS without you having to closely manage the underlying EC2 infrastructure (although a knowledge of the underlying infrastructure will always be useful). While AWS Batch does not have an understanding of CWL like a full-on workflow engine, it does provide one of the simplest ways to run a large number of Dockstore tools at scale. Additionally, it provides an opportunity to run tools and manage resources almost totally from a GUI.


This is a companion discussion topic for the original entry at https://docs.dockstore.org/docs/publisher-tutorials/aws-batch/

hi just to clarify you chose an ECS-optimized image and then you added a second EBS volume and mounted it as /datastore and then created a custom AMI image from that?

Sorry, totally missed this so you probably figured this out already.
I think we expanded the root volume, so one EBS volume in total still.

Separately, the /datastore was specified as one of the directories to mount into Docker containers so that cwltool can function properly.

You can use ebs-autoscale so you don’t have to pre-allocate disk space, GitHub - awslabs/amazon-ebs-autoscale: Don't run out of disk space on your EC2 instance when generating or working with large files. Automatically add EBS volumes to a filesystem mount point in response to disk utilization.. It will grow automatically.