This post was contributed by Doruk Ozturk and Ankur Srivastava at AWS, and Neel Patel at NVIDIA. Introduction Drug discovery is a long and expensive process. Pharmaceutical companies must sift through thousands of compound possibilities to find potential new drugs to treat diseases. This process takes multiple years and costs billions of dollars, with the majority of the candidates failing during clinical trials. As generative artificial intelligence (generative AI) continues to transform industries, the life sciences sector is leveraging these advanced technologies to accelerate drug discover ..read more

This post was contributed by Sai Sunku, Software Engineer, Annapurna Labs, Matt Koop, Principal Engineer and Karthik Raman, Principal Performance Engineer, HPC Engineering The hpc7a instance type is the latest generation AMD-based HPC instance type offered by AWS. It’s available in multiple sizes and offers superior performance compared to the previous generation hpc6a instance type. See our previous post for a deep dive on the hpc7a instance itself and this other post that describes the different instance sizes. If you’re an MPI user, you’ll want to know that Hpc7a’s network bandwidth of 300 ..read more

High throughput shared files systems are an essential part of any HPC or AI environment. Whether you want to train large language models (LLMs), find effective new drugs, or lead the search for new sources of energy, shared file systems are responsible for feeding compute and storing the hard-won results. If you manage or use an on-premise environment, you know the complexity, value, and cost of providing a high throughput persistent data repository. You probably also know it typically takes weeks or months to plan and provision a large Lustre environment from scratch. In this post, I’ll show ..read more

Run simulations using multiple containers in a single AWS Batch job Matthew Hansen, Principal Solutions Architect, AWS Advanced Computing & Simulation Recently, AWS Batch launched a new feature that makes it possible to run multiple containers within a single job. This enables new scenarios customers have asked about like simulations for autonomous vehicles, multi-robot collaboration, and other advanced simulations. For autonomous system (AS) developers, this means you can keep your simulation and test scenario code in separate containers from the autonomy and sensor pipelines you want to ..read more

Nextflow is a popular domain-specific language (DSL) and runtime used to define workflows that string together multiple processing steps into a pipeline. This allows it to perform quite complex genomics or scientific analyses including for machine-learning. Workflows defined in Nextflow code can leverage container orchestration technologies to deploy containerized workloads across clusters, clouds, or HPC environments. As an interpreted language, errors in a Nextflow script are only revealed at runtime. This increases the time and cost of developing and debugging a workflow which could be redu ..read more

This post was contributed by Olivia Choudhury, PhD and Aniket Deshpande from AWS; Sanchit Misra, PhD, Vasimuddin Md., PhD, Narendra Chaudhary, PhD, Saurabh Kalikar, PhD, and Manasi Tiwari, PhD, Research Scientists at Intel Labs India; Ashish Kumar Patel, contingent worker with Intel Technology India Pvt. Ltd. We are living in the exciting times of the rapidly growing field of omics, including genomics, proteomics, transcriptomics, and metabolomics data. Our ability to measure omics data is increasing at a dramatic pace and new data science (AI and data management) pipelines are being deve ..read more

This post was contributed by Ross Pivovar, Solution Architect, and Adam Rasheed, Head of Emerging Workloads & Technologies, AWS and Orang Vahid, Director of Engineering Services and Kayla Rossi, Application Engineer, Maplesoft In our previous posts we discussed how to setup a Level 4 Digital Twin (DT) that adapts to changing environments and how to use the L4 DT to perform forecasting, scenario analysis, and risk assessment based on incoming measurement data. In this post today, we’ll discuss methods to find the optimal number, type, and placement of sensors to maximize the accuracy of you ..read more

Research organizations around the world run large-scale simulations, analyses, models, and other distributed, compute-intensive workloads on AWS every day. These jobs depend on an orchestration layer to coordinate tasks across the compute fleet. As a researcher or systems administrator providing services for researchers, it can be difficult to choose which AWS service or solution to use because there are various options for different kinds of workloads. In this post, we’ll describe some typical research use cases and explain which AWS tool we think best fits that workload. Understanding your ..read more

As many readers will know, AWS Batch provides functionality that enables you to run batch workloads on the managed container orchestration services in AWS: Amazon ECS and Amazon EKS. One of the core concepts of Batch is that it provides a job queue you can submit your work to. Batch is designed to transition your jobs from SUBMITTED to RUNNABLE states if they pass preliminary checks, and from RUNNING to either FAILED or SUCCEEDED after the job is placed on a compute resource and completes. Batch is also sends an event to Amazon CloudWatch Events for each corresponding job state update. Sometim ..read more

HPC workloads like genome sequencing and protein folding involve processing huge amounts of input data. Genome sequencing aims to determine an organism’s complete DNA sequence by analyzing extensive genome databases containing gene and genome reference sequences from thousands of species. Protein folding uses molecular dynamics simulations to model the physical movements of atoms and molecules in a protein. These workloads require analyzing massive input datasets. To support these kinds of applications that need high bandwidth, low latency, and parallel access to lots of data, AWS offers ..read more