Launch Simulations in Seconds
Following this Step-by-Step Guide
- 1. Obtain an Amazon Web Services (AWS) account. A credit card is required. Renting multi-core virtual machines is billed by the hour. You pay for only the hardware you use. There are no fees for software.
- 2. Login to AWS. First, choose your region. Second, create a key pair for the Elastic Compute Cloud (EC2) and download the private key (.pem file). To use a different region, you must create a different set of keys.
- 3. Subscribe to the free Simpetus Electromagnetic Simulation Platform from the AWS Marketplace. This is an Amazon Machine Image (AMI) which includes the latest, pre-installed versions of MEEP, MPB, and SCUFF-EM on Ubuntu. AWS provides free use of a t2.micro instance for 1 year as well as credits for academic research. These video tutorials provide a demonstration of launching a virtual instance using or . Connect to the instance via SSH from your local shell:
> ssh -i <your .pem file name> ubuntu@<your instance DNS>MEEP's scaling performance on EC2 for a multi-node virtual cluster is shown below. The benchmarking test involves time stepping the fields in 3D. The simulation time decreases linearly with the number of virtual cores and is not limited by inter-node communication.
- 4. (optional) Launch a multi-node virtual cluster using StarCluster. This provides a demonstration. First, make a note of your Account ID. Second, create an Access Key, ensure that its status is active, and download the credentials. After clicking 'Continue', select 'Manual Launch' and note the 'AMI ID' for your region. Click on 'Accept Terms'.
- 5. (optional) Install StarCluster on your local machine or EC2 client. StarCluster is an open-source package for automating the setup and management of multi-node virtual clusters on EC2.
> git clone https://github.com/jtriley/StarCluster.git
> cd StarCluster
> sudo python distribute_setup.py
> sudo python setup.py install
- 6. (optional) Customize StarCluster's configuration file to specify the instance type and size of your cluster. Use this template to get started. Spot instances, typically pennies per hour, can provide up to 90% cost savings compared to On-Demand rates.
- 7. (optional) Setup and manage your cluster using StarCluster
a) Provision your EC2 cluster from your local/client shell:
> starcluster start mycluster
b) Transfer files to your cluster:
> starcluster put mycluster /local/path/*.ctl /remote/path
c) Login to the master node of your cluster:
> starcluster sshmaster -u sgeadmin mycluster
d) Launch simulations using the Open Grid Scheduler:
> qsub -V -cwd -pe orte <# of processors> <your job shell script>
- 8. (optional) Resize your cluster on-the-fly as necessary:
> starcluster [addnode OR removenode] mycluster Alternatively, let StarCluster automatically handle the load balancing:
> starcluster loadbalance mycluster
- 9. (optional) When finished, terminate your cluster to stop paying hourly EC2 instance charges:
> starcluster terminate mycluster