Launch Simulations using AWS following this Step-by-Step Guide

1.     Obtain an Amazon Web Services (AWS) account. A credit card is required. Renting multi-core virtual machines is billed by the second. You pay for only the hardware you use. There are no fees for software. Simpetus provides full technical support and training for $100/month with no contracts and cancel anytime. AWS provides instructions for launching a virtual machine using Windows, macOS, or Linux.

2.     Login to the AWS console. First, choose your region. Second, create a key pair for the Elastic Compute Cloud (EC2) and download the private key (.pem file). To use a different region, you must create a different set of keys.

3.     Subscribe to the free Simpetus Electromagnetic Simulation Platform from the AWS Marketplace. This is an Amazon Machine Image (AMI) which includes the latest, pre-installed versions of Meep, MPB, and SCUFF-EM on Ubuntu. AWS provides free use of a t2.micro instance for 1 year as well as credits for academic research. These video tutorials provide a demonstration of launching a virtual instance using 1-Click or EC2 Console. Connect to the instance with IP address XX.XXX.XXX.XX via SSH from your local shell terminal. As a requirement for SSH, make sure that the private key has the correct file permissions:
		      
chmod 400 private_key.pem
ssh -i private_key.pem ubuntu@XX.XXX.XXX.XX
		      
		    
After logging into your virtual instance, you can launch Meep simulations using Python 3 from the default Bash terminal via:
  
mpirun -np 2 python3 myscript.py |tee output.out
  
Note: the Bash script which is launched automatically at start /home/ubuntu/.bashrc sets the PYTHONPATH environment variable. This is necessary to use the Meep module which is installed in the system directory. For simulations involving the Scheme interface, the command is:
  
mpirun -np 2 meep myscript.ctl |tee output.out
  
Meep's linear-scaling performance on EC2 for a multi-node virtual cluster is shown below. The benchmarking test involves time stepping the fields in 3d. The simulation time decreases linearly with the number of virtual cores. For sufficiently large problems, internode communication is not a bottleneck.



4. optional     Launch a multi-node virtual cluster using StarCluster. This video provides a demonstration. First, make a note of your Account ID. Second, create an Access Key, ensure that its status is active, and download the credentials. After clicking 'Continue', select 'Manual Launch' and note the 'AMI ID' for your region. Click on 'Accept Terms'.

5. optional     Install StarCluster on your local machine or EC2 client. StarCluster is an open-source package for automating the setup and management of multi-node virtual clusters on EC2.
		      
git clone https://github.com/jtriley/StarCluster.git
cd StarCluster
sudo python distribute_setup.py
sudo python setup.py install
		      
		    
6. optional     Customize StarCluster's configuration file to specify the instance type and size of your cluster. Use this template to get started. Spot instances, typically pennies per hour, can provide up to 90% cost savings compared to On-Demand rates.

7. optional     Setup and manage your cluster using StarCluster

(a) Provision your EC2 cluster from your local/client shell:
starcluster start mycluster
(b) Transfer files to your cluster:
starcluster put mycluster /local/path/*.ctl /remote/path
(c) Login to the master node of your cluster:
starcluster sshmaster -u sgeadmin mycluster
(d) Launch simulations using the Open Grid Scheduler:
qsub -V -cwd -pe orte <# of processors> <your job shell script>
8. optional     Resize your cluster on-the-fly as necessary:
starcluster [addnode OR removenode] mycluster
Alternatively, let StarCluster automatically handle the load balancing:
starcluster loadbalance mycluster
9. optional     When finished, terminate your cluster to stop paying EC2 instance charges:
starcluster terminate mycluster