Home   Notes   Contact Me

EC2 (Amazon)

Local

External


Setting up a new account

Setup

0a) have java installed 0b) have an Amazon Web Services Account, and sign up for S3 and EC2 1) get access key identitifiers: Access Identifier Secret access identifier 2) get X.509 certificate: cert-XXX.pem pk-XXX.pem example names: cert-HKZYKTAIG2ECMXYIBH3HXV4ZBZQ55CLO.pem pk-HKZYKTAIG2ECMXYIBH3HXV4ZBZQ55CLO.pem eng privatekey seems to be unix text, sales privatekey seems to be dos text. 3) get account number, also get account id (which is the number without the dashes) example: 4952-1993-3132 495219933132 4) get teh command line tools 5) set env EC2_HOME to the directory where the tools are unzipped 6) add the bin dir of EC2_HOME to your path 7) set env EC2_PRIVATE_KEY and EC2_CERT for example (using the example names from above): EC2_PRIVATE_KEY=c:\ec2\pk-HKZYKTAIG2ECMXYIBH3HXV4ZBZQ55CLO.pem EC2_CERT=c:\ec2\cert-HKZYKTAIG2ECMXYIBH3HXV4ZBZQ55CLO.pem

Test run

run an instance 8) find an instance to run (a getting started example instance from amazon) (and note the ami key): ec2-describe-images -a | grep started 9) generate a key pair to use with it (for passwordless login), and copy most of the output (including the lines that start with the dashes) to a file name getting-started: ec2-add-keypair getting-started 10) use cygwin to chmod the private key file to 600 11) run the instance using the ami key ec2-run-instances ami-235fba4a -k getting-started It output: RESERVATION r-e75e838e 220626908882 default INSTANCE i-8662cfef ami-235fba4a pending getting-started 0 m1.small 2008-10-07T14:21:12+0000us-east-1b 12) check the instances status via (using the instance number): ec2-describe-instances i-8662cfef after some time it output: RESERVATION r-e75e838e 220626908882 default INSTANCE i-8662cfef ami-235fba4a ec2-75-101-238-115.compute-1.amazonaws.com ip-10-250-7-0.ec2.internal running getting-started 0 m1.small 2008-10-07T14:21:12+0000 us-east-1b 13) Network authorize to allow access to various ports (I guess this is an account global command): ec2-authorize default -p 22 ec2-authorize default -p 443 ec2-authorize default -p 80 returns: GROUP default PERMISSION default ALLOWS tcp 22 22 FROM CIDR 0.0.0.0/0 14) connect: ssh -i getting-started root@ec2-75-101-238-115.compute-1.amazonaws.com

Create a new Image from an existing one

# Notes: # Below, when I mention "pk-XXX.pem" I mean the pk key you got from Amazon # Below, when I mention "cert-XXX.pem" I mean the cert you got from Amazon # Substute the real info for what ever is inside of { } # Start the existing image and modify it into whatever you want to save as a new image. # Copy to the instance you want to save keys so you can encrypt the new instance: # from non ec2 machine: # BAD VERSION, NEEDS Externally defined password less login key: ####scp -i id-rsa-gsg-keypair pk-XXX.pem cert-XXX.pem {username}@{ec2machineAddress}:/mnt scp pk-XXX.pem cert-XXX.pem {username}@{ec2machineAddress}:/mnt # from within the ec2 machine you want to save: ec2-bundle-vol -d /mnt -k /mnt/pk-XXX.pem -c /mnt/cert-XXX.pem -u {amazonUserIdWithoutDashes} -r i386 # see section below these instructions for example output of bundling # verify that a full set of files was made ls -l /mnt/image.* # you should see an 'image.manifest' file, and some 'part.nn' files # copy the image up to S3 ec2-upload-bundle -b {newBucketName} -m /mnt/image.manifest.xml -a {aws-access-key-id} -s {aws-secret-access-key} # !!! from the NON-EC2 machine: # register the image to EC2 ec2-register {newBucketName}/image.manifest.xml # This will return the AMI identifier for the image (you don't need it if you use Elasticfox) # (in Elasticfox you can just look for the newBucketName you used.) # you should be able to use the image from Elasticfox now

Bundling Example Output

# ec2-bundle-vol -d /mnt -k /mnt/pk-HKZYKTAIG2ECMXYIBH3HXV4ZBZQ55CLO.pem -c /mnt/cert-HKZYKTAIG2ECMXYIBH3HXV4ZBZQ55CLO.pem -u 495219933132 -r i386 Copying / into the image file /mnt/image.img... Excluding: /sys /dev/shm /proc /dev/pts /proc/sys/fs/binfmt_misc /dev /media /mnt /proc /sys /tmp/image.img /mnt/img-mnt 1+0 records in 1+0 records out mke2fs 1.38 (30-Jun-2005) warning: 256 blocks unused. Splitting /mnt/image.gz.crypt... Created image.part.00 Created image.part.01 Created image.part.02 Created image.part.03 ... Created image.part.22 Created image.part.23 Generating digests for each part... Digests generated. Creating bundle manifest... Bundle Volume complete.

Gotchas

Gotcha 1

While bundling an instance: Unable to read instance meta-data for product-codes

This can be ignored. There may also be other similar statements. The line seems to be that this is something that is not required and that you simply don't have it. There is hope that they will improve the messages at some point. Sample:

root@xxx# ec2-bundle-vol -d /mnt -k /mnt/pk-xxx.pem -c /mnt/cert-xxx.pem -u nnnnnn -r i386 Copying / into the image file /mnt/image... Excluding: /sys /proc /dev/pts /proc/sys/fs/binfmt_misc /dev /media /mnt /proc /sys /mnt/image /etc/udev/rules.d/70-persistent-net.rules /mnt/img-mnt 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.002466 s, 425 MB/s mke2fs 1.40.8 (13-Mar-2008) NOTE: rsync with preservation of extended file attributes failed. Retrying rsync without attempting to preserve extended file attributes... /etc/fstab: # Legacy /etc/fstab # Supplied by: ec2-ami-tools-1.3-20041 /dev/sda1 / ext3 defaults 1 1 /dev/sda2 /mnt ext3 defaults 0 0 /dev/sda3 swap swap defaults 0 0 none /proc proc defaults 0 0 none /sys sysfs defaults 0 0 Bundling image file... Splitting /mnt/image.tar.gz.enc... Created image.part.00 Created image.part.01 Created image.part.02 Created image.part.03 Created image.part.04 Created image.part.05 Created image.part.06 Created image.part.07 Created image.part.08 Created image.part.09 Created image.part.10 Created image.part.11 Created image.part.12 Created image.part.13 Created image.part.14 Created image.part.15 Created image.part.16 Created image.part.17 Created image.part.18 Created image.part.19 Created image.part.20 Created image.part.21 Created image.part.22 Created image.part.23 Created image.part.24 Created image.part.25 Created image.part.26 Created image.part.27 Created image.part.28 Created image.part.29 Created image.part.30 Created image.part.31 Created image.part.32 Created image.part.33 Created image.part.34 Created image.part.35 Created image.part.36 Created image.part.37 Created image.part.38 Created image.part.39 Created image.part.40 Created image.part.41 Created image.part.42 Created image.part.43 Created image.part.44 Created image.part.45 Created image.part.46 Created image.part.47 Created image.part.48 Created image.part.49 Created image.part.50 Created image.part.51 Created image.part.52 Created image.part.53 Created image.part.54 Created image.part.55 Created image.part.56 Created image.part.57 Created image.part.58 Created image.part.59 Created image.part.60 Created image.part.61 Created image.part.62 Created image.part.63 Created image.part.64 Created image.part.65 Created image.part.66 Created image.part.67 Generating digests for each part... Digests generated. Unable to read instance meta-data for product-codes Creating bundle manifest... ec2-bundle-vol complete.