EC2 (Amazon)
Local
External
Setting up a new account
Setup
0a) have java installed
0b) have an Amazon Web Services Account, and sign up for S3 and EC2
1) get access key identitifiers:
Access Identifier
Secret access identifier
2) get X.509 certificate:
cert-XXX.pem
pk-XXX.pem
example names:
cert-HKZYKTAIG2ECMXYIBH3HXV4ZBZQ55CLO.pem
pk-HKZYKTAIG2ECMXYIBH3HXV4ZBZQ55CLO.pem
eng privatekey seems to be unix text, sales privatekey seems to be dos text.
3) get account number, also get account id (which is the number without the dashes)
example:
4952-1993-3132
495219933132
4) get teh command line tools
5) set env EC2_HOME to the directory where the tools are unzipped
6) add the bin dir of EC2_HOME to your path
7) set env EC2_PRIVATE_KEY and EC2_CERT for example (using the example names from above):
EC2_PRIVATE_KEY=c:\ec2\pk-HKZYKTAIG2ECMXYIBH3HXV4ZBZQ55CLO.pem
EC2_CERT=c:\ec2\cert-HKZYKTAIG2ECMXYIBH3HXV4ZBZQ55CLO.pem
Test run
run an instance
8) find an instance to run (a getting started example instance from amazon) (and note the ami key):
ec2-describe-images -a | grep started
9) generate a key pair to use with it (for passwordless login), and copy most of the output (including the lines that start with the dashes) to a file name getting-started:
ec2-add-keypair getting-started
10) use cygwin to chmod the private key file to 600
11) run the instance using the ami key
ec2-run-instances ami-235fba4a -k getting-started
It output:
RESERVATION r-e75e838e 220626908882 default
INSTANCE i-8662cfef ami-235fba4a pending getting-started 0 m1.small 2008-10-07T14:21:12+0000us-east-1b
12) check the instances status via (using the instance number):
ec2-describe-instances i-8662cfef
after some time it output:
RESERVATION r-e75e838e 220626908882 default
INSTANCE i-8662cfef ami-235fba4a ec2-75-101-238-115.compute-1.amazonaws.com ip-10-250-7-0.ec2.internal running getting-started 0 m1.small 2008-10-07T14:21:12+0000 us-east-1b
13) Network authorize to allow access to various ports (I guess this is an account global command):
ec2-authorize default -p 22
ec2-authorize default -p 443
ec2-authorize default -p 80
returns:
GROUP default
PERMISSION default ALLOWS tcp 22 22 FROM CIDR 0.0.0.0/0
14) connect:
ssh -i getting-started root@ec2-75-101-238-115.compute-1.amazonaws.com
Create a new Image from an existing one
# Notes:
# Below, when I mention "pk-XXX.pem" I mean the pk key you got from Amazon
# Below, when I mention "cert-XXX.pem" I mean the cert you got from Amazon
# Substute the real info for what ever is inside of { }
# Start the existing image and modify it into whatever you want to save as a new image.
# Copy to the instance you want to save keys so you can encrypt the new instance:
# from non ec2 machine:
# BAD VERSION, NEEDS Externally defined password less login key:
####scp -i id-rsa-gsg-keypair pk-XXX.pem cert-XXX.pem {username}@{ec2machineAddress}:/mnt
scp pk-XXX.pem cert-XXX.pem {username}@{ec2machineAddress}:/mnt
# from within the ec2 machine you want to save:
ec2-bundle-vol -d /mnt -k /mnt/pk-XXX.pem -c /mnt/cert-XXX.pem -u {amazonUserIdWithoutDashes} -r i386
# see section below these instructions for example output of bundling
# verify that a full set of files was made
ls -l /mnt/image.*
# you should see an 'image.manifest' file, and some 'part.nn' files
# copy the image up to S3
ec2-upload-bundle -b {newBucketName} -m /mnt/image.manifest.xml -a {aws-access-key-id} -s {aws-secret-access-key}
# !!! from the NON-EC2 machine:
# register the image to EC2
ec2-register {newBucketName}/image.manifest.xml
# This will return the AMI identifier for the image (you don't need it if you use Elasticfox)
# (in Elasticfox you can just look for the newBucketName you used.)
# you should be able to use the image from Elasticfox now
Bundling Example Output
# ec2-bundle-vol -d /mnt -k /mnt/pk-HKZYKTAIG2ECMXYIBH3HXV4ZBZQ55CLO.pem -c /mnt/cert-HKZYKTAIG2ECMXYIBH3HXV4ZBZQ55CLO.pem -u 495219933132 -r i386
Copying / into the image file /mnt/image.img...
Excluding:
/sys
/dev/shm
/proc
/dev/pts
/proc/sys/fs/binfmt_misc
/dev
/media
/mnt
/proc
/sys
/tmp/image.img
/mnt/img-mnt
1+0 records in
1+0 records out
mke2fs 1.38 (30-Jun-2005)
warning: 256 blocks unused.
Splitting /mnt/image.gz.crypt...
Created image.part.00
Created image.part.01
Created image.part.02
Created image.part.03
...
Created image.part.22
Created image.part.23
Generating digests for each part...
Digests generated.
Creating bundle manifest...
Bundle Volume complete.
Gotchas
- Gotcha 1 While bundling an instance:
Unable to read instance meta-data for product-codes
Gotcha 1
While bundling an instance: Unable to read instance meta-data for product-codes
This can be ignored. There may also be other similar statements.
The line seems to be that this is something that is not required and
that you simply don't have it. There is hope that they will improve
the messages at some point. Sample:
root@xxx# ec2-bundle-vol -d /mnt -k /mnt/pk-xxx.pem -c /mnt/cert-xxx.pem -u nnnnnn -r i386
Copying / into the image file /mnt/image...
Excluding:
/sys
/proc
/dev/pts
/proc/sys/fs/binfmt_misc
/dev
/media
/mnt
/proc
/sys
/mnt/image
/etc/udev/rules.d/70-persistent-net.rules
/mnt/img-mnt
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.002466 s, 425 MB/s
mke2fs 1.40.8 (13-Mar-2008)
NOTE: rsync with preservation of extended file attributes failed. Retrying rsync
without attempting to preserve extended file attributes...
/etc/fstab:
# Legacy /etc/fstab
# Supplied by: ec2-ami-tools-1.3-20041
/dev/sda1 / ext3 defaults 1 1
/dev/sda2 /mnt ext3 defaults 0 0
/dev/sda3 swap swap defaults 0 0
none /proc proc defaults 0 0
none /sys sysfs defaults 0 0
Bundling image file...
Splitting /mnt/image.tar.gz.enc...
Created image.part.00
Created image.part.01
Created image.part.02
Created image.part.03
Created image.part.04
Created image.part.05
Created image.part.06
Created image.part.07
Created image.part.08
Created image.part.09
Created image.part.10
Created image.part.11
Created image.part.12
Created image.part.13
Created image.part.14
Created image.part.15
Created image.part.16
Created image.part.17
Created image.part.18
Created image.part.19
Created image.part.20
Created image.part.21
Created image.part.22
Created image.part.23
Created image.part.24
Created image.part.25
Created image.part.26
Created image.part.27
Created image.part.28
Created image.part.29
Created image.part.30
Created image.part.31
Created image.part.32
Created image.part.33
Created image.part.34
Created image.part.35
Created image.part.36
Created image.part.37
Created image.part.38
Created image.part.39
Created image.part.40
Created image.part.41
Created image.part.42
Created image.part.43
Created image.part.44
Created image.part.45
Created image.part.46
Created image.part.47
Created image.part.48
Created image.part.49
Created image.part.50
Created image.part.51
Created image.part.52
Created image.part.53
Created image.part.54
Created image.part.55
Created image.part.56
Created image.part.57
Created image.part.58
Created image.part.59
Created image.part.60
Created image.part.61
Created image.part.62
Created image.part.63
Created image.part.64
Created image.part.65
Created image.part.66
Created image.part.67
Generating digests for each part...
Digests generated.
Unable to read instance meta-data for product-codes
Creating bundle manifest...
ec2-bundle-vol complete.