If you are like me, you are pleased by the fact that amazon made it even cheaper to store information in Amazon S3 through their reduced redundancy storage model. Unfortunately until just recently there wasn’t a simple way to convert your old objects to use the reduced redundancy storage model. Using a new revision of boto, the python amazon aws package, I wrote a script (derived from one by the boto author) that will automatically convert all your old objects in a bucket to use the reduced redundancy storage model.
The script, shown at the bottom, currently requires a svn version of boto with revision of at least 1595. Assuming you have python and subversion installed, the following will get you up and running with the script, which can be downloaded here. Running the script concurrent times will take significantly much less time, therefore stopping the script midway through is of minor consequence.
svn checkout http://boto.googlecode.com/svn/trunk/@1595 boto-read-only cd boto-read-only # (as root) python setup.py install cd .. rm -rf boto-read-only ./convert_to_rss.py your_bucket_name [aws_access_key_id aws_secret_access_key]
Note that you may alternatively put your aws_access_key_id and aws_secret_access_key into the environment variables AWS_ACCESS_KEY_ID and AWS_SECRET_ACCESS_KEY respectively.
#!/usr/bin/env python import os, sys try: import boto.s3 boto.s3.key.Key.change_storage_class except ImportError, e: sys.stderr.write('Package boto (svn rev. >= 1595) must be installed.\n') sys.exit(1) except AttributeError, e: sys.stderr.write('Invalid version of boto. Required svn rev. >= 1595.\n') sys.exit(1) def convert(bucket_name, aws_id, aws_key): s3 = boto.connect_s3(aws_id, aws_key) bucket = s3.lookup(bucket_name) if not bucket: sys.stderr.write('Invalid authentication, or bucketname. Try again.\n') sys.exit(1) print 'Found bucket: %s' % bucket_name sys.stdout.write('Converting: ') sys.stdout.flush() found = converted = 0 try: for key in bucket.list(): found += 1 if key.storage_class != 'REDUCED_REDUNDANCY': key.change_storage_class('REDUCED_REDUNDANCY') converted += 1 if found % 100 == 0: sys.stdout.write('.') sys.stdout.flush() except KeyboardInterrupt: pass print '\nConverted %d items out of %d to reduced redundancy storage.' % (converted, found) def main(): def usage(msg=None): if msg: sys.stderr.write('\n%s\n\n' % msg) sys.stderr.write(''.join(['Usage: %s bucket [aws_access_key_id ', 'aws_secret_access_key]\n']) % sys.argv) sys.exit(1) if len(sys.argv) == 2: bucket = sys.argv msg = '' if 'AWS_ACCESS_KEY_ID' in os.environ: aws_id = os.environ['AWS_ACCESS_KEY_ID'] else: msg += 'Environment does not contain AWS_ACCESS_KEY_ID.\n' if 'AWS_SECRET_ACCESS_KEY' in os.environ: aws_key = os.environ['AWS_SECRET_ACCESS_KEY'] else: msg += 'Environment does not contain AWS_SECRET_ACCESS_KEY.\n' if msg: usage(msg + 'Please set values in environment or pass them in.') elif len(sys.argv) == 4: bucket = sys.argv aws_id = sys.argv aws_key = sys.argv else: usage() convert(bucket, aws_id, aws_key) if __name__ == '__main__': main()