Welcome message from author
This document is posted to help you gain knowledge. Please leave a comment to let me know what you think about it! Share it to your friends and learn new things together.
Transcript
Welcome to the Frozen Wasteland
Amazon Glacier: “cold” data storage.
Cheap: $0.01/month/GB.
CYA backups of input data.
Tax data.
Not for recovery backups.
No dashboard
No realtime access.
Use code for all operations.
Java & RESTful API's.
Perl uses RESTful.
Net::Amazon::Glacier
Stops at the API.
No high-level operations.
We all get to re-invent the wheel.
Common problem with Perl:
building-blocks, not frameworks.
Net::Amazon::Glacier
Bulky, complex object.
Multiple sub-objects.
Really hard to debug.
Designing the low-level interface.
Mimic the API.
Simple interface.
Croaks to caller on errors.
What the API does
List of jobs.
Describe single job.
Request an inventory.
Acquire job content.
Upload single file.
Multi-part upload.
Low-level issues
Number of jobs in a listing.
Continuation marker for lists.
Long wait for available data.
Example: Current jobs.my $api = Net::AWS::Glacier::API>new( qw( useast1 018362920384739 blahblah ));
my $marker = '';my @found = ();
for(;;){ my $json = $api>list_jobs ( $comp, $count, $status, $marker ); push @found, @{ $json>{ JobList } }; $marker = $json>{ Marker } || last;}
return @found;
Designing the high-level module.
What are the things you don't want to do?
Iterate alist of jobs, checking which ones are done, checking if they have already been processed, downloading their contents if necessary, validating the download, checking continuation markers...
Designing the high-level module.
What are the things you don't want to do?
What the API does.
Designing the high-level module.
What are the things you want to do?
Designing the high-level module.
What are the things you want to do?
Download an archive.
Designing the high-level module.
What are the things you want to do?
Download an archive.
Download an inventory.
Designing the high-level module.
What are the things you want to do?
Download whatever is ready.
Designing the high-level module.
What are the things you want to do?
Download whatever is ready.
Get a current inventory.
Bad design: Use API as base class.
API uses positional notation.
Errors tied to API structure.
No defaults.
Minimal sanity checking.
Not suitable for high-level interface.
Fix: use the API.
High-level interface is more complex:
Named args.
Sanity checks.
Detailed errors.
Uses dispatcher for low-level calls.
High-level interface
“Vault” object.
Const copy of the vault name.
Everything else managed inside the class.
Inside out data keeps object minimal.
Control data inherited from factory object.
Construct: minimal structure.
Guarantee data & type.
Ensure inside-out data is present.
sub construct{ my $proto = shift; my $class = blessed $proto;
my $vault = bless \( my $a = '' ), $class || $proto;
$vault_argz{ refaddr $vault } = $class ? $vault_argz{ refaddr $proto } : [] ;
$vault}
Initialize: Populate the object...
sub initialize{ my $vault = shift; $$vault = shift;
dlock $vault;
…
}
dlock() freezes object contents.
Initialize: … then the data.sub initialize{ …
if( @_ ) { my %initz = @_;
for( @arg_fieldz ) { my $value = $initz{ $_ } // next;
$vault>$_( $value ); } } $vault}
Use array of field names to avoid copying cruft.
Method simplifies inheritence.
NEXT to LAST
NEXT module replaces “SUPER”.
Pseudo-class:
“EVERY” “EVERY::LAST”
New calls:
$obj->EVERY::LAST::initialize( @_ );
DESTROY calls:
$obj->EVERY::cleanup( @_ );
Derived classes layer initializesub new{ # everyone gets the same structure.
my $vault = &construct;
# called in bottomup order. # derived classes can interrogate object # using methods at each level.
$vault>EVERY::LAST::initialize( @_ );
$vault}
Looping over vaults
Vault object inherits factory cred's.
my $proto= Net::AWS::Glacier>new(
'' => key => “...”, secret => “...”);
for my $name ( @vault_namz ){
my $vault = $proto>new( $name );$vault>request_current_inventory;
}
for my $name ( @vault_namz ){
$proto>new( $name ) >download_complete_jobs;}
Resolution
Two-state solution: API + High-Level.
API looks like the published standard:
array messages, croaks up front.
High-level is saner:
factory with inheritence, named args.
Net::AWS
Alpha code currently lives in github:https://github.com/lembark/netaws
this inclues the TreeHash & Signature modules.
Main doc's are in ./lib/Net/AWS/Glacier.pm.
Feedback welcome.
Related Documents
