Top Banner
Hashing THEN AND NOW MIKE SMORUL – ADAPT PROJECT
14

Pasig - Hashing presentation-2013

May 26, 2015

Download

Technology

Mike Smorul

Presentation to the 2013 Pasig meeting in Washington DC
Welcome message from author
This document is posted to help you gain knowledge. Please leave a comment to let me know what you think about it! Share it to your friends and learn new things together.
Transcript
Page 1: Pasig - Hashing presentation-2013

HashingTHEN AND NOW

MIKE SMORUL – ADAPT PROJECT

Page 2: Pasig - Hashing presentation-2013

Commodity Storage Performance

2003 JetStor III IDE-FC62MB/s large block

2013 218MB/s workstation SSDPerc 6/MD1000, 400MB/s+

Page 3: Pasig - Hashing presentation-2013

Chip Speed

2003: Pentium 4 3.2Ghz

2013: Core i7 Extreme3.5Ghz

Page 4: Pasig - Hashing presentation-2013

Hashing Performance

SHA-256 HashingJava: 85MB/sCrypto++: 111-134MB/s

Real World PenaltyJava: 20-40% penalty on

slow seek disk

Page 5: Pasig - Hashing presentation-2013

Implications

Flipped bottlenecks

Page 6: Pasig - Hashing presentation-2013

Parallelize Digesting

Independent IO and digest threads

Always have work for the digest algorithm.

Large files saw over 95% of algorithm potential.

Small files unchanged.

Page 7: Pasig - Hashing presentation-2013

Securing Data in Motion

?

Page 8: Pasig - Hashing presentation-2013

Integrity across the network

Internal AuditingProve your hardware

Peer-AuditingProve your friends

Digital SignaturesProve identity

Token BasedProve time

Page 9: Pasig - Hashing presentation-2013

Chronopolis Integrity

Current:Producer supplied

authoritative manifestPeers locally monitor

integrityManually trace back to

point of ingest

Page 10: Pasig - Hashing presentation-2013

Chronopolis Integrity

In-progressSingle integrity token back

to ingestIdeal

Tokens issued prior to arrival‘Prove’ the state of data to

point before Chronopolis

Page 11: Pasig - Hashing presentation-2013

Manifests 2.0

Token manifestsPortable, embeddable

Python, etc

Page 12: Pasig - Hashing presentation-2013

Integrity supporting Provenance

Digests in a cloud validate transfer only

Http headers can pass extended integrity informationEnd-user verification

Page 13: Pasig - Hashing presentation-2013

Closing

Why are you hashing?What do you want to

prove?Hashing Cost/performance

Page 14: Pasig - Hashing presentation-2013

Contact

Mike Smorul

[email protected]

http://adapt.umiacs.umd.edu/ace