Distributed Systems...... Risks and how to tackle them.

Distributed Systems . . .Distributed Systems . . .

. . . . . . Risks and how to tackle Risks and how to tackle themthem

Slides bySlides by

Peter Thanisch & Jyrki NummenmaaPeter Thanisch & Jyrki Nummenmaa‘‘

Internet Commerce -Internet Commerce -Distributed Application Example AreaDistributed Application Example Area

To exemplify the potential risks in safety To exemplify the potential risks in safety and credibility of distributed systems, and credibility of distributed systems, we will discuss an example application we will discuss an example application area.area.

Internet commerce is a good example Internet commerce is a good example area, because it deals with money and area, because it deals with money and there is a lot of interest in application there is a lot of interest in application development.development.

Internet Commerce definedInternet Commerce defined

The use of the global Internet forThe use of the global Internet for

purchase and sale of goods and services, purchase and sale of goods and services,

including service and support after theincluding service and support after the

sale. sale.

InternetInternet Commerce: our focus Commerce: our focus

AdvertisingAdvertising BrowsingBrowsing Purchasing Purchasing BillingBilling PaymentsPayments

Electronic Commerce: Electronic Commerce: the old waythe old way

CustomerCustomer

Financial AdviserFinancial Adviser

MortgageMortgageLendersLenders

LifeLifeInsurersInsurers

RentalRentalCompanies’Companies’Web SitesWeb Sites

Exhibition Hall’sExhibition Hall’sWeb siteWeb site

standsstands

BrokerageBrokerageserviceservice

ExhibitorExhibitorPC Web browserPC Web browser

Internet Commerce Example: Internet Commerce Example: Exhibition HallExhibition Hall

computerscomputers communicationscommunications furniturefurniture

So what is changing?So what is changing?

Electronic commerceElectronic commerce• Fixed set of Fixed set of

participating participating companiescompanies

• Proprietary, special-Proprietary, special-purpose protocols.purpose protocols.

• Specialist agent Specialist agent drives the dialogue, drives the dialogue, with special-purpose with special-purpose softwaresoftware

Internet commerceInternet commerce• Transient sets of Transient sets of

companies, maybe companies, maybe with brokers.with brokers.

• Protocols are Protocols are Internet standardsInternet standards

• The customer drives The customer drives the dialogue from a the dialogue from a general-purpose general-purpose Web browser.Web browser.

The state of the marketThe state of the market

Projections about the growth of Internet Projections about the growth of Internet commerce have been wildly optimistic.commerce have been wildly optimistic.

Not many retailers have been making Not many retailers have been making big bucks.big bucks.

Market for Internet commerce software Market for Internet commerce software is not hugely profitable either.is not hugely profitable either.

Internet CommerceInternet Commerce

A person, running a web browser on a A person, running a web browser on a desktop computer, electronically purchases a desktop computer, electronically purchases a set of goods or services from several vendors set of goods or services from several vendors at different web sitesat different web sites..• This person wants either the This person wants either the complete setcomplete set

of purchases to go through, or of purchases to go through, or none none of of them.them.

Technical Problems with Technical Problems with Internet CommerceInternet Commerce

SecuritySecurity FailureFailure Multiple sitesMultiple sites Protocol problemsProtocol problems Server product limitationsServer product limitations Response timeResponse time

SecuritySecurity

Security: the end user’s viewSecurity: the end user’s view ConfidentialityConfidentiality: : Preventing sniffing on Preventing sniffing on

your communication.your communication. IdentificationIdentification: Verifying that the sender Verifying that the sender

truly is who it is stated to be.truly is who it is stated to be. AuthenticationAuthentication: : Verifying that the Verifying that the

message has not be altered.message has not be altered. Non-repudiationNon-repudiation: : Ensuring that the Ensuring that the

sender cannot deny sending the sender cannot deny sending the message.message.

YourYourPCPC

YourYourInternetInternetServiceServiceProviderProvider

Web siteWeb siteof companyof companyselling theselling theproduct product you want you want to buyto buy

Internet BackboneInternet Backbone

ConfidentialityConfidentiality

A snifferA sniffer

YourYourPCPC

YourYourInternetInternetServiceServiceProviderProvider

Web siteWeb siteof companyof companyselling theselling theproduct product you want you want to buyto buy

Internet BackboneInternet Backbone

IdentificationIdentificationandand

AuthenticationAuthentication

XYZXYZ

XYZXYZ

ABCABC

ABCABC

XYZXYZ

Security: some solutionsSecurity: some solutions ConfidentialityConfidentiality: Encryption: Encryption.. AuthenticationAuthentication: CertificationCertification.. IntegrityIntegrity: Digitally signed message : Digitally signed message

digest codesdigest codes.. Non-repudiationNon-repudiation: Receipts containing a : Receipts containing a

digital signaturedigital signature.. You can do these through SSL/TLS or You can do these through SSL/TLS or

using the Java APIs.using the Java APIs.

FailureFailure

Failures: single computerFailures: single computer

Hardware failureHardware failure Software crashSoftware crash User switched off the PCUser switched off the PC Active attackActive attack

Failure: Additional Failure: Additional Problems for Multiple SitesProblems for Multiple Sites

Network failureNetwork failure• Or is it just congestion?Or is it just congestion?• Or has the remote computer crashed?Or has the remote computer crashed?• Or is it just running slowly?Or is it just running slowly?

Message loss?Message loss? Denial-of-service attack?Denial-of-service attack? Typically, these failures are partial.Typically, these failures are partial.

Distributed Distributed TransactionTransaction

Changes Changes two or moretwo or more autonomousautonomous databases from one consistent state to databases from one consistent state to another consistent state.another consistent state.

Server Autonomy - any server can Server Autonomy - any server can unilaterally decide to abort the transaction.unilaterally decide to abort the transaction.

Changes must be Changes must be durabledurable: information is : information is preserved despite system failures.preserved despite system failures.

System failures are typically System failures are typically partialpartial..

Subtle Difference: transactionSubtle Difference: transaction

Traditional data Traditional data processing processing transaction:transaction:

set of read and update set of read and update operations collectively operations collectively transform the database transform the database from one consistent from one consistent state to another.state to another.

Internet Internet Commerce Commerce transaction:transaction:

set of read and update set of read and update operations collectively operations collectively provide the user with provide the user with his/her required his/her required packagepackage

Protocol ProblemsProtocol Problems

TIP: Transaction Internet ProtocolTIP: Transaction Internet Protocol

Proposed as an Internet Standard.Proposed as an Internet Standard.• Backed by Microsoft and Tandem.Backed by Microsoft and Tandem.

Heterogeneous Transaction Managers Heterogeneous Transaction Managers can implement TIP to communicate with can implement TIP to communicate with each other.each other.

TIP: Two-pipe modelTIP: Two-pipe model

Site ASite A

ApplicationApplicationProgramProgram

TIP APITIP API

TIP txnTIP txnmanagermanager

Site BSite B

ApplicationApplicationProgramProgram

TIP APITIP API

TIP txnTIP txnmanagermanager

Pipe 1Pipe 1

Pipe 2Pipe 2

TIP commit protocolTIP commit protocol

A Browsing TransactionA Browsing Transaction

User’sWebBrowser

Server A

Server B

Server C

(1) Initiate txn

(2) txn URL

(3) PUSHtxn

(4) txnURL

(5) PULLtxn

AA

CC

PUSH ‘txn1a’PUSH ‘txn1a’

PUSH ‘txn1c’PUSH ‘txn1c’

DD

PUSH ‘txn1b’PUSH ‘txn1b’

BB

PUSH ‘txn1a’PUSH ‘txn1a’

Multiple inclusions of a siteMultiple inclusions of a site

TIP vulnerabilityTIP vulnerability

Communication is pairwise point-to-Communication is pairwise point-to-point.point.

Vulnerable to single link failures.Vulnerable to single link failures.

The Commit Protocol:The Commit Protocol:Ensuring AtomicityEnsuring Atomicity

Once the pushing and pulling is over, a Once the pushing and pulling is over, a coordinator must ensure that all sites coordinator must ensure that all sites can complete their work, writing their can complete their work, writing their results into their databases.results into their databases.

The method used to achieve this is The method used to achieve this is called a Commit Protocol.called a Commit Protocol.

The Commit Protocol must behave The Commit Protocol must behave sensibly even when there are failures.sensibly even when there are failures.

Transaction Commit (no failures!)Transaction Commit (no failures!)

Coordinator

Participants

VOTE-REQUEST

COMMIT or ABORT votes

Multicast decision

Two-Phase Commit BlocksTwo-Phase Commit Blocks

(1) COMMIT sent

(2) C crashes

C

(3) P1 crashes (4) P2 is blocked

COMMIT COMMIT

P1 P2

TIP SecurityTIP Security

Requires Secure-HTTP/SSL/TLS withRequires Secure-HTTP/SSL/TLS with• encryption and encryption and • end-to-end authentication.end-to-end authentication.

Operator intervention is needed when Operator intervention is needed when the commit protocol fouls up. the commit protocol fouls up. • How will this work on the Internet?How will this work on the Internet?

Internet Transaction SecurityInternet Transaction Security

Big value transactions will not be Big value transactions will not be conducted in this way.conducted in this way.

Thus any scams will take the form of Thus any scams will take the form of having a small effect on a large number having a small effect on a large number of transactions. (Salami scams.)of transactions. (Salami scams.)

SSL/TLS does NOT solve all of SSL/TLS does NOT solve all of the problemsthe problems

TIP with TLS does not ensure non-TIP with TLS does not ensure non-repudiation.repudiation.

Various Denial-of-Service attacks are Various Denial-of-Service attacks are possible.possible.

A rogue participant could block A rogue participant could block progress by refusing to commit.progress by refusing to commit.

Denial-of-ServiceDenial-of-Service

PULL-based:PULL-based:• A rogue company that knows the A rogue company that knows the

transaction ID sends a PULL to a site then transaction ID sends a PULL to a site then closes the connection.closes the connection.

PUSH-basedPUSH-based• Flood a sites with PUSHes so that it cannot Flood a sites with PUSHes so that it cannot

service legitimate requests.service legitimate requests.

Broken connectionBroken connection

If a site loses its connection to its If a site loses its connection to its superior, the rogue sites sends it a superior, the rogue sites sends it a RECONNECT command and tells it the RECONNECT command and tells it the wrong result of the commit.wrong result of the commit.

RepudiationRepudiation

General point about how to repudiate:General point about how to repudiate:

The site that wants to repudiate a The site that wants to repudiate a transaction can always cause itself to transaction can always cause itself to crash and then recover, meanwhile crash and then recover, meanwhile losing all information that was in losing all information that was in vulnerable storage.vulnerable storage.

RepudiationRepudiation

Interaction of 2PC and authenticated protocol messages • The semantics of the authenticated

messages only apply if the txn is committed.

RepudiationRepudiation

If a message from A to B is part of a 2PC protocol, then B’s possession of the digital signature proves nothing.• A can claim: Yes, that was sent, but the

action was rolled back. • B must prove that the action was

committed. B must also prove that the message was part of that txn.

Implications for Internet CommerceImplications for Internet Commerce

Existing protocols are inappropriate for the Existing protocols are inappropriate for the way people expect to be able to do business way people expect to be able to do business on the Internet.on the Internet.

The TIP approach looks promising, but ... The TIP approach looks promising, but ... For particular business sectors, a detailed For particular business sectors, a detailed

analysis of likely transaction behaviour will be analysis of likely transaction behaviour will be needed.needed.

Market opportunities for brokerage Market opportunities for brokerage companies.companies.

ConclusionsConclusions

Security:Security: techniques exist, but you have to techniques exist, but you have to know when to use them and howknow when to use them and how

Failure:Failure: Protocols exist, but they have Protocols exist, but they have several shortcomings, some more and several shortcomings, some more and some less serioussome less serious

We did not discuss We did not discuss performanceperformance this time, this time, but performance can be strongly related to but performance can be strongly related to failure (and perhaps to an extent to failure (and perhaps to an extent to security).security).