Page 1
© Alexander Stage 1JASS 05 Synchronization and Replication in the Context of Mobile Applications
Joint Advanced Students School 2005
Replication and synchronization techniques in the context of mobile applications
By Alexander Stage
Software Engineering betrieblicher Informationssysteme (sebis)Ernst Denert-StiftungslehrstuhlLehrstuhl für Informatik 19 Institut für InformatikTU München
wwwmatthes.in.tum.de
Page 2
© Alexander Stage 2JASS 05 Synchronization and Replication in the Context of Mobile Applications
Agenda
Motivation
Mobile Computing
Synchronization Techniques
Replication Techniques in Databases
Oracle Lite 10g – A Mobile Database Solution
Peer to Peer Mobile Computing
Future Applications and Developments
Page 3
© Alexander Stage 3JASS 05 Synchronization and Replication in the Context of Mobile Applications
Motivation – A Scenario
The disconnected sales-person problem: a typical day
Customer A Customer B Customer C
Company
Central DB
travelling path
Mobile Device
Local DB replicate
Mobile Device
Local DBupdate
no communication link
Mobile Device
Local DBsynchronize
Page 4
© Alexander Stage 4JASS 05 Synchronization and Replication in the Context of Mobile Applications
Agenda
Motivation
Mobile Computing
Synchronization Techniques
Replication Techniques in Databases
Oracle Lite 10g – A Mobile Database Solution
Peer to Peer Mobile Computing
Future Applications and Developments
Page 5
© Alexander Stage 5JASS 05 Synchronization and Replication in the Context of Mobile Applications
Mobile Computing - Characteristics
Mobile devices:
Laptops
Palmtops
Smart cell phones
Requirements:
Data access:
- Anywhere
- Anytime
Nomadic users
Constraints:
Limited ressources
Variable connectivty:
- Performance
- Reliability
Page 6
© Alexander Stage 6JASS 05 Synchronization and Replication in the Context of Mobile Applications
Client Server Model Peer to Peer Model
Mobile Computing – Communication Models
Server
Client Client/Server
Client/Server
Client/Server
Client/Server
Client
Client Client
Page 7
© Alexander Stage 7JASS 05 Synchronization and Replication in the Context of Mobile Applications
Two Types of Synchronization
Process synchronization Data synchronization
Data item 1
Process A Process B
Process A
Process B
Data item 1
Data item 1
System BSystem A
Main System
Process Process
Data Item 1 Data Item 1
Data Item 1
Time
Time
Synchronization
1: 1
Syn
chro
niza
tion
1: 1
Syn
chro
niza
tion
correspondance
modification
modification
modification
modification modification
Page 8
© Alexander Stage 8JASS 05 Synchronization and Replication in the Context of Mobile Applications
Aspects of Synchronization
Amount of participating system:
1:1
1:n
Amount of exchanged data:
Incremental
- Usage of timestamps, attached to data items
- Comparison of sub data structures and exchange
Full
Conflict resolution mechanisms
Questions: who wins ?
Merging
System time synchronization in distributed systems
Page 9
© Alexander Stage 9JASS 05 Synchronization and Replication in the Context of Mobile Applications
Merging Data During Synchronization: The Product - Sales Example
System A
Main System
Stock Product A: 100
Order 50 units of product A
Stock Product A: 100
System B
Order 25 units of product A
Stock Product A: 100
System A
Main System
Stock Product A: 25
Stock Product A: 50
System B
Stock Product A: 75
Order Product A: 50 Order Product A: 25
Apply updatesNo communiation
Reduce stock,Insert order
Reduce stock,Insert order
No conflict, minimum stock is preserved
Page 10
© Alexander Stage 10JASS 05 Synchronization and Replication in the Context of Mobile Applications
Synchronization solutions
File system oriented:
Unix tool rsync:
- Incremental exchange of files
Coda file system
Version control systems:
CVS
WEBDav protocol:
- HTTP based file transfer protocol
- Locking and Versioning included
Mobile Application oriented:
Microsoft Active Sync:
- Synchronization between Desktop and Windows CE mobile devices
Page 11
© Alexander Stage 11JASS 05 Synchronization and Replication in the Context of Mobile Applications
Standardization Efforts in Data Synchronization: SyncML
SyncML defines client server based protocol for synchronization
SyncML is XML based, main elements:
SyncMessage
Sync Commands:
- Modification commands: add, copy, delete, put and replace
- Query commands: read and search
- Response commands: status and results
- Execution specific commands: sequence and atomic
Synchronization models:
Two way sync
Slow sync
One way sync
Refresh sync
Server-alerted sync
Page 12
© Alexander Stage 12JASS 05 Synchronization and Replication in the Context of Mobile Applications
SyncML Architecture
Source: SyncML Representation Protocol, version 1.0.1, http://www.syncml.org/docs/syncml_represent_v101_20010615.pdf
Page 13
© Alexander Stage 13JASS 05 Synchronization and Replication in the Context of Mobile Applications
Agenda
Motivation
Mobile Computing
Synchronization Techniques
Replication Techniques in Databases
Oracle Lite 10g – A Mobile Database Solution
Peer to Peer Mobile Computing
Future Applications and Developments
Page 14
© Alexander Stage 14JASS 05 Synchronization and Replication in the Context of Mobile Applications
What is Replication ?
Usage of multiple copies of server system (replicas)
Goals:
Higher availability
Better performance
Areas of usage: Distributed Systems and Databases
Replicated Server Only
Ressource
Server 1 Server n........
Replicated Ressource and Server
Ressource nRessource 1 ........
Server nServer 1 ........
Page 15
© Alexander Stage 15JASS 05 Synchronization and Replication in the Context of Mobile Applications
Replication in Databases
Replication of the resource replicating the database
Elimination of single point of failure
Single server behaviour
Correctness criterion: one-copy serializability
Synchronization in databases: replication
Constant trade-off between consistency and efficiency
Update anywhere, anytime, any way replication scheme
Simple case:
- Only a few replica nodes
- Low transaction rates
- Small amount of deadlocks and reconciliation
System scale up:
- Problem complexity grows drastically at cubic rates
- Inconsistent replicas and system delusion (no way to repair the inconsistencies)
Page 16
© Alexander Stage 16JASS 05 Synchronization and Replication in the Context of Mobile Applications
Replication in Databases – A Classification Scheme
Classification dimensions:
Where
- Dedicated replica:
- Single Master (primary master): primary and secondaries
- Every replica:
- Multi Master (update everywhere)
When
- Synchronous (eager replication)
- Asynchronous (lazy replication)
Other (not considered here)
- Interaction activities (network traffic)
- Constant or linear growth
- Transaction termination:
- Voting or Non-Voting techniques
Page 17
© Alexander Stage 17JASS 05 Synchronization and Replication in the Context of Mobile Applications
Abstract Functional Replication Model
Distinction between different replication solutions:
Phase ordering
Phase skipping
Request Phase:
Client sends a request to
one or all replicas
Agreement Coordination
Phase:
Replicas ensure that they all have done the same thing (2pc)
Client Response
Phase:
Client receives a response
Server Coordination
Phase:
Replicas coordinate their work
with each other (ordering of Execution
steps)
ExecutionPhase:
Replicas execute
operation
Page 18
© Alexander Stage 18JASS 05 Synchronization and Replication in the Context of Mobile Applications
Synchronous Single Master Replication I
Characteristics
Keeps all replicas synchronous in consistent state
No update anomalies
Reduces performance
Increase response times
Possible solution
Trigger on selected tables
Updates remotely all replicas
Fails if one replica fails
Usage of 2pc
Implication:
- No control over when replication should take place
- Only databases with 2pc support
Page 19
© Alexander Stage 19JASS 05 Synchronization and Replication in the Context of Mobile Applications
Synchronous Single Master Replication II
(primary)
(secondary)
(secondary)
Source: Wiesmann, Pedone, Schiper, Kemme, Alonso: Understanding Replication in Databases and Distributed Systems,
Proceedings of 20th International Conference on Distributed Computing Systems (ICDCS'2000)
Page 20
© Alexander Stage 20JASS 05 Synchronization and Replication in the Context of Mobile Applications
Asynchronous Single Master Replication I
Characteristics
Response after primary processed request:
- Primary takes care about transaction ordering
Drawback: stale data items versions
Performance is main goal
Possible solutions
Database system log based propagation:
- Committed updates only
- Restriction: Wait for current transactions
- Further improvement: only send data fields
Log table:
- Trigger records update and insert actions
- Periodically recreate log table
- Send log table to replicas (preserving oder)
Page 21
© Alexander Stage 21JASS 05 Synchronization and Replication in the Context of Mobile Applications
Asynchronous Single Master Replication II
(primary)
(secondary)
(secondary)
Source: Wiesmann, Pedone, Schiper, Kemme, Alonso: Understanding Replication in Databases and Distributed Systems,
Proceedings of 20th International Conference on Distributed Computing Systems (ICDCS'2000)
Page 22
© Alexander Stage 22JASS 05 Synchronization and Replication in the Context of Mobile Applications
Handling Failures
Two types of failure:
Replica fails:
- Secondary
System not affected
Secondary catch up
- Primary
Re-election of primary
Most up-to-date replica: common system state
- Possible data loss
Network partition
- Problem: Which partition is allowed to process further requests ?
Majority consensus algorithms
Quorum consensus algorithms
Danger: Dramatic system performance loss
Page 23
© Alexander Stage 23JASS 05 Synchronization and Replication in the Context of Mobile Applications
Synchronous Multi Master Replication I
Similar to single master synchronous replication
Updates allowed at every replica
Page 24
© Alexander Stage 24JASS 05 Synchronization and Replication in the Context of Mobile Applications
Synchronous Multi Master Replication II
Source: Wiesmann, Pedone, Schiper, Kemme, Alonso: Understanding Replication in Databases and Distributed Systems,
Proceedings of 20th International Conference on Distributed Computing Systems (ICDCS'2000)
Page 25
© Alexander Stage 25JASS 05 Synchronization and Replication in the Context of Mobile Applications
Asynchronous Multi Master Replication I
Characteristics
Multi version concurrency control mechanisms
Conflicts
Reconciliation
- Automatic
- Manually
Possible solution
Unique timestamps
- Concatenation of local replica system clock to replica identifier
- Thomas Write Rule:
Update applied if timestamp numerical greater than local timestamp
- Also known as optimistic replication
Page 26
© Alexander Stage 26JASS 05 Synchronization and Replication in the Context of Mobile Applications
Asynchronous Multi Master Replication II
(primary)
(primary)
(primary)
Source: Wiesmann, Pedone, Schiper, Kemme, Alonso: Understanding Replication in Databases and Distributed Systems,
Proceedings of 20th International Conference on Distributed Computing Systems (ICDCS'2000)
Page 27
© Alexander Stage 27JASS 05 Synchronization and Replication in the Context of Mobile Applications
Replication Requirements in Mobile Context
Disconnected operations require
Asynchronous Multi Master Replication
More frequent reconciliation because of
- Lazy replication
- Message delays
Page 28
© Alexander Stage 28JASS 05 Synchronization and Replication in the Context of Mobile Applications
Replication Techniques - Summary
Replication classification scheme
Eager / synchronous
primary server
Lazy / asynchronous
primary server
Eager / synchronous
Update everywhere
Lazy / asynchromnous
Update everywhere
Update propagation
Upd
ate
loca
tion
Page 29
© Alexander Stage 29JASS 05 Synchronization and Replication in the Context of Mobile Applications
Agenda
Motivation
Mobile Computing
Synchronization techniques
Replication techniques in databases
Oracle Lite 10g – A Mobile Database Solution
Peer to Peer Mobile Computing
Future Applications and Developments
Page 30
© Alexander Stage 30JASS 05 Synchronization and Replication in the Context of Mobile Applications
Oracle Lite 10g – A Mobile Database Solution
Supports disconnected mobile database applications
Local database (snapshot) on mobile device
Content: subset of enterprise database
Synchronization with enterprise database
Central component: Mobile Server
Application Model
Publication
- Deployed on Mobile Server
- Distribution of application code
- Distribution of SQL defined snapshots (controllable by variable)
Page 31
© Alexander Stage 31JASS 05 Synchronization and Replication in the Context of Mobile Applications
Oracle Lite 10g Architecture
Source: Oracle® Database Lite Developer's Guide 10g (10.0.0) Part No. B13788-0
Page 32
© Alexander Stage 32JASS 05 Synchronization and Replication in the Context of Mobile Applications
Oracle Lite 10g Synchronization Process I
Snapshot
Keeps track of local modifications
Three types of publication items (application depended)
Complete refresh
- All data items
Fast refresh
- Incremental (updates and inserts)
Queue based refresh
- No refresh (e.g. data collection: newly created data items)
Asynchronous synchronization
Central component Mobile Server
Via „In queues“ and „Out queues“
Page 33
© Alexander Stage 33JASS 05 Synchronization and Replication in the Context of Mobile Applications
Oracle Lite 10g Synchronization Process II
Source: Oracle® Database Lite Developer's Guide 10g (10.0.0) Part No. B13788-0
Page 34
© Alexander Stage 34JASS 05 Synchronization and Replication in the Context of Mobile Applications
Oracle Lite 10g Evaluation
Well suited for „build from scratch“
Enterprise database directly accessible from Mobile Server
„Mobile only“ or „Designed for mobility“ applications
Existing data access layers not supported (EJB, CORBA, other)
Duplication of mission-critical data access logic
Possible code portation efforts
Maintenance efforts increase
Benefits reduced or outweighted
Page 35
© Alexander Stage 35JASS 05 Synchronization and Replication in the Context of Mobile Applications
Agenda
Motivation
Mobile Computing
Synchronization techniques
Replication techniques in databases
Oracle Lite 10g – A Mobile Database Solution
Peer to Peer Mobile Computing
Future Applications and Developments
Page 36
© Alexander Stage 36JASS 05 Synchronization and Replication in the Context of Mobile Applications
Peer to Peer Mobile Computing
Shortcommings of client server model
No true mobility
No communication between mobile devices
Requirements
Any to any communication
- All devices equal
Large replication factors
Careful data replication
Page 37
© Alexander Stage 37JASS 05 Synchronization and Replication in the Context of Mobile Applications
The Ward Model I
Ward 2 Ward 3
Ward 1Ward member
Ward master
Overlapped ward member
Page 38
© Alexander Stage 38JASS 05 Synchronization and Replication in the Context of Mobile Applications
The Ward Model II
Dynamically reconfigurable synchronization topology
All units equal mobile peers:
Intra ward mobility
Inter ward mobility
- Ward changing (heavyweight)
- Ward overlapping (lightweight)
Ward master:
Re-electable
Knows ward members
No storage of ward member data
Manages data import / export for ward
Page 39
© Alexander Stage 39JASS 05 Synchronization and Replication in the Context of Mobile Applications
The Ward Model III
Second, top-level ward: all ward master
Ward set: data of all ward members within one ward
Changes over time
Optimistically replicated within ward
Selective replication: intra- and inter-ward
Page 40
© Alexander Stage 40JASS 05 Synchronization and Replication in the Context of Mobile Applications
Two Tier Replication
Characteristics
Comparable with ward model
Static backbone of stationary replicated database servers
Mobile replicas connect eventually at all stationary replicas
Benefit
Compromise between efficiency and true mobility
Possible improvement:
Combination with ward model (selective replication)
Page 41
© Alexander Stage 41JASS 05 Synchronization and Replication in the Context of Mobile Applications
Agenda
Motivation
Mobile Computing
Synchronization techniques
Replication techniques in databases
Oracle Lite 10g – A Mobile Database Solution
Peer to Peer Mobile Computing
Future Applications and Developments
Page 42
© Alexander Stage 42JASS 05 Synchronization and Replication in the Context of Mobile Applications
Future Applications and Developments
Car to car communication (http://www.car-to-car.org)
Mobile network of communicating cars
Security services
Development of replication schemes, based on ward model ?
Moving wards
Multi level ward hierachy
Page 43
© Alexander Stage 43JASS 05 Synchronization and Replication in the Context of Mobile Applications
Thanks for your attention !
Page 44
© Alexander Stage 44JASS 05 Synchronization and Replication in the Context of Mobile Applications
Discussion
Try to describe other real world usage scenarios of mobile applications, where synchronization and replication techniques need to be used
Which technologies could be helpful for application level data replication and integration. Think of technologies in the field of Enterprise Application Integration (EAI). Which advantages and disadvantages can be identified with respect to requirements in mobile applications
Page 45
© Alexander Stage 45JASS 05 Synchronization and Replication in the Context of Mobile Applications
Appendix I: SyncML Example
Source: SyncML Representation Protocol, version 1.0.1, http://www.syncml.org/docs/syncml_represent_v101_20010615.pdf
Page 46
© Alexander Stage 46JASS 05 Synchronization and Replication in the Context of Mobile Applications
Appendix II: Majority Consensus Algorithm I
Secondary 1
Primary
Secondary 4
Secondary 3
Secondary 2
Secondary 1
Primary
Secondary 4
Secondary 3
Secondary 2NetworkPartition
Page 47
© Alexander Stage 47JASS 05 Synchronization and Replication in the Context of Mobile Applications
Appendix II: Majority Consensus Algorithm II
Secondary 1
Old Primary
Secondary 4
New Primary
Secondary 2
Workingpartition
InactivePartition
2 32,5 <
Page 48
© Alexander Stage 48JASS 05 Synchronization and Replication in the Context of Mobile Applications
Appendix III: Quorum Consensus Algorithm I
Secondary 1
Primary
Weight: 2
Weight: 1
Primary
Weight: 2
Secondary 1
Weight: 1
Page 49
© Alexander Stage 49JASS 05 Synchronization and Replication in the Context of Mobile Applications
Appendix III: Quorum Consensus Algorithm II
Secondary 1Primary
Weight: 2 Weight: 1Workingpartition
Inactive partition
Partition weight = 2 Partition weight = 1>