1 DocFlow - kick off - 2007 - Moni toring 1 Distributed Monitoring in P2P Systems Serge Abiteboul, Bogdan Marinoiu INRIA-Futurs and Univ. Paris 11
1
DocFlow - kick off - 2007 - Monitoring 1
Distributed Monitoring in P2P Systems
Serge Abiteboul, Bogdan MarinoiuINRIA-Futurs and Univ. Paris 11
2
DocFlow - kick off - 2007 - Monitoring 2
Organization
Motivation
An algebra over XML streams
The P2PMonitor system
Optimization
Conclusion
3
Motivation
4
DocFlow - kick off - 2007 - Monitoring 4
P2P systems
Peer-to-peer: A large and varying number of computers cooperate to solve some particular task without any centralized authority
seti@home; kazaa; cabal
Interactivity between the peers (Web 2.0)
Content sharing communities
Distributed applications (e.g., games)
New services
5
DocFlow - kick off - 2007 - Monitoring 5
P2P systems
Benefit from technologies like XML and Web services
Complex and hard to control systems• Many peers
• Peers are distributed
• Peers are autonomous
• Peers are often unreliable and selfish
• Peers come and leave
• Intense communications between peers
• Content changes rapidly
Goal: monitor such systems
6
DocFlow - kick off - 2007 - Monitoring 6
Why monitor P2P systems?
P2P Monitoring, e.g., supply chain or manufacturing system
Web Intelligence: e.g., business intelligence and surveillance of competing companies
P2P Security: e.g., against intrusion, spamming
P2P optimization and tuning: e.g., gather statistics, control indexing
P2P system administration: e.g., choose where to log or control replication
P2P error management: e.g., error detection, error, diagnosis
Business: e.g., billing for using resources, procurement
7
An algebra over XML streams
8
DocFlow - kick off - 2007 - Monitoring 8
XM
L
strea
m
Monitoring one peer - Alerters
Detect events at the peer level
Data changes• Database updates• Events in RSS feed• Web page changes
Web service calls• Out call• In call
Each event is represented as an XML document
XML Stream
9
DocFlow - kick off - 2007 - Monitoring 9
An algebra over streams
Efficient Filter (can provide many selections simultaneously)
Binary operators: union, join, etc.
Operator with memory: duplicate elimination
Same plans meaningful for query processing or monitoring
= outer join & = s=“Lhasa”
s1 s2 s3 s4
s5
10
DocFlow - kick off - 2007 - Monitoring 10
ActiveXML algebra in short
Used both for query and monitoring plans
Local computation - Algebra for queries over trees:• Pick your favorite
• Has to work on streams
Manage replication• Generic data and services
Glue• Send, receive, eval
Essential• Include service calls – typically query calls
• Exchange streams of ActiveXML Algebraic Expressions
11
DocFlow - kick off - 2007 - Monitoring 11
AX
ML
strea
mAXML stream operator
r1
t s
r1
t s
r1
t s
rr1@p1Music@p3Music@p2
a
Peer p1
Q
“Easy case”Monotone in-streamMonotone query
Stream of ActiveXML
12
P2P Monitor system
13
DocFlow - kick off - 2007 - Monitoring 13
Stream specification language
for $c on local: outCOM
let $timeCall := $c.call.time
and $duration := $c.response.time - $timeCall
where
$c.call.method = "GetTemp“and $duration > 300
and $c.call.site = "http://meteofrance.fr"
return <longGetTemp>
<callTime>{$timeCall}</callTime>
<duration>{$duration}</duration>
</longGetTemp>
14
DocFlow - kick off - 2007 - Monitoring 14
Compiled into a monitoring plan
Compile the specification into a monitoring plan• Some alerters produce input streams
• Stream operators
Evaluate it
The monitoring plan is distributed
15
DocFlow - kick off - 2007 - Monitoring 15
What to do with the result streams
Publish a stream• RSS, email, web site
Turn a stream into a channel• Users can subscribe to it
Store stream – distributed tracing• Provide traces of the computation
• Basis for querying a run
• Basis for analyzing a run: diagnosis
16
DocFlow - kick off - 2007 - Monitoring 16
Example: monitoring involving many peers
For $com1 on http://peer.a.com :outCOM | http://peer.b.com :outCOM,
$com2 on http://meteo.fr : inCOM
Let $duration := $com1.response.timestamp - $com1.call.timestamp
Where
$duration > 10 and
$com1.call.method = « getTemp » and Filter
$com1.call.site = http://meteo.fr and
$com1.call.id = $com2.call.id Join
Return Report($com1, $com2)
By email to [email protected];
A
B
M
17
DocFlow - kick off - 2007 - Monitoring 17
Monitoring Task Global Plan
alerter@A
Filter
alerter@B
Filter
∪
Join
alerter@M
Reporter
Publisher
18
DocFlow - kick off - 2007 - Monitoring 18
Distributed Monitoring Task
alerter@A
Filter@Aalerter@B
Filter@B
∪@B
Join@M
alerter@M
Reporter@M
Publisher@M
Publisher@A
Publisher@B
Peer A:For $c on local:outCOMLet $duration = … Where .Return $cBy publish as channel X
Peer B:For $c on local:outCOM | XLet $duration = … Where ….Return $cBy publish as channel Y
Peer M:For $c1 on local:inCOM , $c2 on YWhere $c1.call.id = $c2.call.id Return Report($c1, $c2)By email to [email protected]
Y
X
19
Optimization
20
DocFlow - kick off - 2007 - Monitoring 20
Optimize the plan
Distribute the work among the peers
Try to place computation close to data if possible
Try to reduce redundancy
21
DocFlow - kick off - 2007 - Monitoring 21
= outer join & = s=“Lhasa”
r1[t,s] r2[s,at] r3[t,s] r4[t,s]
r1[t,s] r3[t,s] r4[t,s]
r1[t,s] r3[t,s] r4[t,s]
r5[t,s]
r5[t,s]
r5[t,s]
monitoring plan (a)
monitoring plan (b)
monitoring plan (c)
Optimization based on ActiveXML algebra
22
Conclusion
23
DocFlow - kick off - 2007 - Monitoring 23
The plan
streams
Specify monitoringGenerate data streams Store tracesQuery tracesMine traces (e.g., diagnosis)
24
DocFlow - kick off - 2007 - Monitoring 24
Merci
Merci