G. Cowan Statistical Methods in Particle Physics 1 Statistical Methods in Particle Physics Day 2: Multivariate Methods (I) 清清清清清清清清清清清清 2010 清 4 清 12—16 清 Glen Cowan Physics Department Royal Holloway, University of Lond [email protected]www.pp.rhul.ac.uk/~cowan
55
Embed
G. Cowan Statistical Methods in Particle Physics1 Statistical Methods in Particle Physics Day 2: Multivariate Methods (I) 清华大学高能物理研究中心 2010 年 4 月 12—16.
This document is posted to help you gain knowledge. Please leave a comment to let me know what you think about it! Share it to your friends and learn new things together.
Transcript
G. Cowan Statistical Methods in Particle Physics 1
Statistical Methods in Particle PhysicsDay 2: Multivariate Methods (I)
清华大学高能物理研究中心2010 年 4 月 12—16日
Glen CowanPhysics DepartmentRoyal Holloway, University of [email protected]/~cowan
do quick sifting, record ~200 events/secsingle event ~ 1 Mbyte1 “year” 107 s, 1016 pp collisions / year2 109 events recorded / year (~2 Pbyte / year)
For new/rare processes, rates at LHC can be vanishingly smalle.g. Higgs bosons detectable per year could be ~103
→ 'needle in a haystack'
For Standard Model and (many) non-SM processes we can generatesimulated data with Monte Carlo programs (including simulationof the detector).
G. Cowan Statistical Methods in Particle Physics page 10
A simulated SUSY event in ATLAS
high pT
muons
high pT jets
of hadrons
missing transverse energy
p p
G. Cowan Statistical Methods in Particle Physics page 11
Background events
This event from Standard Model ttbar production alsohas high p
T jets and muons,
and some missing transverseenergy.
→ can easily mimic a SUSY event.
G. Cowan Statistical Methods in Particle Physics page 12
A simulated event
PYTHIA Monte Carlopp → gluino-gluino
.
.
.
G. Cowan Statistical Methods in Particle Physics page 13
Event selection as a statistical testFor each event we measure a set of numbers: nx,,x=x 1
x1 = jet p
T
x2 = missing energyx
3 = particle i.d. measure, ...
x follows some n-dimensional joint probability density, which
depends on the type of event produced, i.e., was it ,ttpp ,g~g~pp
x i
x jE.g. hypotheses H
0, H
1, ...
Often simply “signal”, “background”
1H|xp
0H|xp
G. Cowan Statistical Methods in Particle Physics page 14
Finding an optimal decision boundary
In particle physics usually startby making simple “cuts”:
xi < c
i
xj < c
j
Maybe later try some other type of decision boundary:
H0 H
0
H0
H1
H1
H1
G. Cowan Statistical Methods in Particle Physics page 15
G. Cowan Statistical Methods in Particle Physics page 16
G. Cowan Statistical Methods in Particle Physics page 17
G. Cowan Statistical Methods in Particle Physics page 18
G. Cowan Statistical Methods in Particle Physics page 19
G. Cowan Statistical Methods in Particle Physics page 20
Two distinct event selection problemsIn some cases, the event types in question are both known to exist.
Example: separation of different particle types (electron vs muon)Use the selected sample for further study.
In other cases, the null hypothesis H0 means "Standard Model" events,and the alternative H1 means "events of a type whose existence isnot yet established" (to do so is the goal of the analysis).
Many subtle issues here, mainly related to the heavy burdenof proof required to establish presence of a new phenomenon.
Typically require p-value of background-only hypothesis below ~ 10 (a 5 sigma effect) to claim discovery of "New Physics".
G. Cowan Statistical Methods in Particle Physics page 21
Using classifier output for discovery
y
f(y)
y
N(y)
Normalized to unity Normalized to expected number of events
excess?
signal
background background
searchregion
Discovery = number of events found in search region incompatiblewith background-only hypothesis.
p-value of background-only hypothesis can depend crucially distribution f(y|b) in the "search region".
ycut
G. Cowan Statistical Methods in Particle Physics page 22
Example of a "cut-based" studyIn the 1990s, the CDF experiment at Fermilab (Chicago) measuredthe number of hadron jets produced in proton-antiproton collisionsas a function of their momentum perpendicular to the beam direction:
Prediction low relative to data forvery high transverse momentum.
"jet" ofparticles
G. Cowan Statistical Methods in Particle Physics page 23
High pT jets = quark substructure?Although the data agree remarkably well with the Standard Model(QCD) prediction overall, the excess at high pT appears significant:
The fact that the variable is "understandable" leads directly to a plausible explanation for the discrepancy, namely, that quarks could possess an internal substructure.
Would not have been the case if the variable plotted was a complicated combination of many inputs.
G. Cowan Statistical Methods in Particle Physics page 24
High pT jets from parton model uncertaintyFurthermore the physical understanding of the variable led oneto a more plausible explanation, namely, an uncertain modeling ofthe quark (and gluon) momentum distributions inside the proton.
When model adjusted, discrepancy largely disappears:
Can be regarded as a "success" of the cut-based approach. Physicalunderstanding of output variable led to solution of apparent discrepancy.
G. Cowan Statistical Methods in Particle Physics page 25
G. Cowan Statistical Methods in Particle Physics page 26
G. Cowan Statistical Methods in Particle Physics page 27
G. Cowan Statistical Methods in Particle Physics page 28
G. Cowan Statistical Methods in Particle Physics page 29
G. Cowan Statistical Methods in Particle Physics page 30
G. Cowan Statistical Methods in Particle Physics page 31
G. Cowan Statistical Methods in Particle Physics page 32
G. Cowan Statistical Methods in Particle Physics page 33
G. Cowan Statistical Methods in Particle Physics page 34
G. Cowan Statistical Methods in Particle Physics page 35
G. Cowan Statistical Methods in Particle Physics page 36
G. Cowan Statistical Methods in Particle Physics page 37
Neural network example from LEP IISignal: ee → WW (often 4 well separated hadron jets)
Background: ee → qqgg (4 less well separated hadron jets)
← input variables based on jetstructure, event shape, ...none by itself gives much separation.
Neural network output:
(Garrido, Juste and Martinez, ALEPH 96-144)
G. Cowan Statistical Methods in Particle Physics page 38
Some issues with neural networksIn the example with WW events, goal was to select these eventsso as to study properties of the W boson.
Needed to avoid using input variables correlated to theproperties we eventually wanted to study (not trivial).
In principle a single hidden layer with an sufficiently large number ofnodes can approximate arbitrarily well the optimal test variable (likelihoodratio).
Usually start with relatively small number of nodes and increaseuntil misclassification rate on validation data sample ceasesto decrease.
Often MC training data is cheap -- problems with getting stuck in local minima, overtraining, etc., less important than concerns of systematic differences between the training data and Nature, and concerns aboutthe ease of interpretation of the output.
G. Cowan Statistical Methods in Particle Physics page 39
Overtraining
training sample independent test sample
If decision boundary is too flexible it will conform too closelyto the training points → overtraining.
Monitor by applying classifier to independent test sample.
G. Cowan Statistical Methods in Particle Physics page 40
validation sample
training sample
Monitoring overtrainingWe can monitor the misclassification rate (or value of the error function) as a function of some parameter related to the level of flexibility of the decision boundary, such as the number of nodes in the hidden layer.
For the data sample used to train the network, the error rate continues to decrease, but for an independent validation sample, it will level off and even increase.
error rate
number of nodes
G. Cowan Statistical Methods in Particle Physics page 41
G. Cowan Statistical Methods in Particle Physics page 42
G. Cowan Statistical Methods in Particle Physics 43
G. Cowan Statistical Methods in Particle Physics 44
G. Cowan Statistical Methods in Particle Physics 45
G. Cowan Statistical Methods in Particle Physics 46
G. Cowan Statistical Methods in Particle Physics 47
G. Cowan Statistical Methods in Particle Physics 48
G. Cowan Statistical Methods in Particle Physics 49
G. Cowan Statistical Methods in Particle Physics 50
G. Cowan Statistical Methods in Particle Physics 51
G. Cowan Statistical Methods in Particle Physics 52
G. Cowan Statistical Methods in Particle Physics 53
G. Cowan Statistical Methods in Particle Physics 54
G. Cowan Statistical Methods in Particle Physics page 55
Summary
Information from many variables can be used to distinguishbetween event types.
Try to exploit as much information as possible.Try to keep method as simple as possible.Often start with: cuts, linear classifiersAnd then try less simple methods: neural networks
Tomorrow we will see some more multivariate classifiers:Probability density estimation methodsBoosted Decision TreesSupport Vector Machines