1 MDC status Overall concept: –The FarDet Mock data challenge ‘dataset’ has been generated with unknown values of m 2 and sin 2 2 which are to be determined.

1

MDC status• Overall concept:

– The FarDet Mock data challenge ‘dataset’ has been generated with unknown values of m2 and sin22 which are to be determined by a fit to the FD CC-like energy spectrum

– In addition, systematic ‘tweaks’ to various cross-section and beam parameters have been introduced into the ND and FD challenge sets, and these must be determined by fits to sensitive distributions in the Near Detector and then applied to the oscillation fit. These parameters are

• Quasi-elastic axial vector mass (3% variation)• Resonance production axial vector mass (3% variation)• Dis-resonance scale factor (4% variation)• BMPT hadron production uncertainties (~25 parameters, 1 variations)

• Tools available– Mad package encapsulates ‘standard’ CC event selection algorithm and energy

reconstruction. (see talks from previous meetings for a description of the standard CC analysis method)

– MCReweight package provides cross-section and beam weights (see Chris’s talk)– Physics Analysis Ntuple (PAN) contains all relevant CC quantities to perform

event reweighting and oscillation fits.

D.A. Petyt 17th March 2005

2

CC MDC Analysis Procedure• Perform match-up between ND MC and MDC datasets to

assess the level of agreement with nominal systematic parameters

• Perform a fit with a reduced set of systematic parameters to:– Determine if the level of agreement between MC/MDC samples can be

improved– Obtain best-fit values and uncertainties on the systematic parameters

• Use central values of these parameters in an oscillation fit to the FD MDC data set

• Perform match-ups between FD MDC distributions and best-fit FD MC

• Perform simultaneous ND/FD fit with systematics as ‘nuisance parameters’

*Oscillation parameters will be revealed in Saturday morning MDC talk

*This has not yet been done

3

Data samples used in this analysis

• PDF construction:– ~50 ND files (550 snarls/file)– 10 FD numu files + ~40 ‘NC’ nutau files

• Event reweighting and parameter fitting:– ND:

• 179 MC files – 78875 selected events (PID parameter>-0.2)• 88 Challenge set files• MC/Data ratio=2.05/1

– FD: • 19 MC files @ 6.5e20 pot – 34992 selected events (PID

parameter>-0.4)• 1 Challenge set file @ 7.4e20 pot• MC/Data ratio=16.7/1

4

ND MC/MDC matchup – before PID cutNB – MC statistical error is not negligible in these plots!

MC

MDC

5

Event length distribution

All MC eventsTrue CC eventsTrue NC eventsChallenge set

6

ND MC/MDC matchup – after PID cut

MC

MDC

7

Reco_enu distribution – after cuts

All MC eventsTrue CC eventsTrue NC eventsChallenge set

8

MDC/MC matchup – nominal parameters

Match-up is pretty good – implies that FD fit with nominal beam/xsec parameters will be OK. ND fit is required to determine the allowed range of these parameters, however.

9

Overall fit philosophy• Cross-section and beam uncertainties can be treated as nuisance

parameters in oscillation fit.– Define in the FD as a function of oscillation parameters and

beam/cross-section parameters. Minimise chisq with respect to these systematic parameters to yield dmsq,s2theta contours

– Can also apply ‘penalty terms’ to in order to constrain the values of these nuisance parameters. FD therefore looks like this:

– Can add additional term for ND which depends only on the nuisance parameters. The idea here is that the ND will help to constrain these parameters since they will, in general, be correlated with dmsq,s2t in FD-only fits.

– In the fits presented here, I just fit the ND distributions to determine the systematic parameters and apply them to the FD MC. The combined ND/FD simultaneous fit is in development…

nsyst

j j

jjnbins

i i

AiiA

ssmmmnmm

12

20

12

222222 )(,...)),2sin,((

,...),2sin,(

10

Fit method• The ND fit is performed on the 2D E_reco vs reconstructed_y distribution, where reco_y

=reco_shw/reco_enu. The reco_y dimension is necessary to provide some discrimination between QEL, RES & DIS events. It is expected that the e_reco distribution will provide discrimination between BMPT beam systematic parameters.

• A total of 51 bins of variable bin-size are employed in the fit (17 in e_reco and 3 in y) and a simple chisq is calculated between the observed and expected distributions.

• The fit uses the ‘many loops’ (or ‘brute force’) method to find the chisq minimum. Numerous tricks have been employed to reduce the execution time to the absolute minimum (a 5 parameter fit currently takes ~30 mins on a single node on the FNAL Linux Cluster). Other techniques, such as the Marquardt fit advocated by Brian, might be necessary if the number of free parameters becomes too large (i.e. >8)

QEL RES DIS

11

BMPT parameterization

TR pxaTRTR

RRR

epxbpxa

xBxxAdp

dE

)(2

3

3

))()(1(

)1()1()(

12

Slide from Alysia’s talk at the March 4th NC meeting…

Parameter errors were determined from a fit to NA20/NA56(SPY) data

13

Effect of ma_qel9%

MDC/nominal MC

Weighted MC/nominal MC

14

Effect of ma_res9%

MDC/nominal MC

Weighted MC/nominal MC

15

Effect of disfact9%

MDC/nominal MC

Weighted MC/nominal MC

16

Effect of A_pi5%

MDC/nominal MC

Weighted MC/nominal MC

17

Effect of B_pi25%

MDC/nominal MC

Weighted MC/nominal MC

18

Effect of alpha_pi5%

Strong correlation between B_pi and alpha_pi expected in fits

MDC/nominal MC

Weighted MC/nominal MC

19

Effect of a_pi6%

MDC/nominal MC

Weighted MC/nominal MC

20

Fit parameters and rangesParameter Nominal value Range constraint

ma_qel 1.032 30% 3%

ma_res 1.032 30% 3%

Disfact* 1.0 30% 4%

A_pi* 62.7 20% 20%

Alpha_pi 3.45 10% 5%

a_pi 6.10 20% 3%

* can be calculated outside of MCReweight

• In ‘unconstrained’ fits, the parameters are allowed to vary freely within the ranges specified above, with no chisq penalty applied if they range far from the nominal values

• In ‘constrained’ fits, a chisq penalty is applied when parameters deviate from their nominal values – the 1 sigma error is given by the ‘constraint’ column in the table above. (BMPT errors are taken from Alysia’s fits

21

2 parameter fit – ma_qel & ma_res

x Best fit

90% CL

Discrimination between ma_qel and ma_res is provided by y-distribution

1d projections

22

3 parameter fit – adding disfact

• Adding extra parameters will inflate the uncertainties on the systematic parameters due to correlations and/or degeneracies between the variables.

• In this case, the size of the error contour in the ma_qel, ma_res chisq projection is significantly larger than the 2 parameter fit.

• The best fit value of ma_qel remains the same, although the value of ma_res is higher by 3%. This is compensated by a 3% decrease in best-fit value of disfact from nominal (0.97 instead of 1.0)

x Best fit

90% CL

23

Result of 5 parameter unconstrained fit

Best fit:

Ma_qel = 1.06 +0.04-0.044

Ma_res = 1.06 +0.05-0.042

alpha_pi = 3.55 +0.16-0.06

a_pi = 6.47 +0.28-0.18

A_pi = 66.0 +4.7-2.5

24

Parameter correlations

ma_qel

ma_res

alpha_pi

a_pi

ma_res alpha_pi a_pi A_pi

x

68% CL90% CLBest fit

25

Comparison with ND MDC spectrum

Fit was already pretty good – additional parameters don’t improve it significantly

nominal

best fit

26

Result of constrained fit

Best fit:

Ma_qel = 1.032 +0.023-0.022

Ma_res = 1.032 +0.034-0.016

alpha_pi = 3.52 +0.05-0.11

a_pi = 6.22 +0.14-0.15

A_pi = 64.8 +1.6-3.6

27

Constrained fit MDC matchup

nominal

best fit

28

Near and Far ratios

unconstrainedconstrained

Near unconstrained Far unconstrainedR

atio

of

we

igh

ted

/no

min

al

Ind

ivid

ual

co

mp

on

ents

Ove

rall

29

Comments on the fits• Firstly, the fits as currently implemented are very slow

– This limits the number of parameters that can be varied and the step size (about 5 or 6 is the current practical limit).

– The ND/FD fit is much slower as it involves the two additional oscillation parameters – this is the main reason why I do not have this fit ready at this time.

– We will need to implement a more time-efficient fitting method (such as Brian’s fit method) if we want to include additional parameters.

• Some degeneracy/reduncancy between parameters– Using e_reco and y factorises some of the dependencies, but

further studies into sensitive variables and event sub-samples are needed.

– This fit used a zeroth order set of systematic parameters (inspired by some trial fits of my own and advice from Alberto on the important BMPT parameters). A more detailed study to determine what the minimal set of important parameters are would be useful.

30

First look at FD challenge set

All MC eventsTrue CC eventsTrue NC eventsChallenge set

Distribution seems consistent with numu disappearance at a level that is expected for SK-like oscillation parameters…

31

FD matchup (nominal) – PID parameter

All MC events

True CC events (oscillated)True NC eventsChallenge set

All MC events (oscillated)

32

FD matchup (nominal) – Reconstructed y

33

FD matchup (nominal) – Event length

Some ‘notchiness’ in the MDC distribution. Is this pathological, or just statistics?

34

FD matchup (best fit syst) – PID parameter

This is caused by best-fit values of ma_qel and ma_res 3% higher than nominal.

35

FD matchup (constrained fit) – PID parameter

Problem alleviated in constrained fit. Get slightly better chisq than nominal.

36

Conclusions/Next steps• I have made a first pass at using the ND to constrain the values of several of the

systematic parameters used in the generation of the MDC• Fits to the 2D ND (ereco,y) distribution with nominal systematic parameters

yield acceptable values of chisq. This implies that the systematic shifts are either small, or conspire in such a way to cancel each other out in these variables.– This also implies that performing a FD fit with nominal systematic parameters is not

an unreasonable thing to do.

• Fardet-only oscillation fits with either nominal or best-fit systematic parameters yield good agreement with the MDC challenge sets in most of the variables I have examined.– Oscillation parameter values will be revealed at Saturday’s MDC talk

• Future work should be focussed in the following areas:– Techniques to speed up the fits – this will allow us to add additional parameters and

reduce the step size– Look for variables/data-sets that can further constrain fits.– Determine some minimal set of systematic parameters that are needed for the fit.

Some generalisation of the BMPT parameters in terms of a reduced set of shape/normalisation variables would also be useful

– Perform combined ND/FD fit where N/F correlations are properly accounted for