Status of EPICs Operations

Pedro Calderón/Guillermo BuenadichaPedro Calderón/Guillermo BuenadichaSOC Operations Support Group

Page 1

XMM-NewtonXMM-Newton

Status of EPICs OperationsStatus of EPICs Operations

Pedro Calderón Riaño Pedro Calderón Riaño

Guillermo BuenadichaGuillermo Buenadicha

MALLORCA 05/11/2007MALLORCA 05/11/2007


Page 2


ContentsContents

Status of operations since PalermoStatus of operations since PalermoPedro CalderonPedro Calderon

Future activities and open pointsFuture activities and open pointsGuillermo BuenadichaGuillermo Buenadicha


Page 3


OverviewOverview

Smooth operations, in generalSmooth operations, in general MOS2 +32 Volts line dropMOS2 +32 Volts line drop New hot stuff, 3X3, CCD6 checkNew hot stuff, 3X3, CCD6 check New MOS CCD thermal controlNew MOS CCD thermal control Future procedure changesFuture procedure changes


Page 4


Routine OperationsRoutine Operations

RBI clock resync every ~194 daysRBI clock resync every ~194 days– Last on May 29Last on May 29thth, next on December 10, next on December 10thth

Eclipse season fully nominal.Eclipse season fully nominal.– All performed automatically by timeline. All performed automatically by timeline.

A MOS-2 exp in Full Frame 2 Nodes modeA MOS-2 exp in Full Frame 2 Nodes mode– The exposure was erroneously planed. This The exposure was erroneously planed. This

mode is not commissioned.mode is not commissioned.– A TC failed during the reconfiguration after the A TC failed during the reconfiguration after the

end of the exposure. The activity has a flaw at end of the exposure. The activity has a flaw at that point -> to be corrected.that point -> to be corrected.


Page 5


MOS2 +32V drop MOS2 +32V drop

The voltage in the +32V line drop instantaneously The voltage in the +32V line drop instantaneously to 0 volts in the middle of a normal observation, to 0 volts in the middle of a normal observation, the CCD stop generating events, and no other the CCD stop generating events, and no other telemetry parameter showed any change or spike.telemetry parameter showed any change or spike.

After a switch After a switch OFFOFF / switch / switch ONON the instrument the instrument returned to full nominal.returned to full nominal.

Happened a similar incident in MOS-1 on 2002 Happened a similar incident in MOS-1 on 2002 (NCR#105), but never again.(NCR#105), but never again.

MRB with Saclay, Leicester and ESACMRB with Saclay, Leicester and ESAC Suspected a SEU in EMVC electronic ??Suspected a SEU in EMVC electronic ??


Page 6


New Hot StuffNew Hot Stuff

New versions of the MOS Offset TablesNew versions of the MOS Offset Tables– Now at version V12Now at version V12

New versions of the PN Bad Pixel TablesNew versions of the PN Bad Pixel Tables– Fixed the bright 64 column in CCD 11Fixed the bright 64 column in CCD 11

Changes in place since Rev 1389 (July 11Changes in place since Rev 1389 (July 11thth) ) but they were not in full use until Rev 1408 but they were not in full use until Rev 1408 (August 16(August 16thth))


Page 7


MOS 3x3 pointed obsMOS 3x3 pointed obs

Mode tested on March 30Mode tested on March 30thth

Real observation performed in May 2Real observation performed in May 2ndnd on on PSR B0833-45 (rev 1354, OBS_ID PSR B0833-45 (rev 1354, OBS_ID 0510390101, 0510390101, routine calibration obs))

Operations OKOperations OK


Page 8


New ED for CCD6 checksNew ED for CCD6 checks

New ED now availableNew ED now available Last check performed on October 29Last check performed on October 29thth

using the new EDusing the new ED The operation was quick and easyThe operation was quick and easy No changes in results (still full of AFFA hex No changes in results (still full of AFFA hex

events)!!!events)!!!


Page 9


CCD6 imageCCD6 image


Page 10


New MOS CDD thermal controlNew MOS CDD thermal control

Exist only 1 circuit (plus redundant) for the 3 Exist only 1 circuit (plus redundant) for the 3 cameras CCD substitution heaters. If heater are cameras CCD substitution heaters. If heater are ONON for one camera, they are for one camera, they are ONON for all. for all.

If a camera has problems, all cameras are heated If a camera has problems, all cameras are heated => bad science=> bad science

The new idea : avoid heating the CCD of the The new idea : avoid heating the CCD of the cameras that are still workingcameras that are still working


Page 11



How to do that? How to do that? Allowing the unpowered CCD to cold down to the Allowing the unpowered CCD to cold down to the

minimum tested temperature (-140 degC). minimum tested temperature (-140 degC). That is near to the external radiator temp, so a lot That is near to the external radiator temp, so a lot

of time is needed.of time is needed. Then trigger heaters Then trigger heaters ONON up to operational up to operational

temperature, and switch temperature, and switch OFFOFF again again Heaters will be Heaters will be ONON for limited time, so the impact for limited time, so the impact

in the other cameras is reducedin the other cameras is reduced All this is done via manual procedureAll this is done via manual procedure


Page 12



-160

-140

-120

-100

-80

-60

-40

-20

0

PNMOS 2MOS 1


Page 13


Future actionsFuture actions

Change of procedures to recover PN from Change of procedures to recover PN from Safe StandbySafe Standby

Now PN never goes normally to SSB, but Now PN never goes normally to SSB, but the procedure should be up to date any the procedure should be up to date any wayway

The idea is switch off all PN, switch on the The idea is switch off all PN, switch on the substitution heaters, and when all substitution heaters, and when all temperatures are good (remember the temperatures are good (remember the problem with the currents when the EPVC problem with the currents when the EPVC is cold, NCR#90), do a full switch on.is cold, NCR#90), do a full switch on.


Page 14


Future activities & open issuesFuture activities & open issues

MOS redundant node readoutMOS redundant node readout MOS and PN, comparison of timing MOS and PN, comparison of timing

fluxflux Rev 1325 Crab timingRev 1325 Crab timing Periodic CCD-6 health checkPeriodic CCD-6 health check PN activities foreseenPN activities foreseen Slew surveySlew survey S2K supportS2K support


Page 15


MOS redundant node readoutMOS redundant node readout

Action pending since TTD@MPE, May 2006Action pending since TTD@MPE, May 2006 The rationale is twofold:The rationale is twofold:

– Assess the capability to react against a HW failure in Assess the capability to react against a HW failure in MOS (similar to RGS-2 case)MOS (similar to RGS-2 case)

– Gain expertise at ESAC building MOS sequencesGain expertise at ESAC building MOS sequences Sequences currently in DB? Sequences currently in DB? Too many…Too many…

– All delivered sequences were kept since launch (and All delivered sequences were kept since launch (and even since FM)even since FM)

– Naming convention has never been consistentNaming convention has never been consistent– Not all of them used In FlightNot all of them used In Flight– However, there are some sequences capable of reverse However, there are some sequences capable of reverse

clocking clocking tests in April 2005, MOS-1 CCD-6tests in April 2005, MOS-1 CCD-6


Page 16


MOS redundant node readoutMOS redundant node readout

How to proceed?How to proceed?– WhenWhen

– It is a task not too urgent, but desirable to be done before T. Abbey runs away

– If current DB sequences are able to do the job, then the design part is not an issue

– Do we aim for Spring 2008?– WhoWho

– Ideally engineers at ESAC and T. Abbey + TBD replacement?

– HowHow– Either developing new sequences from scratch, or

using the DB ones, followed by tests at the Spare and/or on the real unit.

– Last step would be the creation of the DB items (ED’s) to cope with eventual node failure.


Page 17


Timing events PN/M1/M2Timing events PN/M1/M2

Project started by M. Kirsch and John Olaffson (trainee at ESAC).

The idea is to analyze and compare the normalized flux in timing observations among EPIC cameras

Study is still ongoing, some preliminary cases required some support to understand deviations from a nominal ratio– Cases in which MOS-1 deviates (above) from MOS-2 and PN

(possible influence of MOS-1 hot column under weak sources)– Cases in which both MOS deviate (below) from PN, all in

counting. Analysis is required to assess influence of MOS CCD stuck mechanism and the SAS processing of GTI.

Screening of data is ongoing, also the Long Term Archive for HK analysis is being populated.


Page 18


1325 Crab timing…1325 Crab timing…

M. Kirsch reported a M. Kirsch reported a 600 microseconds600 microseconds absolute absolute time deviation in the Crab observation in rev time deviation in the Crab observation in rev 1325.1325.

Nothing abnormal detected in ODF Timing files Nothing abnormal detected in ODF Timing files and no G/S problem found.and no G/S problem found.

A TCS generation problem after the given epoch A TCS generation problem after the given epoch was found at MOC due to old Orbit products.was found at MOC due to old Orbit products.

However, TCS vs. TCX analysis seems OK, shows However, TCS vs. TCX analysis seems OK, shows TCX is independent on the MOC orbit file.TCX is independent on the MOC orbit file.

Mission long TCS vs. TCX analysis is required just Mission long TCS vs. TCX analysis is required just to find different behaviors among files.to find different behaviors among files.

TCX is available and in use for timing observations TCX is available and in use for timing observations through SAS.through SAS.


Page 19


Periodic CCD-6 health checkPeriodic CCD-6 health check

Past operationsPast operations– Tests in April 2005 after CCD failureTests in April 2005 after CCD failure– TTD in MPE May 2006 recommended periodic switch onTTD in MPE May 2006 recommended periodic switch on– Done twice in 2007 (March and October) without Done twice in 2007 (March and October) without

success…success… Is it still worth it? Is it still worth it?

– It is not really time consuming at MOC or SOC, since now It is not really time consuming at MOC or SOC, since now is almost fully automatic.is almost fully automatic.

Can we link this activity to some periodic Can we link this activity to some periodic calibration or diagnostic for MOS?calibration or diagnostic for MOS?– Helps programming the activity regularlyHelps programming the activity regularly


Page 20


PN activities foreseenPN activities foreseen

Past non routine operations since PalermoPast non routine operations since Palermo– Minimal (not to say none!!!)Minimal (not to say none!!!)

Situation seems very stable for the Situation seems very stable for the instrumentinstrument

Any idea or suggestion to:Any idea or suggestion to:– Improve operations?Improve operations?– Exercise operations under failure cases?Exercise operations under failure cases?


Page 21


Slew survey, are we done throughSlew survey, are we done through

Past operationsPast operations– Already reported by Pedro, real target in May.Already reported by Pedro, real target in May.

Future activities and open pointsFuture activities and open points– Offset maps required? Offset maps required? current ones are still current ones are still

theoreticaltheoretical– Calibration items? Calibration items? What would be the What would be the

calibration efforts and type of tests?calibration efforts and type of tests?– Others?Others?


Page 22


S2K in Spare supportS2K in Spare support

Support in the validation of the S2K once it Support in the validation of the S2K once it is integrated in the Spare chain.is integrated in the Spare chain.

Adaptation of DB items to spare (offsets, Adaptation of DB items to spare (offsets, BP’s, clocks, voltage settings).BP’s, clocks, voltage settings).

Define a mirroring mechanism to upgrade Define a mirroring mechanism to upgrade Spare DB in line with future operational Spare DB in line with future operational concepts.concepts.

Status of EPICs Operations

Documents