1 Solar Data Analysis F. Breitling, G. Mann, C. Vocks 3 nd Workshop of the Solar Key Science Project of LOFAR Astrophysikalisches Institut Potsdam 2010 July 05 2010 July 5 Solar Data Analysis F. Breitling
Feb 01, 2016
1
Solar Data Analysis
F. Breitling, G. Mann, C. Vocks
3nd Workshop
of the
Solar Key Science Project of LOFAR
Astrophysikalisches Institut Potsdam
2010 July 05
2010 July 5 Solar Data Analysis F. Breitling
OverviewThe three LOFAR data types
1. Dynamic spectra
2. Single station imaging
3. Standard imaging
RCU: Receiver unit
RSP: Remote station processing board
TBB: Transient buffer boards
2010 July 5 Solar Data Analysis F. Breitling
2
Dynamic spectraData
• Beam-formed data (currently)– Container format: Hirachical Data Format 5 (HDF5)
– Specification: Interface Control Document 3
lofar-usg-icd-003.pdf
• Dynamic spectrum data (maybe in future)– Container format: Hirachical Data Format 5 (HDF5)
– Specification: Interface Control Document 6
lofar-usg-icd-006.pdf
=> both formats are currently under heavy development
Ref: http://usg.lofar.org/wiki/doku.php?id=documents:lofar_data_products
2010 July 5 Solar Data Analysis F. Breitling
3
Dynamic spectraSoftware
• Dynspec package
• developed by Solar KSP (Frank Breitling)
• part of LOFAR User Software (http://usg.lofar.org)
• build on Data Access Layer (DAL)
• Python scripts
• Components – specview.py (can read Fit and HDF5 files)
– downsample.py (can also create Fit files)
– rebin.py
2010 July 5 Solar Data Analysis F. Breitling
4
Dynamic spectraFirst solar observations
Date: 2009-09-11 10:08:52 (ID108)
2009-09-11 13:09:39 (ID109)
Duration: 1h
LOFAR stations: CS302-LBA
RS307-HBA
RS503-HBA
Measured param.: voltages of both polarizations
Subbands/station: 62
Down sampling: 128
2010 July 5 Solar Data Analysis F. Breitling
5
Dynamic spectraFirst results
2010 July 5 Solar Data Analysis F. Breitling
6
Dynamic spectraIdentified problems
– stripe of high intensity in CS302, shifting from subband 30-40 for obs108 to subband 50-60 for obs109
– intensity peaks and drops for subband 0-20 in obs109_CS302LBA
– intensity gradient in obs109_RS503HBA
2010 July 5 Solar Data Analysis F. Breitling
7
Dynamic spectraNext steps
• New versions of the beam-formed and dynamic spectra data format has to be finalized
• Dynspec software can be adapted
• Discussion with Michael Wise and Lars Bähren about their further development plans
• Travel to ASTRON is already planed for July 12-16
2010 July 5 Solar Data Analysis F. Breitling
8
Single station imagingData
• Visibilities from LCU correlator– 96 receiver units (RCUs) x 2 polarizations x 16 byte
– Format: Binary
– Specification: • Menno Norden <norden at astron.nl>• rspctl --help (running on LCU)
Reference: http://usg.lofar.org/forum/index.php?topic=202.0
2010 July 5 Solar Data Analysis F. Breitling
9
Single station imagingSoftware
• acm2skyimage.m
• developed by Stefan Wijnholds <wijnholds at astron.nl>
• runs with Matlab and Octave
• Extended components by Frank Breitling– SolarAzEl.m to add the Sun
– RADec2AzEl.m to add other astronomical objects
(also published as Matlab package at http://www.mathworks.com/matlabcentral/fileexchange/25617-right-ascensiondeclination-to-azimuthelevation)
– make-movies.sh
2010 July 5 Solar Data Analysis F. Breitling
10
Single station imagingPotsdam commissioning
2010 July 5 Solar Data Analysis F. Breitling
11
Single station imagingResults from Effelsberg
2010 July 5 Solar Data Analysis F. Breitling
12
Single station datacombining with GURT
2010 July 5 Solar Data Analysis F. Breitling
13
Standard imagingBasics
• Basic data analysis software– CASA (Common Astronomy Software Applications)
http://casa.nrao.edu/
• Data format– Measurements Sets (MS) from CASA
• Data processing: Standard Imaging Pipeline– The LOFAR wiki
http://www.lofar.org/wiki/doku.php?id=software:standard_imaging_pipeline
– The LOFAR Imaging Cookbookhttp://www.mpa-garching.mpg.de/~fdg/LOFAR_cookbook/LOFAR_Cookbook_v2.3.pdf
2010 July 5 Solar Data Analysis F. Breitling
14
Standard imagingPipeline
Components
• OLAP: Online Application Processing (Blue Gene/P correlator)
• DP3/DPPP: Default Pre-Processing Pipeline (for flagging and averaging)
• BBS: BlackBoard Selfcal (calibration)
• CImager: Imager
Control
• via parset files
2010 July 5 Solar Data Analysis F. Breitling
15
Standard imagingFirst solar observations
Date: 2010-06-09 10:17:00, 11:41:00, 13:40:00, 15:40:00
Duration: 4 x 10 Min
LOFAR stations: 20 CS +3 RS, i.e.
RS106HBA, RS306HBA, RS503HBA
[CS00{1-7}, CS0{21,30}, CS103] HBA{0,1}
Measured param.: complex visibilities, 2 polarizations
Frequency range: ~150-220 MHz
Subbands /station: 7
2010 July 5 Solar Data Analysis F. Breitling
16
Standard imagingData Inspection with casaplotms
• a lot of RFI• flagging with
flagdata• creation of
NDPPP.parset reprocessing
with NDPPP
2010 July 5 Solar Data Analysis F. Breitling
17
Standard imagingSky model
Calibration using BlackBoard Selfcal requires skymodel
File contents of 078xx.skymodel:
# (Name, Type, Ra, Dec, I, Q, U, V, ReferenceFrequency='60e6',
SpectralIndexDegree='0', SpectralIndex:0='0.0', MajorAxis, MinorAxis, Orientation) = format
# The above line defines the field order and is required.
TauA.neb, GAUSSIAN, 05:34:32.00, +22.00.52.00, 1888.55, 0.0, 0.0, 0.0,81.5e6, 0, -0.299, 420.0, 290.0, 0.0
3C123, POINT, 04:37:04.72, +29.40.15.6, 454.97
3C133, POINT, 05:02:58.58, +25.16.22.7, 44.67
3C131, POINT, 04:53:23.26, +31.29.26.4, 37.54
3C139.2, POINT, 05:24:26.92, +28.12.54.2, 35.88
3C141, POINT, 05:26:43.55, +32.50.02.0, 31.76
3C132, POINT, 04:56:43.00, +22.49.23.7, 24.39
4C29.17, POINT, 05:10:40.10, +29.09.00.5, 19.48
4C29.16, POINT, 05:09:29.50, +29.57.56.3, 17.43
3C138, POINT, 05:21:10.46, +16.38.26.0, 14.63
PKS0531, POINT, 05:34:44.65, +19.27.21.9, 14.52
Taken from “The VLA Low-Frequency Sky Survey” catalog (http://www.cv.nrao.edu/4mass/VLSSlist.shtml)
2010 July 5 Solar Data Analysis F. Breitling
18
Standard imagingBBS calibration
After running the BBS calibration
amplitudes and phases are stable
2010 July 5 Solar Data Analysis F. Breitling
19
Standard imagingmwimager
after calibration the master worker imager creates:
– PSF – dirty image– clean image
2010 July 5 Solar Data Analysis F. Breitling
20
Standard imagingDirty image
2010 July 5 Solar Data Analysis F. Breitling
21
Dirty image
not yet sucessful
Standard imagingNext steps
• investigation of 2s clock offset of CS002-CS007
(found by Olaf Wucknitz)
• BBS calibration without short baselines
(suggested by James Anderson)
• automatic flagging of RFI
• travel to ASTRON on July 12-16 and discuss with the experts Ger van Diepen and Ger de Bruyn
• take more observations, e.g. new data on July 03
2010 July 5 Solar Data Analysis F. Breitling
22
Progress documentation
2010 July 5 Solar Data Analysis F. Breitling
23
2010 July 5 Solar Data Analysis F. Breitling
24
Solar Science Data CenterYearly data volume
for solar observations with LOFAR
(assuming a maximum observation time of 8h/d)
Observation timegranted to the SKSP
Data per year[TB/yr]
Data rate[MB/s]
33% (maximum) 104 100
5% 16 100
3% 9.5 100
Summary
• The SKSP is engaged in 3 types of data analysis
• Dynamic spectra can already be taken, next the new format has to be completed
• Single station imaging is important for commissioning, single station operations and for our colleagues from GURT
• Standard imaging is the newest data for the SKSP and will need a lot of attention in the future
• Ultimately we plan for a Solar Science Data Center and work is ongoing
2010 July 5 Solar Data Analysis F. Breitling
25
References
• User Software Group: http://usg.lofar.org/• LOFAR User Forum: http://usg.lofar.org/forum/index.php• LOFAR Interface Control Documents (ICDs)
http://usg.lofar.org/wiki/doku.php?id=documents:lofar_data_products• Documentation for the Standard Imaging Pipeline
http://www.lofar.org/wiki/doku.php?id=software:standard_imaging_pipeline• The LOFAR Imaging Cookbook
http://www.mpa-garching.mpg.de/~fdg/LOFAR_cookbook/LOFAR_Cookbook_v2.3.pdf
• Frank Breitling <fbreitling at aip.de> http://www.aip.de/People/fbreitling/
2010 July 5 Solar Data Analysis F. Breitling
26
2010 July 5 Solar Data Analysis F. Breitling
27
Yearly observation time
at Exloo (core stations) altitude >10° above horizon
=> tobs=~8 h/d =10x106 s/y
(Maximum possible)