Data Handling, Compression and Telemetry Formatting...Data Formats • Transmission scheme and multiplexing • Telemetry modelling and optimization • Data Compression • Pipeline

Data Handling, Data Handling, Compression and Compression and Telemetry FormattingTelemetry Formatting

Institute of Space Studies of Catalonia (IEEC)Institute of Space Studies of Catalonia (IEEC)University of Barcelona (UB)University of Barcelona (UB)

Technical University of Catalonia (UPC)Technical University of Catalonia (UPC)

Presented by Jordi Portell (IEEC/UPC)Presented by Jordi Portell (IEEC/UPC)

Gaia SWG/RVS Meeting J. Portell et al.Nice (France), 10-12 December 2003

ScienceScience Data Data Pipeline Pipeline in in GaiaGaia

Instruments [Instruments [CCDsCCDs]]

Data Handling [PDHS/PDHE]Data Handling [PDHS/PDHE]

Formatting and codificationFormatting and codification

Data compressionData compression

TM system / RFTM system / RFDB

MultiplexingMultiplexing

ASTROASTRO MBPMBP RVSRVS


Payload Payload Data Data Handling SystemHandling System•• Payload Data Payload Data

Handling System Handling System (PDHS)(PDHS)

•• Data FormatsData Formats•• Transmission Transmission

scheme and scheme and multiplexingmultiplexing

•• TelemetryTelemetrymodellingmodelling and and optimizationoptimization

•• Data CompressionData Compression•• Pipeline of Pipeline of

Telemetry Telemetry SimulationsSimulations

Full design available (system level)Full design available (system level)–– Modules andModules and submodulessubmodules–– Data fluxData flux

Consistent with GASS, GIBIS and PDHEConsistent with GASS, GIBIS and PDHEA good reference for the final mission: A good reference for the final mission: Processes/modules that generate and Processes/modules that generate and modify the data in the payload ofmodify the data in the payload of GaiaGaiaASTRO and PDHU described. SPECTRO can ASTRO and PDHU described. SPECTRO can easily be included with a similar designeasily be included with a similar design


PDHS PDHS SchematicSchematic•• Payload Data Payload Data








Data Data Formats for SimulationsFormats for Simulations•• Payload Data Payload Data







GAIAGAIA--BCNBCN--007007: : ““OnOn--board and telemetryboard and telemetry data data formatsformats””RecommendedRecommended Data Data Formats for simulations implying the Formats for simulations implying the Telemetry StreamTelemetry Stream::

–– GASSGASS–– Telemetry formatting and packetingTelemetry formatting and packeting–– Data Data compressioncompression–– ......

Current versionCurrent version (1.5b):(1.5b):–– ASTRO (GDAASASTRO (GDAAS--2)2)–– MBP MBP andand RVS RVS temptative codingtemptative coding

Forthcoming versionsForthcoming versions::–– MBP+RVS MBP+RVS stablestable (GDAAS(GDAAS--2). HK+2). HK+AttitudeAttitude??–– FlightFlight--realisticrealistic–– ......


Data Data Formats for SimulationsFormats for Simulations•• Payload Data Payload Data






Telemetry Telemetry SimulationsSimulations CodingCoding//transmission scheme fortransmission scheme for

RVS RVS shouldshould be be decideddecided::–– NowNow 1 1 patchpatch / 1 / 1 starstar ((simplisticsimplistic))–– TransmissionTransmission by by readoutreadout time?time?–– Transmission ofTransmission of 1 1 patchpatch / / star star excluding overlapsexcluding overlaps??


Transmission schemeTransmission scheme•• Payload Data Payload Data







GAIAGAIA--BCNBCN--006006: : Proposal ofProposal of a a transmissiontransmission, , multiplexing and coding scheme for the payload of Gaiamultiplexing and coding scheme for the payload of Gaia::

–– Compilation of requirementsCompilation of requirements–– Multiplexing ofMultiplexing of data data sourcessources–– Integration within Packet TelemetryIntegration within Packet Telemetry–– RobustRobust ((transmission errorstransmission errors, , packet orderpacket order))–– Coding ofCoding of data data fields complyingfields complying CCSDS CCSDS standardsstandards

Optimized forOptimized for a a minimal bandwidth consumptionminimal bandwidth consumptionPrepared forPrepared for data data compression applied afterwardscompression applied afterwardsAlgorithm for compiling theAlgorithm for compiling the data data from instrumentsfrom instrumentsClock imperfections willClock imperfections will be be taken into accounttaken into account ((now now being studied underbeing studied under a Master a Master ThesisThesis))


MultiplexingMultiplexing•• Payload Data Payload Data








RobustnessRobustness•• Payload Data Payload Data








Source PacketsSource Packets ((PacketPacket TM)TM)•• Payload Data Payload Data








Telemetry modelTelemetry model•• Payload Data Payload Data







Model of theModel of the average average telemetry consumptiontelemetry consumption((simplisticsimplistic))

Optimization of codification parametersOptimization of codification parameters: : ResolutionsResolutions, , length oflength of Time Time SlotsSlots, , maximum sourcesmaximum sources in a SP...in a SP...MatLab simulations for optimizing these parametersMatLab simulations for optimizing these parametersGUI GUI framework for optimizing telemetry parametersframework for optimizing telemetry parameters


Optimization of parametersOptimization of parameters•• Payload Data Payload Data








Average Average resultsresults•• Payload Data Payload Data






Telemetry Telemetry SimulationsSimulations Preliminary optimizationsPreliminary optimizations::

–– HigherHigher time time resolutionsresolutions, CCSDS, CCSDS--compatible data compatible data fieldsfields–– > 2.5 > 2.5 kbpskbps average average savingsaving (GDAAS2(GDAAS2--like modified codinglike modified coding))–– > 32 > 32 kbpskbps average average savingsaving (GDAAS2 original (GDAAS2 original codingcoding))–– Adaptive codification recommendedAdaptive codification recommended ((depending on star ratedepending on star rate))

ASTRO average data ASTRO average data raterate: ~1.9 : ~1.9 MbpsMbps (no (no comprcompr.).)

< Only averaged (simplistic) simulations >

“Telemetry coder” under development...“flight-realistic” telemetry curves


Data Data compressioncompression•• Payload Data Payload Data




•• Telemetry Telemetry modellingmodelling and and optimizationoptimization

•• Data Data CompressionCompression

•• Pipeline of Pipeline of Telemetry Telemetry SimulationsSimulations

Redesign ofRedesign of GaiaGaia has relaxed the requirements:has relaxed the requirements:Lower spin rate Lower spin rate lower star rate lower star rate llowerower data ratedata rate

Anyway:Anyway: ComprCompr. ratio of ~. ratio of ~44 is required (is required (losslesslossless))

Preliminary results are satisfactory:Preliminary results are satisfactory:

Differential Differential adaptiveadaptive prepre--compressorscompressors+ standard compression algorithms+ standard compression algorithms

3.0 to 3.83.0 to 3.8using GASS binary data (raw data),using GASS binary data (raw data), GaiaGaia--1 design1 design


Data Data compressioncompression•• Payload Data Payload Data





•• Data Data CompressionCompression

•• Pipeline of Pipeline of Telemetry Telemetry SimulationsSimulations

Telemetry stream readyTelemetry stream readyfor including thefor including the DataDataCompressionCompression ““blackblack boxbox””::

New designs coming soonNew designs coming soon::–– Use Use of newof new data data fromfrom GASS GASS usingusingGaiaGaia--2 2 and other improvementsand other improvements–– Improved preImproved pre--compressorscompressors

–– Persistency of TreesPersistency of Trees & & DictionariesDictionaries are are taken into accounttaken into account(data (data integrity even with txintegrity even with tx. . errorserrors))–– Cosmic rays mustCosmic rays must be be taken into accounttaken into account–– Optimized algorithmsOptimized algorithms((onon--board processing requirementsboard processing requirements!)!)

Data Compression

...


Telemetry pipelineTelemetry pipeline•• Payload Data Payload Data







Telemetry coder

DataCompression

[GAIA-BCN-007]

Txt2Bin

Binarydata

[GAIA-BCN-006]

Format Mplex. Packet

Source Packets (uncompressed science data)

[CCSDSPacket TM]

SPs (compr.sci. data) Tr. Frames

composerTFs [Transmiss.

Channel]TFs with errors

[CCSDSPacket TM][HRTF][GAIA-BCN-001 and ...]

Ground StationReceive Error Correct.

TFs OK

(GDAAS)

(GDAAS)

DB

ASCIIoutput

Raw sci. data

[Errors, cosmic,windows,

SourceId...]

[Error rate]

GASS

[Errors, cosmic,windows,

SourceId...]

DecoderSPsTFs

[PDHS, clocks...]


ConclusionsConclusionsPDHS PDHS designdesign finishedfinished

Data Data sourcessources, , modifiers and main processes identifiedmodifiers and main processes identified[ SPECTRO [ SPECTRO instrument toinstrument to be be includedincluded ]]

Data Data formattingformatting defineddefinedStandarisation for simulators makingStandarisation for simulators making use use of telemetry streamof telemetry stream

[ [ ConsistentConsistent MBP MBP andand RVS data RVS data mustmust be be includedincluded ]][ [ Document must evolve towards flightDocument must evolve towards flight--realisticrealistic ]]

Transmission schemeTransmission scheme defineddefined, , modelled and optimizedmodelled and optimizedImprovementsImprovements in data in data qualityquality, , reliability and compatibilityreliability and compatibilityReducedReduced data data raterate

[ [ ‘‘Telemetry CoderTelemetry Coder’’ toto be be developeddeveloped ]]

Key factors forKey factors for data data compressioncompression are are identifiedidentifiedDevelopment of an optimal system is well plannedDevelopment of an optimal system is well planned

[ Final Data [ Final Data Compression System to Compression System to be be developeddeveloped ]]

Telemetry pipelineTelemetry pipeline identifiedidentifiedTelemetry simulations willTelemetry simulations will be be easily concatenatedeasily concatenatedImprovementsImprovements//updatesupdates//adaptationsadaptations can be can be easily introduced between simulatorseasily introduced between simulators

Data Handling, Compression and Telemetry Formatting...Data Formats • Transmission scheme and multiplexing • Telemetry modelling and optimization • Data Compression • Pipeline

Documents