Learning-based approach for designing error-correcting codes Shan LU Gifu University ็ฌฌ๏ผๅ ่ชคใ่จๆญฃ็ฌฆๅทใฎใฏใผใฏใทใงใใ ๏ผ๏ผ๏ผ๏ผๅนด๏ผๆ๏ผๆฅ๏ฝ๏ผๆ๏ผๆฅ ๏ผ ใชใณใฉใคใณ้ๅฌ 1
Learning-based approach for designing error-correcting codes
Shan LU Gifu University
็ฌฌ๏ผๅ ่ชคใ่จๆญฃ็ฌฆๅทใฎใฏใผใฏใทใงใใ
๏ผ๏ผ๏ผ๏ผๅนด๏ผๆ๏ผๆฅ๏ฝ๏ผๆ๏ผๆฅ๏ผ ใชใณใฉใคใณ้ๅฌ 1
Noisy-Channel Coding Theorem
Given a noisy channel with channel capacity C,for arbitrary small ๐๐ > 0, if information transmitted rate ๐ ๐ < ๐ถ๐ถ and code length n is sufficiently large, there exists an [n, k] code of rate ๐๐/๐๐ โฅ ๐ ๐ with error probability ๐๐
๐๐ โค ๐๐.
Transmitter NoisyChannel Receiver
Noisy-Channel Coding Theorem
2
The model of coding system
Encoder๐๐๐๐
NoisyChannel
Decoder๐๐๐๐
message codeword
3
Traditional error-correcting codes (linear codes)โข Generator Matrix G
โข Parity-Check Matrix
G: generator matrixw: message vector
x: codeword
Short code (design Generator Matrix or Parity-Check Matrix)Hamming Distance: block codes using finite field algebra
Such as: Hamming codes, Golay codes, RM codes, BCH codes, RS codes, etc.Free distance: convolutional codes by increasing memory order/selecting polynomials
Such as: convolutional code/ Turbo code Long code: design the property for code ensemble, choose one code from code ensemble.
โข Fixed design of error-correcting codes
(If failed to transmit, ARQ (Automatic repeat-request) is used.) 4
Flexible design of error-correcting codes
ACK/NACK
โข HARQ:(Hybrid Automatic repeat-request)โข Rateless codeโข Rate-compatibility code
Encoder๐๐๐๐
NoisyChannel
Decoder๐๐๐๐
message codeword
5
Learning design of error-correcting code
โข AI techniques: machine learning/ deep learning/Reinforcement learningโข AI techniques is a natural choice for learning the encoding and decoding
functions due to their ability to perform universal function approximation.
โข How: Neural network + optimization algorithm (ใใฅใผใฉใซใใใใฏใผใฏ + ๆ้ฉๅใขใซใดใชใบใ )
โข fixed design of error-correcting codes โข Supervised learning: Learning with a labeled training set
โข flexible design of error-correcting codes โข Reinforcement learning: Learn to act based on feedback/reward
6
Design of error-correcting codes
Fixed design Flexible designtraditional
design Differential evolution
algorithmRateless code
Rate-compatibility code
Design by learning
Supervised learning Reinforcement learning
7
Traditional design of error-correcting code
Fixed design Flexible designDifferential evolution algorithm
ๅทฎๅ้ฒๅๆณ
(Example of LDPC)
Rateless codeRate-compatibility code
8
LDPC code๏ผLow-Density Parity-Check code
Design a code with coderate ๐น๐น = ๐๐/๐๐ is to find a parity check matrix ๐ฏ๐ฏ โ {๐๐,๐๐}(๐๐โ๐๐)ร๐๐
Find a sparse Hor
a sparse Tannergraph
Sparse parity check matrix
sparse Tanner graph
9
Idea: fixed design of code ensembleโข Long code: design the code ensembleโข Code ensemble: a set of code with same property.Example of property: convolutional/Turbo code : weight enumerator
LDPC code : degree/ degree distribution/girth
Idea of design code ensemble:โ Random choose one parameter of code ensemble.โก Estimate the average performance of code ensemble.โข Iteratively update the parameter of the code ensemble until
obtain the excellent performance.โฃ Pick a code at random from the ensemble and expect
excellent performance.
Constructor:code ensemble
Estimator:average
performance
10
Example: Regular (๐๐๐๐,๐๐๐๐)-LDPC Code Ensemble
11
protograph
Tanner graph(a specific code)
(code ensemble: codes set with all of possible permutations)
Fixed permutation
permutations codelength๏ผn = 3Q
code rate๏ผR = 1/3
copy
Q = 3
permute
Example: Matrix presentation of regular LDPC Code Ensemble
12
(code ensemble: all of possible permutations)
Base matrix
copy
Q = 3
Fixed permutation
permute
codelength๏ผn = 3Q
code rate๏ผR = 1/3
Tanner graph(a specific code)
Example: Differential Evolution Algorithm(ๅทฎๅ้ฒๅๆณ)
โ Initiation: Given a degree (๐๐๐๐,๐๐๐๐) of LDPC code.
โก Estimate the average performance (Decoding Threshold: in terms of the noise standard deviation) by density evolution(ๅฏๅบฆ้ฒๅ)/EXIT chart over a code ensemble (๐๐๐๐,๐๐๐๐)
โข Update the degree (๐๐๐๐,๐๐๐๐) , find the parameter of code ensemble with excellent performance.
โฃ Pick a code at random from the ensemble and expect excellent performance.
Constructor(degree (๐๐๐๐, ๐๐๐๐) )
Evaluator (Decoding Threshold)
13
Idea: Flexible Design of error-correcting code (HARQ)
ACK/NACK
Encoder๐๐๐๐
NoisyChannel
message codeword Decoder๐๐๐๐
HARQ
14
Transmitter
Receiver
ACK๏ผ acknowledgement ่ฏๅฎๅฟ็ญNACK: negative-acknowledgement, ๅฆๅฎๅฟ็ญ
Flexible design: Rateless code/Rate-compatibility code
Parity check matrix of Rate-compatibility code
Encoder๐๐๐๐
NoisyChannel
message codeword Decoder๐๐๐๐
Rateless code (fountain code)
ACK/NACK
15
Learning-based approach of designing error-correcting codes
Fixed design: Supervised Learning (Autoencoder)
16
Supervised Learning Basic: Training and Testingโข Training stage: ( Learning with a labeled training set: input data, desired results)
โข Testing stage:
Input Data Learning system
Output Data
NewInput Data
Learning system
Output/Testing
Neural Network(NN)ใใฅใผใฉใซใใใใฏใผใฏ
17
Neural Network
18
The operation in the neural networkWeights
Activation functions
๐พ๐พ๐๐, ๐๐๐๐ ๐พ๐พ๐๐ , ๐๐๐๐ ๐พ๐พ๐๐, ๐๐๐๐
๐๐๐๐ ๐๐๐๐
Inputx
Iteration of โLinearโ and โnon-linearโ operation
๐๐๐๐
๐พ๐พ1x +b1 ๐๐๐๐(๐พ๐พ1x+b1)x ๐พ๐พ2๐๐๐๐(๐พ๐พ1x+b1) +b2 โฆ โฆInput Linear
TransformNon-LinearTransform
LinearTransform
๐๐๐๐(๐พ๐พ2๐๐๐๐(๐พ๐พ1x+b1) +b2)Non-LinearTransform
Example: activation functions (ๆดปๆงๅ้ขๆฐ)(non-linear)
ReLU(Rectified Linear Unit):
Sigmoid function:
19
How Neural Network Learning: Backpropagation่ชคๅทฎ้ไผๆญๆณ
โข Forward Pass:
Input Data Neural Network Prediction
โข Backward Pass: Neural Network
Measure of Error
(loss function)Update weights
by SGD to decrease loss function
quantifies gap between prediction and desired results.
Constructor Evaluator
SGD: stochastic gradient descent็ขบ็็ๅพ้ ้ไธๆณ
Loss functionFor regression: Mean Squared Error
For classification:Cross Entropy Loss
20
Autoencoders of error-correcting codes system
Encoder๐๐๐๐
NoisyChannel
Decoder๐๐๐๐
message codeword
Tim OโShea, and Jakob Hoydis, โAn Introduction to Deep Learning for the Physical Layerโ, IEEE Transactions on Cognitive Communications and Networking, Vol. 3-4, PP. 563-575, Dec. 2017.
Encoder
Decoder
energy constraint
โข An autoencoder is a neural network that is trained to attempt to copy its input to its output.
21
๐๐ โค ๐๐
Autoencoders of error-correcting codes system (training process)
โ Loss function:
โก Update weights by SGD:
Constructor (autoencoder)
Noisy channel
NN Encoder NN Decoder
EvaluatorUpdated weights
22
BLER vs Eb/N0 for the autoencoder and baseline communication schemes
Training Eb/N0 = 7dB
Tim OโShea, and Jakob Hoydis, โAn Introduction to Deep Learning for the Physical Layerโ, IEEE Transactions on Cognitive Communications and Networking, Vol. 3-4, PP. 563-575, Dec. 2017.
23
Autoencoder for NN Error-Correcting Systems
https://mlc.committees.comsoc.org/research-library/
Advantages:โข Possible to design non-linear code with good performance
(suitable for coding and decoding for multi-access channel)
โข Simper design for construction(suitable for designing decoding algorithm of the code)
Disadvantage:โข Difficult to design long code
(try to design the parameters of the code ensemble)
24
Further design of NN for error-correcting systemsDecoding algorithmโข E. Nachmani, E. Marciano, L. Lugosch, W. J. Gross, D. Burshtein and Y. Beโery, โDeep learning methods for improved decoding of linear codes,โ IEEE
Journal of Selected Topics in Signal Processing, vol. 12, no. 1, pp.119-131, February 2018.
โข L. Lugosch and W. J. Gross, โNeural offset min-sum decoding,โ in Proc. IEEE International Symposium on Information Theory (ISIT), June 2017.
โข F. Liang, C. Shen and F. Wu, โAn iterative BP-CNN architecture for channel decoding,โ in IEEE Journal of Selected Topics in Signal Processing, vol. 12, no. 1, pp. 144-159, February 2018.
โข W. Xu, X. You, C. Zhang and Y. Beโery, โPolar decoding on sparse graphs with deep learning,โ in Proc. IEEE Asilomar Conference on Signal, System, Computers, October 2018.
Coding and decoding for multi-access channel โข L. WEI, S. Lu, H. Kamabe, J. Cheng, โUser Identification and Channel Estimation by DNN-Based Decoder on Multiple-Access Channel,โ to be presented
in 2020 IEEE Global Communications Conference.
โข S. Takabe, Y. Yamauchi, and T. Wadayama, โTrainable projected gradient detector for sparsely spread code division multiple access,โ preprint arXiv:1910.10336, 2019.
โข J. Lin, S. Feng, Z. Yang, Y. Zhang and Y. Zhang, โA novel deep neural network-based approach for sparse code multiple access,โ preprint arXiv:1906.03169, 2019.
โข I. Abidi, M. Hizem, I. Ahriz, M. Cherif and R. Bouallegue, โConvolutional neural networks for blind decoding in sparse code multiple access,โ in Proc.International Wireless Communications & Mobile Computing Conference (IWCMC), 2019.
Joint design of source-channel coding โข Y. M. Saidutta, A. Abdi and F. Fekri, โM to 1 joint source-channel coding of Gaussian sources via dichotomy of the input space based on deep learning,โ in
Proc. Data Compression Conference (DCC), 2019.
https://mlc.committees.comsoc.org/research-library/25
Learning-based approach of designing error-correcting codes
Reinforcement learning (ๅผทๅๅญฆ็ฟ)(Flexible design of error-correcting codes)
Instead of trying to produce a program to simulate the adult mind, why not rather try to produce one which simulates the childโs ?
โโAlan Turing 26
Reinforcement Learningโข Task
โข Learn how to behave successfully to achieve a goal while interacting withan external environment
โข How to design? Reinforcement Learning problems can be modeled by a so-called
Markov Decision Process (MDP) (ใใซใณใๆฑบๅฎใใญใปใน)
โข Examplesโข Game playing: player knows whether it win or lose, but not know how to move at each stepโข Control: a traffic system can measure the delay of cars, but not know how to decrease it.โข Error-correcting codes: a system knows how to measure the performance of the codes, but not know
how to construct it at each system .
27
Markov Decision Process (MDP) modelโข State: s / S: state setโข Action: ๐๐ / A: actions space
Agentโข Policy(ๆน้): ๐๐(๐๐|๐ ๐ )
function to map states ๐ ๐ to actions ๐๐
Environmentโข ๐ท๐ท๐๐(๐๐โฒ|๐๐,๐๐)
probability of action ๐๐ in state ๐ ๐ at time ๐ก๐กto state ๐ ๐ โฒ at time ๐ก๐ก + 1.
โข ๐๐(๐ ๐ , ๐๐, ๐ ๐ โฒ)immediate reward(ๅณๆๅ ฑ้ ฌ): feedback after
transitioning from state ๐ ๐ to state ๐ ๐ ๐ , triggered by action ๐๐.
Environment๐ท๐ท๐๐(๐๐โฒ|๐๐,๐๐)๐๐(๐ ๐ , ๐๐, ๐ ๐ โฒ)
Agent๐๐(๐๐|๐ ๐ )
action areward r
new state sโ
Neural network
28
Return (long-run reward้ทๆๅ ฑ้ ฌ)
MDP model to reinforcement learning
โข How to define the rewards ๐๐(๐ ๐ , ๐๐, ๐ ๐ โฒ) and returnโข How to change the policy based on experience โข How to change transitions ๐๐(๐ ๐ โฒ|๐ ๐ , ๐๐)
The agent task: To find an optimal policy ๐๐(๐๐|๐ ๐ ) that maximize Return (long-run reward)
๐๐ 1
๐๐ 2 ๐๐ 3 ๐๐ 4 ๐๐ 5๐๐ 1
๐๐ 2 ๐๐ 3 ๐๐ 4S: state set
A: actions space
๐๐(๐๐|๐ ๐ )
29
Example of RL for designing error-correcting code
Feedback
Encoder๐๐๐๐
NoisyChannel
message codeword Decoder๐๐๐๐
Environment(Estimator)๐ท๐ท๐๐(๐๐โฒ|๐๐,๐๐)๐๐(๐ ๐ , ๐๐, ๐ ๐ โฒ)
Agent(constructor)
๐๐(๐๐|๐ ๐ ) action a
reward rnew state Sโ
Neural network
โข S (state set): SNR (by step)โข A: actions space: parity check matrix H
(โ1โโs positions of P in H=[IK , P])
โข Reward ๐๐(๐ ๐ , ๐๐, ๐ ๐ โฒ) : BERโข Return:
โข ๐๐(๐๐|๐ ๐ ): depend parity check matrix by NNโข ๐๐(๐ ๐ โฒ|๐ ๐ , ๐๐) : depend SNR by NNโข How to change the ๐๐(๐ ๐ โฒ|๐ ๐ , ๐๐)/๐๐(๐๐|๐ ๐ ):
update based on SGDโฆ 30
System model of error-correcting codes with feedback
Encoder๐๐๐๐
NoisyChannel
Decoder๐๐๐๐
message codeword
energy constraint
31
Error-correcting codes RL model (training)
Encoder๐๐๐๐
NoisyChannel
Decoder๐๐๐๐
codeword
energy constraint
๐๐(๐ ๐ , ๐๐, ๐ ๐ โ): immediate reward
Learning ๐๐(๐๐|๐ ๐ )
32
p(SNR1)p(SNR2)
...
Learning ๐๐(๐ ๐ โ|๐ ๐ , ๐๐)
p(H1)p(H2) ...
Hi
Compute Loss
Supervised Learning
Unsupervised Learning system also can be constructed.
Error-correcting codes after training
SNR1
H 2 H 3 H4 H 5H 1
S: SNR set
A: Code space
๐๐(๐๐|๐ ๐ )
SNR 2 SNR3 SNR4
33
Extension of RL for error-correcting systems
Advantages:โข Possible to design code corresponding state of channel information(SCI)
Polar codes: (such as frozen positions of polar codes)โข L. Huang, H. Zhang, R. Li, Y. Ge and J. Wang, โReinforcement learning for nested polar code construction,โ preprint
arXiv:1904.07511, 2019โข F. Carpi, C. Hรคger, M. Martalรฒ, R. Raheli and H. D. Pfister, โReinforcement learning for channel coding: Learned bit-flipping
decoding,โ in Proc. 57th Annual Allerton Conference on Communication, Control, and Computing (Allerton), 2019.MIMO system (state of channel information )โข Y.-S. Jeon, J. Li, N. Tavangaran, and H. V. Poor, โData-Aided Channel Estimator for MIMO Systems via Reinforcement
Learning,โ preprint arXiv:2003.10084, 2020.โข Y.-S. Jeon, N. Lee and H. V. Poor, โRobust data detection for MIMO systems with one-bit ADCs: A reinforcement learning
approach,โ preprint arXiv:1903.12546, 2019.โข M. Goutay, F. Ait Aoudia and J. Hoydis, โDeep reinforcement learning autoencoder with noisy feedback,โ preprint
arXiv:1810.05419, 2018.
Disadvantage:โข Difficult to design long code
34
Conclusionfixed design flexible design
Traditional design Differential evolution algorithm
Rateless code/Rate-compatibility code
Design by learningโข perform universal function
approximationโข Automatic design
Supervised learning Reinforcement learning(for feedback channel)
Constructor
Estimator
โข Code design is a code property optimization problems.
35
Constructor: code/ code ensemble (G or NN)Estimator: BER/threshold performance (G or NN)