Enhancing Urban Flow Maps via Neural ODEs Fan Zhou 1 , Liang Li 1 , Ting Zhong 1 , Goce Trajcevski 2 , Kunpeng Zhang 3 and Jiahao Wang 1 1 School of Information and Software Engineering, University of Electronic Science and Technology of China 2 Iowa State University, Ames IA 3 University of Maryland, College Park MD {fan.zhou, zhongting, wangjh}@uestc.edu.cn, [email protected], [email protected], [email protected] Abstract Flow super-resolution (FSR) enables inferring fine- grained urban flows with coarse-grained observa- tions and plays an important role in traffic moni- toring and prediction. The existing FSR solutions rely on deep CNN models (e.g., ResNet) for learn- ing spatial correlation, incurring excessive mem- ory cost and numerous parameter updates. We pro- pose to tackle the urban flows inference using dy- namic systems paradigm and present a new method FODE – FSR with Ordinary Differential Equations (ODEs). FODE extends neural ODEs by introduc- ing an affine coupling layer to overcome the prob- lem of numerically unstable gradient computation, which allows more accurate and efficient spatial correlation estimation, without extra memory cost. In addition, FODE provides a flexible balance be- tween flow inference accuracy and computational efficiency. A FODE-based augmented normaliza- tion mechanism is further introduced to constrain the flow distribution with the influence of exter- nal factors. Experimental evaluations on two real- world datasets demonstrate that FODE significantly outperforms several baseline approaches. 1 Introduction Urban flow super-resolution (FSR) aims at inferring fine- grained crowd flows in a city based on coarse-grained obser- vations. As a variant of image SR in the traffic domain [Cai et al., 2019; Wang et al., 2019] it has practical significance in urban planning and traffic monitoring. Despite its close re- lationship to image SR, FSR has certain constraints the most important one being the structural constraint – i.e., the sum of the flow volumes in surrounding regions in the inferred fine-grained map strictly equals that of their corresponding superregion in the original map. However, in practice, the distribution of the flows in a given region is affected by many external factors, e.g., weather, time-of-day, etc. A recent work [Liang et al., 2019] addresses FSR problem based on the residual networks [He et al., 2016] and uses a simple normalization scheme to constrain the structural dis- tribution of flows. However, the proposed architecture heav- ily relies on empirically stacking deep neural networks and, consequently, lacks principles to guide the design of effec- tive and interpretable FSR networks. Furthermore, it pays no attention to the computational overheads as the network goes deeper, which requires significantly more memory cost and network parameters – making it hard to be optimized and prone to be overfitting. Recently, there has been a growing interest in bridging the gap between neural networks and dynamic systems [E, 2017; Lu et al., 2018]. In particular, a recent study [Chen et al., 2018] has shown that ResNet [He et al., 2016] can be in- terpreted as discretized Neural Ordinary Differential Equa- tions (NODE), which provides a new perspective of improv- ing stability and trainability of neural networks. For exam- ple, an additional state (called adjoint) is used in [Chen et al., 2018] to solve the ODEs. In this way, it does not need to store the intermediate states of the forward pass, resulting in O(1) memory cost in each ODE block. A few studies have been proposed to improve the performance of NODE [Liu et al., 2019b] and/or to apply NODE in different domains such as graph neural networks [Poli et al., 2019], time series learning [Rubanova et al., 2019; De Brouwer et al., 2019] and generative models [Heinonen and L¨ ahdesm¨ aki, 2019; Grathwohl et al., 2019]. However, the dynamics of either the hidden state or the adjoint are usually unstable, due to the numerical instability of solving the backward ODEs. AN- ODE [Gholami et al., 2019] and its variant [Zhang et al., 2019] improve the robustness and generalization of the ad- joint method by adding a few of the intermediate states from the forward pass however, these cannot fundamentally solve the incorrect gradient problem. In this work, we introduce a new SR method – FODE (FSR with ODEs) for fine-grained urban flow inference. FODE is a more general neural ODE architecture that addresses the numerical instability problem of previous methods, while not incurring extra memory cost. The main idea of FODE is to in- corporate an affine coupling layer in each ODE block to avoid the inaccurate gradient issue. The input to a FODE block can then be accurately reconstructed from its outputs without the need of storing intermediate states. We show theoretically and empirically that the proposed FODE model can be used to learn spatial correlations among urban regions and the in- fluence of external features. The main contributions of this work can be summarized as: • We present a new neural ODE framework that can accu- Proceedings of the Twenty-Ninth International Joint Conference on Artificial Intelligence (IJCAI-20) 1295
Welcome message from author
This document is posted to help you gain knowledge. Please leave a comment to let me know what you think about it! Share it to your friends and learn new things together.
Transcript
Enhancing Urban Flow Maps via Neural ODEsFan Zhou1 , Liang Li1 , Ting Zhong1 , Goce Trajcevski2 , Kunpeng Zhang3 and Jiahao Wang1
1School of Information and Software Engineering,University of Electronic Science and Technology of China
2Iowa State University, Ames IA3University of Maryland, College Park MD
{fan.zhou, zhongting, wangjh}@uestc.edu.cn, [email protected], [email protected],[email protected]
AbstractFlow super-resolution (FSR) enables inferring fine-grained urban flows with coarse-grained observa-tions and plays an important role in traffic moni-toring and prediction. The existing FSR solutionsrely on deep CNN models (e.g., ResNet) for learn-ing spatial correlation, incurring excessive mem-ory cost and numerous parameter updates. We pro-pose to tackle the urban flows inference using dy-namic systems paradigm and present a new methodFODE – FSR with Ordinary Differential Equations(ODEs). FODE extends neural ODEs by introduc-ing an affine coupling layer to overcome the prob-lem of numerically unstable gradient computation,which allows more accurate and efficient spatialcorrelation estimation, without extra memory cost.In addition, FODE provides a flexible balance be-tween flow inference accuracy and computationalefficiency. A FODE-based augmented normaliza-tion mechanism is further introduced to constrainthe flow distribution with the influence of exter-nal factors. Experimental evaluations on two real-world datasets demonstrate that FODE significantlyoutperforms several baseline approaches.
1 IntroductionUrban flow super-resolution (FSR) aims at inferring fine-grained crowd flows in a city based on coarse-grained obser-vations. As a variant of image SR in the traffic domain [Caiet al., 2019; Wang et al., 2019] it has practical significancein urban planning and traffic monitoring. Despite its close re-lationship to image SR, FSR has certain constraints the mostimportant one being the structural constraint – i.e., the sumof the flow volumes in surrounding regions in the inferredfine-grained map strictly equals that of their correspondingsuperregion in the original map. However, in practice, thedistribution of the flows in a given region is affected by manyexternal factors, e.g., weather, time-of-day, etc.
A recent work [Liang et al., 2019] addresses FSR problembased on the residual networks [He et al., 2016] and uses asimple normalization scheme to constrain the structural dis-tribution of flows. However, the proposed architecture heav-ily relies on empirically stacking deep neural networks and,
consequently, lacks principles to guide the design of effec-tive and interpretable FSR networks. Furthermore, it paysno attention to the computational overheads as the networkgoes deeper, which requires significantly more memory costand network parameters – making it hard to be optimized andprone to be overfitting.
Recently, there has been a growing interest in bridging thegap between neural networks and dynamic systems [E, 2017;Lu et al., 2018]. In particular, a recent study [Chen et al.,2018] has shown that ResNet [He et al., 2016] can be in-terpreted as discretized Neural Ordinary Differential Equa-tions (NODE), which provides a new perspective of improv-ing stability and trainability of neural networks. For exam-ple, an additional state (called adjoint) is used in [Chen etal., 2018] to solve the ODEs. In this way, it does not need tostore the intermediate states of the forward pass, resulting inO(1) memory cost in each ODE block. A few studies havebeen proposed to improve the performance of NODE [Liuet al., 2019b] and/or to apply NODE in different domainssuch as graph neural networks [Poli et al., 2019], time serieslearning [Rubanova et al., 2019; De Brouwer et al., 2019]and generative models [Heinonen and Lahdesmaki, 2019;Grathwohl et al., 2019]. However, the dynamics of eitherthe hidden state or the adjoint are usually unstable, due to thenumerical instability of solving the backward ODEs. AN-ODE [Gholami et al., 2019] and its variant [Zhang et al.,2019] improve the robustness and generalization of the ad-joint method by adding a few of the intermediate states fromthe forward pass however, these cannot fundamentally solvethe incorrect gradient problem.
In this work, we introduce a new SR method – FODE (FSRwith ODEs) for fine-grained urban flow inference. FODE isa more general neural ODE architecture that addresses thenumerical instability problem of previous methods, while notincurring extra memory cost. The main idea of FODE is to in-corporate an affine coupling layer in each ODE block to avoidthe inaccurate gradient issue. The input to a FODE block canthen be accurately reconstructed from its outputs without theneed of storing intermediate states. We show theoreticallyand empirically that the proposed FODE model can be usedto learn spatial correlations among urban regions and the in-fluence of external features. The main contributions of thiswork can be summarized as:• We present a new neural ODE framework that can accu-
Proceedings of the Twenty-Ninth International Joint Conference on Artificial Intelligence (IJCAI-20)
1295
rately compute the gradients in each block.
• Our novel FSR method requires significantly less mem-ory for fine-grained flow inference compared withResNet-based deep neural network.
• The augmented normalization scheme disentangles theinfluence of external factors on different regions.
• Experiments conducted on two real-world datasetsdemonstrate that our FODE not only outperforms strongbaselines on FSR task, but also provides a flexible bal-ance between computational cost and inference accu-racy.
2 Architecture and MethodologyWe now present the problem settings and discuss in detail themain aspects of the proposed FODE approach.
We assume a grid-based segmentation, similar to [Zhanget al., 2017], which divides a city mapM into H×W spatialgrid-cells based on their geographical locations, i.e., M ={rij}H×M . Each cell rij corresponds to a spatial region –the ith row and the jth column ofM. Let X ∈ RH×W+ be theflow map at a given time, where each entry xij ∈ R+ denotesthe volume of the flow in the region rij .
Definition 1 (Urban Flow Super-Resolution (FSR)). Given acoarse-grained flow map Xc, an upscaling factor N as wellas the external factors E (e.g., weather, events, etc.), the FSRtask is to learn a model F mapping Xc into a fine-grainedflow map Xf ∈ RNH×NW+ :
Xf = F (Xc|E,N ; θ) , (1)
where θ represent all learnable parameters.
Figure 1 illustrates an example of converting a coarse-grained flow map to a fine-grained flow map in the city ofBeijing. The coarse-grained flow map (20km×20km) con-sists of a total of 32×32 cells, each of which denotes a su-perregion. In the fine-grained flow map, there are 128×128subregions in total. At the top of Figure 1, a superregion iscomposed of N2 subregions (N = 4 here). Note that the sumof the flow volume in theN2 subregions (top right) is equal tothe flow volume of the corresponding superregion (top left).
82010 1281 95
16 599 87
86 98 95 7518 15 12 16
SuperregionScaling factor N
<latexit sha1_base64="xC4WQGpW4aYStRlrheUI+mglWFM=">AAAB6HicbVBNS8NAEJ3Ur1q/qh69LBbBU0lE0JMUvHiSFuwHtKFstpN27WYTdjdCCf0FXjwo4tWf5M1/47bNQVsfDDzem2FmXpAIro3rfjuFtfWNza3idmlnd2//oHx41NJxqhg2WSxi1QmoRsElNg03AjuJQhoFAtvB+Hbmt59QaR7LBzNJ0I/oUPKQM2qs1Ljvlytu1Z2DrBIvJxXIUe+Xv3qDmKURSsME1brruYnxM6oMZwKnpV6qMaFsTIfYtVTSCLWfzQ+dkjOrDEgYK1vSkLn6eyKjkdaTKLCdETUjvezNxP+8bmrCaz/jMkkNSrZYFKaCmJjMviYDrpAZMbGEMsXtrYSNqKLM2GxKNgRv+eVV0rqoem7Va1xWajd5HEU4gVM4Bw+uoAZ3UIcmMEB4hld4cx6dF+fd+Vi0Fpx85hj+wPn8AaVdjM4=</latexit><latexit sha1_base64="xC4WQGpW4aYStRlrheUI+mglWFM=">AAAB6HicbVBNS8NAEJ3Ur1q/qh69LBbBU0lE0JMUvHiSFuwHtKFstpN27WYTdjdCCf0FXjwo4tWf5M1/47bNQVsfDDzem2FmXpAIro3rfjuFtfWNza3idmlnd2//oHx41NJxqhg2WSxi1QmoRsElNg03AjuJQhoFAtvB+Hbmt59QaR7LBzNJ0I/oUPKQM2qs1Ljvlytu1Z2DrBIvJxXIUe+Xv3qDmKURSsME1brruYnxM6oMZwKnpV6qMaFsTIfYtVTSCLWfzQ+dkjOrDEgYK1vSkLn6eyKjkdaTKLCdETUjvezNxP+8bmrCaz/jMkkNSrZYFKaCmJjMviYDrpAZMbGEMsXtrYSNqKLM2GxKNgRv+eVV0rqoem7Va1xWajd5HEU4gVM4Bw+uoAZ3UIcmMEB4hld4cx6dF+fd+Vi0Fpx85hj+wPn8AaVdjM4=</latexit><latexit sha1_base64="xC4WQGpW4aYStRlrheUI+mglWFM=">AAAB6HicbVBNS8NAEJ3Ur1q/qh69LBbBU0lE0JMUvHiSFuwHtKFstpN27WYTdjdCCf0FXjwo4tWf5M1/47bNQVsfDDzem2FmXpAIro3rfjuFtfWNza3idmlnd2//oHx41NJxqhg2WSxi1QmoRsElNg03AjuJQhoFAtvB+Hbmt59QaR7LBzNJ0I/oUPKQM2qs1Ljvlytu1Z2DrBIvJxXIUe+Xv3qDmKURSsME1brruYnxM6oMZwKnpV6qMaFsTIfYtVTSCLWfzQ+dkjOrDEgYK1vSkLn6eyKjkdaTKLCdETUjvezNxP+8bmrCaz/jMkkNSrZYFKaCmJjMviYDrpAZMbGEMsXtrYSNqKLM2GxKNgRv+eVV0rqoem7Va1xWajd5HEU4gVM4Bw+uoAZ3UIcmMEB4hld4cx6dF+fd+Vi0Fpx85hj+wPn8AaVdjM4=</latexit><latexit sha1_base64="xC4WQGpW4aYStRlrheUI+mglWFM=">AAAB6HicbVBNS8NAEJ3Ur1q/qh69LBbBU0lE0JMUvHiSFuwHtKFstpN27WYTdjdCCf0FXjwo4tWf5M1/47bNQVsfDDzem2FmXpAIro3rfjuFtfWNza3idmlnd2//oHx41NJxqhg2WSxi1QmoRsElNg03AjuJQhoFAtvB+Hbmt59QaR7LBzNJ0I/oUPKQM2qs1Ljvlytu1Z2DrBIvJxXIUe+Xv3qDmKURSsME1brruYnxM6oMZwKnpV6qMaFsTIfYtVTSCLWfzQ+dkjOrDEgYK1vSkLn6eyKjkdaTKLCdETUjvezNxP+8bmrCaz/jMkkNSrZYFKaCmJjMviYDrpAZMbGEMsXtrYSNqKLM2GxKNgRv+eVV0rqoem7Va1xWajd5HEU4gVM4Bw+uoAZ3UIcmMEB4hld4cx6dF+fd+Vi0Fpx85hj+wPn8AaVdjM4=</latexit>
= 4Subregions
Figure 1: Coarse-grained and fine-grained flow maps in Beijing.
2.1 ArchitectureFigure 2 illustrates the overall architecture of FODE, whichconsists of three main components:
• FODE block which extracts spatial correlations amongflow regions with the proposed new ODEs.• Feature fusion network (FFN) which combines the
FODE blocks and fully connected networks for fusingthe influence of external factors.• Fine-grained flow inference (FFI) which leverages an
augmented N2-normalization scheme (AN2) to esti-mate the distribution of both flows and external factorsfor generating the fine-grained flow maps.
2.2 FODE BlockResNet [He et al., 2016] and its variants have been widelyemployed for image super-resolution [Ledig et al., 2017;Zhang et al., 2018]. Recently, [Liang et al., 2019] ex-ploited residual block for regional correlation learning andfine-grained urban flow inference, which share the same ideaof image SR. Suppose we obtain the high-dimensional hiddenstate z0 from the coarse-grained flow Xc by some convolu-tional layers. The residual block then transforms the hiddenstates zn according to:
zn+1 = zn + f (zn; θ) . (2)
While achieving promising results on SR tasks, residualnetworks still confront the problem of intensive computationand require to tune a huge amount of parameters. There is agrowing interest in bridging the gap between discrete neuralnetworks and continuous dynamic systems. For example, theiterative updates in the above residual block can be viewedas a discretization of a continuous ODE operator [Lu et al.,2018; Chen et al., 2018], if we take time t ∈ T as a continu-ous variable:
dz(t)
dt= f(z(t), t; θ), where z(tn) = zn, (3)
z(T ) = z(0) +
∫ T0
f(z(t), t; θ)dt. (4)
Solving the above ODEs requires computing the gradi-ents through backpropagation, which would be memory pro-hibitive if the time is reduced into infinite steps. In prin-ciple, it requires O(N ) cost to store the all N intermedi-ate activations. NODE [Chen et al., 2018] proposes to ad-dress this problem with the adjoint method. Considering aloss function L, an additional state referred to as the adjointa (t) = ∂L/∂z (t) can be used to compute the gradient w.r.t.parameters θ:
da(t)
dt= −a(t)ᵀ
∂f(z(t), t; θ)
∂z(t)dt, (5)
∂L
∂θ= a(t)
∂z(t)
∂θ= −
∫ 0
Ta(t)ᵀ
∂f(z(t), t; θ)
∂θdt, (6)
where we only need to store the final state z(T ). Hence thisstrategy successfully reduces the memory cost. However, italso introduces other problems – the inaccurate gradient and
Proceedings of the Twenty-Ninth International Joint Conference on Artificial Intelligence (IJCAI-20)
1296
Coarse-grained Flows
Fine-grained Flows
.<latexit sha1_base64="xPJ3I8QuSzHh9z8zb3kPhLdsbeI=">AAAB6HicbVBNS8NAEJ34WetX1aOXxSJ4CokIeix40VsL9gPaUDbbSbt2swm7G6GE/gIvHhTx6k/y5r9x2+agrQ8GHu/NMDMvTAXXxvO+nbX1jc2t7dJOeXdv/+CwcnTc0kmmGDZZIhLVCalGwSU2DTcCO6lCGocC2+H4dua3n1BpnsgHM0kxiOlQ8ogzaqzUcPuVqud6c5BV4hekCgXq/cpXb5CwLEZpmKBad30vNUFOleFM4LTcyzSmlI3pELuWShqjDvL5oVNybpUBiRJlSxoyV39P5DTWehKHtjOmZqSXvZn4n9fNTHQT5FymmUHJFouiTBCTkNnXZMAVMiMmllCmuL2VsBFVlBmbTdmG4C+/vEpal67vuX7jqlq7L+IowSmcwQX4cA01uIM6NIEBwjO8wpvz6Lw4787HonXNKWZO4A+czx94LIy5</latexit><latexit sha1_base64="xPJ3I8QuSzHh9z8zb3kPhLdsbeI=">AAAB6HicbVBNS8NAEJ34WetX1aOXxSJ4CokIeix40VsL9gPaUDbbSbt2swm7G6GE/gIvHhTx6k/y5r9x2+agrQ8GHu/NMDMvTAXXxvO+nbX1jc2t7dJOeXdv/+CwcnTc0kmmGDZZIhLVCalGwSU2DTcCO6lCGocC2+H4dua3n1BpnsgHM0kxiOlQ8ogzaqzUcPuVqud6c5BV4hekCgXq/cpXb5CwLEZpmKBad30vNUFOleFM4LTcyzSmlI3pELuWShqjDvL5oVNybpUBiRJlSxoyV39P5DTWehKHtjOmZqSXvZn4n9fNTHQT5FymmUHJFouiTBCTkNnXZMAVMiMmllCmuL2VsBFVlBmbTdmG4C+/vEpal67vuX7jqlq7L+IowSmcwQX4cA01uIM6NIEBwjO8wpvz6Lw4787HonXNKWZO4A+czx94LIy5</latexit><latexit sha1_base64="xPJ3I8QuSzHh9z8zb3kPhLdsbeI=">AAAB6HicbVBNS8NAEJ34WetX1aOXxSJ4CokIeix40VsL9gPaUDbbSbt2swm7G6GE/gIvHhTx6k/y5r9x2+agrQ8GHu/NMDMvTAXXxvO+nbX1jc2t7dJOeXdv/+CwcnTc0kmmGDZZIhLVCalGwSU2DTcCO6lCGocC2+H4dua3n1BpnsgHM0kxiOlQ8ogzaqzUcPuVqud6c5BV4hekCgXq/cpXb5CwLEZpmKBad30vNUFOleFM4LTcyzSmlI3pELuWShqjDvL5oVNybpUBiRJlSxoyV39P5DTWehKHtjOmZqSXvZn4n9fNTHQT5FymmUHJFouiTBCTkNnXZMAVMiMmllCmuL2VsBFVlBmbTdmG4C+/vEpal67vuX7jqlq7L+IowSmcwQX4cA01uIM6NIEBwjO8wpvz6Lw4787HonXNKWZO4A+czx94LIy5</latexit><latexit sha1_base64="xPJ3I8QuSzHh9z8zb3kPhLdsbeI=">AAAB6HicbVBNS8NAEJ34WetX1aOXxSJ4CokIeix40VsL9gPaUDbbSbt2swm7G6GE/gIvHhTx6k/y5r9x2+agrQ8GHu/NMDMvTAXXxvO+nbX1jc2t7dJOeXdv/+CwcnTc0kmmGDZZIhLVCalGwSU2DTcCO6lCGocC2+H4dua3n1BpnsgHM0kxiOlQ8ogzaqzUcPuVqud6c5BV4hekCgXq/cpXb5CwLEZpmKBad30vNUFOleFM4LTcyzSmlI3pELuWShqjDvL5oVNybpUBiRJlSxoyV39P5DTWehKHtjOmZqSXvZn4n9fNTHQT5FymmUHJFouiTBCTkNnXZMAVMiMmllCmuL2VsBFVlBmbTdmG4C+/vEpal67vuX7jqlq7L+IowSmcwQX4cA01uIM6NIEBwjO8wpvz6Lw4787HonXNKWZO4A+czx94LIy5</latexit>
Df⇡
<latexit sha1_base64="ymPu5urvCtUJCK9gJWJLmRF9h6c=">AAAB+XicbVDLSsNAFL2pr1pfUZduBovgqiQi6LKgC91VsA9oYphMJ+3QySTMTAol9E/cuFDErX/izr9x0mahrQcGDufcyz1zwpQzpR3n26qsrW9sblW3azu7e/sH9uFRRyWZJLRNEp7IXogV5UzQtmaa014qKY5DTrvh+KbwuxMqFUvEo56m1I/xULCIEayNFNi2F2M9CqP8dhZ4KXuKArvuNJw50CpxS1KHEq3A/vIGCcliKjThWKm+66Taz7HUjHA6q3mZoikmYzykfUMFjqny83nyGTozygBFiTRPaDRXf2/kOFZqGodmssiplr1C/M/rZzq69nMm0kxTQRaHoowjnaCiBjRgkhLNp4ZgIpnJisgIS0y0KatmSnCXv7xKOhcN12m4D5f15n1ZRxVO4BTOwYUraMIdtKANBCbwDK/wZuXWi/VufSxGK1a5cwx/YH3+ALG4k7Q=</latexit><latexit sha1_base64="ymPu5urvCtUJCK9gJWJLmRF9h6c=">AAAB+XicbVDLSsNAFL2pr1pfUZduBovgqiQi6LKgC91VsA9oYphMJ+3QySTMTAol9E/cuFDErX/izr9x0mahrQcGDufcyz1zwpQzpR3n26qsrW9sblW3azu7e/sH9uFRRyWZJLRNEp7IXogV5UzQtmaa014qKY5DTrvh+KbwuxMqFUvEo56m1I/xULCIEayNFNi2F2M9CqP8dhZ4KXuKArvuNJw50CpxS1KHEq3A/vIGCcliKjThWKm+66Taz7HUjHA6q3mZoikmYzykfUMFjqny83nyGTozygBFiTRPaDRXf2/kOFZqGodmssiplr1C/M/rZzq69nMm0kxTQRaHoowjnaCiBjRgkhLNp4ZgIpnJisgIS0y0KatmSnCXv7xKOhcN12m4D5f15n1ZRxVO4BTOwYUraMIdtKANBCbwDK/wZuXWi/VufSxGK1a5cwx/YH3+ALG4k7Q=</latexit><latexit sha1_base64="ymPu5urvCtUJCK9gJWJLmRF9h6c=">AAAB+XicbVDLSsNAFL2pr1pfUZduBovgqiQi6LKgC91VsA9oYphMJ+3QySTMTAol9E/cuFDErX/izr9x0mahrQcGDufcyz1zwpQzpR3n26qsrW9sblW3azu7e/sH9uFRRyWZJLRNEp7IXogV5UzQtmaa014qKY5DTrvh+KbwuxMqFUvEo56m1I/xULCIEayNFNi2F2M9CqP8dhZ4KXuKArvuNJw50CpxS1KHEq3A/vIGCcliKjThWKm+66Taz7HUjHA6q3mZoikmYzykfUMFjqny83nyGTozygBFiTRPaDRXf2/kOFZqGodmssiplr1C/M/rZzq69nMm0kxTQRaHoowjnaCiBjRgkhLNp4ZgIpnJisgIS0y0KatmSnCXv7xKOhcN12m4D5f15n1ZRxVO4BTOwYUraMIdtKANBCbwDK/wZuXWi/VufSxGK1a5cwx/YH3+ALG4k7Q=</latexit><latexit sha1_base64="ymPu5urvCtUJCK9gJWJLmRF9h6c=">AAAB+XicbVDLSsNAFL2pr1pfUZduBovgqiQi6LKgC91VsA9oYphMJ+3QySTMTAol9E/cuFDErX/izr9x0mahrQcGDufcyz1zwpQzpR3n26qsrW9sblW3azu7e/sH9uFRRyWZJLRNEp7IXogV5UzQtmaa014qKY5DTrvh+KbwuxMqFUvEo56m1I/xULCIEayNFNi2F2M9CqP8dhZ4KXuKArvuNJw50CpxS1KHEq3A/vIGCcliKjThWKm+66Taz7HUjHA6q3mZoikmYzykfUMFjqny83nyGTozygBFiTRPaDRXf2/kOFZqGodmssiplr1C/M/rZzq69nMm0kxTQRaHoowjnaCiBjRgkhLNp4ZgIpnJisgIS0y0KatmSnCXv7xKOhcN12m4D5f15n1ZRxVO4BTOwYUraMIdtKANBCbwDK/wZuXWi/VufSxGK1a5cwx/YH3+ALG4k7Q=</latexit>
Hf<latexit sha1_base64="HmXh09BBdv0Pxoy3doDfoh9bWDg=">AAAB83icbVDLSsNAFL3xWeur6tLNYBFclUQEXRbc1F0F+4Cmlsl00g6dTMLMjVBCf8ONC0Xc+jPu/BsnbRbaemDgcM693DMnSKQw6Lrfztr6xubWdmmnvLu3f3BYOTpumzjVjLdYLGPdDajhUijeQoGSdxPNaRRI3gkmt7nfeeLaiFg94DTh/YiOlAgFo2gl348ojoMwa8wew0Gl6tbcOcgq8QpShQLNQeXLH8YsjbhCJqkxPc9NsJ9RjYJJPiv7qeEJZRM64j1LFY246WfzzDNybpUhCWNtn0IyV39vZDQyZhoFdjLPaJa9XPzP66UY3vQzoZIUuWKLQ2EqCcYkL4AMheYM5dQSyrSwWQkbU00Z2prKtgRv+curpH1Z89yad39Vrd8VdZTgFM7gAjy4hjo0oAktYJDAM7zCm5M6L86787EYXXOKnRP4A+fzBzH6kcs=</latexit><latexit sha1_base64="HmXh09BBdv0Pxoy3doDfoh9bWDg=">AAAB83icbVDLSsNAFL3xWeur6tLNYBFclUQEXRbc1F0F+4Cmlsl00g6dTMLMjVBCf8ONC0Xc+jPu/BsnbRbaemDgcM693DMnSKQw6Lrfztr6xubWdmmnvLu3f3BYOTpumzjVjLdYLGPdDajhUijeQoGSdxPNaRRI3gkmt7nfeeLaiFg94DTh/YiOlAgFo2gl348ojoMwa8wew0Gl6tbcOcgq8QpShQLNQeXLH8YsjbhCJqkxPc9NsJ9RjYJJPiv7qeEJZRM64j1LFY246WfzzDNybpUhCWNtn0IyV39vZDQyZhoFdjLPaJa9XPzP66UY3vQzoZIUuWKLQ2EqCcYkL4AMheYM5dQSyrSwWQkbU00Z2prKtgRv+curpH1Z89yad39Vrd8VdZTgFM7gAjy4hjo0oAktYJDAM7zCm5M6L86787EYXXOKnRP4A+fzBzH6kcs=</latexit><latexit sha1_base64="HmXh09BBdv0Pxoy3doDfoh9bWDg=">AAAB83icbVDLSsNAFL3xWeur6tLNYBFclUQEXRbc1F0F+4Cmlsl00g6dTMLMjVBCf8ONC0Xc+jPu/BsnbRbaemDgcM693DMnSKQw6Lrfztr6xubWdmmnvLu3f3BYOTpumzjVjLdYLGPdDajhUijeQoGSdxPNaRRI3gkmt7nfeeLaiFg94DTh/YiOlAgFo2gl348ojoMwa8wew0Gl6tbcOcgq8QpShQLNQeXLH8YsjbhCJqkxPc9NsJ9RjYJJPiv7qeEJZRM64j1LFY246WfzzDNybpUhCWNtn0IyV39vZDQyZhoFdjLPaJa9XPzP66UY3vQzoZIUuWKLQ2EqCcYkL4AMheYM5dQSyrSwWQkbU00Z2prKtgRv+curpH1Z89yad39Vrd8VdZTgFM7gAjy4hjo0oAktYJDAM7zCm5M6L86787EYXXOKnRP4A+fzBzH6kcs=</latexit><latexit sha1_base64="HmXh09BBdv0Pxoy3doDfoh9bWDg=">AAAB83icbVDLSsNAFL3xWeur6tLNYBFclUQEXRbc1F0F+4Cmlsl00g6dTMLMjVBCf8ONC0Xc+jPu/BsnbRbaemDgcM693DMnSKQw6Lrfztr6xubWdmmnvLu3f3BYOTpumzjVjLdYLGPdDajhUijeQoGSdxPNaRRI3gkmt7nfeeLaiFg94DTh/YiOlAgFo2gl348ojoMwa8wew0Gl6tbcOcgq8QpShQLNQeXLH8YsjbhCJqkxPc9NsJ9RjYJJPiv7qeEJZRM64j1LFY246WfzzDNybpUhCWNtn0IyV39vZDQyZhoFdjLPaJa9XPzP66UY3vQzoZIUuWKLQ2EqCcYkL4AMheYM5dQSyrSwWQkbU00Z2prKtgRv+curpH1Z89yad39Vrd8VdZTgFM7gAjy4hjo0oAktYJDAM7zCm5M6L86787EYXXOKnRP4A+fzBzH6kcs=</latexit>
Xcup
<latexit sha1_base64="QdmMy78XPqyMcGDCyWH4fKaAIYU=">AAAB/3icbVDLSsNAFJ3UV62vqODGTbAIrkoigi4LbnRXwT6giWEynbRDJ5MwcyOWmIW/4saFIm79DXf+jZM2C209MHA4517umRMknCmw7W+jsrS8srpWXa9tbG5t75i7ex0Vp5LQNol5LHsBVpQzQdvAgNNeIimOAk67wfiy8Lv3VCoWi1uYJNSL8FCwkBEMWvLNAzfCMArCrJf7LtAHyNIkvyO+Wbcb9hTWInFKUkclWr755Q5ikkZUAOFYqb5jJ+BlWAIjnOY1N1U0wWSMh7SvqcARVV42zZ9bx1oZWGEs9RNgTdXfGxmOlJpEgZ4s0qp5rxD/8/ophBdexkSSAhVkdihMuQWxVZRhDZikBPhEE0wk01ktMsISE9CV1XQJzvyXF0nntOHYDefmrN68LuuookN0hE6Qg85RE12hFmojgh7RM3pFb8aT8WK8Gx+z0YpR7uyjPzA+fwASn5bK</latexit><latexit sha1_base64="QdmMy78XPqyMcGDCyWH4fKaAIYU=">AAAB/3icbVDLSsNAFJ3UV62vqODGTbAIrkoigi4LbnRXwT6giWEynbRDJ5MwcyOWmIW/4saFIm79DXf+jZM2C209MHA4517umRMknCmw7W+jsrS8srpWXa9tbG5t75i7ex0Vp5LQNol5LHsBVpQzQdvAgNNeIimOAk67wfiy8Lv3VCoWi1uYJNSL8FCwkBEMWvLNAzfCMArCrJf7LtAHyNIkvyO+Wbcb9hTWInFKUkclWr755Q5ikkZUAOFYqb5jJ+BlWAIjnOY1N1U0wWSMh7SvqcARVV42zZ9bx1oZWGEs9RNgTdXfGxmOlJpEgZ4s0qp5rxD/8/ophBdexkSSAhVkdihMuQWxVZRhDZikBPhEE0wk01ktMsISE9CV1XQJzvyXF0nntOHYDefmrN68LuuookN0hE6Qg85RE12hFmojgh7RM3pFb8aT8WK8Gx+z0YpR7uyjPzA+fwASn5bK</latexit><latexit sha1_base64="QdmMy78XPqyMcGDCyWH4fKaAIYU=">AAAB/3icbVDLSsNAFJ3UV62vqODGTbAIrkoigi4LbnRXwT6giWEynbRDJ5MwcyOWmIW/4saFIm79DXf+jZM2C209MHA4517umRMknCmw7W+jsrS8srpWXa9tbG5t75i7ex0Vp5LQNol5LHsBVpQzQdvAgNNeIimOAk67wfiy8Lv3VCoWi1uYJNSL8FCwkBEMWvLNAzfCMArCrJf7LtAHyNIkvyO+Wbcb9hTWInFKUkclWr755Q5ikkZUAOFYqb5jJ+BlWAIjnOY1N1U0wWSMh7SvqcARVV42zZ9bx1oZWGEs9RNgTdXfGxmOlJpEgZ4s0qp5rxD/8/ophBdexkSSAhVkdihMuQWxVZRhDZikBPhEE0wk01ktMsISE9CV1XQJzvyXF0nntOHYDefmrN68LuuookN0hE6Qg85RE12hFmojgh7RM3pFb8aT8WK8Gx+z0YpR7uyjPzA+fwASn5bK</latexit><latexit sha1_base64="QdmMy78XPqyMcGDCyWH4fKaAIYU=">AAAB/3icbVDLSsNAFJ3UV62vqODGTbAIrkoigi4LbnRXwT6giWEynbRDJ5MwcyOWmIW/4saFIm79DXf+jZM2C209MHA4517umRMknCmw7W+jsrS8srpWXa9tbG5t75i7ex0Vp5LQNol5LHsBVpQzQdvAgNNeIimOAk67wfiy8Lv3VCoWi1uYJNSL8FCwkBEMWvLNAzfCMArCrJf7LtAHyNIkvyO+Wbcb9hTWInFKUkclWr755Q5ikkZUAOFYqb5jJ+BlWAIjnOY1N1U0wWSMh7SvqcARVV42zZ9bx1oZWGEs9RNgTdXfGxmOlJpEgZ4s0qp5rxD/8/ophBdexkSSAhVkdihMuQWxVZRhDZikBPhEE0wk01ktMsISE9CV1XQJzvyXF0nntOHYDefmrN68LuuookN0hE6Qg85RE12hFmojgh7RM3pFb8aT8WK8Gx+z0YpR7uyjPzA+fwASn5bK</latexit>
Nearest-neighbor Upsampling
1
32
128
32
128
32
128
128
1
128
External factors
Upsa
mpl
ing
Ef<latexit sha1_base64="0NoOZfESLPf6U8/hQYY++naUGLI=">AAAB83icbVDLSsNAFL3xWeur6tLNYBFclUQEXRZE0F0F+4Cmlsl00g6dTMLMjVBCf8ONC0Xc+jPu/BsnbRbaemDgcM693DMnSKQw6Lrfzsrq2vrGZmmrvL2zu7dfOThsmTjVjDdZLGPdCajhUijeRIGSdxLNaRRI3g7G17nffuLaiFg94CThvYgOlQgFo2gl348ojoIwu5k+hv1K1a25M5Bl4hWkCgUa/cqXP4hZGnGFTFJjup6bYC+jGgWTfFr2U8MTysZ0yLuWKhpx08tmmafk1CoDEsbaPoVkpv7eyGhkzCQK7GSe0Sx6ufif100xvOplQiUpcsXmh8JUEoxJXgAZCM0ZyokllGlhsxI2opoytDWVbQne4peXSeu85rk17/6iWr8r6ijBMZzAGXhwCXW4hQY0gUECz/AKb07qvDjvzsd8dMUpdo7gD5zPHy1lkcg=</latexit><latexit sha1_base64="0NoOZfESLPf6U8/hQYY++naUGLI=">AAAB83icbVDLSsNAFL3xWeur6tLNYBFclUQEXRZE0F0F+4Cmlsl00g6dTMLMjVBCf8ONC0Xc+jPu/BsnbRbaemDgcM693DMnSKQw6Lrfzsrq2vrGZmmrvL2zu7dfOThsmTjVjDdZLGPdCajhUijeRIGSdxLNaRRI3g7G17nffuLaiFg94CThvYgOlQgFo2gl348ojoIwu5k+hv1K1a25M5Bl4hWkCgUa/cqXP4hZGnGFTFJjup6bYC+jGgWTfFr2U8MTysZ0yLuWKhpx08tmmafk1CoDEsbaPoVkpv7eyGhkzCQK7GSe0Sx6ufif100xvOplQiUpcsXmh8JUEoxJXgAZCM0ZyokllGlhsxI2opoytDWVbQne4peXSeu85rk17/6iWr8r6ijBMZzAGXhwCXW4hQY0gUECz/AKb07qvDjvzsd8dMUpdo7gD5zPHy1lkcg=</latexit><latexit sha1_base64="0NoOZfESLPf6U8/hQYY++naUGLI=">AAAB83icbVDLSsNAFL3xWeur6tLNYBFclUQEXRZE0F0F+4Cmlsl00g6dTMLMjVBCf8ONC0Xc+jPu/BsnbRbaemDgcM693DMnSKQw6Lrfzsrq2vrGZmmrvL2zu7dfOThsmTjVjDdZLGPdCajhUijeRIGSdxLNaRRI3g7G17nffuLaiFg94CThvYgOlQgFo2gl348ojoIwu5k+hv1K1a25M5Bl4hWkCgUa/cqXP4hZGnGFTFJjup6bYC+jGgWTfFr2U8MTysZ0yLuWKhpx08tmmafk1CoDEsbaPoVkpv7eyGhkzCQK7GSe0Sx6ufif100xvOplQiUpcsXmh8JUEoxJXgAZCM0ZyokllGlhsxI2opoytDWVbQne4peXSeu85rk17/6iWr8r6ijBMZzAGXhwCXW4hQY0gUECz/AKb07qvDjvzsd8dMUpdo7gD5zPHy1lkcg=</latexit><latexit sha1_base64="0NoOZfESLPf6U8/hQYY++naUGLI=">AAAB83icbVDLSsNAFL3xWeur6tLNYBFclUQEXRZE0F0F+4Cmlsl00g6dTMLMjVBCf8ONC0Xc+jPu/BsnbRbaemDgcM693DMnSKQw6Lrfzsrq2vrGZmmrvL2zu7dfOThsmTjVjDdZLGPdCajhUijeRIGSdxLNaRRI3g7G17nffuLaiFg94CThvYgOlQgFo2gl348ojoIwu5k+hv1K1a25M5Bl4hWkCgUa/cqXP4hZGnGFTFJjup6bYC+jGgWTfFr2U8MTysZ0yLuWKhpx08tmmafk1CoDEsbaPoVkpv7eyGhkzCQK7GSe0Sx6ufif100xvOplQiUpcsXmh8JUEoxJXgAZCM0ZyokllGlhsxI2opoytDWVbQne4peXSeu85rk17/6iWr8r6ijBMZzAGXhwCXW4hQY0gUECz/AKb07qvDjvzsd8dMUpdo7gD5zPHy1lkcg=</latexit>
Upsa
mpl
ing
Conv
Conv
FODE FFI
FNN
Figure 2: Overview of the proposed FSR architecture.
numerical instability. Taking Eq. (2) for example, we com-pare the difference between the two calculated gradients:(1) Calculate the gradient w.r.t. zn by Eq. (6):
a (tn) = a (tn+1) +
∫ tn
tn+1
−a(t)∂f(z(t), t; θ)
∂z(t)dt. (7)
(2) Calculate the exact gradient w.r.t. zn by chain rule:
an = an+1 + an+1∂f(zn, θ)
∂zn. (8)
As can be seen, the second terms of Eq. (7) and Eq. (8) aredifferent, which, therefore, makes the state a (tn) 6= an. Inother words, the adjoint method leads to inaccurate and un-stable gradient ∂L/∂θ – compared to direct backpropagationthat computes correct and stable gradient, but suffers a pro-hibitive memory cost. ANODE and its variant [Gholami etal., 2019; Zhang et al., 2019] mitigate this problem by split-ting a block into time batches and stores in memory a fewintermediate states from the forward pass, which, in a sense,is a compromise between time and storage.
In this work, we bridge this gap by introducing an affinecoupling layer in the computations of ODEs, to avoid the in-accurate gradient issue while only requiring O(1) memoryin each block. Affine coupling layer referred to a family ofneural network whose forward function is a bijective map-ping and has been widely used in invertible generative mod-els [Dinh et al., 2015; Dinh et al., 2017; Kingma and Dhari-wal, 2018], where the input to a bijective block can be accu-rately reconstructed from its outputs.
As illustrated in Figure 3, we divide the input zt(let zt = z (t)) into two parts of same size zat , zbt ∈RC/2×H×W – where C is the number of channels. In the for-ward pass functions in each FODE block with time step size∆t, we have:
F :
{hat = zathbt = I (zat )× zbt + J (zat )
G :
{zbt+∆t = hbtzat+∆t = S
(hbt)× hat +K
(hbt),
(9)
zat<latexit sha1_base64="dVZjVrzk1gDs0XK78bOifaldnSk=">AAAB9XicbVDJSgNBEK2JW4xb1KOXxiDoJcx4MceAF71FMAtko6fTkzTp6Rm6a9Q45D+8eFDEq7/gN3jzb+wsB018UPB4r4qqen4shUHX/XYyK6tr6xvZzdzW9s7uXn7/oGaiRDNeZZGMdMOnhkuheBUFSt6INaehL3ndH15O/Pod10ZE6hZHMW+HtK9EIBhFK3VayB/QD9LHcYd2sZsvuEV3CrJMvDkplM9Kn2BR6ea/Wr2IJSFXyCQ1pum5MbZTqlEwyce5VmJ4TNmQ9nnTUkVDbtrp9OoxObFKjwSRtqWQTNXfEykNjRmFvu0MKQ7MojcR//OaCQaldipUnCBXbLYoSCTBiEwiID2hOUM5soQyLeythA2opgxtUDkbgrf48jKpnRc9t+jd2DSuYYYsHMExnIIHF1CGK6hAFRhoeIIXeHXunWfnzXmftWac+cwh/IHz8QMQ4pRX</latexit><latexit sha1_base64="orkq9fkXzn+cB6MDMAK+hrWqdy0=">AAAB9XicbVDLSgNBEJxVozG+oh69DAZBD4ZdL+YkAS96i2AekGzC7KQ3GTL7YKZXjUv+w4sHRbz6C36DN//GyeOgiQUNRVU33V1eLIVG2/62lpZXMqtr2fXcxubW9k5+d6+mo0RxqPJIRqrhMQ1ShFBFgRIasQIWeBLq3uBy7NfvQGkRhbc4jMENWC8UvuAMjdRuITyg56ePozbrYCdfsIv2BHSRODNSKJ+UPjMXp16lk/9qdSOeBBAil0zrpmPH6KZMoeASRrlWoiFmfMB60DQ0ZAFoN51cPaJHRulSP1KmQqQT9fdEygKth4FnOgOGfT3vjcX/vGaCfslNRRgnCCGfLvITSTGi4whoVyjgKIeGMK6EuZXyPlOMowkqZ0Jw5l9eJLWzomMXnRuTxjWZIksOyCE5Jg45J2VyRSqkSjhR5Im8kFfr3nq23qz3aeuSNZvZJ39gffwAKWOVKQ==</latexit><latexit sha1_base64="orkq9fkXzn+cB6MDMAK+hrWqdy0=">AAAB9XicbVDLSgNBEJxVozG+oh69DAZBD4ZdL+YkAS96i2AekGzC7KQ3GTL7YKZXjUv+w4sHRbz6C36DN//GyeOgiQUNRVU33V1eLIVG2/62lpZXMqtr2fXcxubW9k5+d6+mo0RxqPJIRqrhMQ1ShFBFgRIasQIWeBLq3uBy7NfvQGkRhbc4jMENWC8UvuAMjdRuITyg56ePozbrYCdfsIv2BHSRODNSKJ+UPjMXp16lk/9qdSOeBBAil0zrpmPH6KZMoeASRrlWoiFmfMB60DQ0ZAFoN51cPaJHRulSP1KmQqQT9fdEygKth4FnOgOGfT3vjcX/vGaCfslNRRgnCCGfLvITSTGi4whoVyjgKIeGMK6EuZXyPlOMowkqZ0Jw5l9eJLWzomMXnRuTxjWZIksOyCE5Jg45J2VyRSqkSjhR5Im8kFfr3nq23qz3aeuSNZvZJ39gffwAKWOVKQ==</latexit><latexit sha1_base64="rTTE2KHfqN1Ze/bw2qNesJ5c6og=">AAAB9XicbVC7TsNAEDzzDOEVoKQ5ESFRRTYNlJFooAsSeUiJE50v6+SU89m6WwPByn/QUIAQLf9Cx99wSVxAwkgrjWZ2tbsTJFIYdN1vZ2V1bX1js7BV3N7Z3dsvHRw2TJxqDnUey1i3AmZACgV1FCihlWhgUSChGYyupn7zHrQRsbrDcQJ+xAZKhIIztFK3g/CIQZg9Tbqsh71S2a24M9Bl4uWkTHLUeqWvTj/maQQKuWTGtD03QT9jGgWXMCl2UgMJ4yM2gLalikVg/Gx29YSeWqVPw1jbUkhn6u+JjEXGjKPAdkYMh2bRm4r/ee0Uw0s/EypJERSfLwpTSTGm0whoX2jgKMeWMK6FvZXyIdOMow2qaEPwFl9eJo3ziudWvFu3XL3J4yiQY3JCzohHLkiVXJMaqRNONHkmr+TNeXBenHfnY9664uQzR+QPnM8fM8GS9g==</latexit>
zbt<latexit sha1_base64="hTEaeK8al6jz0BdYk5g8e9qvllg=">AAAB9XicbVC7SgNBFL3rM8ZX1NJmMAjahF0bUwZstItgHpAXs5PZZMjs7DJzV4lL/sPGQhFbf8FvsPNvnE1SaOKBgcM593LPHD+WwqDrfjsrq2vrG5u5rfz2zu7efuHgsG6iRDNeY5GMdNOnhkuheA0FSt6MNaehL3nDH11lfuOeayMidYfjmHdCOlAiEIyilbrtkOLQD9LHSdfvYa9QdEvuFGSZeHNSrJyXP8Gi2it8tfsRS0KukElqTMtzY+ykVKNgkk/y7cTwmLIRHfCWpYqG3HTSaeoJObVKnwSRtk8hmaq/N1IaGjMOfTuZpTSLXib+57USDMqdVKg4Qa7Y7FCQSIIRySogfaE5Qzm2hDItbFbChlRThraovC3BW/zyMqlflDy35N3aNm5ghhwcwwmcgQeXUIFrqEINGGh4ghd4dR6cZ+fNeZ+NrjjznSP4A+fjB+hZlD0=</latexit><latexit sha1_base64="bQjDTPkMvoNX762SXYXSMddgAbA=">AAAB9XicbVDLSgMxFL1Trdb6qrp0EyyCLiwzbuxKCm50V8E+oC+SNNOGZjJDklHq0P9w40IRt/6C3+DOvzHTdqGtBwKHc+7lnhwSCa6N6347mZXV7Np6biO/ubW9s1vY26/rMFaU1WgoQtUkWDPBJasZbgRrRorhgAjWIKOr1G/cM6V5KO/MOGKdAA8k9znFxkrddoDNkPjJ46RLeqZXKLoldwq0TLw5KVZOy5/ZyzNS7RW+2v2QxgGThgqsdctzI9NJsDKcCjbJt2PNIkxHeMBalkocMN1Jpqkn6NgqfeSHyj5p0FT9vZHgQOtxQOxkmlIveqn4n9eKjV/uJFxGsWGSzg75sUAmRGkFqM8Vo0aMLcFUcZsV0SFWmBpbVN6W4C1+eZnUz0ueW/JubRs3MEMODuEITsCDC6jANVShBhQUPMELvDoPzrPz5rzPRjPOfOcA/sD5+AEA6ZUP</latexit><latexit sha1_base64="bQjDTPkMvoNX762SXYXSMddgAbA=">AAAB9XicbVDLSgMxFL1Trdb6qrp0EyyCLiwzbuxKCm50V8E+oC+SNNOGZjJDklHq0P9w40IRt/6C3+DOvzHTdqGtBwKHc+7lnhwSCa6N6347mZXV7Np6biO/ubW9s1vY26/rMFaU1WgoQtUkWDPBJasZbgRrRorhgAjWIKOr1G/cM6V5KO/MOGKdAA8k9znFxkrddoDNkPjJ46RLeqZXKLoldwq0TLw5KVZOy5/ZyzNS7RW+2v2QxgGThgqsdctzI9NJsDKcCjbJt2PNIkxHeMBalkocMN1Jpqkn6NgqfeSHyj5p0FT9vZHgQOtxQOxkmlIveqn4n9eKjV/uJFxGsWGSzg75sUAmRGkFqM8Vo0aMLcFUcZsV0SFWmBpbVN6W4C1+eZnUz0ueW/JubRs3MEMODuEITsCDC6jANVShBhQUPMELvDoPzrPz5rzPRjPOfOcA/sD5+AEA6ZUP</latexit><latexit sha1_base64="PIqGlvElopvX6Fq4aL5GAynMQgE=">AAAB9XicbVC7TsMwFL0pr1JeBUYWiwqJqUpYYKzEAluR6EPqS47rtFYdJ7JvQCXqf7AwgBAr/8LG3+C0GaDlSJaOzrlX9/j4sRQGXffbKaytb2xuFbdLO7t7+wflw6OmiRLNeINFMtJtnxouheINFCh5O9achr7kLX9ynfmtB66NiNQ9TmPeC+lIiUAwilbqd0OKYz9In2Z9f4CDcsWtunOQVeLlpAI56oPyV3cYsSTkCpmkxnQ8N8ZeSjUKJvms1E0Mjymb0BHvWKpoyE0vnaeekTOrDEkQafsUkrn6eyOloTHT0LeTWUqz7GXif14nweCqlwoVJ8gVWxwKEkkwIlkFZCg0ZyinllCmhc1K2JhqytAWVbIleMtfXiXNi6rnVr07t1K7zesowgmcwjl4cAk1uIE6NICBhmd4hTfn0Xlx3p2PxWjByXeO4Q+czx8LR5Lc</latexit>
I<latexit sha1_base64="0WuYMklVLO5nsJ+oGNzkG7iDz44=">AAAB8nicbVDLSsNAFL3xWeur6tLNYBFclcSN7iy4sbsK9gFtKJPppB06mYSZG6GEfoYbF4q49WPEnX/jJO1CWw8MHM65lzn3BIkUBl3321lb39jc2i7tlHf39g8OK0fHbROnmvEWi2WsuwE1XArFWyhQ8m6iOY0CyTvB5Db3O49cGxGrB5wm3I/oSIlQMIpW6vUjimNGZdaYDSpVt+YWIKvEW5DqzScUaA4qX/1hzNKIK2SSGtPz3AT9jGoUTPJZuZ8anlA2oSPes1TRiBs/KyLPyLlVhiSMtX0KSaH+3shoZMw0CuxkHtEse7n4n9dLMbz2M6GSFLli84/CVBKMSX4/GQrNGcqpJZRpYbMSNqaaMrQtlW0J3vLJq6R9WfPcmnfvVuuNeRtQglM4gwvw4ArqcAdNaAGDGJ7gBV4ddJ6dN+d9PrrmLHZO4A+cjx/s9JJx</latexit><latexit sha1_base64="0WuYMklVLO5nsJ+oGNzkG7iDz44=">AAAB8nicbVDLSsNAFL3xWeur6tLNYBFclcSN7iy4sbsK9gFtKJPppB06mYSZG6GEfoYbF4q49WPEnX/jJO1CWw8MHM65lzn3BIkUBl3321lb39jc2i7tlHf39g8OK0fHbROnmvEWi2WsuwE1XArFWyhQ8m6iOY0CyTvB5Db3O49cGxGrB5wm3I/oSIlQMIpW6vUjimNGZdaYDSpVt+YWIKvEW5DqzScUaA4qX/1hzNKIK2SSGtPz3AT9jGoUTPJZuZ8anlA2oSPes1TRiBs/KyLPyLlVhiSMtX0KSaH+3shoZMw0CuxkHtEse7n4n9dLMbz2M6GSFLli84/CVBKMSX4/GQrNGcqpJZRpYbMSNqaaMrQtlW0J3vLJq6R9WfPcmnfvVuuNeRtQglM4gwvw4ArqcAdNaAGDGJ7gBV4ddJ6dN+d9PrrmLHZO4A+cjx/s9JJx</latexit><latexit sha1_base64="0WuYMklVLO5nsJ+oGNzkG7iDz44=">AAAB8nicbVDLSsNAFL3xWeur6tLNYBFclcSN7iy4sbsK9gFtKJPppB06mYSZG6GEfoYbF4q49WPEnX/jJO1CWw8MHM65lzn3BIkUBl3321lb39jc2i7tlHf39g8OK0fHbROnmvEWi2WsuwE1XArFWyhQ8m6iOY0CyTvB5Db3O49cGxGrB5wm3I/oSIlQMIpW6vUjimNGZdaYDSpVt+YWIKvEW5DqzScUaA4qX/1hzNKIK2SSGtPz3AT9jGoUTPJZuZ8anlA2oSPes1TRiBs/KyLPyLlVhiSMtX0KSaH+3shoZMw0CuxkHtEse7n4n9dLMbz2M6GSFLli84/CVBKMSX4/GQrNGcqpJZRpYbMSNqaaMrQtlW0J3vLJq6R9WfPcmnfvVuuNeRtQglM4gwvw4ArqcAdNaAGDGJ7gBV4ddJ6dN+d9PrrmLHZO4A+cjx/s9JJx</latexit><latexit sha1_base64="cHrVSakdgDWvBMg2pDYOtrWdUUM=">AAAB8nicbVDLSsNAFL3xWeur6tLNYBFclcSNLgtu7K6CfUAaymQ6aYdOJmHmRiihn+HGhSJu/Rp3/o2TNgttPTBwOOde5twTplIYdN1vZ2Nza3tnt7JX3T84PDqunZx2TZJpxjsskYnuh9RwKRTvoEDJ+6nmNA4l74XTu8LvPXFtRKIecZbyIKZjJSLBKFrJH8QUJ4zKvDUf1upuw12ArBOvJHUo0R7WvgajhGUxV8gkNcb33BSDnGoUTPJ5dZAZnlI2pWPuW6pozE2QLyLPyaVVRiRKtH0KyUL9vZHT2JhZHNrJIqJZ9QrxP8/PMLoNcqHSDLliy4+iTBJMSHE/GQnNGcqZJZRpYbMSNqGaMrQtVW0J3urJ66R73fDchvfg1putso4KnMMFXIEHN9CEe2hDBxgk8Ayv8Oag8+K8Ox/L0Q2n3DmDP3A+fwB9UZFi</latexit> J<latexit sha1_base64="ooz7r4CrhEzjgJGN9dDmj2CgQ30=">AAAB8nicbVC7SgNBFL0bXzG+4qOzGQyCVdi10TJgo1YRzAOSJcxOZpMhs7PrzF0hLPkMGwtFbP0Z7fwbZ5MUmnhg4HDOvcy5J0ikMOi6305hZXVtfaO4Wdra3tndK+8fNE2casYbLJaxbgfUcCkUb6BAyduJ5jQKJG8Fo6vcbz1ybUSs7nGccD+iAyVCwShaqdONKA4ZldntpFeuuFV3CrJMvDmp1I4ePmsAUO+Vv7r9mKURV8gkNabjuQn6GdUomOSTUjc1PKFsRAe8Y6miETd+No08IadW6ZMw1vYpJFP190ZGI2PGUWAn84hm0cvF/7xOiuGlnwmVpMgVm30UppJgTPL7SV9ozlCOLaFMC5uVsCHVlKFtqWRL8BZPXibN86rnVr0728YNzFCEYziBM/DgAmpwDXVoAIMYnuAFXh10np035302WnDmO4fwB87HD/lzkzk=</latexit><latexit sha1_base64="DALIhsvk0IfQ47wl4+6GDypl3vE=">AAAB8nicbVDLSgMxFM3UV62v+ti5CRbBVZlxoYKbght1IRXsA9qhZNJMG5pJxuSOUIZ+hhsXirj1V1zozr8x03ahrQcCh3PuJeeeIBbcgOt+O7mFxaXllfxqYW19Y3OruL1TNyrRlNWoEko3A2KY4JLVgINgzVgzEgWCNYLBReY3Hpg2XMk7GMbMj0hP8pBTAlZqtSMCfUpEej3qFEtu2R0DzxNvSkqVvfvPm/OPk2qn+NXuKppETAIVxJiW58bgp0QDp4KNCu3EsJjQAemxlqWSRMz46TjyCB9apYtDpe2TgMfq742URMYMo8BOZhHNrJeJ/3mtBMIzP+UyToBJOvkoTAQGhbP7cZdrRkEMLSFUc5sV0z7RhIJtqWBL8GZPnif147Lnlr1b28YVmiCP9tEBOkIeOkUVdImqqIYoUugRPaMXB5wn59V5m4zmnOnOLvoD5/0HtTqUhQ==</latexit><latexit sha1_base64="DALIhsvk0IfQ47wl4+6GDypl3vE=">AAAB8nicbVDLSgMxFM3UV62v+ti5CRbBVZlxoYKbght1IRXsA9qhZNJMG5pJxuSOUIZ+hhsXirj1V1zozr8x03ahrQcCh3PuJeeeIBbcgOt+O7mFxaXllfxqYW19Y3OruL1TNyrRlNWoEko3A2KY4JLVgINgzVgzEgWCNYLBReY3Hpg2XMk7GMbMj0hP8pBTAlZqtSMCfUpEej3qFEtu2R0DzxNvSkqVvfvPm/OPk2qn+NXuKppETAIVxJiW58bgp0QDp4KNCu3EsJjQAemxlqWSRMz46TjyCB9apYtDpe2TgMfq742URMYMo8BOZhHNrJeJ/3mtBMIzP+UyToBJOvkoTAQGhbP7cZdrRkEMLSFUc5sV0z7RhIJtqWBL8GZPnif147Lnlr1b28YVmiCP9tEBOkIeOkUVdImqqIYoUugRPaMXB5wn59V5m4zmnOnOLvoD5/0HtTqUhQ==</latexit><latexit sha1_base64="lPPOYpMnAIzaplxv9ynUX2mXB8U=">AAAB8nicbVBNS8NAFHypX7V+VT16CRbBU0m86LHgRT1VsLWQhrLZbtqlm92w+yKU0J/hxYMiXv013vw3btoctHVgYZh5j503USq4Qc/7dipr6xubW9Xt2s7u3v5B/fCoa1SmKetQJZTuRcQwwSXrIEfBeqlmJIkEe4wm14X/+MS04Uo+4DRlYUJGksecErRS0E8IjikR+d1sUG94TW8Od5X4JWlAifag/tUfKpolTCIVxJjA91IMc6KRU8FmtX5mWErohIxYYKkkCTNhPo88c8+sMnRjpe2T6M7V3xs5SYyZJpGdLCKaZa8Q//OCDOOrMOcyzZBJuvgozoSLyi3ud4dcM4piagmhmtusLh0TTSjalmq2BH/55FXSvWj6XtO/9xqt27KOKpzAKZyDD5fQghtoQwcoKHiGV3hz0Hlx3p2PxWjFKXeO4Q+czx9+1pFj</latexit>
hat
<latexit sha1_base64="oeORQizBCtoSBNB8wP/kzgQFALQ=">AAAB9XicbVC7SgNBFL0bXzG+opY2g0HQJuzamDJgo10E84BkE2Yns8mQ2dll5q4SlvyHjYUitv6C32Dn3zh5FJp4YOBwzr3cMydIpDDout9Obm19Y3Mrv13Y2d3bPygeHjVMnGrG6yyWsW4F1HApFK+jQMlbieY0CiRvBqPrqd984NqIWN3jOOF+RAdKhIJRtFK3E1EcBmE2nHRpD3vFklt2ZyCrxFuQUvWi8gkWtV7xq9OPWRpxhUxSY9qem6CfUY2CST4pdFLDE8pGdMDblioaceNns9QTcmaVPgljbZ9CMlN/b2Q0MmYcBXZymtIse1PxP6+dYljxM6GSFLli80NhKgnGZFoB6QvNGcqxJZRpYbMSNqSaMrRFFWwJ3vKXV0njsuy5Ze/OtnELc+ThBE7hHDy4gircQA3qwEDDE7zAq/PoPDtvzvt8NOcsdo7hD5yPH8sxlCo=</latexit><latexit sha1_base64="TJi3UaFmDCn5UeIsRRO8/CFoGOs=">AAAB9XicbVDLSsNAFL1Rq7W+qi7dBIugC0vixq6k4EZ3FewD2rRMppN26GQSZm6UEvofblwo4tZf8Bvc+TdO2i609cDA4Zx7uWeOHwuu0XG+rZXVtdz6Rn6zsLW9s7tX3D9o6ChRlNVpJCLV8olmgktWR46CtWLFSOgL1vRH15nffGBK80je4zhmXkgGkgecEjRStxMSHPpBOpx0SQ97xZJTdqawl4k7J6XqWeUzd3Xu13rFr04/oknIJFJBtG67ToxeShRyKtik0Ek0iwkdkQFrGypJyLSXTlNP7BOj9O0gUuZJtKfq742UhFqPQ99MZin1opeJ/3ntBIOKl3IZJ8gknR0KEmFjZGcV2H2uGEUxNoRQxU1Wmw6JIhRNUQVTgrv45WXSuCi7Ttm9M23cwgx5OIJjOAUXLqEKN1CDOlBQ8AQv8Go9Ws/Wm/U+G12x5juH8AfWxw/jspT8</latexit><latexit sha1_base64="TJi3UaFmDCn5UeIsRRO8/CFoGOs=">AAAB9XicbVDLSsNAFL1Rq7W+qi7dBIugC0vixq6k4EZ3FewD2rRMppN26GQSZm6UEvofblwo4tZf8Bvc+TdO2i609cDA4Zx7uWeOHwuu0XG+rZXVtdz6Rn6zsLW9s7tX3D9o6ChRlNVpJCLV8olmgktWR46CtWLFSOgL1vRH15nffGBK80je4zhmXkgGkgecEjRStxMSHPpBOpx0SQ97xZJTdqawl4k7J6XqWeUzd3Xu13rFr04/oknIJFJBtG67ToxeShRyKtik0Ek0iwkdkQFrGypJyLSXTlNP7BOj9O0gUuZJtKfq742UhFqPQ99MZin1opeJ/3ntBIOKl3IZJ8gknR0KEmFjZGcV2H2uGEUxNoRQxU1Wmw6JIhRNUQVTgrv45WXSuCi7Ttm9M23cwgx5OIJjOAUXLqEKN1CDOlBQ8AQv8Go9Ws/Wm/U+G12x5juH8AfWxw/jspT8</latexit><latexit sha1_base64="NBjUILDa21Zntyt6OTPvyjgPhLM=">AAAB9XicbVDLSsNAFL3xWeur6tLNYBFclcSNLgtudFfBPqBNy2Q6aYdOJmHmRimh/+HGhSJu/Rd3/o2TNgttPTBwOOde7pkTJFIYdN1vZ219Y3Nru7RT3t3bPzisHB23TJxqxpsslrHuBNRwKRRvokDJO4nmNAokbweTm9xvP3JtRKwecJpwP6IjJULBKFqp34sojoMwG8/6dICDStWtuXOQVeIVpAoFGoPKV28YszTiCpmkxnQ9N0E/oxoFk3xW7qWGJ5RN6Ih3LVU04sbP5qln5NwqQxLG2j6FZK7+3shoZMw0CuxkntIse7n4n9dNMbz2M6GSFLlii0NhKgnGJK+ADIXmDOXUEsq0sFkJG1NNGdqiyrYEb/nLq6R1WfPcmnfvVut3RR0lOIUzuAAPrqAOt9CAJjDQ8Ayv8OY8OS/Ou/OxGF1zip0T+APn8wfuEJLJ</latexit>
hbt
<latexit sha1_base64="y2PF2eRKm7D1x5vIZDVLGakCLnw=">AAAB9XicbVC7SgNBFL0bXzG+opY2g0HQJuzamDJgo10E84BkE2Yns8mQ2dll5q4SlvyHjYUitv6C32Dn3zh5FJp4YOBwzr3cMydIpDDout9Obm19Y3Mrv13Y2d3bPygeHjVMnGrG6yyWsW4F1HApFK+jQMlbieY0CiRvBqPrqd984NqIWN3jOOF+RAdKhIJRtFK3E1EcBmE2nHSDHvaKJbfszkBWibcgpepF5RMsar3iV6cfszTiCpmkxrQ9N0E/oxoFk3xS6KSGJ5SN6IC3LVU04sbPZqkn5MwqfRLG2j6FZKb+3shoZMw4CuzkNKVZ9qbif147xbDiZ0IlKXLF5ofCVBKMybQC0heaM5RjSyjTwmYlbEg1ZWiLKtgSvOUvr5LGZdlzy96dbeMW5sjDCZzCOXhwBVW4gRrUgYGGJ3iBV+fReXbenPf5aM5Z7BzDHzgfP8y3lCs=</latexit><latexit sha1_base64="wGYwxAxI+0HwnW2pRuTkZJYs0s8=">AAAB9XicbVDLSgMxFL2jVmt9VV26CRZBF5YZN3YlBTe6q2Af0E5Lkmba0ExmSDJKGfofblwo4tZf8Bvc+Tdm2i609UDgcM693JNDYsG1cd1vZ2V1Lbe+kd8sbG3v7O4V9w8aOkoUZXUaiUi1CNZMcMnqhhvBWrFiOCSCNcnoOvObD0xpHsl7M46ZH+KB5AGn2Fip2wmxGZIgHU66pGd6xZJbdqdAy8Sbk1L1rPKZuzontV7xq9OPaBIyaajAWrc9NzZ+ipXhVLBJoZNoFmM6wgPWtlTikGk/naaeoBOr9FEQKfukQVP190aKQ63HIbGTWUq96GXif147MUHFT7mME8MknR0KEoFMhLIKUJ8rRo0YW4Kp4jYrokOsMDW2qIItwVv88jJpXJQ9t+zd2TZuYYY8HMExnIIHl1CFG6hBHSgoeIIXeHUenWfnzXmfja44851D+APn4wflOJT9</latexit><latexit sha1_base64="wGYwxAxI+0HwnW2pRuTkZJYs0s8=">AAAB9XicbVDLSgMxFL2jVmt9VV26CRZBF5YZN3YlBTe6q2Af0E5Lkmba0ExmSDJKGfofblwo4tZf8Bvc+Tdm2i609UDgcM693JNDYsG1cd1vZ2V1Lbe+kd8sbG3v7O4V9w8aOkoUZXUaiUi1CNZMcMnqhhvBWrFiOCSCNcnoOvObD0xpHsl7M46ZH+KB5AGn2Fip2wmxGZIgHU66pGd6xZJbdqdAy8Sbk1L1rPKZuzontV7xq9OPaBIyaajAWrc9NzZ+ipXhVLBJoZNoFmM6wgPWtlTikGk/naaeoBOr9FEQKfukQVP190aKQ63HIbGTWUq96GXif147MUHFT7mME8MknR0KEoFMhLIKUJ8rRo0YW4Kp4jYrokOsMDW2qIItwVv88jJpXJQ9t+zd2TZuYYY8HMExnIIHl1CFG6hBHSgoeIIXeHUenWfnzXmfja44851D+APn4wflOJT9</latexit><latexit sha1_base64="6hIDhuB78RtGkzplZ2L2egJKyuc=">AAAB9XicbVDLSsNAFL3xWeur6tLNYBFclcSNLgtudFfBPqBNy2Q6aYdOJmHmRimh/+HGhSJu/Rd3/o2TNgttPTBwOOde7pkTJFIYdN1vZ219Y3Nru7RT3t3bPzisHB23TJxqxpsslrHuBNRwKRRvokDJO4nmNAokbweTm9xvP3JtRKwecJpwP6IjJULBKFqp34sojoMwG8/6wQAHlapbc+cgq8QrSBUKNAaVr94wZmnEFTJJjel6boJ+RjUKJvms3EsNTyib0BHvWqpoxI2fzVPPyLlVhiSMtX0KyVz9vZHRyJhpFNjJPKVZ9nLxP6+bYnjtZ0IlKXLFFofCVBKMSV4BGQrNGcqpJZRpYbMSNqaaMrRFlW0J3vKXV0nrsua5Ne/erdbvijpKcApncAEeXEEdbqEBTWCg4Rle4c15cl6cd+djMbrmFDsn8AfO5w/vlpLK</latexit>
zat+�t<latexit sha1_base64="j1mipoxl3AKzh7fT0M1PayI9hk0=">AAACAnicbVA9SwNBEJ2LXzF+Ra3EZjEIghDubLQMaKFdBKNCEsPeZi5Z3Ptgd06MR7Dxr9hYKGJrb2/nv3GTWGjig4HHezPMzPMTJQ257peTm5qemZ3LzxcWFpeWV4qra+cmTrXAmohVrC99blDJCGskSeFlopGHvsIL//pw4F/coDYyjs6ol2Az5J1IBlJwslKruNEgvCU/yO76V7yV0W7jCBVxRv1WseSW3SHYJPF+SKnifiwzAKi2ip+NdizSECMSihtT99yEmhnXJIXCfqGRGky4uOYdrFsa8RBNMxu+0GfbVmmzINa2ImJD9fdExkNjeqFvO0NOXTPuDcT/vHpKwUEzk1GSEkZitChIFaOYDfJgbalRkOpZwoWW9lYmulxzQTa1gg3BG395kpzvlT237J3aNE5ghDxswhbsgAf7UIFjqEINBNzDIzzDi/PgPDmvztuoNef8zKzDHzjv34dWmPc=</latexit><latexit sha1_base64="2uz+Dxq0YNIzfAz8Ji/d5l8QlEc=">AAACAnicbVA9SwNBEN2LXzFqPLWSNItBEIRwZ6NlQAvFJoJRIYlhbzOXLNn7YHdOjEew8a/YWCgidvb2dpb+EzeJhSY+GHi8N8PMPC+WQqPjfFqZqemZ2bnsfG5hcSm/bK+snukoURyqPJKRuvCYBilCqKJACRexAhZ4Es697v7AP78CpUUUnmIvhkbA2qHwBWdopKa9Xke4Rs9Pb/qXrJnidv0AJDKK/aZddErOEHSSuD+kWHbe84Xj169K0/6otyKeBBAil0zrmuvE2EiZQsEl9HP1REPMeJe1oWZoyALQjXT4Qp9uGqVF/UiZCpEO1d8TKQu07gWe6QwYdvS4NxD/82oJ+nuNVIRxghDy0SI/kRQjOsiDtoQCjrJnCONKmFsp7zDFOJrUciYEd/zlSXK2U3Kdknti0jgiI2RJgWyQLeKSXVImh6RCqoSTW3JPHsmTdWc9WM/Wy6g1Y/3MrJE/sN6+AQLLmtM=</latexit><latexit sha1_base64="2uz+Dxq0YNIzfAz8Ji/d5l8QlEc=">AAACAnicbVA9SwNBEN2LXzFqPLWSNItBEIRwZ6NlQAvFJoJRIYlhbzOXLNn7YHdOjEew8a/YWCgidvb2dpb+EzeJhSY+GHi8N8PMPC+WQqPjfFqZqemZ2bnsfG5hcSm/bK+snukoURyqPJKRuvCYBilCqKJACRexAhZ4Es697v7AP78CpUUUnmIvhkbA2qHwBWdopKa9Xke4Rs9Pb/qXrJnidv0AJDKK/aZddErOEHSSuD+kWHbe84Xj169K0/6otyKeBBAil0zrmuvE2EiZQsEl9HP1REPMeJe1oWZoyALQjXT4Qp9uGqVF/UiZCpEO1d8TKQu07gWe6QwYdvS4NxD/82oJ+nuNVIRxghDy0SI/kRQjOsiDtoQCjrJnCONKmFsp7zDFOJrUciYEd/zlSXK2U3Kdknti0jgiI2RJgWyQLeKSXVImh6RCqoSTW3JPHsmTdWc9WM/Wy6g1Y/3MrJE/sN6+AQLLmtM=</latexit><latexit sha1_base64="imLn78DUhOY0t/xgvbTm4EdD+eM=">AAACAnicbVDJSgNBEO2JW4xb1JN4aQyCIIQZL3oM6EFvEcwCmTj0dGqSJj0L3TViHIIXf8WLB0W8+hXe/Bs7y0ETHxQ83quiqp6fSKHRtr+t3MLi0vJKfrWwtr6xuVXc3qnrOFUcajyWsWr6TIMUEdRQoIRmooCFvoSG3z8f+Y07UFrE0Q0OEmiHrBuJQHCGRvKKey7CPfpB9jC8ZV6Gx+4FSGQUh16xZJftMeg8caakRKaoesUvtxPzNIQIuWRatxw7wXbGFAouYVhwUw0J433WhZahEQtBt7PxC0N6aJQODWJlKkI6Vn9PZCzUehD6pjNk2NOz3kj8z2ulGJy1MxElKULEJ4uCVFKM6SgP2hEKOMqBIYwrYW6lvMcU42hSK5gQnNmX50n9pOzYZefaLlWupnHkyT45IEfEIaekQi5JldQIJ4/kmbySN+vJerHerY9Ja86azuySP7A+fwCkVJeS</latexit>
zbt+�t<latexit sha1_base64="tTG66e3nuOFwFjNXQr++OMJA5pE=">AAACAnicbVA7SwNBEJ6Lrxhfp1ZisxgEQQh3NloGtNAugnlAEo+9zV6yZO/B7pwQj2DjX7GxUMTW3t7Of+PmUWjiBwMf3zfDzHx+IoVGx/m2cguLS8sr+dXC2vrG5pa9vVPTcaoYr7JYxqrhU82liHgVBUreSBSnoS953e+fj/z6HVdaxNENDhLeDmk3EoFgFI3k2XutkGLPD7L74a3vZXjcuuASKcGhZxedkjMGmSfulBTLzucmAYCKZ3+1OjFLQx4hk1Trpusk2M6oQsEkHxZaqeYJZX3a5U1DIxpy3c7GLwzJoVE6JIiVqQjJWP09kdFQ60Hom87RwXrWG4n/ec0Ug7N2JqIkRR6xyaIglQRjMsqDdITiDOXAEMqUMLcS1qOKMjSpFUwI7uzL86R2UnKdkntt0riCCfKwDwdwBC6cQhkuoQJVYPAAT/ACr9aj9Wy9We+T1pw1ndmFP7A+fgBdvpjd</latexit><latexit sha1_base64="RKPDKfZ0N60mHWZRT17jvmKyrPM=">AAACAnicbVA9SwNBEN2LXzF+nVpJmsUgCEK4s9EyoIViE8F8QC4ee5u9ZMneB7tzQjyCjX/FxkIRsbO3t7P0n7iXpNDEBwOP92aYmefFgiuwrC8jNze/sLiUXy6srK6tb5ibW3UVJZKyGo1EJJseUUzwkNWAg2DNWDISeII1vP5J5jdumFQ8Cq9gELN2QLoh9zkloCXX3HECAj3PT2+H156bwoFzygQQDEPXLFllawQ8S+wJKVWsj/Xixdt31TU/nU5Ek4CFQAVRqmVbMbRTIoFTwYYFJ1EsJrRPuqylaUgCptrp6IUh3tNKB/uR1BUCHqm/J1ISKDUIPN2ZHaymvUz8z2sl4B+3Ux7GCbCQjhf5icAQ4SwP3OGSURADTQiVXN+KaY9IQkGnVtAh2NMvz5L6Ydm2yvalTuMcjZFHRbSL9pGNjlAFnaEqqiGK7tADekLPxr3xaLwYr+PWnDGZ2UZ/YLz/ANkkmrk=</latexit><latexit sha1_base64="RKPDKfZ0N60mHWZRT17jvmKyrPM=">AAACAnicbVA9SwNBEN2LXzF+nVpJmsUgCEK4s9EyoIViE8F8QC4ee5u9ZMneB7tzQjyCjX/FxkIRsbO3t7P0n7iXpNDEBwOP92aYmefFgiuwrC8jNze/sLiUXy6srK6tb5ibW3UVJZKyGo1EJJseUUzwkNWAg2DNWDISeII1vP5J5jdumFQ8Cq9gELN2QLoh9zkloCXX3HECAj3PT2+H156bwoFzygQQDEPXLFllawQ8S+wJKVWsj/Xixdt31TU/nU5Ek4CFQAVRqmVbMbRTIoFTwYYFJ1EsJrRPuqylaUgCptrp6IUh3tNKB/uR1BUCHqm/J1ISKDUIPN2ZHaymvUz8z2sl4B+3Ux7GCbCQjhf5icAQ4SwP3OGSURADTQiVXN+KaY9IQkGnVtAh2NMvz5L6Ydm2yvalTuMcjZFHRbSL9pGNjlAFnaEqqiGK7tADekLPxr3xaLwYr+PWnDGZ2UZ/YLz/ANkkmrk=</latexit><latexit sha1_base64="bLDO2YZHaov3P5Dy2BNLMHN2lKc=">AAACAnicbVDLSsNAFJ34rPUVdSVuBosgCCVxo8uCLnRXwT6giWEynbRDJw9mboQaght/xY0LRdz6Fe78GydtFtp64MLhnHu59x4/EVyBZX0bC4tLyyurlbXq+sbm1ra5s9tWcSopa9FYxLLrE8UEj1gLOAjWTSQjoS9Yxx9dFH7nnknF4+gWxglzQzKIeMApAS155r4TEhj6QfaQ3/leBifOJRNAMOSeWbPq1gR4ntglqaESTc/8cvoxTUMWARVEqZ5tJeBmRAKnguVVJ1UsIXREBqynaURCptxs8kKOj7TSx0EsdUWAJ+rviYyESo1DX3cWB6tZrxD/83opBOduxqMkBRbR6aIgFRhiXOSB+1wyCmKsCaGS61sxHRJJKOjUqjoEe/bledI+rdtW3b6xao3rMo4KOkCH6BjZ6Aw10BVqohai6BE9o1f0ZjwZL8a78TFtXTDKmT30B8bnD3q8l3g=</latexit>
S<latexit sha1_base64="FRCZWif0bwx30LPCONxkeu1Se5g=">AAAB8nicbVDLSsNAFL2pr1pfVZdugkVwVRIRdFlwo7uK9gFtKJPppB06mQkzN0IJ/Qw3LhRx69e482+ctFlo64GBwzn3MueeMBHcoOd9O6W19Y3NrfJ2ZWd3b/+genjUNirVlLWoEkp3Q2KY4JK1kKNg3UQzEoeCdcLJTe53npg2XMlHnCYsiMlI8ohTglbq9WOCY0pE9jAbVGte3ZvDXSV+QWpQoDmofvWHiqYxk0gFMabnewkGGdHIqWCzSj81LCF0QkasZ6kkMTNBNo88c8+sMnQjpe2T6M7V3xsZiY2ZxqGdzCOaZS8X//N6KUbXQcZlkiKTdPFRlAoXlZvf7w65ZhTF1BJCNbdZXTommlC0LVVsCf7yyaukfVH3vbp/f1lr3BV1lOEETuEcfLiCBtxCE1pAQcEzvMKbg86L8+58LEZLTrFzDH/gfP4AjcORcA==</latexit><latexit sha1_base64="FRCZWif0bwx30LPCONxkeu1Se5g=">AAAB8nicbVDLSsNAFL2pr1pfVZdugkVwVRIRdFlwo7uK9gFtKJPppB06mQkzN0IJ/Qw3LhRx69e482+ctFlo64GBwzn3MueeMBHcoOd9O6W19Y3NrfJ2ZWd3b/+genjUNirVlLWoEkp3Q2KY4JK1kKNg3UQzEoeCdcLJTe53npg2XMlHnCYsiMlI8ohTglbq9WOCY0pE9jAbVGte3ZvDXSV+QWpQoDmofvWHiqYxk0gFMabnewkGGdHIqWCzSj81LCF0QkasZ6kkMTNBNo88c8+sMnQjpe2T6M7V3xsZiY2ZxqGdzCOaZS8X//N6KUbXQcZlkiKTdPFRlAoXlZvf7w65ZhTF1BJCNbdZXTommlC0LVVsCf7yyaukfVH3vbp/f1lr3BV1lOEETuEcfLiCBtxCE1pAQcEzvMKbg86L8+58LEZLTrFzDH/gfP4AjcORcA==</latexit><latexit sha1_base64="FRCZWif0bwx30LPCONxkeu1Se5g=">AAAB8nicbVDLSsNAFL2pr1pfVZdugkVwVRIRdFlwo7uK9gFtKJPppB06mQkzN0IJ/Qw3LhRx69e482+ctFlo64GBwzn3MueeMBHcoOd9O6W19Y3NrfJ2ZWd3b/+genjUNirVlLWoEkp3Q2KY4JK1kKNg3UQzEoeCdcLJTe53npg2XMlHnCYsiMlI8ohTglbq9WOCY0pE9jAbVGte3ZvDXSV+QWpQoDmofvWHiqYxk0gFMabnewkGGdHIqWCzSj81LCF0QkasZ6kkMTNBNo88c8+sMnQjpe2T6M7V3xsZiY2ZxqGdzCOaZS8X//N6KUbXQcZlkiKTdPFRlAoXlZvf7w65ZhTF1BJCNbdZXTommlC0LVVsCf7yyaukfVH3vbp/f1lr3BV1lOEETuEcfLiCBtxCE1pAQcEzvMKbg86L8+58LEZLTrFzDH/gfP4AjcORcA==</latexit><latexit sha1_base64="FRCZWif0bwx30LPCONxkeu1Se5g=">AAAB8nicbVDLSsNAFL2pr1pfVZdugkVwVRIRdFlwo7uK9gFtKJPppB06mQkzN0IJ/Qw3LhRx69e482+ctFlo64GBwzn3MueeMBHcoOd9O6W19Y3NrfJ2ZWd3b/+genjUNirVlLWoEkp3Q2KY4JK1kKNg3UQzEoeCdcLJTe53npg2XMlHnCYsiMlI8ohTglbq9WOCY0pE9jAbVGte3ZvDXSV+QWpQoDmofvWHiqYxk0gFMabnewkGGdHIqWCzSj81LCF0QkasZ6kkMTNBNo88c8+sMnQjpe2T6M7V3xsZiY2ZxqGdzCOaZS8X//N6KUbXQcZlkiKTdPFRlAoXlZvf7w65ZhTF1BJCNbdZXTommlC0LVVsCf7yyaukfVH3vbp/f1lr3BV1lOEETuEcfLiCBtxCE1pAQcEzvMKbg86L8+58LEZLTrFzDH/gfP4AjcORcA==</latexit>
K<latexit sha1_base64="iG5QDhzEdzzsoItycVrxeMEn0Es=">AAAB8nicbVDLSsNAFL2pr1pfVZdugkVwVRIRdFlwo7ipYB/QhjKZTtqhk5kwcyOU0M9w40IRt36NO//GSZuFth4YOJxzL3PuCRPBDXret1NaW9/Y3CpvV3Z29/YPqodHbaNSTVmLKqF0NySGCS5ZCzkK1k00I3EoWCec3OR+54lpw5V8xGnCgpiMJI84JWilXj8mOKZEZPezQbXm1b053FXiF6QGBZqD6ld/qGgaM4lUEGN6vpdgkBGNnAo2q/RTwxJCJ2TEepZKEjMTZPPIM/fMKkM3Uto+ie5c/b2RkdiYaRzayTyiWfZy8T+vl2J0HWRcJikySRcfRalwUbn5/e6Qa0ZRTC0hVHOb1aVjoglF21LFluAvn7xK2hd136v7D5e1xl1RRxlO4BTOwYcraMAtNKEFFBQ8wyu8Oei8OO/Ox2K05BQ7x/AHzucPgZuRaA==</latexit><latexit sha1_base64="iG5QDhzEdzzsoItycVrxeMEn0Es=">AAAB8nicbVDLSsNAFL2pr1pfVZdugkVwVRIRdFlwo7ipYB/QhjKZTtqhk5kwcyOU0M9w40IRt36NO//GSZuFth4YOJxzL3PuCRPBDXret1NaW9/Y3CpvV3Z29/YPqodHbaNSTVmLKqF0NySGCS5ZCzkK1k00I3EoWCec3OR+54lpw5V8xGnCgpiMJI84JWilXj8mOKZEZPezQbXm1b053FXiF6QGBZqD6ld/qGgaM4lUEGN6vpdgkBGNnAo2q/RTwxJCJ2TEepZKEjMTZPPIM/fMKkM3Uto+ie5c/b2RkdiYaRzayTyiWfZy8T+vl2J0HWRcJikySRcfRalwUbn5/e6Qa0ZRTC0hVHOb1aVjoglF21LFluAvn7xK2hd136v7D5e1xl1RRxlO4BTOwYcraMAtNKEFFBQ8wyu8Oei8OO/Ox2K05BQ7x/AHzucPgZuRaA==</latexit><latexit sha1_base64="iG5QDhzEdzzsoItycVrxeMEn0Es=">AAAB8nicbVDLSsNAFL2pr1pfVZdugkVwVRIRdFlwo7ipYB/QhjKZTtqhk5kwcyOU0M9w40IRt36NO//GSZuFth4YOJxzL3PuCRPBDXret1NaW9/Y3CpvV3Z29/YPqodHbaNSTVmLKqF0NySGCS5ZCzkK1k00I3EoWCec3OR+54lpw5V8xGnCgpiMJI84JWilXj8mOKZEZPezQbXm1b053FXiF6QGBZqD6ld/qGgaM4lUEGN6vpdgkBGNnAo2q/RTwxJCJ2TEepZKEjMTZPPIM/fMKkM3Uto+ie5c/b2RkdiYaRzayTyiWfZy8T+vl2J0HWRcJikySRcfRalwUbn5/e6Qa0ZRTC0hVHOb1aVjoglF21LFluAvn7xK2hd136v7D5e1xl1RRxlO4BTOwYcraMAtNKEFFBQ8wyu8Oei8OO/Ox2K05BQ7x/AHzucPgZuRaA==</latexit><latexit sha1_base64="iG5QDhzEdzzsoItycVrxeMEn0Es=">AAAB8nicbVDLSsNAFL2pr1pfVZdugkVwVRIRdFlwo7ipYB/QhjKZTtqhk5kwcyOU0M9w40IRt36NO//GSZuFth4YOJxzL3PuCRPBDXret1NaW9/Y3CpvV3Z29/YPqodHbaNSTVmLKqF0NySGCS5ZCzkK1k00I3EoWCec3OR+54lpw5V8xGnCgpiMJI84JWilXj8mOKZEZPezQbXm1b053FXiF6QGBZqD6ld/qGgaM4lUEGN6vpdgkBGNnAo2q/RTwxJCJ2TEepZKEjMTZPPIM/fMKkM3Uto+ie5c/b2RkdiYaRzayTyiWfZy8T+vl2J0HWRcJikySRcfRalwUbn5/e6Qa0ZRTC0hVHOb1aVjoglF21LFluAvn7xK2hd136v7D5e1xl1RRxlO4BTOwYcraMAtNKEFFBQ8wyu8Oei8OO/Ox2K05BQ7x/AHzucPgZuRaA==</latexit>
zt+�t<latexit sha1_base64="UzO4G9mH0d14nrnW2PlrXyujM88=">AAACAHicbVDLSsNAFJ3UV62vqAsXbgaLIAglEUGXBV3oroJ9QBPCZDpph04ezNwINWTjr7hxoYhbP8Odf+OkzUJbD1w4nHMv997jJ4IrsKxvo7K0vLK6Vl2vbWxube+Yu3sdFaeSsjaNRSx7PlFM8Ii1gYNgvUQyEvqCdf3xVeF3H5hUPI7uYZIwNyTDiAecEtCSZx44IYGRH2SPuZfBqXPNBBAMuWfWrYY1BV4kdknqqETLM7+cQUzTkEVABVGqb1sJuBmRwKlgec1JFUsIHZMh62sakZApN5s+kONjrQxwEEtdEeCp+nsiI6FSk9DXncW5at4rxP+8fgrBpZvxKEmBRXS2KEgFhhgXaeABl4yCmGhCqOT6VkxHRBIKOrOaDsGef3mRdM4attWw787rzdsyjio6REfoBNnoAjXRDWqhNqIoR8/oFb0ZT8aL8W58zForRjmzj/7A+PwB++CWqA==</latexit><latexit sha1_base64="UzO4G9mH0d14nrnW2PlrXyujM88=">AAACAHicbVDLSsNAFJ3UV62vqAsXbgaLIAglEUGXBV3oroJ9QBPCZDpph04ezNwINWTjr7hxoYhbP8Odf+OkzUJbD1w4nHMv997jJ4IrsKxvo7K0vLK6Vl2vbWxube+Yu3sdFaeSsjaNRSx7PlFM8Ii1gYNgvUQyEvqCdf3xVeF3H5hUPI7uYZIwNyTDiAecEtCSZx44IYGRH2SPuZfBqXPNBBAMuWfWrYY1BV4kdknqqETLM7+cQUzTkEVABVGqb1sJuBmRwKlgec1JFUsIHZMh62sakZApN5s+kONjrQxwEEtdEeCp+nsiI6FSk9DXncW5at4rxP+8fgrBpZvxKEmBRXS2KEgFhhgXaeABl4yCmGhCqOT6VkxHRBIKOrOaDsGef3mRdM4attWw787rzdsyjio6REfoBNnoAjXRDWqhNqIoR8/oFb0ZT8aL8W58zForRjmzj/7A+PwB++CWqA==</latexit><latexit sha1_base64="UzO4G9mH0d14nrnW2PlrXyujM88=">AAACAHicbVDLSsNAFJ3UV62vqAsXbgaLIAglEUGXBV3oroJ9QBPCZDpph04ezNwINWTjr7hxoYhbP8Odf+OkzUJbD1w4nHMv997jJ4IrsKxvo7K0vLK6Vl2vbWxube+Yu3sdFaeSsjaNRSx7PlFM8Ii1gYNgvUQyEvqCdf3xVeF3H5hUPI7uYZIwNyTDiAecEtCSZx44IYGRH2SPuZfBqXPNBBAMuWfWrYY1BV4kdknqqETLM7+cQUzTkEVABVGqb1sJuBmRwKlgec1JFUsIHZMh62sakZApN5s+kONjrQxwEEtdEeCp+nsiI6FSk9DXncW5at4rxP+8fgrBpZvxKEmBRXS2KEgFhhgXaeABl4yCmGhCqOT6VkxHRBIKOrOaDsGef3mRdM4attWw787rzdsyjio6REfoBNnoAjXRDWqhNqIoR8/oFb0ZT8aL8W58zForRjmzj/7A+PwB++CWqA==</latexit><latexit sha1_base64="UzO4G9mH0d14nrnW2PlrXyujM88=">AAACAHicbVDLSsNAFJ3UV62vqAsXbgaLIAglEUGXBV3oroJ9QBPCZDpph04ezNwINWTjr7hxoYhbP8Odf+OkzUJbD1w4nHMv997jJ4IrsKxvo7K0vLK6Vl2vbWxube+Yu3sdFaeSsjaNRSx7PlFM8Ii1gYNgvUQyEvqCdf3xVeF3H5hUPI7uYZIwNyTDiAecEtCSZx44IYGRH2SPuZfBqXPNBBAMuWfWrYY1BV4kdknqqETLM7+cQUzTkEVABVGqb1sJuBmRwKlgec1JFUsIHZMh62sakZApN5s+kONjrQxwEEtdEeCp+nsiI6FSk9DXncW5at4rxP+8fgrBpZvxKEmBRXS2KEgFhhgXaeABl4yCmGhCqOT6VkxHRBIKOrOaDsGef3mRdM4attWw787rzdsyjio6REfoBNnoAjXRDWqhNqIoR8/oFb0ZT8aL8W58zForRjmzj/7A+PwB++CWqA==</latexit>
zt<latexit sha1_base64="6UpXO1SHbfyvjsxMsVUcyUsXXP8=">AAAB9XicbVDLSsNAFL2pr1pfVZdugkVwVRIRdFkQwWUF+4A2lsl00g6dTMLMjVJD/sONC0Xc+i/u/BsnbRbaemDgcM693DPHjwXX6DjfVmlldW19o7xZ2dre2d2r7h+0dZQoylo0EpHq+kQzwSVrIUfBurFiJPQF6/iTq9zvPDCleSTvcBozLyQjyQNOCRrpvh8SHPtB+pQNUswG1ZpTd2awl4lbkBoUaA6qX/1hRJOQSaSCaN1znRi9lCjkVLCs0k80iwmdkBHrGSpJyLSXzlJn9olRhnYQKfMk2jP190ZKQq2noW8m85R60cvF/7xegsGll3IZJ8gknR8KEmFjZOcV2EOuGEUxNYRQxU1Wm46JIhRNURVTgrv45WXSPqu7Tt29Pa81ros6ynAEx3AKLlxAA26gCS2goOAZXuHNerRerHfrYz5asoqdQ/gD6/MHYH2TFA==</latexit><latexit sha1_base64="6UpXO1SHbfyvjsxMsVUcyUsXXP8=">AAAB9XicbVDLSsNAFL2pr1pfVZdugkVwVRIRdFkQwWUF+4A2lsl00g6dTMLMjVJD/sONC0Xc+i/u/BsnbRbaemDgcM693DPHjwXX6DjfVmlldW19o7xZ2dre2d2r7h+0dZQoylo0EpHq+kQzwSVrIUfBurFiJPQF6/iTq9zvPDCleSTvcBozLyQjyQNOCRrpvh8SHPtB+pQNUswG1ZpTd2awl4lbkBoUaA6qX/1hRJOQSaSCaN1znRi9lCjkVLCs0k80iwmdkBHrGSpJyLSXzlJn9olRhnYQKfMk2jP190ZKQq2noW8m85R60cvF/7xegsGll3IZJ8gknR8KEmFjZOcV2EOuGEUxNYRQxU1Wm46JIhRNURVTgrv45WXSPqu7Tt29Pa81ros6ynAEx3AKLlxAA26gCS2goOAZXuHNerRerHfrYz5asoqdQ/gD6/MHYH2TFA==</latexit><latexit sha1_base64="6UpXO1SHbfyvjsxMsVUcyUsXXP8=">AAAB9XicbVDLSsNAFL2pr1pfVZdugkVwVRIRdFkQwWUF+4A2lsl00g6dTMLMjVJD/sONC0Xc+i/u/BsnbRbaemDgcM693DPHjwXX6DjfVmlldW19o7xZ2dre2d2r7h+0dZQoylo0EpHq+kQzwSVrIUfBurFiJPQF6/iTq9zvPDCleSTvcBozLyQjyQNOCRrpvh8SHPtB+pQNUswG1ZpTd2awl4lbkBoUaA6qX/1hRJOQSaSCaN1znRi9lCjkVLCs0k80iwmdkBHrGSpJyLSXzlJn9olRhnYQKfMk2jP190ZKQq2noW8m85R60cvF/7xegsGll3IZJ8gknR8KEmFjZOcV2EOuGEUxNYRQxU1Wm46JIhRNURVTgrv45WXSPqu7Tt29Pa81ros6ynAEx3AKLlxAA26gCS2goOAZXuHNerRerHfrYz5asoqdQ/gD6/MHYH2TFA==</latexit><latexit sha1_base64="6UpXO1SHbfyvjsxMsVUcyUsXXP8=">AAAB9XicbVDLSsNAFL2pr1pfVZdugkVwVRIRdFkQwWUF+4A2lsl00g6dTMLMjVJD/sONC0Xc+i/u/BsnbRbaemDgcM693DPHjwXX6DjfVmlldW19o7xZ2dre2d2r7h+0dZQoylo0EpHq+kQzwSVrIUfBurFiJPQF6/iTq9zvPDCleSTvcBozLyQjyQNOCRrpvh8SHPtB+pQNUswG1ZpTd2awl4lbkBoUaA6qX/1hRJOQSaSCaN1znRi9lCjkVLCs0k80iwmdkBHrGSpJyLSXzlJn9olRhnYQKfMk2jP190ZKQq2noW8m85R60cvF/7xegsGll3IZJ8gknR8KEmFjZOcV2EOuGEUxNYRQxU1Wm46JIhRNURVTgrv45WXSPqu7Tt29Pa81ros6ynAEx3AKLlxAA26gCS2goOAZXuHNerRerHfrYz5asoqdQ/gD6/MHYH2TFA==</latexit>
Figure 3: Computational graph for a FODE block.
where hat and hbt are the intermediate states, zat+∆t and zbt+∆tdenote the outputs, I, J , S and K are differentiable neuralnetworks. The reverse computations are therefore:{
hbt = zbt+∆t
hat =(zat+∆t −K
(hbt))/S(hbt){
zat = hatzbt = (hat − J (zat )) /I (zat )
(10)
s.t. 0 < si ∈ S(hbt), 0 < ii ∈ I (zat ) .
To ensure the reversibility of Eq. (10), all elements inS(hbt)
and I (zat ) need to be greater than zero, which canbe met by making a simple transformation for each elemente as exp(loge), e ∈ I or e ∈ S . Then we have followingproperty which ensures that the affine transformations in theforward pass in Eq. (9) are reversible:Proposition 1. The forward pass functions Eq. (9) in theFODE block is reversible as long as each element of I(zat )and S(hbt) is non-zero.
Proof. Let JF and JG be the Jacobians of the transformationsin Eq. (9), which can be computed as:
JF(zat , z
bt
)=
[I O
∂hbt
∂zat
diag(I (zat ))
], (11)
JG(hbt ,h
at
)=
[I O
∂zat+∆t
∂hbt
diag(S(hbt))
], (12)
where I and O are identity and zero matrix, respectively, anddiag(·) denotes the diagonal matrix whose diagonal elementscorrespond to the elements in I (zat ) or S(hbt). Since JFand JG are lower triangular matrices, their determinants are
Proceedings of the Twenty-Ninth International Joint Conference on Artificial Intelligence (IJCAI-20)
1297
Algorithm 1 Gradient calculation in FODE.Input: Initial value: za0 , zb0; parameters θ; integration timet ∈ T ; time step size: ∆tOutput: Gradient dLdθ ;
1: Forward Pass:2: for t := 0 to T do3: zat+∆t, z
bt+∆t ← ODESolve
(Eq.(9), [zat , z
bt ], θ
);
4: Delete zat , zbt and all intermediate activations;
5: t← t+ ∆t;6: end for7: return zaT and zbT .8: Backward:9: for t := T to 0 do
10: Restore zat−∆t and zbt−∆t with zat and zbt by Eq.(10);11: Let zat−∆t = zat−∆t, z
bt−∆t = zbt−∆t;
12: Compute zat and zbt by Eq.(9) ;13: Compute gradients5t by Eq. (13) and Eq. (14);14: Update model gradients dL
dθ ← dLdθ +5t;
15: Delete zat , zbt ;
16: t← t−∆t;17: end for18: return dL
dθ .
computed as |JF | = diag(I (zat )) and |JG| = diag(S(hbt)).While each element e in S(hbt) and I (zat ) is greater than 0 af-ter transformation exp(loge), Jacobians JF and JG are there-fore invertible.
Due to the reversibility of FODE, in the forward pass withODE solver, we only save the output (zat+∆t,z
bt+∆t) with-
out the needs to store other variables and intermediate ac-tivations. In the backward stage, we first restore the input(zat , zbt) from the output (zat+∆t, z
bt+∆t) by Eq. (10). Then,
we perform one-time step forward pass to obtain the output(zat+∆t,z
bt+∆t), and then calculate corresponding gradients
∂[zat+∆t,z
bt+∆t]
∂[zan,z
bn]
and∂[za
t+∆t,zbt+∆t]
∂[θI ,θJ ,θS ,θK] . Subsequently, the gradi-ents w.r.t. zt and θ of one-time step are computed as:
∂L
∂[zat , z
bt
] =∂L
∂[zat+∆t, z
bt+∆t
] ∂ [zat+∆t, zbt+∆t
]∂[zat , z
bt
] , (13)
∂L
∂ [θI , θJ , θS , θK]=
∂L
∂[zat+∆t, z
bt+∆t
] ∂ [zat+∆t, zbt+∆t
]∂ [θI , θJ , θS , θK]
. (14)
The process of calculating accurate gradients in FODE issummarized in Algorithm 1, where ODESolve can be anynumerical solutions, e.g., Euler, Runge-Kutta [Butcher andWanner, 1996] and Dopris Solver [Ascher et al., 1997]. Af-ter FODE block, layer normalization [Ba et al., 2016] is em-ployed to normalize the feature maps in the channel. Thenwe leverage SubPixel blocks [Liang et al., 2019] to upscalethe hidden state from coarse-grained to fine-grained with up-scaling factor N , which obtains the fine-grained hidden stateoutput Hf after the convolutional layer.
External factors Ef<latexit sha1_base64="Xex2g6bKrgS4t1RUSW7SZNZ8+PI=">AAAB83icbVDLSsNAFL3xWeur6tLNYBFclUQEXRZUcFnBPqCpZTKdtEMnkzBzI5TQ33DjQhG3/ow7/8ZJm4W2Hhg4nHMv98wJEikMuu63s7K6tr6xWdoqb+/s7u1XDg5bJk41400Wy1h3Amq4FIo3UaDknURzGgWSt4Pxde63n7g2IlYPOEl4L6JDJULBKFrJ9yOKoyDMbqePYb9SdWvuDGSZeAWpQoFGv/LlD2KWRlwhk9SYrucm2MuoRsEkn5b91PCEsjEd8q6likbc9LJZ5ik5tcqAhLG2TyGZqb83MhoZM4kCO5lnNIteLv7ndVMMr3qZUEmKXLH5oTCVBGOSF0AGQnOGcmIJZVrYrISNqKYMbU1lW4K3+OVl0jqveW7Nu7+o1m+KOkpwDCdwBh5cQh3uoAFNYJDAM7zCm5M6L8678zEfXXGKnSP4A+fzByvkkcM=</latexit><latexit sha1_base64="Xex2g6bKrgS4t1RUSW7SZNZ8+PI=">AAAB83icbVDLSsNAFL3xWeur6tLNYBFclUQEXRZUcFnBPqCpZTKdtEMnkzBzI5TQ33DjQhG3/ow7/8ZJm4W2Hhg4nHMv98wJEikMuu63s7K6tr6xWdoqb+/s7u1XDg5bJk41400Wy1h3Amq4FIo3UaDknURzGgWSt4Pxde63n7g2IlYPOEl4L6JDJULBKFrJ9yOKoyDMbqePYb9SdWvuDGSZeAWpQoFGv/LlD2KWRlwhk9SYrucm2MuoRsEkn5b91PCEsjEd8q6likbc9LJZ5ik5tcqAhLG2TyGZqb83MhoZM4kCO5lnNIteLv7ndVMMr3qZUEmKXLH5oTCVBGOSF0AGQnOGcmIJZVrYrISNqKYMbU1lW4K3+OVl0jqveW7Nu7+o1m+KOkpwDCdwBh5cQh3uoAFNYJDAM7zCm5M6L8678zEfXXGKnSP4A+fzByvkkcM=</latexit><latexit sha1_base64="Xex2g6bKrgS4t1RUSW7SZNZ8+PI=">AAAB83icbVDLSsNAFL3xWeur6tLNYBFclUQEXRZUcFnBPqCpZTKdtEMnkzBzI5TQ33DjQhG3/ow7/8ZJm4W2Hhg4nHMv98wJEikMuu63s7K6tr6xWdoqb+/s7u1XDg5bJk41400Wy1h3Amq4FIo3UaDknURzGgWSt4Pxde63n7g2IlYPOEl4L6JDJULBKFrJ9yOKoyDMbqePYb9SdWvuDGSZeAWpQoFGv/LlD2KWRlwhk9SYrucm2MuoRsEkn5b91PCEsjEd8q6likbc9LJZ5ik5tcqAhLG2TyGZqb83MhoZM4kCO5lnNIteLv7ndVMMr3qZUEmKXLH5oTCVBGOSF0AGQnOGcmIJZVrYrISNqKYMbU1lW4K3+OVl0jqveW7Nu7+o1m+KOkpwDCdwBh5cQh3uoAFNYJDAM7zCm5M6L8678zEfXXGKnSP4A+fzByvkkcM=</latexit><latexit sha1_base64="Xex2g6bKrgS4t1RUSW7SZNZ8+PI=">AAAB83icbVDLSsNAFL3xWeur6tLNYBFclUQEXRZUcFnBPqCpZTKdtEMnkzBzI5TQ33DjQhG3/ow7/8ZJm4W2Hhg4nHMv98wJEikMuu63s7K6tr6xWdoqb+/s7u1XDg5bJk41400Wy1h3Amq4FIo3UaDknURzGgWSt4Pxde63n7g2IlYPOEl4L6JDJULBKFrJ9yOKoyDMbqePYb9SdWvuDGSZeAWpQoFGv/LlD2KWRlwhk9SYrucm2MuoRsEkn5b91PCEsjEd8q6likbc9LJZ5ik5tcqAhLG2TyGZqb83MhoZM4kCO5lnNIteLv7ndVMMr3qZUEmKXLH5oTCVBGOSF0AGQnOGcmIJZVrYrISNqKYMbU1lW4K3+OVl0jqveW7Nu7+o1m+KOkpwDCdwBh5cQh3uoAFNYJDAM7zCm5M6L8678zEfXXGKnSP4A+fzByvkkcM=</latexit>
...<latexit sha1_base64="1dttRt72SwZcxQYiVfLx2+ZMyS0=">AAAB6nicbVBNS8NAEJ3Ur1q/qh69LBbBU0hE0GPBi94q2g9oQ9lsN+3SzSbsToQS+hO8eFDEq7/Im//GbZuDtj4YeLw3w8y8MJXCoOd9O6W19Y3NrfJ2ZWd3b/+genjUMkmmGW+yRCa6E1LDpVC8iQIl76Sa0ziUvB2Ob2Z++4lrIxL1iJOUBzEdKhEJRtFKD67r9qs1z/XmIKvEL0gNCjT61a/eIGFZzBUySY3p+l6KQU41Cib5tNLLDE8pG9Mh71qqaMxNkM9PnZIzqwxIlGhbCslc/T2R09iYSRzazpjiyCx7M/E/r5thdB3kQqUZcsUWi6JMEkzI7G8yEJozlBNLKNPC3krYiGrK0KZTsSH4yy+vktaF63uuf39Zq98VcZThBE7hHHy4gjrcQgOawGAIz/AKb450Xpx352PRWnKKmWP4A+fzB03/jSk=</latexit><latexit sha1_base64="1dttRt72SwZcxQYiVfLx2+ZMyS0=">AAAB6nicbVBNS8NAEJ3Ur1q/qh69LBbBU0hE0GPBi94q2g9oQ9lsN+3SzSbsToQS+hO8eFDEq7/Im//GbZuDtj4YeLw3w8y8MJXCoOd9O6W19Y3NrfJ2ZWd3b/+genjUMkmmGW+yRCa6E1LDpVC8iQIl76Sa0ziUvB2Ob2Z++4lrIxL1iJOUBzEdKhEJRtFKD67r9qs1z/XmIKvEL0gNCjT61a/eIGFZzBUySY3p+l6KQU41Cib5tNLLDE8pG9Mh71qqaMxNkM9PnZIzqwxIlGhbCslc/T2R09iYSRzazpjiyCx7M/E/r5thdB3kQqUZcsUWi6JMEkzI7G8yEJozlBNLKNPC3krYiGrK0KZTsSH4yy+vktaF63uuf39Zq98VcZThBE7hHHy4gjrcQgOawGAIz/AKb450Xpx352PRWnKKmWP4A+fzB03/jSk=</latexit><latexit sha1_base64="1dttRt72SwZcxQYiVfLx2+ZMyS0=">AAAB6nicbVBNS8NAEJ3Ur1q/qh69LBbBU0hE0GPBi94q2g9oQ9lsN+3SzSbsToQS+hO8eFDEq7/Im//GbZuDtj4YeLw3w8y8MJXCoOd9O6W19Y3NrfJ2ZWd3b/+genjUMkmmGW+yRCa6E1LDpVC8iQIl76Sa0ziUvB2Ob2Z++4lrIxL1iJOUBzEdKhEJRtFKD67r9qs1z/XmIKvEL0gNCjT61a/eIGFZzBUySY3p+l6KQU41Cib5tNLLDE8pG9Mh71qqaMxNkM9PnZIzqwxIlGhbCslc/T2R09iYSRzazpjiyCx7M/E/r5thdB3kQqUZcsUWi6JMEkzI7G8yEJozlBNLKNPC3krYiGrK0KZTsSH4yy+vktaF63uuf39Zq98VcZThBE7hHHy4gjrcQgOawGAIz/AKb450Xpx352PRWnKKmWP4A+fzB03/jSk=</latexit><latexit sha1_base64="1dttRt72SwZcxQYiVfLx2+ZMyS0=">AAAB6nicbVBNS8NAEJ3Ur1q/qh69LBbBU0hE0GPBi94q2g9oQ9lsN+3SzSbsToQS+hO8eFDEq7/Im//GbZuDtj4YeLw3w8y8MJXCoOd9O6W19Y3NrfJ2ZWd3b/+genjUMkmmGW+yRCa6E1LDpVC8iQIl76Sa0ziUvB2Ob2Z++4lrIxL1iJOUBzEdKhEJRtFKD67r9qs1z/XmIKvEL0gNCjT61a/eIGFZzBUySY3p+l6KQU41Cib5tNLLDE8pG9Mh71qqaMxNkM9PnZIzqwxIlGhbCslc/T2R09iYSRzazpjiyCx7M/E/r5thdB3kQqUZcsUWi6JMEkzI7G8yEJozlBNLKNPC3krYiGrK0KZTsSH4yy+vktaF63uuf39Zq98VcZThBE7hHHy4gjrcQgOawGAIz/AKb450Xpx352PRWnKKmWP4A+fzB03/jSk=</latexit> ...<latexit sha1_base64="1dttRt72SwZcxQYiVfLx2+ZMyS0=">AAAB6nicbVBNS8NAEJ3Ur1q/qh69LBbBU0hE0GPBi94q2g9oQ9lsN+3SzSbsToQS+hO8eFDEq7/Im//GbZuDtj4YeLw3w8y8MJXCoOd9O6W19Y3NrfJ2ZWd3b/+genjUMkmmGW+yRCa6E1LDpVC8iQIl76Sa0ziUvB2Ob2Z++4lrIxL1iJOUBzEdKhEJRtFKD67r9qs1z/XmIKvEL0gNCjT61a/eIGFZzBUySY3p+l6KQU41Cib5tNLLDE8pG9Mh71qqaMxNkM9PnZIzqwxIlGhbCslc/T2R09iYSRzazpjiyCx7M/E/r5thdB3kQqUZcsUWi6JMEkzI7G8yEJozlBNLKNPC3krYiGrK0KZTsSH4yy+vktaF63uuf39Zq98VcZThBE7hHHy4gjrcQgOawGAIz/AKb450Xpx352PRWnKKmWP4A+fzB03/jSk=</latexit><latexit sha1_base64="1dttRt72SwZcxQYiVfLx2+ZMyS0=">AAAB6nicbVBNS8NAEJ3Ur1q/qh69LBbBU0hE0GPBi94q2g9oQ9lsN+3SzSbsToQS+hO8eFDEq7/Im//GbZuDtj4YeLw3w8y8MJXCoOd9O6W19Y3NrfJ2ZWd3b/+genjUMkmmGW+yRCa6E1LDpVC8iQIl76Sa0ziUvB2Ob2Z++4lrIxL1iJOUBzEdKhEJRtFKD67r9qs1z/XmIKvEL0gNCjT61a/eIGFZzBUySY3p+l6KQU41Cib5tNLLDE8pG9Mh71qqaMxNkM9PnZIzqwxIlGhbCslc/T2R09iYSRzazpjiyCx7M/E/r5thdB3kQqUZcsUWi6JMEkzI7G8yEJozlBNLKNPC3krYiGrK0KZTsSH4yy+vktaF63uuf39Zq98VcZThBE7hHHy4gjrcQgOawGAIz/AKb450Xpx352PRWnKKmWP4A+fzB03/jSk=</latexit><latexit sha1_base64="1dttRt72SwZcxQYiVfLx2+ZMyS0=">AAAB6nicbVBNS8NAEJ3Ur1q/qh69LBbBU0hE0GPBi94q2g9oQ9lsN+3SzSbsToQS+hO8eFDEq7/Im//GbZuDtj4YeLw3w8y8MJXCoOd9O6W19Y3NrfJ2ZWd3b/+genjUMkmmGW+yRCa6E1LDpVC8iQIl76Sa0ziUvB2Ob2Z++4lrIxL1iJOUBzEdKhEJRtFKD67r9qs1z/XmIKvEL0gNCjT61a/eIGFZzBUySY3p+l6KQU41Cib5tNLLDE8pG9Mh71qqaMxNkM9PnZIzqwxIlGhbCslc/T2R09iYSRzazpjiyCx7M/E/r5thdB3kQqUZcsUWi6JMEkzI7G8yEJozlBNLKNPC3krYiGrK0KZTsSH4yy+vktaF63uuf39Zq98VcZThBE7hHHy4gjrcQgOawGAIz/AKb450Xpx352PRWnKKmWP4A+fzB03/jSk=</latexit><latexit sha1_base64="1dttRt72SwZcxQYiVfLx2+ZMyS0=">AAAB6nicbVBNS8NAEJ3Ur1q/qh69LBbBU0hE0GPBi94q2g9oQ9lsN+3SzSbsToQS+hO8eFDEq7/Im//GbZuDtj4YeLw3w8y8MJXCoOd9O6W19Y3NrfJ2ZWd3b/+genjUMkmmGW+yRCa6E1LDpVC8iQIl76Sa0ziUvB2Ob2Z++4lrIxL1iJOUBzEdKhEJRtFKD67r9qs1z/XmIKvEL0gNCjT61a/eIGFZzBUySY3p+l6KQU41Cib5tNLLDE8pG9Mh71qqaMxNkM9PnZIzqwxIlGhbCslc/T2R09iYSRzazpjiyCx7M/E/r5thdB3kQqUZcsUWi6JMEkzI7G8yEJozlBNLKNPC3krYiGrK0KZTsSH4yy+vktaF63uuf39Zq98VcZThBE7hHHy4gjrcQgOawGAIz/AKb450Xpx352PRWnKKmWP4A+fzB03/jSk=</latexit> ...<latexit sha1_base64="1dttRt72SwZcxQYiVfLx2+ZMyS0=">AAAB6nicbVBNS8NAEJ3Ur1q/qh69LBbBU0hE0GPBi94q2g9oQ9lsN+3SzSbsToQS+hO8eFDEq7/Im//GbZuDtj4YeLw3w8y8MJXCoOd9O6W19Y3NrfJ2ZWd3b/+genjUMkmmGW+yRCa6E1LDpVC8iQIl76Sa0ziUvB2Ob2Z++4lrIxL1iJOUBzEdKhEJRtFKD67r9qs1z/XmIKvEL0gNCjT61a/eIGFZzBUySY3p+l6KQU41Cib5tNLLDE8pG9Mh71qqaMxNkM9PnZIzqwxIlGhbCslc/T2R09iYSRzazpjiyCx7M/E/r5thdB3kQqUZcsUWi6JMEkzI7G8yEJozlBNLKNPC3krYiGrK0KZTsSH4yy+vktaF63uuf39Zq98VcZThBE7hHHy4gjrcQgOawGAIz/AKb450Xpx352PRWnKKmWP4A+fzB03/jSk=</latexit><latexit sha1_base64="1dttRt72SwZcxQYiVfLx2+ZMyS0=">AAAB6nicbVBNS8NAEJ3Ur1q/qh69LBbBU0hE0GPBi94q2g9oQ9lsN+3SzSbsToQS+hO8eFDEq7/Im//GbZuDtj4YeLw3w8y8MJXCoOd9O6W19Y3NrfJ2ZWd3b/+genjUMkmmGW+yRCa6E1LDpVC8iQIl76Sa0ziUvB2Ob2Z++4lrIxL1iJOUBzEdKhEJRtFKD67r9qs1z/XmIKvEL0gNCjT61a/eIGFZzBUySY3p+l6KQU41Cib5tNLLDE8pG9Mh71qqaMxNkM9PnZIzqwxIlGhbCslc/T2R09iYSRzazpjiyCx7M/E/r5thdB3kQqUZcsUWi6JMEkzI7G8yEJozlBNLKNPC3krYiGrK0KZTsSH4yy+vktaF63uuf39Zq98VcZThBE7hHHy4gjrcQgOawGAIz/AKb450Xpx352PRWnKKmWP4A+fzB03/jSk=</latexit><latexit sha1_base64="1dttRt72SwZcxQYiVfLx2+ZMyS0=">AAAB6nicbVBNS8NAEJ3Ur1q/qh69LBbBU0hE0GPBi94q2g9oQ9lsN+3SzSbsToQS+hO8eFDEq7/Im//GbZuDtj4YeLw3w8y8MJXCoOd9O6W19Y3NrfJ2ZWd3b/+genjUMkmmGW+yRCa6E1LDpVC8iQIl76Sa0ziUvB2Ob2Z++4lrIxL1iJOUBzEdKhEJRtFKD67r9qs1z/XmIKvEL0gNCjT61a/eIGFZzBUySY3p+l6KQU41Cib5tNLLDE8pG9Mh71qqaMxNkM9PnZIzqwxIlGhbCslc/T2R09iYSRzazpjiyCx7M/E/r5thdB3kQqUZcsUWi6JMEkzI7G8yEJozlBNLKNPC3krYiGrK0KZTsSH4yy+vktaF63uuf39Zq98VcZThBE7hHHy4gjrcQgOawGAIz/AKb450Xpx352PRWnKKmWP4A+fzB03/jSk=</latexit><latexit sha1_base64="1dttRt72SwZcxQYiVfLx2+ZMyS0=">AAAB6nicbVBNS8NAEJ3Ur1q/qh69LBbBU0hE0GPBi94q2g9oQ9lsN+3SzSbsToQS+hO8eFDEq7/Im//GbZuDtj4YeLw3w8y8MJXCoOd9O6W19Y3NrfJ2ZWd3b/+genjUMkmmGW+yRCa6E1LDpVC8iQIl76Sa0ziUvB2Ob2Z++4lrIxL1iJOUBzEdKhEJRtFKD67r9qs1z/XmIKvEL0gNCjT61a/eIGFZzBUySY3p+l6KQU41Cib5tNLLDE8pG9Mh71qqaMxNkM9PnZIzqwxIlGhbCslc/T2R09iYSRzazpjiyCx7M/E/r5thdB3kQqUZcsUWi6JMEkzI7G8yEJozlBNLKNPC3krYiGrK0KZTsSH4yy+vktaF63uuf39Zq98VcZThBE7hHHy4gjrcQgOawGAIz/AKb450Xpx352PRWnKKmWP4A+fzB03/jSk=</latexit>FC Network
E<latexit sha1_base64="Hr7ynn7H4wg8xFUSapFwyKj6tqY=">AAAB8XicbVBNS8NAFHypX7V+VT16WSyCp5KIoMeCCh4r2FpsQ9lsX9qlm03Y3Qgl9F948aCIV/+NN/+NmzYHbR1YGGbeY+dNkAiujet+O6WV1bX1jfJmZWt7Z3evun/Q1nGqGLZYLGLVCahGwSW2DDcCO4lCGgUCH4LxVe4/PKHSPJb3ZpKgH9Gh5CFn1FjpsRdRMwrC7Gbar9bcujsDWSZeQWpQoNmvfvUGMUsjlIYJqnXXcxPjZ1QZzgROK71UY0LZmA6xa6mkEWo/myWekhOrDEgYK/ukITP190ZGI60nUWAn84R60cvF/7xuasJLP+MySQ1KNv8oTAUxMcnPJwOukBkxsYQyxW1WwkZUUWZsSRVbgrd48jJpn9U9t+7dndca10UdZTiCYzgFDy6gAbfQhBYwkPAMr/DmaOfFeXc+5qMlp9g5hD9wPn8AsFSQ6w==</latexit><latexit sha1_base64="Hr7ynn7H4wg8xFUSapFwyKj6tqY=">AAAB8XicbVBNS8NAFHypX7V+VT16WSyCp5KIoMeCCh4r2FpsQ9lsX9qlm03Y3Qgl9F948aCIV/+NN/+NmzYHbR1YGGbeY+dNkAiujet+O6WV1bX1jfJmZWt7Z3evun/Q1nGqGLZYLGLVCahGwSW2DDcCO4lCGgUCH4LxVe4/PKHSPJb3ZpKgH9Gh5CFn1FjpsRdRMwrC7Gbar9bcujsDWSZeQWpQoNmvfvUGMUsjlIYJqnXXcxPjZ1QZzgROK71UY0LZmA6xa6mkEWo/myWekhOrDEgYK/ukITP190ZGI60nUWAn84R60cvF/7xuasJLP+MySQ1KNv8oTAUxMcnPJwOukBkxsYQyxW1WwkZUUWZsSRVbgrd48jJpn9U9t+7dndca10UdZTiCYzgFDy6gAbfQhBYwkPAMr/DmaOfFeXc+5qMlp9g5hD9wPn8AsFSQ6w==</latexit><latexit sha1_base64="Hr7ynn7H4wg8xFUSapFwyKj6tqY=">AAAB8XicbVBNS8NAFHypX7V+VT16WSyCp5KIoMeCCh4r2FpsQ9lsX9qlm03Y3Qgl9F948aCIV/+NN/+NmzYHbR1YGGbeY+dNkAiujet+O6WV1bX1jfJmZWt7Z3evun/Q1nGqGLZYLGLVCahGwSW2DDcCO4lCGgUCH4LxVe4/PKHSPJb3ZpKgH9Gh5CFn1FjpsRdRMwrC7Gbar9bcujsDWSZeQWpQoNmvfvUGMUsjlIYJqnXXcxPjZ1QZzgROK71UY0LZmA6xa6mkEWo/myWekhOrDEgYK/ukITP190ZGI60nUWAn84R60cvF/7xuasJLP+MySQ1KNv8oTAUxMcnPJwOukBkxsYQyxW1WwkZUUWZsSRVbgrd48jJpn9U9t+7dndca10UdZTiCYzgFDy6gAbfQhBYwkPAMr/DmaOfFeXc+5qMlp9g5hD9wPn8AsFSQ6w==</latexit><latexit sha1_base64="Hr7ynn7H4wg8xFUSapFwyKj6tqY=">AAAB8XicbVBNS8NAFHypX7V+VT16WSyCp5KIoMeCCh4r2FpsQ9lsX9qlm03Y3Qgl9F948aCIV/+NN/+NmzYHbR1YGGbeY+dNkAiujet+O6WV1bX1jfJmZWt7Z3evun/Q1nGqGLZYLGLVCahGwSW2DDcCO4lCGgUCH4LxVe4/PKHSPJb3ZpKgH9Gh5CFn1FjpsRdRMwrC7Gbar9bcujsDWSZeQWpQoNmvfvUGMUsjlIYJqnXXcxPjZ1QZzgROK71UY0LZmA6xa6mkEWo/myWekhOrDEgYK/ukITP190ZGI60nUWAn84R60cvF/7xuasJLP+MySQ1KNv8oTAUxMcnPJwOukBkxsYQyxW1WwkZUUWZsSRVbgrd48jJpn9U9t+7dndca10UdZTiCYzgFDy6gAbfQhBYwkPAMr/DmaOfFeXc+5qMlp9g5hD9wPn8AsFSQ6w==</latexit>
FODEFODESubPixel Block
Ec<latexit sha1_base64="cSwyOISso9+QAbR151gloHM/u9Y=">AAAB9HicbVDLSgMxFL3js9ZX1aWbYBFclRkRdFlQwWUF+4B2LJk004ZmMmNyp1CGfocbF4q49WPc+Tdm2i609UDgcM693JMTJFIYdN1vZ2V1bX1js7BV3N7Z3dsvHRw2TJxqxusslrFuBdRwKRSvo0DJW4nmNAokbwbD69xvjrg2IlYPOE64H9G+EqFgFK3kk05EcRCE2e3kkXVLZbfiTkGWiTcnZZij1i19dXoxSyOukElqTNtzE/QzqlEwySfFTmp4QtmQ9nnbUkUjbvxsGnpCTq3SI2Gs7VNIpurvjYxGxoyjwE7mGc2il4v/ee0Uwys/EypJkSs2OxSmkmBM8gZIT2jOUI4toUwLm5WwAdWUoe2paEvwFr+8TBrnFc+tePcX5erNvI4CHMMJnIEHl1CFO6hBHRg8wTO8wpszcl6cd+djNrrizHeO4A+czx9+UpHq</latexit><latexit sha1_base64="cSwyOISso9+QAbR151gloHM/u9Y=">AAAB9HicbVDLSgMxFL3js9ZX1aWbYBFclRkRdFlQwWUF+4B2LJk004ZmMmNyp1CGfocbF4q49WPc+Tdm2i609UDgcM693JMTJFIYdN1vZ2V1bX1js7BV3N7Z3dsvHRw2TJxqxusslrFuBdRwKRSvo0DJW4nmNAokbwbD69xvjrg2IlYPOE64H9G+EqFgFK3kk05EcRCE2e3kkXVLZbfiTkGWiTcnZZij1i19dXoxSyOukElqTNtzE/QzqlEwySfFTmp4QtmQ9nnbUkUjbvxsGnpCTq3SI2Gs7VNIpurvjYxGxoyjwE7mGc2il4v/ee0Uwys/EypJkSs2OxSmkmBM8gZIT2jOUI4toUwLm5WwAdWUoe2paEvwFr+8TBrnFc+tePcX5erNvI4CHMMJnIEHl1CFO6hBHRg8wTO8wpszcl6cd+djNrrizHeO4A+czx9+UpHq</latexit><latexit sha1_base64="cSwyOISso9+QAbR151gloHM/u9Y=">AAAB9HicbVDLSgMxFL3js9ZX1aWbYBFclRkRdFlQwWUF+4B2LJk004ZmMmNyp1CGfocbF4q49WPc+Tdm2i609UDgcM693JMTJFIYdN1vZ2V1bX1js7BV3N7Z3dsvHRw2TJxqxusslrFuBdRwKRSvo0DJW4nmNAokbwbD69xvjrg2IlYPOE64H9G+EqFgFK3kk05EcRCE2e3kkXVLZbfiTkGWiTcnZZij1i19dXoxSyOukElqTNtzE/QzqlEwySfFTmp4QtmQ9nnbUkUjbvxsGnpCTq3SI2Gs7VNIpurvjYxGxoyjwE7mGc2il4v/ee0Uwys/EypJkSs2OxSmkmBM8gZIT2jOUI4toUwLm5WwAdWUoe2paEvwFr+8TBrnFc+tePcX5erNvI4CHMMJnIEHl1CFO6hBHRg8wTO8wpszcl6cd+djNrrizHeO4A+czx9+UpHq</latexit><latexit sha1_base64="cSwyOISso9+QAbR151gloHM/u9Y=">AAAB9HicbVDLSgMxFL3js9ZX1aWbYBFclRkRdFlQwWUF+4B2LJk004ZmMmNyp1CGfocbF4q49WPc+Tdm2i609UDgcM693JMTJFIYdN1vZ2V1bX1js7BV3N7Z3dsvHRw2TJxqxusslrFuBdRwKRSvo0DJW4nmNAokbwbD69xvjrg2IlYPOE64H9G+EqFgFK3kk05EcRCE2e3kkXVLZbfiTkGWiTcnZZij1i19dXoxSyOukElqTNtzE/QzqlEwySfFTmp4QtmQ9nnbUkUjbvxsGnpCTq3SI2Gs7VNIpurvjYxGxoyjwE7mGc2il4v/ee0Uwys/EypJkSs2OxSmkmBM8gZIT2jOUI4toUwLm5WwAdWUoe2paEvwFr+8TBrnFc+tePcX5erNvI4CHMMJnIEHl1CFO6hBHRg8wTO8wpszcl6cd+djNrrizHeO4A+czx9+UpHq</latexit>
Figure 4: Feature fusion network.
2.3 Feature Fusion NetworkIt has been demonstrated that external features (e.g., windspeed, temperature, weather and holidays) affect the traf-fic distribution of the flows [Liang et al., 2019]. Here wealso take these factors into consideration for improving per-formance. In addition to a simple fully connected network(FCN) for feature fusion, we use the proposed FODE blocksto improve the ability of estimating the feature influence. Asshown in Figure 4, we obtain the coarse-grained feature mapEc ∈ RH×W+ and fine-grained feature map Ef ∈ RNH×NW+by appending two FODE blocks:
Ec = FODE (E)⊕E, (15)
Ef = FODE (SP (Ec))⊕ SP (Ec) , (16)
where SP indicates a SubPixel block used for upsampling.
2.4 Augmented Flow NormalizationOne of the main differences between FSR and image SR tasksis that there is a structural constraint in FSR, i.e., the amountof flows in the subregions should equal the flows in the re-spective superregion:
xcij =∑i′j′
xfi′j′ s.t.
⌊i′
N
⌋= i,
⌊j′
N
⌋= j, (17)
where xcij (resp. xfi′j′ ) denotes the flow volume in a super-region (resp. subregion) of the coarse-grained (resp. fine-grained) grid map. This is a straightforward method that nor-malizes the flow in the N2 subregions to meet the constraint,a.k.a. N2-Normalization [Liang et al., 2019]. However, itignores the influence of external factors on the subregions,and to address this problem, we propose an augmented N2
normalization method which takes the distribution of externalfactors into account when constraining the flow, as illustratedin Figure 5.
More specifically, we replace the xfi′j′ in a subregion withprobability value αi′,j′ , and we modify Eq. (17) as follows:
xci,j =∑i′,j′
αi′,j′xci,j , e
ci,j =
∑i′,j′
βi′,j′eci,j . α, β ∈ R+,
s.t.∑
αi′,j′ = 1,∑
βi′,j′ = 1,
⌊i′
N
⌋= i,
⌊j′
N
⌋= j, (18)
where eci,j indicates the degree of influence of complex fac-tors on the region ri,j . αi′,j′ and βi′,j′ denote the proportionof flow and factor influence assigned to the subregions fromcorresponding superregion, respectively.
Now, we learn the joint distribution of factors’ influenceDfe and flow Df
h to obtain the final flow distribution Dfπ .
Here we present a Distribution Gating Mechanism (DGM)
Proceedings of the Twenty-Ninth International Joint Conference on Artificial Intelligence (IJCAI-20)
1298
N2<latexit sha1_base64="CIriErnqCCjgick8tvNOnaCrKFY=">AAAB6nicbZC7SgNBFIbPeo3xFrW0GQyCVdhNo50BEa0korlAsobZyWwyZHZ2mTkrhJBHsLFQxNYHsba0s/RNnFwKTfxh4OP/z2HOOUEihUHX/XIWFpeWV1Yza9n1jc2t7dzObtXEqWa8wmIZ63pADZdC8QoKlLyeaE6jQPJa0Dsb5bV7ro2I1S32E+5HtKNEKBhFa91c3RVbubxbcMci8+BNIX/68X3xDgDlVu6z2Y5ZGnGFTFJjGp6boD+gGgWTfJhtpoYnlPVohzcsKhpx4w/Gow7JoXXaJIy1fQrJ2P3dMaCRMf0osJURxa6ZzUbmf1kjxfDEHwiVpMgVm3wUppJgTEZ7k7bQnKHsW6BMCzsrYV2qKUN7naw9gje78jxUiwXPLXjXbr50DhNlYB8O4Ag8OIYSXEIZKsCgAw/wBM+OdB6dF+d1UrrgTHv24I+ctx/EwpBm</latexit><latexit sha1_base64="vCTLH4WzRo8cWxknP7+lKdsPQvU=">AAAB6nicbZC7SgNBFIbPxluMt/XS2QwGwSrsptHOgIhWEtFcIFnD7GQ2GTI7u8zMCnHJI9hYKGJr7YNY2lnmTZxcCk38YeDj/89hzjl+zJnSjvNtZRYWl5ZXsqu5tfWNzS17e6eqokQSWiERj2Tdx4pyJmhFM81pPZYUhz6nNb93Nspr91QqFolb3Y+pF+KOYAEjWBvr5uqu2LLzTsEZC82DO4X86efw4mPvYVhu2V/NdkSSkApNOFaq4Tqx9lIsNSOcDnLNRNEYkx7u0IZBgUOqvHQ86gAdGqeNgkiaJzQau787Uhwq1Q99Uxli3VWz2cj8L2skOjjxUibiRFNBJh8FCUc6QqO9UZtJSjTvG8BEMjMrIl0sMdHmOjlzBHd25XmoFguuU3CvnXzpHCbKwj4cwBG4cAwluIQyVIBABx7hGV4sbj1Zr9bbpDRjTXt24Y+s9x+WLZHD</latexit><latexit sha1_base64="vCTLH4WzRo8cWxknP7+lKdsPQvU=">AAAB6nicbZC7SgNBFIbPxluMt/XS2QwGwSrsptHOgIhWEtFcIFnD7GQ2GTI7u8zMCnHJI9hYKGJr7YNY2lnmTZxcCk38YeDj/89hzjl+zJnSjvNtZRYWl5ZXsqu5tfWNzS17e6eqokQSWiERj2Tdx4pyJmhFM81pPZYUhz6nNb93Nspr91QqFolb3Y+pF+KOYAEjWBvr5uqu2LLzTsEZC82DO4X86efw4mPvYVhu2V/NdkSSkApNOFaq4Tqx9lIsNSOcDnLNRNEYkx7u0IZBgUOqvHQ86gAdGqeNgkiaJzQau787Uhwq1Q99Uxli3VWz2cj8L2skOjjxUibiRFNBJh8FCUc6QqO9UZtJSjTvG8BEMjMrIl0sMdHmOjlzBHd25XmoFguuU3CvnXzpHCbKwj4cwBG4cAwluIQyVIBABx7hGV4sbj1Zr9bbpDRjTXt24Y+s9x+WLZHD</latexit><latexit sha1_base64="Fg+jyDjGVUT484d1S3uYz4RBdv4=">AAAB6nicbVBNS8NAEJ3Ur1q/qh69LBbBU0l60WNBBE9S0X5AG8tmu2mXbjZhdyKU0J/gxYMiXv1F3vw3btsctPXBwOO9GWbmBYkUBl332ymsrW9sbhW3Szu7e/sH5cOjlolTzXiTxTLWnYAaLoXiTRQoeSfRnEaB5O1gfDXz209cGxGrB5wk3I/oUIlQMIpWur99rPXLFbfqzkFWiZeTCuRo9MtfvUHM0ogrZJIa0/XcBP2MahRM8mmplxqeUDamQ961VNGIGz+bnzolZ1YZkDDWthSSufp7IqORMZMosJ0RxZFZ9mbif143xfDSz4RKUuSKLRaFqSQYk9nfZCA0ZygnllCmhb2VsBHVlKFNp2RD8JZfXiWtWtVzq96dW6lf53EU4QRO4Rw8uIA63EADmsBgCM/wCm+OdF6cd+dj0Vpw8plj+APn8wfLS411</latexit>
�<latexit sha1_base64="fbyjEpZSGVU6xX1DH25hjezBdGs=">AAAB7XicbZC7SgNBFIbPeo3rLWppMxgEq7Bro40YtLGMYC6QLGF2MpuMmcsyMyuEJeAj2FgoYuuj2Nv5Nk4uhSb+MPDx/+cw55w45czYIPj2lpZXVtfWCxv+5tb2zm5xb79uVKYJrRHFlW7G2FDOJK1ZZjltpppiEXPaiAfX47zxQLVhSt7ZYUojgXuSJYxg66x627CewJ1iKSgHE6FFCGdQuvz0Lx4BoNopfrW7imSCSks4NqYVBqmNcqwtI5yO/HZmaIrJAPdoy6HEgpoon0w7QsfO6aJEafekRRP3d0eOhTFDEbtKgW3fzGdj87+sldnkPMqZTDNLJZl+lGQcWYXGq6Mu05RYPnSAiWZuVkT6WGNi3YF8d4RwfuVFqJ+Ww6Ac3galyhVMVYBDOIITCOEMKnADVagBgXt4ghd49ZT37L1579PSJW/WcwB/5H38AApmkOw=</latexit><latexit sha1_base64="6TTneXtyobn5Jdp44M9pu6I2aVE=">AAAB7XicbZC7SgNBFIbPxltcb1FLm8EgWIVdG23EoI1lBHOBJITZyWwyZi7LzKwQlryDjYUiNhY+ir2N+DZOLoUm/jDw8f/nMOecKOHM2CD49nJLyyura/l1f2Nza3unsLtXMyrVhFaJ4ko3ImwoZ5JWLbOcNhJNsYg4rUeDq3Fev6faMCVv7TChbYF7ksWMYOusWsuwnsCdQjEoBROhRQhnULz48M+Tty+/0il8trqKpIJKSzg2phkGiW1nWFtGOB35rdTQBJMB7tGmQ4kFNe1sMu0IHTmni2Kl3ZMWTdzfHRkWxgxF5CoFtn0zn43N/7JmauOzdsZkkloqyfSjOOXIKjReHXWZpsTyoQNMNHOzItLHGhPrDuS7I4TzKy9C7aQUBqXwJiiWL2GqPBzAIRxDCKdQhmuoQBUI3MEDPMGzp7xH78V7nZbmvFnPPvyR9/4D++aSYA==</latexit><latexit sha1_base64="6TTneXtyobn5Jdp44M9pu6I2aVE=">AAAB7XicbZC7SgNBFIbPxltcb1FLm8EgWIVdG23EoI1lBHOBJITZyWwyZi7LzKwQlryDjYUiNhY+ir2N+DZOLoUm/jDw8f/nMOecKOHM2CD49nJLyyura/l1f2Nza3unsLtXMyrVhFaJ4ko3ImwoZ5JWLbOcNhJNsYg4rUeDq3Fev6faMCVv7TChbYF7ksWMYOusWsuwnsCdQjEoBROhRQhnULz48M+Tty+/0il8trqKpIJKSzg2phkGiW1nWFtGOB35rdTQBJMB7tGmQ4kFNe1sMu0IHTmni2Kl3ZMWTdzfHRkWxgxF5CoFtn0zn43N/7JmauOzdsZkkloqyfSjOOXIKjReHXWZpsTyoQNMNHOzItLHGhPrDuS7I4TzKy9C7aQUBqXwJiiWL2GqPBzAIRxDCKdQhmuoQBUI3MEDPMGzp7xH78V7nZbmvFnPPvyR9/4D++aSYA==</latexit><latexit sha1_base64="jN6Xkptr9YiQbG0o0aWcfgdui7o=">AAAB7XicbVDLSgNBEOz1GeMr6tHLYBA8hVkvegx68RjBPCBZwuxkNhkzj2VmVghL/sGLB0W8+j/e/BsnyR40saChqOqmuytOBbcO4+9gbX1jc2u7tFPe3ds/OKwcHbeszgxlTaqFNp2YWCa4Yk3HnWCd1DAiY8Ha8fh25refmLFcqwc3SVkkyVDxhFPivNTqWT6UpF+p4hqeA62SsCBVKNDoV756A00zyZSjgljbDXHqopwYx6lg03IvsywldEyGrOupIpLZKJ9fO0XnXhmgRBtfyqG5+nsiJ9LaiYx9pyRuZJe9mfif181cch3lXKWZY4ouFiWZQE6j2etowA2jTkw8IdRwfyuiI2IIdT6gsg8hXH55lbQuayGuhfe4Wr8p4ijBKZzBBYRwBXW4gwY0gcIjPMMrvAU6eAneg49F61pQzJzAHwSfP5sfjx8=</latexit>
+<latexit sha1_base64="5tDwYnANjTPXkg7Hl3+b227HsCw=">AAAB6HicbVDLSgNBEOyNryS+oh69DAZBEMKuFz0GvXhMwDwgWeLspDcZMzu7zMwKYckXePGgSK5+kjf/xsnjoIkFDUVVN91dQSK4Nq777eQ2Nre2d/KF4u7e/sFh6ei4qeNUMWywWMSqHVCNgktsGG4EthOFNAoEtoLR3cxvPaPSPJYPZpygH9GB5CFn1Fipftkrld2KOwdZJ96SlKuFZPoIALVe6avbj1kaoTRMUK07npsYP6PKcCZwUuymGhPKRnSAHUsljVD72fzQCTm3Sp+EsbIlDZmrvycyGmk9jgLbGVEz1KveTPzP66QmvPEzLpPUoGSLRWEqiInJ7GvS5wqZEWNLKFPc3krYkCrKjM2maEPwVl9eJ82riudWvLpN4xYWyMMpnMEFeHANVbiHGjSAAcILvMG78+S8Oh/OdNGac5YzJ/AHzucPuieOXQ==</latexit><latexit sha1_base64="AxCXw2Qlpz7aZrm1qu9e0DAkB9g=">AAAB6HicbVDLSgNBEOyNryS+oh69DAZBEMKuFz0GvXhMwDwwCWF20puMmZ1dZmbFsOQLvHhQJFd/wH/x5tfo5HHQaEFDUdVNd5cfC66N6346mZXVtfWNbC6/ubW9s1vY26/rKFEMaywSkWr6VKPgEmuGG4HNWCENfYENf3g19Rv3qDSP5I0ZxdgJaV/ygDNqrFQ97RaKbsmdgfwl3oIUy7l4cvv+8FXpFj7avYglIUrDBNW65bmx6aRUGc4EjvPtRGNM2ZD2sWWppCHqTjo7dEyOrdIjQaRsSUNm6s+JlIZaj0LfdobUDPSyNxX/81qJCS46KZdxYlCy+aIgEcREZPo16XGFzIiRJZQpbm8lbEAVZcZmk7cheMsv/yX1s5LnlryqTeMS5sjCIRzBCXhwDmW4hgrUgAHCIzzDi3PnPDmvzmTemnEWMwfwC87bN5C4kH0=</latexit><latexit sha1_base64="AxCXw2Qlpz7aZrm1qu9e0DAkB9g=">AAAB6HicbVDLSgNBEOyNryS+oh69DAZBEMKuFz0GvXhMwDwwCWF20puMmZ1dZmbFsOQLvHhQJFd/wH/x5tfo5HHQaEFDUdVNd5cfC66N6346mZXVtfWNbC6/ubW9s1vY26/rKFEMaywSkWr6VKPgEmuGG4HNWCENfYENf3g19Rv3qDSP5I0ZxdgJaV/ygDNqrFQ97RaKbsmdgfwl3oIUy7l4cvv+8FXpFj7avYglIUrDBNW65bmx6aRUGc4EjvPtRGNM2ZD2sWWppCHqTjo7dEyOrdIjQaRsSUNm6s+JlIZaj0LfdobUDPSyNxX/81qJCS46KZdxYlCy+aIgEcREZPo16XGFzIiRJZQpbm8lbEAVZcZmk7cheMsv/yX1s5LnlryqTeMS5sjCIRzBCXhwDmW4hgrUgAHCIzzDi3PnPDmvzmTemnEWMwfwC87bN5C4kH0=</latexit><latexit sha1_base64="1sXRlCCqI5rChQ00zvsAYLHdTlU=">AAAB6HicbVBNS8NAEJ3Ur1q/qh69LBZBEEripR6LXjy2YD+gDWWznbRrN5uwuxFK6C/w4kERr/4kb/4bt20O2vpg4PHeDDPzgkRwbVz32ylsbG5t7xR3S3v7B4dH5eOTto5TxbDFYhGrbkA1Ci6xZbgR2E0U0igQ2Akmd3O/84RK81g+mGmCfkRHkoecUWOl5tWgXHGr7gJknXg5qUCOxqD81R/GLI1QGiao1j3PTYyfUWU4Ezgr9VONCWUTOsKepZJGqP1sceiMXFhlSMJY2ZKGLNTfExmNtJ5Gge2MqBnrVW8u/uf1UhPe+BmXSWpQsuWiMBXExGT+NRlyhcyIqSWUKW5vJWxMFWXGZlOyIXirL6+T9nXVc6te063Ub/M4inAG53AJHtSgDvfQgBYwQHiGV3hzHp0X5935WLYWnHzmFP7A+fwBcEWMqw==</latexit>
⇥<latexit sha1_base64="hWJn7XnKJ9olvb3nID+auGAk1ds=">AAAB7XicbVDLSgNBEOyNryS+oh69DAbBU9j1osegF48RzAOSJc5OJsmY2ZllplcIS/7Bi4eIePV/vPk3Th4HTSxoKKq66e6KEiks+v63l9vY3NreyReKu3v7B4elo+OG1alhvM601KYVUculULyOAiVvJYbTOJK8GY1uZ37zmRsrtHrAccLDmA6U6AtG0UmNDoqY226p7Ff8Ocg6CZakXC0k00cAqHVLX52eZmnMFTJJrW0HfoJhRg0KJvmk2EktTygb0QFvO6qoWxJm82sn5NwpPdLXxpVCMld/T2Q0tnYcR64zpji0q95M/M9rp9i/DjOhkhS5YotF/VQS1GT2OukJwxnKsSOUGeFuJWxIDWXoAiq6EILVl9dJ47IS+JXg3qVxAwvk4RTO4AICuIIq3EEN6sDgCV5gCm+e9l69d+9j0ZrzljMn8Afe5w/+zZDi</latexit><latexit sha1_base64="rsBppLPxs+qaKV2a9OvPsDFH+rE=">AAAB7XicbVC7TsNAEFzzTMIrQEljESFRRTYNlBE0lEEiD5FY0flyTo6c76y7NSKy8g80FEGIlpJ/oeNr4PIoIGGklUYzu9rdCRPBDXrel7Oyura+sZnLF7a2d3b3ivsHdaNSTVmNKqF0MySGCS5ZDTkK1kw0I3EoWCMcXE38xgPThit5i8OEBTHpSR5xStBK9TbymJlOseSVvSncZeLPSamST8Z3H4/f1U7xs91VNI2ZRCqIMS3fSzDIiEZOBRsV2qlhCaED0mMtSyWxS4Jseu3IPbFK142UtiXRnaq/JzISGzOMQ9sZE+ybRW8i/ue1UowugozLJEUm6WxRlAoXlTt53e1yzSiKoSWEam5vdWmfaELRBlSwIfiLLy+T+lnZ98r+jU3jEmbIwREcwyn4cA4VuIYq1IDCPTzBGF4c5Tw7r87brHXFmc8cwh847z/VXpMC</latexit><latexit sha1_base64="rsBppLPxs+qaKV2a9OvPsDFH+rE=">AAAB7XicbVC7TsNAEFzzTMIrQEljESFRRTYNlBE0lEEiD5FY0flyTo6c76y7NSKy8g80FEGIlpJ/oeNr4PIoIGGklUYzu9rdCRPBDXrel7Oyura+sZnLF7a2d3b3ivsHdaNSTVmNKqF0MySGCS5ZDTkK1kw0I3EoWCMcXE38xgPThit5i8OEBTHpSR5xStBK9TbymJlOseSVvSncZeLPSamST8Z3H4/f1U7xs91VNI2ZRCqIMS3fSzDIiEZOBRsV2qlhCaED0mMtSyWxS4Jseu3IPbFK142UtiXRnaq/JzISGzOMQ9sZE+ybRW8i/ue1UowugozLJEUm6WxRlAoXlTt53e1yzSiKoSWEam5vdWmfaELRBlSwIfiLLy+T+lnZ98r+jU3jEmbIwREcwyn4cA4VuIYq1IDCPTzBGF4c5Tw7r87brHXFmc8cwh847z/VXpMC</latexit><latexit sha1_base64="mQpgD4vO+sKTYKCTiWZnuZId88E=">AAAB7XicbVA9SwNBEJ3zM8avqKXNYhCswp2NlkEbywjmA5Ij7G32kjV7u8funBCO/AcbC0Vs/T92/hs3yRWa+GDg8d4MM/OiVAqLvv/tra1vbG5tl3bKu3v7B4eVo+OW1ZlhvMm01KYTUculULyJAiXvpIbTJJK8HY1vZ377iRsrtHrAScrDhA6ViAWj6KRWD0XCbb9S9Wv+HGSVBAWpQoFGv/LVG2iWJVwhk9TabuCnGObUoGCST8u9zPKUsjEd8q6jirolYT6/dkrOnTIgsTauFJK5+nsip4m1kyRynQnFkV32ZuJ/XjfD+DrMhUoz5IotFsWZJKjJ7HUyEIYzlBNHKDPC3UrYiBrK0AVUdiEEyy+vktZlLfBrwb1frd8UcZTgFM7gAgK4gjrcQQOawOARnuEV3jztvXjv3seidc0rZk7gD7zPH7TrjzA=</latexit> N2<latexit sha1_base64="CIriErnqCCjgick8tvNOnaCrKFY=">AAAB6nicbZC7SgNBFIbPeo3xFrW0GQyCVdhNo50BEa0korlAsobZyWwyZHZ2mTkrhJBHsLFQxNYHsba0s/RNnFwKTfxh4OP/z2HOOUEihUHX/XIWFpeWV1Yza9n1jc2t7dzObtXEqWa8wmIZ63pADZdC8QoKlLyeaE6jQPJa0Dsb5bV7ro2I1S32E+5HtKNEKBhFa91c3RVbubxbcMci8+BNIX/68X3xDgDlVu6z2Y5ZGnGFTFJjGp6boD+gGgWTfJhtpoYnlPVohzcsKhpx4w/Gow7JoXXaJIy1fQrJ2P3dMaCRMf0osJURxa6ZzUbmf1kjxfDEHwiVpMgVm3wUppJgTEZ7k7bQnKHsW6BMCzsrYV2qKUN7naw9gje78jxUiwXPLXjXbr50DhNlYB8O4Ag8OIYSXEIZKsCgAw/wBM+OdB6dF+d1UrrgTHv24I+ctx/EwpBm</latexit><latexit sha1_base64="vCTLH4WzRo8cWxknP7+lKdsPQvU=">AAAB6nicbZC7SgNBFIbPxluMt/XS2QwGwSrsptHOgIhWEtFcIFnD7GQ2GTI7u8zMCnHJI9hYKGJr7YNY2lnmTZxcCk38YeDj/89hzjl+zJnSjvNtZRYWl5ZXsqu5tfWNzS17e6eqokQSWiERj2Tdx4pyJmhFM81pPZYUhz6nNb93Nspr91QqFolb3Y+pF+KOYAEjWBvr5uqu2LLzTsEZC82DO4X86efw4mPvYVhu2V/NdkSSkApNOFaq4Tqx9lIsNSOcDnLNRNEYkx7u0IZBgUOqvHQ86gAdGqeNgkiaJzQau787Uhwq1Q99Uxli3VWz2cj8L2skOjjxUibiRFNBJh8FCUc6QqO9UZtJSjTvG8BEMjMrIl0sMdHmOjlzBHd25XmoFguuU3CvnXzpHCbKwj4cwBG4cAwluIQyVIBABx7hGV4sbj1Zr9bbpDRjTXt24Y+s9x+WLZHD</latexit><latexit sha1_base64="vCTLH4WzRo8cWxknP7+lKdsPQvU=">AAAB6nicbZC7SgNBFIbPxluMt/XS2QwGwSrsptHOgIhWEtFcIFnD7GQ2GTI7u8zMCnHJI9hYKGJr7YNY2lnmTZxcCk38YeDj/89hzjl+zJnSjvNtZRYWl5ZXsqu5tfWNzS17e6eqokQSWiERj2Tdx4pyJmhFM81pPZYUhz6nNb93Nspr91QqFolb3Y+pF+KOYAEjWBvr5uqu2LLzTsEZC82DO4X86efw4mPvYVhu2V/NdkSSkApNOFaq4Tqx9lIsNSOcDnLNRNEYkx7u0IZBgUOqvHQ86gAdGqeNgkiaJzQau787Uhwq1Q99Uxli3VWz2cj8L2skOjjxUibiRFNBJh8FCUc6QqO9UZtJSjTvG8BEMjMrIl0sMdHmOjlzBHd25XmoFguuU3CvnXzpHCbKwj4cwBG4cAwluIQyVIBABx7hGV4sbj1Zr9bbpDRjTXt24Y+s9x+WLZHD</latexit><latexit sha1_base64="Fg+jyDjGVUT484d1S3uYz4RBdv4=">AAAB6nicbVBNS8NAEJ3Ur1q/qh69LBbBU0l60WNBBE9S0X5AG8tmu2mXbjZhdyKU0J/gxYMiXv1F3vw3btsctPXBwOO9GWbmBYkUBl332ymsrW9sbhW3Szu7e/sH5cOjlolTzXiTxTLWnYAaLoXiTRQoeSfRnEaB5O1gfDXz209cGxGrB5wk3I/oUIlQMIpWur99rPXLFbfqzkFWiZeTCuRo9MtfvUHM0ogrZJIa0/XcBP2MahRM8mmplxqeUDamQ961VNGIGz+bnzolZ1YZkDDWthSSufp7IqORMZMosJ0RxZFZ9mbif143xfDSz4RKUuSKLRaFqSQYk9nfZCA0ZygnllCmhb2VsBHVlKFNp2RD8JZfXiWtWtVzq96dW6lf53EU4QRO4Rw8uIA63EADmsBgCM/wCm+OdF6cd+dj0Vpw8plj+APn8wfLS411</latexit>
N2<latexit sha1_base64="CIriErnqCCjgick8tvNOnaCrKFY=">AAAB6nicbZC7SgNBFIbPeo3xFrW0GQyCVdhNo50BEa0korlAsobZyWwyZHZ2mTkrhJBHsLFQxNYHsba0s/RNnFwKTfxh4OP/z2HOOUEihUHX/XIWFpeWV1Yza9n1jc2t7dzObtXEqWa8wmIZ63pADZdC8QoKlLyeaE6jQPJa0Dsb5bV7ro2I1S32E+5HtKNEKBhFa91c3RVbubxbcMci8+BNIX/68X3xDgDlVu6z2Y5ZGnGFTFJjGp6boD+gGgWTfJhtpoYnlPVohzcsKhpx4w/Gow7JoXXaJIy1fQrJ2P3dMaCRMf0osJURxa6ZzUbmf1kjxfDEHwiVpMgVm3wUppJgTEZ7k7bQnKHsW6BMCzsrYV2qKUN7naw9gje78jxUiwXPLXjXbr50DhNlYB8O4Ag8OIYSXEIZKsCgAw/wBM+OdB6dF+d1UrrgTHv24I+ctx/EwpBm</latexit><latexit sha1_base64="vCTLH4WzRo8cWxknP7+lKdsPQvU=">AAAB6nicbZC7SgNBFIbPxluMt/XS2QwGwSrsptHOgIhWEtFcIFnD7GQ2GTI7u8zMCnHJI9hYKGJr7YNY2lnmTZxcCk38YeDj/89hzjl+zJnSjvNtZRYWl5ZXsqu5tfWNzS17e6eqokQSWiERj2Tdx4pyJmhFM81pPZYUhz6nNb93Nspr91QqFolb3Y+pF+KOYAEjWBvr5uqu2LLzTsEZC82DO4X86efw4mPvYVhu2V/NdkSSkApNOFaq4Tqx9lIsNSOcDnLNRNEYkx7u0IZBgUOqvHQ86gAdGqeNgkiaJzQau787Uhwq1Q99Uxli3VWz2cj8L2skOjjxUibiRFNBJh8FCUc6QqO9UZtJSjTvG8BEMjMrIl0sMdHmOjlzBHd25XmoFguuU3CvnXzpHCbKwj4cwBG4cAwluIQyVIBABx7hGV4sbj1Zr9bbpDRjTXt24Y+s9x+WLZHD</latexit><latexit sha1_base64="vCTLH4WzRo8cWxknP7+lKdsPQvU=">AAAB6nicbZC7SgNBFIbPxluMt/XS2QwGwSrsptHOgIhWEtFcIFnD7GQ2GTI7u8zMCnHJI9hYKGJr7YNY2lnmTZxcCk38YeDj/89hzjl+zJnSjvNtZRYWl5ZXsqu5tfWNzS17e6eqokQSWiERj2Tdx4pyJmhFM81pPZYUhz6nNb93Nspr91QqFolb3Y+pF+KOYAEjWBvr5uqu2LLzTsEZC82DO4X86efw4mPvYVhu2V/NdkSSkApNOFaq4Tqx9lIsNSOcDnLNRNEYkx7u0IZBgUOqvHQ86gAdGqeNgkiaJzQau787Uhwq1Q99Uxli3VWz2cj8L2skOjjxUibiRFNBJh8FCUc6QqO9UZtJSjTvG8BEMjMrIl0sMdHmOjlzBHd25XmoFguuU3CvnXzpHCbKwj4cwBG4cAwluIQyVIBABx7hGV4sbj1Zr9bbpDRjTXt24Y+s9x+WLZHD</latexit><latexit sha1_base64="Fg+jyDjGVUT484d1S3uYz4RBdv4=">AAAB6nicbVBNS8NAEJ3Ur1q/qh69LBbBU0l60WNBBE9S0X5AG8tmu2mXbjZhdyKU0J/gxYMiXv1F3vw3btsctPXBwOO9GWbmBYkUBl332ymsrW9sbhW3Szu7e/sH5cOjlolTzXiTxTLWnYAaLoXiTRQoeSfRnEaB5O1gfDXz209cGxGrB5wk3I/oUIlQMIpWur99rPXLFbfqzkFWiZeTCuRo9MtfvUHM0ogrZJIa0/XcBP2MahRM8mmplxqeUDamQ961VNGIGz+bnzolZ1YZkDDWthSSufp7IqORMZMosJ0RxZFZ9mbif143xfDSz4RKUuSKLRaFqSQYk9nfZCA0ZygnllCmhb2VsBHVlKFNp2RD8JZfXiWtWtVzq96dW6lf53EU4QRO4Rw8uIA63EADmsBgCM/wCm+OdF6cd+dj0Vpw8plj+APn8wfLS411</latexit>
Df⇡<latexit sha1_base64="iLKgNY2GTJ3/znwy/IU37mlF/IU=">AAAB+XicbVBNS8NAEJ34WetX1KOXYBE8lUQEPRb14LGC/YA2hs120y7dbMLupFhC/4kXD4p49Z9489+4bXPQ1gcDj/dmmJkXpoJrdN1va2V1bX1js7RV3t7Z3du3Dw6bOskUZQ2aiES1Q6KZ4JI1kKNg7VQxEoeCtcLhzdRvjZjSPJEPOE6ZH5O+5BGnBI0U2HYX2ROGUX47Cbopf4wCu+JW3RmcZeIVpAIF6oH91e0lNIuZRCqI1h3PTdHPiUJOBZuUu5lmKaFD0mcdQyWJmfbz2eUT59QoPSdKlCmJzkz9PZGTWOtxHJrOmOBAL3pT8T+vk2F05edcphkySeeLokw4mDjTGJweV4yiGBtCqOLmVocOiCIUTVhlE4K3+PIyaZ5XPbfq3V9UatdFHCU4hhM4Aw8uoQZ3UIcGUBjBM7zCm5VbL9a79TFvXbGKmSP4A+vzB9nTk8g=</latexit><latexit sha1_base64="iLKgNY2GTJ3/znwy/IU37mlF/IU=">AAAB+XicbVBNS8NAEJ34WetX1KOXYBE8lUQEPRb14LGC/YA2hs120y7dbMLupFhC/4kXD4p49Z9489+4bXPQ1gcDj/dmmJkXpoJrdN1va2V1bX1js7RV3t7Z3du3Dw6bOskUZQ2aiES1Q6KZ4JI1kKNg7VQxEoeCtcLhzdRvjZjSPJEPOE6ZH5O+5BGnBI0U2HYX2ROGUX47Cbopf4wCu+JW3RmcZeIVpAIF6oH91e0lNIuZRCqI1h3PTdHPiUJOBZuUu5lmKaFD0mcdQyWJmfbz2eUT59QoPSdKlCmJzkz9PZGTWOtxHJrOmOBAL3pT8T+vk2F05edcphkySeeLokw4mDjTGJweV4yiGBtCqOLmVocOiCIUTVhlE4K3+PIyaZ5XPbfq3V9UatdFHCU4hhM4Aw8uoQZ3UIcGUBjBM7zCm5VbL9a79TFvXbGKmSP4A+vzB9nTk8g=</latexit><latexit sha1_base64="iLKgNY2GTJ3/znwy/IU37mlF/IU=">AAAB+XicbVBNS8NAEJ34WetX1KOXYBE8lUQEPRb14LGC/YA2hs120y7dbMLupFhC/4kXD4p49Z9489+4bXPQ1gcDj/dmmJkXpoJrdN1va2V1bX1js7RV3t7Z3du3Dw6bOskUZQ2aiES1Q6KZ4JI1kKNg7VQxEoeCtcLhzdRvjZjSPJEPOE6ZH5O+5BGnBI0U2HYX2ROGUX47Cbopf4wCu+JW3RmcZeIVpAIF6oH91e0lNIuZRCqI1h3PTdHPiUJOBZuUu5lmKaFD0mcdQyWJmfbz2eUT59QoPSdKlCmJzkz9PZGTWOtxHJrOmOBAL3pT8T+vk2F05edcphkySeeLokw4mDjTGJweV4yiGBtCqOLmVocOiCIUTVhlE4K3+PIyaZ5XPbfq3V9UatdFHCU4hhM4Aw8uoQZ3UIcGUBjBM7zCm5VbL9a79TFvXbGKmSP4A+vzB9nTk8g=</latexit><latexit sha1_base64="iLKgNY2GTJ3/znwy/IU37mlF/IU=">AAAB+XicbVBNS8NAEJ34WetX1KOXYBE8lUQEPRb14LGC/YA2hs120y7dbMLupFhC/4kXD4p49Z9489+4bXPQ1gcDj/dmmJkXpoJrdN1va2V1bX1js7RV3t7Z3du3Dw6bOskUZQ2aiES1Q6KZ4JI1kKNg7VQxEoeCtcLhzdRvjZjSPJEPOE6ZH5O+5BGnBI0U2HYX2ROGUX47Cbopf4wCu+JW3RmcZeIVpAIF6oH91e0lNIuZRCqI1h3PTdHPiUJOBZuUu5lmKaFD0mcdQyWJmfbz2eUT59QoPSdKlCmJzkz9PZGTWOtxHJrOmOBAL3pT8T+vk2F05edcphkySeeLokw4mDjTGJweV4yiGBtCqOLmVocOiCIUTVhlE4K3+PIyaZ5XPbfq3V9UatdFHCU4hhM4Aw8uoQZ3UIcGUBjBM7zCm5VbL9a79TFvXbGKmSP4A+vzB9nTk8g=</latexit>
Dfh
<latexit sha1_base64="LIffKUIpS0XfsFQFMl1BsRPcnDc=">AAAB9XicbVDLSgNBEOyNrxhfUY9eBoPgKeyKoMegHjxGMA/Ii9nJbDJkdnaZ6VXDkv/w4kERr/6LN//GSbIHTSxoKKq66e7yYykMuu63k1tZXVvfyG8WtrZ3dveK+wd1EyWa8RqLZKSbPjVcCsVrKFDyZqw5DX3JG/7oeuo3Hrg2IlL3OI55J6QDJQLBKFqp20b+hH6Q3kx6w27QK5bcsjsDWSZeRkqQodorfrX7EUtCrpBJakzLc2PspFSjYJJPCu3E8JiyER3wlqWKhtx00tnVE3JilT4JIm1LIZmpvydSGhozDn3bGVIcmkVvKv7ntRIMLjupUHGCXLH5oiCRBCMyjYD0heYM5dgSyrSwtxI2pJoytEEVbAje4svLpH5W9tyyd3deqlxlceThCI7hFDy4gArcQhVqwEDDM7zCm/PovDjvzse8NedkM4fwB87nD9VlkrY=</latexit><latexit sha1_base64="LIffKUIpS0XfsFQFMl1BsRPcnDc=">AAAB9XicbVDLSgNBEOyNrxhfUY9eBoPgKeyKoMegHjxGMA/Ii9nJbDJkdnaZ6VXDkv/w4kERr/6LN//GSbIHTSxoKKq66e7yYykMuu63k1tZXVvfyG8WtrZ3dveK+wd1EyWa8RqLZKSbPjVcCsVrKFDyZqw5DX3JG/7oeuo3Hrg2IlL3OI55J6QDJQLBKFqp20b+hH6Q3kx6w27QK5bcsjsDWSZeRkqQodorfrX7EUtCrpBJakzLc2PspFSjYJJPCu3E8JiyER3wlqWKhtx00tnVE3JilT4JIm1LIZmpvydSGhozDn3bGVIcmkVvKv7ntRIMLjupUHGCXLH5oiCRBCMyjYD0heYM5dgSyrSwtxI2pJoytEEVbAje4svLpH5W9tyyd3deqlxlceThCI7hFDy4gArcQhVqwEDDM7zCm/PovDjvzse8NedkM4fwB87nD9VlkrY=</latexit><latexit sha1_base64="LIffKUIpS0XfsFQFMl1BsRPcnDc=">AAAB9XicbVDLSgNBEOyNrxhfUY9eBoPgKeyKoMegHjxGMA/Ii9nJbDJkdnaZ6VXDkv/w4kERr/6LN//GSbIHTSxoKKq66e7yYykMuu63k1tZXVvfyG8WtrZ3dveK+wd1EyWa8RqLZKSbPjVcCsVrKFDyZqw5DX3JG/7oeuo3Hrg2IlL3OI55J6QDJQLBKFqp20b+hH6Q3kx6w27QK5bcsjsDWSZeRkqQodorfrX7EUtCrpBJakzLc2PspFSjYJJPCu3E8JiyER3wlqWKhtx00tnVE3JilT4JIm1LIZmpvydSGhozDn3bGVIcmkVvKv7ntRIMLjupUHGCXLH5oiCRBCMyjYD0heYM5dgSyrSwtxI2pJoytEEVbAje4svLpH5W9tyyd3deqlxlceThCI7hFDy4gArcQhVqwEDDM7zCm/PovDjvzse8NedkM4fwB87nD9VlkrY=</latexit><latexit sha1_base64="LIffKUIpS0XfsFQFMl1BsRPcnDc=">AAAB9XicbVDLSgNBEOyNrxhfUY9eBoPgKeyKoMegHjxGMA/Ii9nJbDJkdnaZ6VXDkv/w4kERr/6LN//GSbIHTSxoKKq66e7yYykMuu63k1tZXVvfyG8WtrZ3dveK+wd1EyWa8RqLZKSbPjVcCsVrKFDyZqw5DX3JG/7oeuo3Hrg2IlL3OI55J6QDJQLBKFqp20b+hH6Q3kx6w27QK5bcsjsDWSZeRkqQodorfrX7EUtCrpBJakzLc2PspFSjYJJPCu3E8JiyER3wlqWKhtx00tnVE3JilT4JIm1LIZmpvydSGhozDn3bGVIcmkVvKv7ntRIMLjupUHGCXLH5oiCRBCMyjYD0heYM5dgSyrSwtxI2pJoytEEVbAje4svLpH5W9tyyd3deqlxlceThCI7hFDy4gArcQhVqwEDDM7zCm/PovDjvzse8NedkM4fwB87nD9VlkrY=</latexit>
Dfe<latexit sha1_base64="tXlhjuEp+E+ES2vWKPS7LsOPj4g=">AAAB9XicbVDLSgNBEJyNrxhfUY9eBoPgKeyKoMegHjxGMA/Ii9lJbzJkdnaZ6VXDkv/w4kERr/6LN//GSbIHTSxoKKq66e7yYykMuu63k1tZXVvfyG8WtrZ3dveK+wd1EyWaQ41HMtJNnxmQQkENBUpoxhpY6Eto+KPrqd94AG1EpO5xHEMnZAMlAsEZWqnbRnhCP0hvJj3oBr1iyS27M9Bl4mWkRDJUe8Wvdj/iSQgKuWTGtDw3xk7KNAouYVJoJwZixkdsAC1LFQvBdNLZ1RN6YpU+DSJtSyGdqb8nUhYaMw592xkyHJpFbyr+57USDC47qVBxgqD4fFGQSIoRnUZA+0IDRzm2hHEt7K2UD5lmHG1QBRuCt/jyMqmflT237N2dlypXWRx5ckSOySnxyAWpkFtSJTXCiSbP5JW8OY/Oi/PufMxbc042c0j+wPn8AdDTkrM=</latexit><latexit sha1_base64="tXlhjuEp+E+ES2vWKPS7LsOPj4g=">AAAB9XicbVDLSgNBEJyNrxhfUY9eBoPgKeyKoMegHjxGMA/Ii9lJbzJkdnaZ6VXDkv/w4kERr/6LN//GSbIHTSxoKKq66e7yYykMuu63k1tZXVvfyG8WtrZ3dveK+wd1EyWaQ41HMtJNnxmQQkENBUpoxhpY6Eto+KPrqd94AG1EpO5xHEMnZAMlAsEZWqnbRnhCP0hvJj3oBr1iyS27M9Bl4mWkRDJUe8Wvdj/iSQgKuWTGtDw3xk7KNAouYVJoJwZixkdsAC1LFQvBdNLZ1RN6YpU+DSJtSyGdqb8nUhYaMw592xkyHJpFbyr+57USDC47qVBxgqD4fFGQSIoRnUZA+0IDRzm2hHEt7K2UD5lmHG1QBRuCt/jyMqmflT237N2dlypXWRx5ckSOySnxyAWpkFtSJTXCiSbP5JW8OY/Oi/PufMxbc042c0j+wPn8AdDTkrM=</latexit><latexit sha1_base64="tXlhjuEp+E+ES2vWKPS7LsOPj4g=">AAAB9XicbVDLSgNBEJyNrxhfUY9eBoPgKeyKoMegHjxGMA/Ii9lJbzJkdnaZ6VXDkv/w4kERr/6LN//GSbIHTSxoKKq66e7yYykMuu63k1tZXVvfyG8WtrZ3dveK+wd1EyWaQ41HMtJNnxmQQkENBUpoxhpY6Eto+KPrqd94AG1EpO5xHEMnZAMlAsEZWqnbRnhCP0hvJj3oBr1iyS27M9Bl4mWkRDJUe8Wvdj/iSQgKuWTGtDw3xk7KNAouYVJoJwZixkdsAC1LFQvBdNLZ1RN6YpU+DSJtSyGdqb8nUhYaMw592xkyHJpFbyr+57USDC47qVBxgqD4fFGQSIoRnUZA+0IDRzm2hHEt7K2UD5lmHG1QBRuCt/jyMqmflT237N2dlypXWRx5ckSOySnxyAWpkFtSJTXCiSbP5JW8OY/Oi/PufMxbc042c0j+wPn8AdDTkrM=</latexit><latexit sha1_base64="tXlhjuEp+E+ES2vWKPS7LsOPj4g=">AAAB9XicbVDLSgNBEJyNrxhfUY9eBoPgKeyKoMegHjxGMA/Ii9lJbzJkdnaZ6VXDkv/w4kERr/6LN//GSbIHTSxoKKq66e7yYykMuu63k1tZXVvfyG8WtrZ3dveK+wd1EyWaQ41HMtJNnxmQQkENBUpoxhpY6Eto+KPrqd94AG1EpO5xHEMnZAMlAsEZWqnbRnhCP0hvJj3oBr1iyS27M9Bl4mWkRDJUe8Wvdj/iSQgKuWTGtDw3xk7KNAouYVJoJwZixkdsAC1LFQvBdNLZ1RN6YpU+DSJtSyGdqb8nUhYaMw592xkyHJpFbyr+57USDC47qVBxgqD4fFGQSIoRnUZA+0IDRzm2hHEt7K2UD5lmHG1QBRuCt/jyMqmflT237N2dlypXWRx5ckSOySnxyAWpkFtSJTXCiSbP5JW8OY/Oi/PufMxbc042c0j+wPn8AdDTkrM=</latexit>Ef
<latexit sha1_base64="SbE7CDty/NIimTZEe1YFI2l7grE=">AAAB83icbVBNS8NAEN3Ur1q/qh69LBbBU0lE0GNRBI8V7Ae0sWy2k3bpZhN2J2IJ/RtePCji1T/jzX/jts1BWx8MPN6bYWZekEhh0HW/ncLK6tr6RnGztLW9s7tX3j9omjjVHBo8lrFuB8yAFAoaKFBCO9HAokBCKxhdT/3WI2gjYnWP4wT8iA2UCAVnaKVuF+EJgzC7mTyEvXLFrboz0GXi5aRCctR75a9uP+ZpBAq5ZMZ0PDdBP2MaBZcwKXVTAwnjIzaAjqWKRWD8bHbzhJ5YpU/DWNtSSGfq74mMRcaMo8B2RgyHZtGbiv95nRTDSz8TKkkRFJ8vClNJMabTAGhfaOAox5YwroW9lfIh04yjjalkQ/AWX14mzbOq51a9u/NK7SqPo0iOyDE5JR65IDVyS+qkQThJyDN5JW9O6rw4787HvLXg5DOH5A+czx9VFJHc</latexit><latexit sha1_base64="SbE7CDty/NIimTZEe1YFI2l7grE=">AAAB83icbVBNS8NAEN3Ur1q/qh69LBbBU0lE0GNRBI8V7Ae0sWy2k3bpZhN2J2IJ/RtePCji1T/jzX/jts1BWx8MPN6bYWZekEhh0HW/ncLK6tr6RnGztLW9s7tX3j9omjjVHBo8lrFuB8yAFAoaKFBCO9HAokBCKxhdT/3WI2gjYnWP4wT8iA2UCAVnaKVuF+EJgzC7mTyEvXLFrboz0GXi5aRCctR75a9uP+ZpBAq5ZMZ0PDdBP2MaBZcwKXVTAwnjIzaAjqWKRWD8bHbzhJ5YpU/DWNtSSGfq74mMRcaMo8B2RgyHZtGbiv95nRTDSz8TKkkRFJ8vClNJMabTAGhfaOAox5YwroW9lfIh04yjjalkQ/AWX14mzbOq51a9u/NK7SqPo0iOyDE5JR65IDVyS+qkQThJyDN5JW9O6rw4787HvLXg5DOH5A+czx9VFJHc</latexit><latexit sha1_base64="SbE7CDty/NIimTZEe1YFI2l7grE=">AAAB83icbVBNS8NAEN3Ur1q/qh69LBbBU0lE0GNRBI8V7Ae0sWy2k3bpZhN2J2IJ/RtePCji1T/jzX/jts1BWx8MPN6bYWZekEhh0HW/ncLK6tr6RnGztLW9s7tX3j9omjjVHBo8lrFuB8yAFAoaKFBCO9HAokBCKxhdT/3WI2gjYnWP4wT8iA2UCAVnaKVuF+EJgzC7mTyEvXLFrboz0GXi5aRCctR75a9uP+ZpBAq5ZMZ0PDdBP2MaBZcwKXVTAwnjIzaAjqWKRWD8bHbzhJ5YpU/DWNtSSGfq74mMRcaMo8B2RgyHZtGbiv95nRTDSz8TKkkRFJ8vClNJMabTAGhfaOAox5YwroW9lfIh04yjjalkQ/AWX14mzbOq51a9u/NK7SqPo0iOyDE5JR65IDVyS+qkQThJyDN5JW9O6rw4787HvLXg5DOH5A+czx9VFJHc</latexit><latexit sha1_base64="SbE7CDty/NIimTZEe1YFI2l7grE=">AAAB83icbVBNS8NAEN3Ur1q/qh69LBbBU0lE0GNRBI8V7Ae0sWy2k3bpZhN2J2IJ/RtePCji1T/jzX/jts1BWx8MPN6bYWZekEhh0HW/ncLK6tr6RnGztLW9s7tX3j9omjjVHBo8lrFuB8yAFAoaKFBCO9HAokBCKxhdT/3WI2gjYnWP4wT8iA2UCAVnaKVuF+EJgzC7mTyEvXLFrboz0GXi5aRCctR75a9uP+ZpBAq5ZMZ0PDdBP2MaBZcwKXVTAwnjIzaAjqWKRWD8bHbzhJ5YpU/DWNtSSGfq74mMRcaMo8B2RgyHZtGbiv95nRTDSz8TKkkRFJ8vClNJMabTAGhfaOAox5YwroW9lfIh04yjjalkQ/AWX14mzbOq51a9u/NK7SqPo0iOyDE5JR65IDVyS+qkQThJyDN5JW9O6rw4787HvLXg5DOH5A+czx9VFJHc</latexit>
Hf<latexit sha1_base64="JS9urkdF89d2iuBBc09rrncy2qo=">AAAB83icbVBNS8NAEJ3Ur1q/qh69LBbBU0lE0GPRS48V7Ae0sWy2m3bpZhN2J2IJ/RtePCji1T/jzX/jts1BWx8MPN6bYWZekEhh0HW/ncLa+sbmVnG7tLO7t39QPjxqmTjVjDdZLGPdCajhUijeRIGSdxLNaRRI3g7GtzO//ci1EbG6x0nC/YgOlQgFo2ilXg/5EwZhVp8+hP1yxa26c5BV4uWkAjka/fJXbxCzNOIKmaTGdD03QT+jGgWTfFrqpYYnlI3pkHctVTTixs/mN0/JmVUGJIy1LYVkrv6eyGhkzCQKbGdEcWSWvZn4n9dNMbz2M6GSFLlii0VhKgnGZBYAGQjNGcqJJZRpYW8lbEQ1ZWhjKtkQvOWXV0nrouq5Ve/uslK7yeMowgmcwjl4cAU1qEMDmsAggWd4hTcndV6cd+dj0Vpw8plj+APn8wdZqZHf</latexit><latexit sha1_base64="JS9urkdF89d2iuBBc09rrncy2qo=">AAAB83icbVBNS8NAEJ3Ur1q/qh69LBbBU0lE0GPRS48V7Ae0sWy2m3bpZhN2J2IJ/RtePCji1T/jzX/jts1BWx8MPN6bYWZekEhh0HW/ncLa+sbmVnG7tLO7t39QPjxqmTjVjDdZLGPdCajhUijeRIGSdxLNaRRI3g7GtzO//ci1EbG6x0nC/YgOlQgFo2ilXg/5EwZhVp8+hP1yxa26c5BV4uWkAjka/fJXbxCzNOIKmaTGdD03QT+jGgWTfFrqpYYnlI3pkHctVTTixs/mN0/JmVUGJIy1LYVkrv6eyGhkzCQKbGdEcWSWvZn4n9dNMbz2M6GSFLlii0VhKgnGZBYAGQjNGcqJJZRpYW8lbEQ1ZWhjKtkQvOWXV0nrouq5Ve/uslK7yeMowgmcwjl4cAU1qEMDmsAggWd4hTcndV6cd+dj0Vpw8plj+APn8wdZqZHf</latexit><latexit sha1_base64="JS9urkdF89d2iuBBc09rrncy2qo=">AAAB83icbVBNS8NAEJ3Ur1q/qh69LBbBU0lE0GPRS48V7Ae0sWy2m3bpZhN2J2IJ/RtePCji1T/jzX/jts1BWx8MPN6bYWZekEhh0HW/ncLa+sbmVnG7tLO7t39QPjxqmTjVjDdZLGPdCajhUijeRIGSdxLNaRRI3g7GtzO//ci1EbG6x0nC/YgOlQgFo2ilXg/5EwZhVp8+hP1yxa26c5BV4uWkAjka/fJXbxCzNOIKmaTGdD03QT+jGgWTfFrqpYYnlI3pkHctVTTixs/mN0/JmVUGJIy1LYVkrv6eyGhkzCQKbGdEcWSWvZn4n9dNMbz2M6GSFLlii0VhKgnGZBYAGQjNGcqJJZRpYW8lbEQ1ZWhjKtkQvOWXV0nrouq5Ve/uslK7yeMowgmcwjl4cAU1qEMDmsAggWd4hTcndV6cd+dj0Vpw8plj+APn8wdZqZHf</latexit><latexit sha1_base64="JS9urkdF89d2iuBBc09rrncy2qo=">AAAB83icbVBNS8NAEJ3Ur1q/qh69LBbBU0lE0GPRS48V7Ae0sWy2m3bpZhN2J2IJ/RtePCji1T/jzX/jts1BWx8MPN6bYWZekEhh0HW/ncLa+sbmVnG7tLO7t39QPjxqmTjVjDdZLGPdCajhUijeRIGSdxLNaRRI3g7GtzO//ci1EbG6x0nC/YgOlQgFo2ilXg/5EwZhVp8+hP1yxa26c5BV4uWkAjka/fJXbxCzNOIKmaTGdD03QT+jGgWTfFrqpYYnlI3pkHctVTTixs/mN0/JmVUGJIy1LYVkrv6eyGhkzCQKbGdEcWSWvZn4n9dNMbz2M6GSFLlii0VhKgnGZBYAGQjNGcqJJZRpYW8lbEQ1ZWhjKtkQvOWXV0nrouq5Ve/uslK7yeMowgmcwjl4cAU1qEMDmsAggWd4hTcndV6cd+dj0Vpw8plj+APn8wdZqZHf</latexit>
Figure 5: Illustration of AN2-Normalization.
to explicitly capture the dynamic spatial dependence betweenthe two distributions:
Dfπ = N2
(Dfh ⊕
(Dfh ⊗ σ
(Dfe
))), (19)
where σ denotes distribution gate, Dfe and Df
h are gener-ated by N2-Normalization with input Ef and Hf . AN2-Normalization not only inherits all the advantages of N2-Normalization (e.g., parallelizable computation and no extraparameters), but also bridges gaps between external factorsand urban flows when normalizing the values of subregions.
Fine-grained flow inference. At this point, we are readyto generate the fine-grained flow Xf from the learned jointdistribution as:
Xf = Xcup �Df
π, (20)
where Xcup ∈ RNH×NW+ is produced by nearest-neighbor up-
sampling with scaling factor N .
Optimization. Finally, we minimize the mean squared er-ror between the ground truth fine-grained flow map Xf andthe model inferred output Xf :
L(θ) =∥∥∥Xf − Xf
∥∥∥2
2=∥∥Xf −F (Xc, |E,N ; θ)
∥∥2
2,
where θ represents all learnable parameters in our model.
3 ExperimentsWe now present the details of our experimental evaluations.
3.1 Experimental SettingsDatasets. We evaluate all the methods using two real-worldurban flow datasets: (1) TaxiBJ [Liang et al., 2019] – a taxiGPS data including taxi flows from July 1, 2014 to October31, 2014; and (2) BikeNYC – collected from an open web-site1 which contains data from January 1, 2019 to June 30,2019. Each dataset contains two sub-datasets: coarse-grainedand fine-grained flows. A detailed description of the datasetsis shown in Table 1. Note that the scaling factors are differentfor the two datasets, i.e., N = 4 and N = 2 for TaxiBJ andBikeNYC, respectively.
Baselines. We compare FODE with the following 10 base-lines:
• Mean Partition (Mean) evenly distributes the flow vol-ume in each subregion.
1https://www.citibikenyc.com/system-data
Dataset TaxiBJ BikeNYCTime range 7/1/2013-10/31/2013 1/1/2019-3/31/2019
Time interval 30 minutes 1 hourCoarse-grained size 32×32 40×16Fine-grained size 128×128 80×32
Upscaling factor (N) 4 2Latitude range 39.82◦N - 39.99◦N 40.65◦N-40.81◦N
Longitude range 116.26◦E-116.49◦E 74.00◦E-74.07◦EExternal Factors (meteorology, time (e.g., hourofday, dayofweek))
Temperature / ◦C [-24.6,41.0] \Wind speed / mph [0,48.6] \Weather conditions 16 types (e.g., Rainy,Sunny) \
Holidays 18 10
Table 1: Statistics of datasets.
• Historical Average (HA) models historical average datato predict the flow in the subregion.
• SRCNN [Dong et al., 2015] is a classic model for imageSR based on CNNs.
• ESPCN [Shi et al., 2016] introduces a sub-pixel convo-lutional layer for image SR.
• VDSR [Kim et al., 2016] employs residual networks tosolve the slow convergence and limited representationproblems in SR.
• SRResNet [Ledig et al., 2017] is a ResNet-based variantof VDSR model, which allows stacking more networklayers.
• OISR [He et al., 2019] is an ODE-inspired SR modelusing Runge-Kutta (RK3) method [Butcher and Wanner,1996] as ODE solver.
• NODE [Chen et al., 2018] is a neural ODE method thatuses adjoint method for discretization and optimal con-trol of ODEs.
• ANODE [Gholami et al., 2019] is an improved versionof NODE by introducing checking points for alleviatingthe incorrect gradient issue.
• UrbanFM [Liang et al., 2019] infers fine-grained ur-ban flow with external factors by stacking ResNet-basedneural networks.
Metrics. We evaluate different methods with three widelyused metrics: Root Mean Squared Error (RMSE), Mean Ab-solute Error (MAE) and Mean Absolute Percentage Error(MAPE).
Implementation details. Adam [Kingma and Ba, 2014] isadopted to train FODE with batch size 16 and learning ratee−4. We leverage Dopri5 numerical method, which can adap-tively choose the step size, as ODESovle in FODE. FODEconsists of 128 channels and 1 ODE block. We also present asimplified version S-FODE which contains 64 channels whileother components are the same as FODE. During training, wehalve the learning rate and perform a test on the validation setevery 20 epochs. For all image SR models numerical methods(NODE and ANODE), we useN2-normalization to constrainthe inferred flow distribution. We note that the details of othernetwork settings are described in the source-implementation2.
2https://github.com/Anewnoob/FODE
Proceedings of the Twenty-Ninth International Joint Conference on Artificial Intelligence (IJCAI-20)
1299
Datasets TaxiBJ BikeNYCMethod RMSE MAE MAPE RMSE MAE MAPE
Mean 20.918 12.019 4.469 4.554 1.379 0.678HA 4.741 2.214 0.332 2.414 0.676 0.216
SRCNN 4.297 2.491 0.714 2.385 0.821 0.433ESPCN 4.206 2.497 0.732 2.356 0.825 0.441VDSR 4.159 2.213 0.467 2.344 0.734 0.285
SRResNet 4.164 2.457 0.713 2.355 0.741 0.430OISR 4.126 2.134 0.421 2.318 0.683 0.246NODE 4.058 2.125 0.408 2.309 0.671 0.243
ANODE 3.967 2.043 0.351 2.246 0.635 0.217UrbanFM 3.951 2.011 0.327 2.234 0.627 0.209S-FODE 3.941 2.001 0.322 1.951 0.517 0.129FODE 3.860 1.963 0.313 1.916 0.512 0.115
Table 2: Performance comparisons on TaxiBJ and BikeNYC.
3.2 ResultsOverall performance. Table 2 reports the FSR results of allthe methods, demonstrating that FODE and its variant outper-form all baselines on all metrics across both datasets. Tak-ing TaxiBJ for example, FODE yields 14.9%, 19.1%, and41.6% improvement on average in terms of RMSE, MAE,and MAPE, respectively. In a relatively smaller area of NYC,FODE achieves larger improvement for inferring the bicycleflow. The performance gain of FODE over baselines demon-strates the effectiveness of continuous-time ODEs, whichprovides an alternative view of improving fine-grained flowinference.
Comparison analysis. Image SR methods are usually notcomparable even withN2 normalization, due to the structuralconstraints and the influence of external factors in FSR appli-cation. This implies that FSR requires specific model designthat seamlessly taking constraints and factors into account.As a specifically tailored FSR model, UrbanFM achievesbest performance among baselines. However, as a ResNet-based model, it models the urban flow in a discrete mannerby stacking deep neural networks, which could be problem-atic since urban flow inherently can be viewed as a contin-uous dynamic system. OISR, NODE and ANODE are nu-merical methods tailored for FSR. OISR uses RK-block asthe network structure, which suffers from a huge number ofparameters and numerical errors that significantly affect theperformance. Additionally, it requires a significant amountof memory for storing intermediate quantities during back-propagation (cf. Table 3). NODE, in contrast, uses adjointmethod for solving ODEs, which only needs O(L) memoryas FODE (note that L = 1 in FODE), while ANODE requiresO(L)+O(N ) memory. Nevertheless, the dynamics of eitherthe hidden state or the adjoint might be unstable, which incursinaccurate gradient computation, as we analyzed in previoussection. The performance gain of FODE over these numeri-cal methods indicates that our method estimates the gradientsmore accurately, due to the introduced affine coupling layerwhile incurring no memory cost.
Memory efficiency. In addition to FSR performance,FODE has a significant memory and parameter efficiency,compared to ResNet-based models. Table 3 outlines thememory cost and parameters required for different methods(we omit other image SR methods due to the similar archi-
Method C #Params (M) memory timeSRResNet 128 5.5 O(LH) O(LH)OISR 128 15.7 O(LH) O(LH)
NODE 128 2.1 O(L) O(N )
ANODE 128 2.4 O(L)+O(N ) O(N )UrbanFM 128 6.2 O(LH) O(LH)
S-FODE 64 0.7 O(L) O(N )
FODE 128 2.1 O(L) O(N )
Table 3: Comparisons of parameters and memory cost. C: the num-ber of channels; L(L): the number of ResNet (ODE blocks); H:the number of layers in each ResNet; N : the number of functionevaluations.
tectures as SRResNet). In particular, FODE requires only1/3× parameters of UrbanFM and reduces the memory costto O(1). It is worthwhile to note that a simplified versionS-FODE contains only 0.7M parameters while still outper-forming all the baseline methods on FSR task.
Ability of factor fusion. External factors play importantroles and should be carefully considered in FSR models. Fig-ure 6(a) and 6(b) compare the influence of factors learned byUrbanFM and FODE, respectively. UrbanFM uses a FCN tofuse external factors, which is too weak to correlate complexfactors with flow distributions. For example, the impact offactors concentrates in a smaller area for UrbanFM – i.e., twomain roads are more affected by external factors than otherregions. In contrast, FODE estimates the influence of factorsby evenly distributing the external influences and is there-fore more robust. This can be further verified by the resultsshown in Figure 6(c) and Figure 6(d), where we observe thatFODE consistently converges while UrbanFM, surprisingly,achieves best performance using temperature only, rather thanall the factors.
Error analysis. Figure 7(a) shows the inference errors ofUrbanFM and FODE on a data sample, where a brighter pixelindicates a larger error. To better visualize the quality of in-ference, we select four subregions (A, B, C, and D), fromwhich we clearly see that the flow inference by FODE per-forms better than UrbanFM in crowded areas. Similarly, Fig-ure 7(b) depicts the overall inference error using two differ-ent normalization schemes. We observe that in most areas,especially in E, F, G subregions, the distribution generatedby AN2-normalization is closer to the ground-truth, whichdemonstrates that the proposed AN2 method is a more effec-tive way of constraining the flow distributions. This improve-ment can be attributed to jointly modeling the distribution ofexternal factors and urban flows inAN2, compared to simplyconstraining the flows in subregions in N2-normalization.
Accuracy vs. efficiency. Another merit of FODE is that itallows to balance the trade off between the flow inference ac-curacy and the computational overhead, by varying the num-ber of function evaluations N . As shown in Figure 8, themore function evaluated in the forward pass, the lower theMSE loss. Accordingly, the time required for training themodel increases linearly with the number of function evalua-tions. This result is important since downstream applications
Proceedings of the Twenty-Ninth International Joint Conference on Artificial Intelligence (IJCAI-20)
1300
(a) UrbanFM. (b) FODE.
0 100 200 300 400 500#Validations
4.0
4.1
4.2
4.3
4.4
4.5
Valid
atio
n RM
SE
UrbanFM-without factorsUrbanFM-wind speedUrbanFM-temperatureUrbanFM-holidayUrbanFM-with factors
(c) UrbanFM fusion capability.
0 200 400 600 800 1000#Validations
3.9
4.0
4.1
4.2
4.3
4.4
4.5
Valid
atio
n RM
SE
UrbanFM-without factorsUrbanFM-with factorsFODE-without factorsFODE-with factors
(d) FODE fusion capability.
Figure 6: Analysis of external factor fusion.
A
B
C
UrbanFM FODE
D
(a) Comparison of inference errors.
-Normalization -NormalizationAN2<latexit sha1_base64="0JmL0Yvl2X9MoY3pbru44DKSkz8=">AAAB63icbVBNSwMxEJ2tX7V+VT16CRbBU9ktQvVW8eJJKtgPaNeSTbNtaJJdkqxQlv4FLx4U8eof8ua/MdvuQVsfDDzem2FmXhBzpo3rfjuFtfWNza3idmlnd2//oHx41NZRoghtkYhHqhtgTTmTtGWY4bQbK4pFwGknmNxkfueJKs0i+WCmMfUFHkkWMoJNJl3fPdYG5YpbdedAq8TLSQVyNAflr/4wIomg0hCOte55bmz8FCvDCKezUj/RNMZkgke0Z6nEgmo/nd86Q2dWGaIwUrakQXP190SKhdZTEdhOgc1YL3uZ+J/XS0x46adMxomhkiwWhQlHJkLZ42jIFCWGTy3BRDF7KyJjrDAxNp6SDcFbfnmVtGtVz6169xeVxlUeRxFO4BTOwYM6NOAWmtACAmN4hld4c4Tz4rw7H4vWgpPPHMMfOJ8/US2NuA==</latexit><latexit sha1_base64="0JmL0Yvl2X9MoY3pbru44DKSkz8=">AAAB63icbVBNSwMxEJ2tX7V+VT16CRbBU9ktQvVW8eJJKtgPaNeSTbNtaJJdkqxQlv4FLx4U8eof8ua/MdvuQVsfDDzem2FmXhBzpo3rfjuFtfWNza3idmlnd2//oHx41NZRoghtkYhHqhtgTTmTtGWY4bQbK4pFwGknmNxkfueJKs0i+WCmMfUFHkkWMoJNJl3fPdYG5YpbdedAq8TLSQVyNAflr/4wIomg0hCOte55bmz8FCvDCKezUj/RNMZkgke0Z6nEgmo/nd86Q2dWGaIwUrakQXP190SKhdZTEdhOgc1YL3uZ+J/XS0x46adMxomhkiwWhQlHJkLZ42jIFCWGTy3BRDF7KyJjrDAxNp6SDcFbfnmVtGtVz6169xeVxlUeRxFO4BTOwYM6NOAWmtACAmN4hld4c4Tz4rw7H4vWgpPPHMMfOJ8/US2NuA==</latexit><latexit sha1_base64="0JmL0Yvl2X9MoY3pbru44DKSkz8=">AAAB63icbVBNSwMxEJ2tX7V+VT16CRbBU9ktQvVW8eJJKtgPaNeSTbNtaJJdkqxQlv4FLx4U8eof8ua/MdvuQVsfDDzem2FmXhBzpo3rfjuFtfWNza3idmlnd2//oHx41NZRoghtkYhHqhtgTTmTtGWY4bQbK4pFwGknmNxkfueJKs0i+WCmMfUFHkkWMoJNJl3fPdYG5YpbdedAq8TLSQVyNAflr/4wIomg0hCOte55bmz8FCvDCKezUj/RNMZkgke0Z6nEgmo/nd86Q2dWGaIwUrakQXP190SKhdZTEdhOgc1YL3uZ+J/XS0x46adMxomhkiwWhQlHJkLZ42jIFCWGTy3BRDF7KyJjrDAxNp6SDcFbfnmVtGtVz6169xeVxlUeRxFO4BTOwYM6NOAWmtACAmN4hld4c4Tz4rw7H4vWgpPPHMMfOJ8/US2NuA==</latexit><latexit sha1_base64="0JmL0Yvl2X9MoY3pbru44DKSkz8=">AAAB63icbVBNSwMxEJ2tX7V+VT16CRbBU9ktQvVW8eJJKtgPaNeSTbNtaJJdkqxQlv4FLx4U8eof8ua/MdvuQVsfDDzem2FmXhBzpo3rfjuFtfWNza3idmlnd2//oHx41NZRoghtkYhHqhtgTTmTtGWY4bQbK4pFwGknmNxkfueJKs0i+WCmMfUFHkkWMoJNJl3fPdYG5YpbdedAq8TLSQVyNAflr/4wIomg0hCOte55bmz8FCvDCKezUj/RNMZkgke0Z6nEgmo/nd86Q2dWGaIwUrakQXP190SKhdZTEdhOgc1YL3uZ+J/XS0x46adMxomhkiwWhQlHJkLZ42jIFCWGTy3BRDF7KyJjrDAxNp6SDcFbfnmVtGtVz6169xeVxlUeRxFO4BTOwYM6NOAWmtACAmN4hld4c4Tz4rw7H4vWgpPPHMMfOJ8/US2NuA==</latexit>
N2<latexit sha1_base64="ukUDUCkHap/9wQwm4AFJq9ukS2w=">AAAB6nicbVBNS8NAEJ34WetX1aOXxSJ4KkkR9CQFL56kov2ANpbNdtIu3WzC7kYooT/BiwdFvPqLvPlv3LY5aOuDgcd7M8zMCxLBtXHdb2dldW19Y7OwVdze2d3bLx0cNnWcKoYNFotYtQOqUXCJDcONwHaikEaBwFYwup76rSdUmsfywYwT9CM6kDzkjBor3d8+VnulsltxZyDLxMtJGXLUe6Wvbj9maYTSMEG17nhuYvyMKsOZwEmxm2pMKBvRAXYslTRC7WezUyfk1Cp9EsbKljRkpv6eyGik9TgKbGdEzVAvelPxP6+TmvDSz7hMUoOSzReFqSAmJtO/SZ8rZEaMLaFMcXsrYUOqKDM2naINwVt8eZk0qxXPrXh35+XaVR5HAY7hBM7AgwuowQ3UoQEMBvAMr/DmCOfFeXc+5q0rTj5zBH/gfP4AynCNcg==</latexit><latexit sha1_base64="ukUDUCkHap/9wQwm4AFJq9ukS2w=">AAAB6nicbVBNS8NAEJ34WetX1aOXxSJ4KkkR9CQFL56kov2ANpbNdtIu3WzC7kYooT/BiwdFvPqLvPlv3LY5aOuDgcd7M8zMCxLBtXHdb2dldW19Y7OwVdze2d3bLx0cNnWcKoYNFotYtQOqUXCJDcONwHaikEaBwFYwup76rSdUmsfywYwT9CM6kDzkjBor3d8+VnulsltxZyDLxMtJGXLUe6Wvbj9maYTSMEG17nhuYvyMKsOZwEmxm2pMKBvRAXYslTRC7WezUyfk1Cp9EsbKljRkpv6eyGik9TgKbGdEzVAvelPxP6+TmvDSz7hMUoOSzReFqSAmJtO/SZ8rZEaMLaFMcXsrYUOqKDM2naINwVt8eZk0qxXPrXh35+XaVR5HAY7hBM7AgwuowQ3UoQEMBvAMr/DmCOfFeXc+5q0rTj5zBH/gfP4AynCNcg==</latexit><latexit sha1_base64="ukUDUCkHap/9wQwm4AFJq9ukS2w=">AAAB6nicbVBNS8NAEJ34WetX1aOXxSJ4KkkR9CQFL56kov2ANpbNdtIu3WzC7kYooT/BiwdFvPqLvPlv3LY5aOuDgcd7M8zMCxLBtXHdb2dldW19Y7OwVdze2d3bLx0cNnWcKoYNFotYtQOqUXCJDcONwHaikEaBwFYwup76rSdUmsfywYwT9CM6kDzkjBor3d8+VnulsltxZyDLxMtJGXLUe6Wvbj9maYTSMEG17nhuYvyMKsOZwEmxm2pMKBvRAXYslTRC7WezUyfk1Cp9EsbKljRkpv6eyGik9TgKbGdEzVAvelPxP6+TmvDSz7hMUoOSzReFqSAmJtO/SZ8rZEaMLaFMcXsrYUOqKDM2naINwVt8eZk0qxXPrXh35+XaVR5HAY7hBM7AgwuowQ3UoQEMBvAMr/DmCOfFeXc+5q0rTj5zBH/gfP4AynCNcg==</latexit><latexit sha1_base64="ukUDUCkHap/9wQwm4AFJq9ukS2w=">AAAB6nicbVBNS8NAEJ34WetX1aOXxSJ4KkkR9CQFL56kov2ANpbNdtIu3WzC7kYooT/BiwdFvPqLvPlv3LY5aOuDgcd7M8zMCxLBtXHdb2dldW19Y7OwVdze2d3bLx0cNnWcKoYNFotYtQOqUXCJDcONwHaikEaBwFYwup76rSdUmsfywYwT9CM6kDzkjBor3d8+VnulsltxZyDLxMtJGXLUe6Wvbj9maYTSMEG17nhuYvyMKsOZwEmxm2pMKBvRAXYslTRC7WezUyfk1Cp9EsbKljRkpv6eyGik9TgKbGdEzVAvelPxP6+TmvDSz7hMUoOSzReFqSAmJtO/SZ8rZEaMLaFMcXsrYUOqKDM2naINwVt8eZk0qxXPrXh35+XaVR5HAY7hBM7AgwuowQ3UoQEMBvAMr/DmCOfFeXc+5q0rTj5zBH/gfP4AynCNcg==</latexit>
E
F
G
(b) N2 vs. AN2.
Figure 7: Inference error visualization.
high efficiencyhigh error
low efficiencylow error
Figure 8: Tradeoff between training time and inference accuracy.
may make flexible solutions by conciliating inference accu-racy with computational cost.
4 ConclusionWe proposed a novel method FODE for inferring fine-grainedurban flow. FODE learns urban flow distribution through anew ODEs parameterized by affine coupling neural networksalleviating the numerical instability gradient computation is-sue, which allows for both memory and model parameter sav-ings. In addition, it is capable of explicitly providing moreflexible prediction performance by adaptively balancing pre-diction accuracy and computation overheads. Furthermore,we believe that FODE is a more general ODE-based archi-tecture and can be better exploited for time series predictionor other fine-grained inference tasks such as single imageSR [Cai et al., 2019; Wang et al., 2019] and air quality in-ference [Liu et al., 2019a].
AcknowledgmentsThis work was supported by National Natural Science Foun-dation of China (Grant No.61602097) and NSF grant CNS1646107.
References[Ascher et al., 1997] Uri M Ascher, Steven J Ruuth, and
Raymond J Spiteri. Implicit-explicit runge-kutta methodsfor time-dependent partial differential equations. AppliedNumerical Mathematics, 25(2-3):151–167, 1997.
[Ba et al., 2016] Jimmy Lei Ba, Jamie Ryan Kiros, and Ge-offrey E Hinton. Layer normalization. arXiv: 1607.06450,2016.
[Butcher and Wanner, 1996] John Charles Butcher and Ger-hard Wanner. Runge-kutta methods: some historical notes.Applied Numerical Mathematics, 22(1-3):113–151, 1996.
[Cai et al., 2019] Jianrui Cai, Shuhang Gu, Radu Timofte,Lei Zhang, and et al. Ntire 2019 challenge on real imagesuper-resolution - methods and results. In CVPR Chal-lenge, pages 2211–2223, 2019.
[Chen et al., 2018] Tian Qi Chen, Yulia Rubanova, JesseBettencourt, and David K Duvenaud. Neural ordinary dif-ferential equations. In NeuIPS, pages 6572–6583, 2018.
Proceedings of the Twenty-Ninth International Joint Conference on Artificial Intelligence (IJCAI-20)
1301
[De Brouwer et al., 2019] Edward De Brouwer, Jaak Simm,Adam Arany, and Yves Moreau. Gru-ode-bayes - contin-uous modeling of sporadically-observed time series. InNeurIPS, pages 7377–7388, 2019.
[Dinh et al., 2015] Laurent Dinh, David Krueger, andYoshua Bengio. NICE: non-linear independent compo-nents estimation. In ICLR, 2015.
[Dinh et al., 2017] Laurent Dinh, Jascha Sohl-Dickstein, andSamy Bengio. Density estimation using real NVP. InICLR, 2017.
[Dong et al., 2015] Chao Dong, Chen Change Loy, KaimingHe, and Xiaoou Tang. Image super-resolution using deepconvolutional networks. TPAMI, 38(2):295–307, 2015.
[E, 2017] Weinan E. A proposal on machine learning viadynamical systems. Communications in Mathematics andStatistics, 5(1):1–11, 2017.
[Gholami et al., 2019] Amir Gholami, Kurt Keutzer, andGeorge Biros. Anode: Unconditionally accurate memory-efficient gradients for neural odes. In IJCAI, pages 730–736, 2019.
[Grathwohl et al., 2019] Will Grathwohl, Ricky T. Q. Chen,Jesse Bettencourt, Ilya Sutskever, and David Duvenaud.FFJORD: free-form continuous dynamics for scalable re-versible generative models. In ICLR, 2019.
[He et al., 2016] Kaiming He, Xiangyu Zhang, ShaoqingRen, and Jian Sun. Deep residual learning for image recog-nition. In CVPR, pages 770–778, 2016.
[He et al., 2019] Xiangyu He, Zitao Mo, Peisong Wang,Yang Liu, Mingyuan Yang, and Jian Cheng. Ode-inspirednetwork design for single image super-resolution. InCVPR, pages 1732–1741, 2019.
[Heinonen and Lahdesmaki, 2019] Markus Heinonen andHarri Lahdesmaki. ODE2VAE: Deep Generative SecondOrder ODEs with Bayesian Neural Networks. In NeurIPS,pages 13412–13421, 2019.
[Kim et al., 2016] Jiwon Kim, Jung Kwon Lee, and KyoungMu Lee. Accurate image super-resolution using verydeep convolutional networks. In CVPR, pages 1646–1654,2016.
[Kingma and Ba, 2014] Diederik P Kingma and Jimmy Ba.Adam: A method for stochastic optimization. arXiv:1412.6980, 2014.
[Kingma and Dhariwal, 2018] Durk P Kingma and PrafullaDhariwal. Glow: Generative flow with invertible 1x1 con-volutions. In NIPS, pages 10236–10245, 2018.
[Ledig et al., 2017] Christian Ledig, Lucas Theis, FerencHuszar, Caballero, and et al. Photo-realistic single imagesuper-resolution using a generative adversarial network. InCVPR, pages 105–114, 2017.
[Liang et al., 2019] Yuxuan Liang, Kun Ouyang, Lin Jing,Sijie Ruan, Ye Liu, Junbo Zhang, David S Rosenblum, andYu Zheng. Urbanfm: Inferring fine-grained urban flows. InKDD, pages 3132–3142, 2019.
[Liu et al., 2019a] Ning Liu, Rui Ma, Yue Wang, and LinZhang. Inferring fine-grained air pollution map via a spa-tiotemporal super-resolution scheme. In UbiqComp, pages498–504, 2019.
[Liu et al., 2019b] Xuanqing Liu, Tesi Xiao, Si Si, Qin Cao,Sanjiv Kumar, and Cho-Jui Hsieh. Neural SDE: Stabi-lizing neural ode networks with stochastic noise. arXiv:1906.02355v1, 2019.
[Lu et al., 2018] Yiping Lu, Aoxiao Zhong, Quanzheng Li,and Bin Dong. Beyond finite layer neural networks: Bridg-ing deep architectures and numerical differential equa-tions. In ICML, pages 3282–3291, 2018.
[Poli et al., 2019] Michael Poli, Stefano Massaroli, Jun-young Park, Atsushi Yamashita, Hajime Asama, andJinkyoo Park. Graph neural ordinary differential equa-tions. arXiv: 1911.07532v1, 2019.
[Rubanova et al., 2019] Yulia Rubanova, Ricky T.Q. Chen,and David Duvenaud. Latent ordinary differential equa-tions for irregularly-sampled time series. In NeurIPS,pages 5321–5331, 2019.
[Shi et al., 2016] Wenzhe Shi, Jose Caballero, FerencHuszar, Johannes Totz, Andrew P Aitken, Rob Bishop,Daniel Rueckert, and Zehan Wang. Real-time single im-age and video super-resolution using an efficient sub-pixelconvolutional neural network. In CVPR, pages 1874–1883, 2016.
[Wang et al., 2019] Zhihao Wang, Jian Chen, and Steven CHHoi. Deep learning for image super-resolution: A survey.arXiv:1902.06068, 2019.
[Zhang et al., 2017] Junbo Zhang, Yu Zheng, and DekangQi. Deep spatio-temporal residual networks for citywidecrowd flows prediction. In AAAI, pages 1655–1661, 2017.
[Zhang et al., 2018] Yulun Zhang, Yapeng Tian, Yu Kong,Bineng Zhong, and Yun Fu. Residual dense network forimage super-resolution. In CVPR, pages 2472–2481, 2018.
[Zhang et al., 2019] Tianjun Zhang, Zhewei Yao, Amir Gho-lami, Kurt Keutzer, Joseph Gonzalez, George Biros, andMichael Mahoney. Anodev2: A coupled neural ode evolu-tion framework. In NeurIPS, 2019.
Proceedings of the Twenty-Ninth International Joint Conference on Artificial Intelligence (IJCAI-20)
1302
Related Documents
