-
The Solution of a FEM Equation in Frequency Domain
Using a Parallel Computing with CUBLAS
R. Dominguez1, A. Medina1, and A. Ramos-Paz1 1Facultad de
Ingeniería Eléctrica, División de Estudios de Posgrado, U.M.S.N.H.,
Ciudad Universitaria, C.P.
58030, Morelia, Michoacán, MEXICO.
Abstract - The recent technological computer advances have
allowed the use of the Finite Element Method (FEM), to
calculate the solution of the Maxwell field equations of
electrical machines or devices. In some cases, an
axisymmetric or a plane symmetry can be assumed to reduce
the complexity of the finite element analysis to be
performed.
Nevertheless, the large size of the matrix equations
derived,
could imply a significant computing effort. In this paper, a
parallel method of solution in frequency domain of a FEM
equation with currents known is proposed. It consists on
implementing the LU method using a parallel computing with
CUBLAS. A normal and a reduced type of FEM equation
proposed by the authors have been solved in the frequency
domain using this parallel computing platform. It is shown
that a significant reduction in the computing time to solve
these FEM equations in the frequency domain is achieved.
Keywords: Finite element method, frequency domain
analysis, parallel processing
1 Introduction
The Finite Element Method (FEM) is a very powerful
tool to solve the electric and magnetic equations of
electrical
machines or devices. The method has been widely used,
since the computational technological advances have
allowed the application of the method on the modeling and
simulation of electrical machines or devices with complex
geometries of configurations [1]-[3].
Nevertheless, the method can be difficult to use in
devices with 3D geometries or in those which need a detailed
geometry model; the reason is the large matrix equations
derived by the finite element analysis, which in turn can be
difficult to solve in the frequency domain or in the time
domain. However, the finite element analysis can be
simplified if a planar or axisymmetric assumption is taking
into account [2], [3].
In an earlier paper, the authors proposed a new form to
solve a FEM equation with currents or voltages known [4].
The method consists on deriving a lesser order equation from
a normal FEM equation. The reduced equivalent equation
obtained is expressed in terms of the time varying
variables,
and it can be easily solved in time domain or in the
frequency
domain [3]. The reduced equation can be calculated from a
normal FEM equation derived from of a finite element
analysis performed on a device with a planar or an
axisymmetric symmetry [4]. The reduced equation is easy to
derive and solve, since it implies the use of simple matrix
operations [4]. These matrix operations can be derived by a
parallel computing. Moreover, the normal and the reduced
FEM equations can be solved in the frequency domain by a
parallel solution. Thus, it is possible to obtain a
significant
computation time reduction. The FEM equations to be
solved correspond to equations that model a device with a
planar or axisymmetric symmetry, and whose conductor
currents are known.
In this paper, the LU method has been implemented in
the CUBLAS parallel platform, in order to solve normal and
reduced FEM equations in the frequency domain.
Specifically, an LU decomposition process was implemented
using parallel processing using routines of the CUBLAS
library. The proposed parallel solution has been tested in
two
devices: a planar conductor and a series reactor with an
axisymmetric symmetry assumption.
The rest of the paper is organized as follows: Section 2
explains the features of the partial differential equations
of
devices modelled by planar or the axisymmetric symmetries.
Section 3 explains the features of the FEM matrix equations,
derived from a finite element analysis performed with the
partial differential equations shown in Section 2. Section 4
explains how the normal and the reduced FEM matrix are
solved in the frequency domain; Section 5 describes how
these equations are solved using the CUBLAS computing
platform; Section 6 describes a case study which consists of
two devices in which the parallel solution has been tested:
the first device is a “T” conductor modelled by a planar
symmetry and the second device is an air series reactor
modelled by an axisymmetric symmetry. Finally, Section 7
contains the main conclusion drawn from this investigation.
2 Partial Differential Equations of a Device with Planar or
Axisymmetric
Symmetries
This investigation is based on the following
assumptions: the frequency of the voltage source of the
device to be modelled is low enough to neglect the
-
displacement current in the Maxwell field equations [2],
[3],
[5]. The permeability and the conductivity of the device are
assumed to be constant. Finally, there are no voltage
difference at different conductor points [5].
In some cases, the modelling of a device can be
simplified by a planar or axisymmetric symmetries [2], [3].
If it is considered that a skin effect exists on the
conductors
of the devices, and that these conductors are excited by
voltage sources, then the partial differential equations for
a
device with a planar or an axisymmetric symmetries are
given by [5]. [6],
�
�� �� ������ � − ��� �� ������ � + ����� = �{��}� (1)
− �� ��� �� ������ � − �� ��� �� ������ � + ��� �� + ����� =
{��}��� (2)
Where Az and Aϕ are the magnetic vector potential of a
device with a planar or an axisymmetric symmetry
assumption, respectively; σ and v are the conductivity and
reluctivity of the materials, respectively. {Uc} is a vector
which contains the voltages applied at the conductors of the
device. If it is considered that the voltages along the
z-axis
are constant for a planar symmetry; and that the voltages
along the ϕ-axis are constant for an axisymmetric symmetry;
then it is possible to derive an equation to relate the
voltage,
current and the magnetic vector potentials at the conductors
of the device [5], [6]. The equation is given by [5], [6],
[!�]#${%&} − ∬ ���� ()*� = {+} (3)
Where {I} is a vector that contains the conductors’
current. The matrix [Δx] for the planar and the axisymmetric
symmetry is defined by the equations (4) and (5),
respectively.
[!�] = -��� �∬ .*�*� �#$/.012 (4)
s
[!�] = [3&] = - �� �∬ ()*� �#$/.012 (5)
Where [Rc] is the conductor matrix resistance if the
device has a planar symmetry assumption. The surface area
Sc of the equations (4) and (5) varies if the device is
modeled
by a planar or an axisymmetric symmetry. For the case of the
planar symmetry, the surface area Sc involves the plane x-y
[5]. For an axisymmetric symmetry, the surface area
involves the plane r-z [6].
3 Finite Element Analysis of the Device
It is possible to perform a finite element analysis on the
partial differential equations defined on (1) and (2). At
the
same time, a Newton Cotes analysis can be performed on the
expression defined in (3). It yields [5], [6],
[)]{��} + [4] .{�5}.� = {6}{%&} (6)
[!�]#${%&} − [7&] .{�5}.� = {+} (7)
Where the matrices [S], [T], [Mc] and the vector {f} are
obtained from the finite element analysis performed for a
planar or axisymmetric symmetry [5], [6]. The vector {I}
contains the currents in the conductors of the device, Ax is
defined in the z-axis and the ϕ-axis for the planar and the
axisymmetric symmetry, respectively.
If the conductor currents in {I} are known, it is possible
to calculate the magnetic vector potentials {Ax} and the
conductor voltages {Uc}. This can be achieved by coupling
the equations (6) and (7) in a unique equation that can be
easily solved in the frequency domain [3], [7]. It gives,
8-[)] −{6}0 [!�]#$/ + :(2=6) -[4] 0−[7&] 0/? @
A�B�CA%D&CE = F
0{+B}G (8)
Where the vector of magnetic potentials {�B�}, the conductor
voltages {%D&} and the conductor currents {+B} are all harmonic
variables defined for frequency f. The equation
(8) can be represented as, ([H] + :(2=6)[I]){JK} = A6BC (9)
Moreover, (9) can be represented in a simpler way, i.e. [�]{JK}
= {LK} (10)
The equation (10) is a normal FEM matrix expression.
It is possible to derive a simpler equation from (10) [4].
This
reduced equation allows to express (10) in terms of its time
varying variables, e.g. the vector of magnetic potentials of
the conductors [4]. The equation is of lesser order than (9)
and can be also solved in the frequency domain. The
reduced equation can be represented by,
[�M]{JKM} = {LKM} (11)
The equations shown in (10) and (11) have a
preprocessing step, where their matrices are formed by a
finite element analysis, and by a calculating step in which
their solution in the frequency domain is derived. These
stages will be discussed next.
4 Solution of the Normal and the Reduced FEM Equations in
the
Frequency Domain
The FEM matrix equation to be solved are the normal
(10) and the reduced types (11). For both equations can be
recognize two specific steps in the process of calculating
-
their solution in the frequency domain, i.e. a preprocessing
and a calculating steps, respectively. These stages will be
explained next.
4.1 Preprocessing Step of the FEM Equations
The preprocessing step of the normal FEM method
consists on deriving the final matrices [K] and [G] and the
vector {f} of the (10). The process consists on first
calculating the FEM matrices and vectors of one finite
element, integrate them into the global matrices and vectors
that model the device [2], [3] and apply the required
boundary conditions.
The preprocessing step of the reduced FEM method
consists on deriving sub-matrices and sub-vectors from the
final matrices and vectors obtained from the preprocessing
step of the normal FEM equation, in order to calculate
matrices of lesser order [4]. These FEM matrices permit to
formulate a FEM equation of lesser order, which allows to
directly solve the time varying variables of the device. The
preprocessing step of a normal and a reduced FEM
equations can be seen in Fig. 1.
Fig. 1 Preprocessing steps of the FEM equations
a) Preprocessing step of the normal FEM equation
b) Preprocessing step of the reduced FEM equation
4.2 Calculating Step of the FEM Equations
Once the matrices and vector of the normal and the
reduced FEM equations are calculated, it is possible to
derive their solution in the frequency domain. The normal
and the reduced equations have the form of the expressions
previously defined in (10) and (11), respectively.
It can be seen that these FEM equations have the form of the
expression N�2OAPQ2C = ALK2C. This matrix equation can be solved
by using the LU method.
The calculating process of the normal and the reduced
FEM equations is performed using the LU method. Thus, the
first step consists on performing a decomposition of the
matrix [Ag] into two matrices [Lg] and [Ug], respectively.
It
yields,
N�2O = NR2ON%2O (12)
After having the matrices [Lg] and [Ug], the solution of
[Ag]{xg}={bg} can be achieved by triangular decomposition
LU; and the normal and reduced FEM equations can be
solved. The difference between these equations is the
preprocessing step and the order of the FEM matrix equation
to be solved by the calculating step.
5 Calculating Process implemented by a Parallel Computing in
CUBLAS
The calculating process for the normal and the reduced
FEM matrix equations are implemented in the CUBLAS
computing platform. Some steps of the preprocessing
process of the reduced FEM equation can also be
implemented by parallel computing. This will be explained
next.
5.1 Decomposition LU implemented in
CUBLAS
Once the complex matrix equation [�]{PQ} = {LK}, that
corresponds to the normal or the reduced FEM equation, has
been formulated, the matrix [A] will be decomposed into the
product of matrices [L] and [U]. This can be achieved by
using the standard LU decomposition process. This process
implies to calculate a pivot located in the main diagonal of
[A], performing a modification of the next rows and,
finally,
eliminating the rows using the Gauss eliminating process.
The decomposition process was implemented by a parallel
computing in CUBLAS. This process is shown in Fig. 2.
-
Fig. 2 Decomposition process implemented in CUBLAS
The CUBLAS routines used for the parallel
computation ot the LU decomposition, correspond to
matrices and vectors composed of single precission complex
numbers [8]. Once the matrix [Ag] is decomposed int the
product of [Lg] and [Ug], the equation N�2OAPQ2C = ALK2C can be
easily solved. This will be explained next.
5.2 Final Solution achieved by CUBLAS
After having the matrixes [Lg] and [Ug], the solution APQ2C can
be calculated by solving the next equations in the CUBLAS computing
platform,
NR2OASQ2C = ALK2C (13)
N%2OAPQ2C = ASQ2C (14)
Equation (13) is solved by using the routine
cublasCstrv, and specifying that the equation to be solved
corresponds to a triangular matrix stored in lower mode [8];
while (14) is also solved using the routine, but specifying
that the equation to be solved corresponds to a triangular
matrix stored in upper mode [8]. It can be seen that the
solution of the complex equation N�2OAPQ2C = ALK2C can be easily
derived by implementing the LU method by a parallel
computing in CUBLAS. The results and the performance of
this method of solution were tested for the case study
described next.
6 Case Study
It consists on analyzing in the frequency domain two
devices modelled by a planar and the axisymmetric
symmetry assumption. The first device to be analyzed is a
“T” planar conductor. The second device is an air series
reactor that can be modelled by an axisymmetric symmetry
assumption. The finite element analysis to be performed on
these devices involves the solution of the normal and the
reduced FEM equations, which have the form of the
expressions shown in (10) and (11), respectively. These
equations will be solved in a sequential and a parallel
computing platform.
6.1 Device modelled by a Planar Symmetry
Assumption
It consists on analyzing a “T” slot-embedded conductor
with a copper conductor and an air region in a frequency
range of 5Hz to 60Hz with a frequency step of 5Hz. The
objective of the example is to analyze how the total source
current density Jct of the conductor varies in this
frequency
range [5]. The source density Jct will be obtained via the
calculating process shown in Fig. 3 (b). The FEM model and
the geometry of the “T” conductor is shown in Fig. 3(a).
Fig. 3 Device with a planar symmetry assumption
a) Geometry and FEM model
b) Calculating process of the device
6.2 Device modelled by an Axisymmetric
Symmetry Assumption
It consists on analyzing in the frequency domain, a
small air-cored reactor [6]. The example consists on finding
how the reactor inductance ratio (RL=Lca/Lcd) varies within
a
-
frequency range [6], defined from 20Hz to 1000Hz with a
frequency step of 20Hz. Lca is defined as the inductance
obtained at a specific frequency; and Lcd is the inductance
in
a near to zero frequency. Here, the inductance ratio will be
obtained via the calculating process shown in Fig. 4(b). The
FEM model and the geometry of the series reactor is shown
in Fig. 4(a).
Fig. 4 Device with an axisymmetric symmetry assumption
a) Geometry and FEM model
b) Calculating process of the device
6.3 Methods of Solution of the FEM equations
The two devices will be solved by the normal and the
reduced FEM equations, which have the form of the
expressions defined in (10) and (11), respectively. The
dimensions and features of both, normal and reduced FEM
equations, are listed in Table I. Please notice that the FEM
equations of each device are required to be solved several
times for the respective frequency range.
Table I. FEM equations to be solved in a frequency range
Device
analyzed
No.
FEM
Eqs
Normal
FEM equation
Reduced
FEM equation
Planar
symmetry 14
[��TT]{PQ�TT} = ALK�TTC N�M,�VWOAPQM,�VWC = ALKM,�VWC
Axisym.
symmetry 51
[�XW�V]{PQXW�V} = ALKXW�VC N�M,$�YVOAPQM,$�YVC = ALKM,$�YVC
In order to measure the performance of the method
implemented in CUBLAS, the normal and the reduced FEM
equations were also solved in a sequential computing
platform. Specifically, the LU routines included in the GSL
computing platform [9]. In the sequential form of the
solution, the preprocessing and the calculating steps were
entirely implemented in the GSL platform [9]. For the
parallel solution, some stages of the preprocessing step
were
calculated by a sequential computing in GSL [9], while the
calculating steps were completely implemented in the
CUBLAS computing platform [8]. Thus, the calculating step
of the normal and the reduced FEM equation will be solved
for each frequency by the LU method implemented in the
CUBLAS.
Table II and III describe the specific routines that are
used for the sequential and the parallel solutions of the
normal and the reduced FEM equations, respectively.
Table II. Routines used in the sequential form of solution
of the FEM Equations
Stage Normal FEM
Equation
Reduced FEM
Equation
Preprocessing Step
Normal
preprocessing step C routines, GSL matrix routines
Deriving
Submatrixes for the
reduced equation
Not applied GSL matrix
routines
Calculating final
matrixes for the
reduced equation
Not applied gsl_blas_dgemm
gsl_blas_dgmev
Calculating Step
Forming equation [�]{PQ} = {LK} GSL matrix routines LU
Decomposition [�] = [R][%] gsl_linalg_complex_LU_decomp Solving
equation [�]{PQ} = {LK} gsl_linalg_complex_LU_solve
-
Table III. Routines used in the parallel form of solution of
the FEM equations
Stage Normal FEM
Equation
Reduced FEM
Equation
Preprocessing Step
Normal
preprocessing step C routines, GSL matrix routines
Deriving
Submatrixes for the
reduced equation
Not applied GSL matrix
routines
Calculating final
matrixes for the
reduced equation
Not applied
(Matrix inverse
calculated using
routine defined
in [10])
cublasSgemm
cublasSgemv
Calculating Step
Forming equation [�]{PQ} = {LK} CUBLAS matrix routines LU
Decomposition [�] = [R][%] See Fig. 2
Solving equation [�]{PQ} = {LK} cublasCtsv:
([R]{SQ} = {LK}) ([%]{PQ} = {SQ})
The computing times obtained from solving the normal
and the reduced FEM equations in the sequential and the
parallel form of solution, it will be shown in the next
section.
6.4 Results and Performance Comparison
It is important to mention that the results obtained from
the solution of the planar and axisymmetric problems, were
validated and compared against simulations performed with
ANSYS in the frequency domain. The results derived by the
normal and the reduced FEM equations are accurate and
validate the proposed parallel form of solution of both
equations.
The normal and the reduced FEM equations were solved
in the computing platforms GSL and CUBLAS. The programs
were implemented in the same computer and operative
system. A Dell Precision R5500 Rack Workstation, GPU
NVIDIA® Quadro® 600, 1 GB RAM and an Ubuntu
Operative System were used.
The total computation time (CPU time) required to
solve the devices with planar and axisymmetric symmetries
in the correspondent frequency range was measured. Fig. 5
illustrates the CPU times needed to solve these equations
using the sequential and the parallel computing platforms.
Fig. 5. CPU times derived for the FEM equations solutions
a) CPU time derived for the planar device
b) CPU time derived for the axysimmetric device
For the case of the device with a planar symmetry, it
can be observed that the reduced FEM equation allows to
derive a faster solution compared to the normal FEM
equation solution. Specifically, when the sequential
computing was used, the CPU time of the normal and the
reduced equation are 1.92sec and 0.89sec, respectively.
Moreover, when the parallel computing was used, the CPU
time of the normal and the reduced equation are 6.36sec and
0.90sec, respectively. Although the reduced FEM equation
allows a faster solution with both computing platforms to be
achieved, a reduction of CPU time was not obtained when
parallel computing with CUBLAS was used. The reason
being is that the reduced and the normal equations of the
planar device are of low order, i.e. 205 and 266,
respectively.
A CPU time reduction cannot be achieved, since the
advantage of using the parallel platform is only evident
when
the size of the equations to be solved is really huge.
For the specific case of the device with an
axisymmetric symmetry, it can be observed that the reduced
FEM equation also permits to derive a faster solution
compared to the normal FEM equation solution. For
example, for sequential processing, the CPU time of the
normal and the reduced equation are 15298.31sec and
-
763.89sec, respectively. Moreover, when parallel
computation was used, the CPU time of the normal and the
reduced equation were 4365.99sec and 226.48sec,
respectively. It can be seen that the parallel processing of
the
reduced FEM equation requires of only 226.48sec. The
sequential computation of a normal FEM equation requires
a CPU time of 15298.3sec. The difference between these
CPU times is really significant, nearly 6760%. The reason is
that the reduced and the normal equations of the planar
device are of higher order, i.e. 3520 and 1270,
respectively.
7 Conclusions
A method of solution of a FEM equation, using the LU
method implemented in the CUBLAS computing platform
has been proposed. It has the following advantages:
1) It can be used to solve a normal and a reduced FEM equation
that models devices that can be simplified by a
planar or an axisymmetric symmetry assumption.
2) Its solution has been compared against a sequential computing
platform. It has allowed a significant
reduction of computer effort, as compared to the
sequential solution, which was implemented by using
the LU routines included in the GSL platform.
3) It allows a significant time reduction when the reduced FEM
equation is solved. A significant reduction of CPU
time to solve larger order FEM equations sets in the
frequency domain has been obtained. The CPU time for
solving this equation using CUBLAS is 67.54 times
lesser, than the time required for solving the normal
FEM equation with GSL.
The parallel solutions of the normal and the reduced
FEM equations have been successfully tested for a case
study where a finite element analysis has been used to
analyze planar and axisymmetric devices. The results
derived by the parallel and the sequential solutions of
these
FEM equations have been against those obtained by finite
element simulations performed in ANSYS in the frequency
domain. An excellent agreement between the results
obtained with both approaches has been achieved.
A significant time reduction has been achieved with the
application of CUBLAS platform for solving the FEM
equations in the frequency domain. For the specific case of
the device modelled by an axisymmetric symmetry
assumption, it has been obtained a CPU time of 226.48s
which is a significant small time, compared with the CPU
time of 15298.31s, which was derived by the sequential
solution with GSL.
8 References
[1] X. Wang, D. Xie, “Analysis of Induction Motor Using
Field-Circuit Coupled Time-Periodic Finite Element Method
Taking Account of Hysteresis”, IEEE Transactions on
Magnetics, Vol. 45, No.3, pp. 1740-1741, March 2009.
[2] Bianchi N., Electrical Machine Analysis Using Finite Element
Method, 1st ed. Taylor & Francis Group, Boca
Raton, 2005.
[3] A. Arkkio, “Analysis of Induction motors based on the
numerical solution of the magnetic field and circuit
equations”, Doctoral thesis, Acta Polytechnica
Scandinavica, Electrical Engineering Series, no. 59,
September 1987.
[4] R. Dominguez, A. Medina, “A Novel Method for the Solution of
a Finite Element-Circuit Coupled Equation using
a Reduced Equivalent Equation”, Proceedings of the 2013
IEEE International Electric Machines and Drives
Conference, pp. 1061-1064, May 2013.
[5] A. Konrad, “Integrodifferential Finite Element Formulation
of Two-Dimensional Steady-State Skin Effect
Problems”, IEEE Transactions On Magnetics, vol. mag-18,
No.1, pp. 284-286, January 1982.
[6] K. Preis, “A Contribution to Eddy Current Calculations in
Plane and Axisymmetric Multiconductor Systems”, IEEE
Transactions On Magnetics, Vol. 19, No.6, pp. 2397-2399,
November 1983.
[7] S.L. Ho, H.L. Li, W.N. Fu, “Inclusion of Interbar Currents
in a Network-Field Coupled Time-Stepping Finite-
Element Model of Skewed-Rotor Induction Motors”, IEEE
Transactions on Magnetics, vol. 35, No.5, pp. 4218-4219,
September 1999.
[8] CUDA Toolkit 5.0 CUBLAS Library, Nvidia Corporation, 2701
San Tomas Express Way Santa Clara CA,
pp. 59-60, 2014.
[9] GNU Scientific Library, GNU Project, 2013.
[10] R. Cisneros-Magaña, A. Medina, V. Dinavahi, “Parallel
Kalman Filter Based Time-Domain Harmonic State
Estimation”, Proceedings of the North American Power
Symposium (NAPS), pp. 1-6, September 2013.