Tracking technology for optoelectronic imaging platform of ...

Tracking technology for optoelectronic imaging platform of tetheredballoon based on DGPS/INS

Y.W. WANG*1,2, Z.Y. WANG1, W.H. ZHOU2, and X.Q. HAN2

1School of Instrumentation Science & Opto−Electronics Engineering, Key Laboratory of PrecisionOpto−Mechatronics Technology, Ministry of Education, Beihang University, Xueyuan Road 37,

Beijing 100191, China2Academy of Opto−Electronics, Chinese Academy of Sciences. Automation Building,

Zhongguancun East Road 95, Haidian District, Beijing 100190, China

In this paper, a tracking method for optoelectronic imaging platform of tethered balloon based on difference global positio−ning system/inertial navigation system is presented in detail. The location and attitude information of optoelectronic imagingplatform, the azimuth and elevation angles of camera’s line of sight are used by this method to locate the ground target at thecentre point of the camera’s field of view through corresponding coordinate transformation. And then, the method makes useof the update position and attitude information to solve the theoretical point of camera’s line of sight inversely. Finally, anangle control commend will be sent to the inertially−stabilized turntable on the optoelectronic imaging platform, which willadjust its azimuth and elevation angle to make the camera’s line of sight to the ground target. A lot of experiments are con−ducted, and the results show that the initial ground target is always in the centre of camera’s field of view no matter how theballoon’s position and attitude change, and the new location data of ground target has little difference with the initiallocation data, while the difference between them is close to 0.

Keywords: tracking, optoelectronic imaging platform, DGPS/INS, tethered balloon.

1. Introduction

Tethered balloon is a lighter−than−air aircraft and has thecharacteristic of long endurance, low prices, and recoveryeasily [1]. It is widely used in the research of high−alti−tude detection, which can carry a variety of equipmentsused for communication, remote sensing, early warning,jamming, surveillance, detection and so on. The applica−tion areas of tethered balloons change with different pay−loads hung below them. For example, tethered balloonsequipped with various radar systems have been used forairborne early warning [2], tethered balloon equippedwith a communication base station has been used forcommunications and television relay [3,4], tethered bal−loon equipped with navigation equipments can be used toincrease the accuracy of navigation combined with thegeosynchronous satellite, which can also become a newindependent network of regional navigation system [5].The optoelectronic imaging platform of tethered balloonmentioned in this paper is similar to the airborne opto−electronic imaging platform. Currently, airborne opto−electronic imaging platform generally is equipped withvisible/infrared camera, laser ranging, image processingequipment, two−dimensional inertial turntable and related

accessories [6,7], which uses television or infraredtracking technology to realize the detection, identifica−tion, location and tracking of the ground targets [8,9].

The optoelectronic imaging platform of tethered bal−loon presented in this paper is mainly made up with visiblelight monitoring system, inertial navigation system (INS)system, difference global positioning system (DGPS), andrelated accessories [10]. As the system does not have thelaser range and image processing equipments, mature tar−get tracking technology cannot be used in this system. Thispaper describes a target tracking method based on triangu−lation principle and DGPS/INS system in detail. Thismethod first uses the location and attitude information ofoptoelectronic imaging platform from DGPS/INS system,the azimuth and elevation angles of camera’s line of sight(LOS) from visible light monitoring system to locate theground target at the centre point of the camera’s field ofview through corresponding coordinate transformation,and then makes use of the update position and attitude in−formation of the optoelectronic imaging platform to solvethe theoretical point of camera’s LOS inversely. Finally,the inertially−stabilized turntable of visible light monito−ring system will adjust its azimuth and elevation angle tomake the LOS of camera to the ground target according tothe theoretical point. Therefore, the tracking of a groundtarget is achieved.

242 Opto−Electron. Rev., 19, no. 2, 2011

OPTO−ELECTRONICS REVIEW 19(2), 242–248

DOI: 10.2478/s11772−011−0005−7

*e−mail: [email protected]

2. Composition of optoelectronic imagingplatform and target tracking principle

Optoelectronic imaging platform is hung at the bottom ofthe tethered balloon, while its power and major communi−cation links are provided by the main cable of the tetheredballoon. Visible light monitoring system, ground moni−toring and control system, INS system, DGPS system arethe major components used for target tracking. Visiblelight monitoring system consists of four inertially−stabi−lized turntables each equipped with a HD (High Defini−tion) camera, while the turntables can output the azimuthand elevation angles of cameras’ LOS. The INS systemconsists of three pairs of orthogonally mounted fiber−op−tic gyroscopes and quartz accelerometers with additionaltwo GPS antennas for course correction to increase theaccuracy of fiber−optic gyroscopes. The DGPS system ismade up of one ground differential base station, two pairsof wireless communication antennas and GPS antennasused for location (one pair of antennas is fixed on the op−toelectronic imaging platform, the other pair is fixed onthe ground) [10]. Figure 1 illustrates the structure of teth−ered balloon system and optoelectronic imaging platform[10].The anchorage car is the supporting equipment forthe operation of tethered balloon. The ground communi−cation control vehicle is used to get the image informa−tion from optical payloads, acquire telemetry data, and tosend control command manually to all the equipments onthe optoelectronic imaging platform.

The raw data of inertially−stabilized turntables, INS sys−tem, and DGPS system are transmitted to ground communi−cation control vehicle. Then, the target location informationat the centre point of camera’s field of view could be ob−tained through corresponding coordinate transformation.Once the position and attitude of optoelectronic imagingplatform change, a new theoretical point of LOS will be cal−culated inversely and sent to the inertially−stabilized turnta−ble in the way of angle commend, which will adjust its azi−muth angle and elevation angle to make the LOS of camerato the ground target. Then, the tracking of ground target isrealized. Data flow chart of target tracking is shown inFig. 2.

Opto−Electron. Rev., 19, no. 2, 2011 Y.W. Wang 243

Fig. 1. Structure of tethered balloon system and optoelectronic imaging platform.

Fig. 2. Data flow chart of target tracking.

2.1. Definition of coordinate system

Coordinate systems involved in target tracking algorithmare shown in Fig. 3. Specifically, definitions of coordinatesystems are as follows:� in the turntable coordinate system (c), the origin is the

intersection of inertially−stabilized turntable’s azimuthaxis and the camera’s LOS (point S in Fig. 3), the X−axisis the direction of the initial zero of inertially−stabilizedturntable’s azimuth encoder and pointing out, the Z−axisis the direction of inertially−stabilized turntable’s azi−muth rotary axis and pointing up, the Y−axis is perpen−dicular to the plane S−XcZc and meets the right−hand(RH) rule.

� in the INS coordinate system (b), X−axis, Y−axis, andZ−axis are consistent with the heading of three fiber−op−tic gyrostabilizers and meet up−right−front (URF) rule,the origin is the geometric centre of INS system (pointO in Fig. 3), the X−axis is the direction of opto−electronicimaging platform’s lateral axis and pointing right; theY−axis is the direction of opto−electronic imaging plat−form’s longitudinal axis, perpendicular to the X−axis andpointing front; the Z−axis is perpendicular to the planeO−XbYb and meets the RH rule.

� in the navigation coordinate system (n), the origin isDGPS antenna Q, the Y−axis points to true north, theZ−axis points toward the interior of the ellipsoid alongthe ellipsoid normal, the X−axis points east to completethe orthogonal, RH rectangular coordinate system [11].There is also a coordinate system at GPS antenna P,whose axes are parallel with the three axes describedabove.

2.2. Target tracking principle and coordinaterotation matrices

In the target tracking algorithm, it is needed to assume thatthe ground target M is at the plane Q−XnYn. In Fig. 3, the tar−get M is the intersection of the line L (camera’s LOS) andthe plane Q−XnYn. At the time t1, the equation of the line Lin the navigation coordinate system could be gotten by solv−ing the equation of the vector ST (a unit vector of the line L),which can be acquired by the output data of inertially−stabi−

lized turntables, INS system, and DGPS system throughcorresponding coordinate transformation. Then, the coordi−nate of the target M in the navigation coordinate system isknown. At the time t2, when the position and attitude ofoptoelectronic imaging platform change, this algorithm usesthe coordinate of the target M, the new information of INSsystem and DGPS system to calculate the theoretical azi−

muth angle and elevation angle of inertially−stabilized turn−table inversely. Figure 4 shows the conversion sequence ofthe three coordinate systems.

Rotation matrix Rcb

The four inertially−stabilized turntables and INS system arefixed on the optoelectronic imaging platform, so the rotationmatrix Rc

b from the turntable coordinate system (c) to theINS coordinate system (b) is due to the fixed angle betweenthe two coordinate systems.

Rotation matrix Rbn

The rotation matrix Rbn from the INS coordinate system (b)

to the navigation coordinate system (n) relates with the yawangle, pitch angle and roll angle of the optoelectronic imag−ing platform [12]. The corner sequence between them isyaw−pitch−roll”

Tracking technology for optoelectronic imaging platform of tethered balloon based on DGPS/INS

244 Opto−Electron. Rev., 19, no. 2, 2011 © 2011 SEP, Warsaw

Fig. 3. Schematic diagram of target tracking algorithm.

Fig. 4. Transformation sequences of coordinate systems.

R

e e e e e e e e

cb

y z y x z y z y

�� cos cos sin sin sin cos sin sin sin e e e e

e e e e e

e

x z y x

x z x z x

y

cos sin cos

cos sin cos cos sin

sin c

�� os cos sin sin sin sin cos sin cos cose e e e e e e e e ez y x z y z y x z� � y xesin

�

�

��

�

3. Acquisition of location information andtracking of static ground target

3.1. Location of static ground targetIn Fig. 3, the vector ST is a unit vector of the line L, while Sis the origin of ST, as shown in Fig. 5. The coordinates of thepoint T in the turntable coordinate system can be expressedby the azimuth angle (clockwise is positive) and elevationangle (vertical down is zero) of the turntable, that is

x

y

z

cT

cT

cT

� �

� � �

� � �

cos( ) cos

cos( ) sin

sin( )

90

90

90

� �

� �

�

, (1)

where � is the azimuth angle of inertially−stabilized turnta−ble, � is the elevation angle of inertially−stabilized turntable.The coordinate of the point T ( , , )x y zc

tct

ct is also the expres−

sion of the vector ST in the turntable coordinate system.The coordinate of the point S in the navigation coordi−

nate system could be acquired by the rotating point S fromINS coordinate system to navigation coordinate system(described by dotted arrow) and plus the coordinate of thepoint P in the navigation coordinate system, that is

x

y

z

R

x

y

z

x

yns

ns

ns

bn

bs

bs

bs

bp

b

�

�

��

�

�

�

�

��

�

� p

bp

np

np

np

z

x

y

z

�

�

��

�

�

�

�

�

��

�

�

�

��

�

, (2)

where ( , , )x y zbs

bs

bs and ( , , )x y z

bp

bp

bp are the coordinates of

the point S and the point P in the INS coordinate system (b),( , , )x y zn

pnp

np is the coordinate of the point P in the naviga−

tion coordinate system (n), ( , , )x y zns

ns

ns is the coordinate of

the point S in the navigation coordinate system (n).The expression of a direction vector in different coordi−

nate systems only involves the rotation among the coordi−nate systems, so the equation of the vector ST could be got−ten by the rotating ST from turntable coordinate system (c)

to INS coordinate system (b) and then to navigation coordi−nate system (n)

x

y

z

R R

x

y

z

nT

nT

nT

bn

cb

cT

cT

cT

�

�

��

�

�

�

�

��

�

, (3)

where ( , , )x y znT

nT

nT is the expression of the vector ST in the

navigation coordinate system (n).Then, the expression of the line L could be acquired by

the coordinate of the point S and the expression of the vectorST in the navigation coordinate system (n), which is

x x

x

y y

y

z z

z

n nS

nT

n nS

nT

n nS

nT

��

��

�, (4)

where ( , , )x y zn n n is the coordinate of any point in the nav−igation coordinate system (n).

Assuming the coordinate of the target M in the naviga−tion coordinate system is ( , , )x y zn

MnM

nM . Substitution of

( , , )x y znM

nM

nM into Eq. (4) yields

x x

x

y y

y

z z

z

nM

nS

nT

nM

nS

nT

nM

nS

nT

��

��

�. (5)

Let znM � 0, then

x x z x z

y y z y z

z

nM

nS

nS

nT

nT

nM

nS

nS

nT

nT

nM

� �

� �

� 0

. (6)

3.2. Tracking of static ground target

When the position and attitude of tethered balloon change,the point of camera’ LOS should be adjusted to ensure thatthe target M is always in the centre of camera’s view offield, which requires the direction of the vector ST and vec−tor S’M are the same, as shown in Fig. 6.


R

y r y p r y p y r y p

bn �

� �cos cos sin sin sin sin cos cos sin sin sin cos

sin cos cos sin sin cos cos sin sin cos si

r

y r y p r y p y r y� � � � n cos

cos sin sin cos cos

p r

p r p p r�

�

�

��

�

Fig. 5. Target, LOS, and turntable coordinate system.

Fig. 6. Tracking of static ground target.

In this way, save the coordinate of the target M in thenavigation coordinate system. When the balloon moves, usethe updated position and attitude information from INS sys−tem and DGPS system to calculate the new coordinates of S’then, the new LOS L’ could be acquired by

� � � �

�

�

��

�

�

�

�

��

�

L S M

x

y

z

x

y

z

nM

nM

nM

nS

nS

nS

. (7)

Rotate S’M to the turntable coordinate system inversely,while the vector S’T’ is proportional with S’M and the direc−tion of the two vectors are the same in the turntable coordi−nate system, which is

x

y

z

x

y

z

R RcT

cT

cT

cM

cM

cM

cb

b

�

�

�

� ��

�

��

�

�

�

�

��

�

� P L �. (8)

Like Eq. (1), the coordinates of T’ in the turntable coor−dinate system could be expressed as

x l

y l

z l

cT

cT

cT

�

�

�

� � � �

� � � � �

� �

cos( ) cos

cos( ) sin

90

90

� �

� �

sin( )90 � ��

. (9)

Then, the theoretical point of LOS L’ is

� � ��

�

�

��

� ��

��arctg

y

x

x

x ycM

cM

cM

cM,

,

, ,

0

180 0 cM

cM

cMx y

��

�

��

��

0

180 0 0� , ,

, (10)

� � ��

�

�

��

� � � ��

� � ��

�90

90

90arctg

z

y

cM

cM

sin ,,

, � � �� 90

. (11)

Send the theoretical point �� and �� to the inertially−sta−bilized turntable in the way of angle command, which willadjust its azimuth angle and elevation angle to track theground target.

4. Test results and analysis

Experiments are carried out at the tethered balloon test base.The test conditions are: working height of tethered balloonis 200 m (above sea level), wind speed is stable, the ca−mera’s LOS is made to the stationary ground target M byadjusting the inertially−stabilized turntable manually. Wechoose a group of data to be the initial information of thetarget M, which are � = –104.65°, � = 15.98°, yaw = 38.8°,pithc = 1.12°, roll = –2.71°, xn

p = 43.82 m, ynp = –45.18 m,

znp = 199.5 m. The coordinate of the target M in a navigation

coordinate system is (28.9495, –5.2726, 0). As the attitudeand position of the tethered balloon are always changed

with the wind direction and speed in the sky, the statisticalrange of balloon’s attitude and position in 30 minutes areshown in Table 1.

Table 1. Statistical range of balloon’s attitude and position.

Range

Yaw 127.52°

(most affected by the wind direction)

Pitch 5.3°

Roll 2.83°

East distance 20 m

North distance 20 m

Zenith distance 1 m

Use a random number to simulate the change of positionand attitude data. Figure 7 shows the changes of the LOS’stheoretical point, balloon’s position information and attitudeinformation. Figure 8 shows certain kind of correlationsbetween yaw angle and azimuth angle.

It could be found in Fig. 7(a) that the azimuth angle ofthe theoretical point of LOS varies a lot due to changes oftethered balloon’s position and attitude shown in Figs. 7(b)and 7(c) while small variation on the pitch angle of the theo−retical point of LOS. At the same time, it can be clearly seenin Fig. 8 that the changes of the balloon’s course has thelargest contribution to the azimuth angle adjustment shaft,and the change direction of the two are opposite. By usingthe theoretical point of camera’s LOS and the current bal−loon’s position and attitude information, we can get the newcoordinates of the centre point of camera’s view of field.Comparing these new coordinates with the initial coordinateof the ground target (28.9495, –5.2726, 0) in Fig. 9, it couldbe found that the difference between them is close to 0,which indicates that the camera’s LOS is always the point tothe initial ground target. In short, the tracking algorithm foroptoelectronic imaging platform of tethered balloon canachieve the tracking of static ground target no matter howthe tethered balloon’s position and attitude change.

5. Conclusions

In this paper, a target tracking method for optoelectronicimaging platform of tethered balloon based on triangulationprinciple and DGPS/INS is described in detail. The trackingalgorithm first solves the location of ground target by usingthe initial point data of camera’s LOS, the position and atti−tude data of balloon through corresponding coordinatetransformation, and then uses the updated position and atti−tude information of tethered balloon to calculate the idealpoint of camera’s LOS reversely. At last, the inertially−sta−bilized turntables on the optoelectronic imaging platform isdriven to adjust the actual point of camera’s LOS to realizethe tracking of ground target. A lot of simulation and experi−ments are conducted at tethered balloon test base. Testresults show that the initial ground target is always in the



centre of camera’s field of view no matter how the balloon’sposition and attitude change, and the new location data ofground target has little difference with the initial locationdata, while the differences between them is close to 0.

Acknowledgements

This work is supported by Shanghai World Expo Scienceand Technology Action Plan from the Ministry of Scienceand Technology of the People’s Republic of China.


Fig. 7. Changes of LOS’s theoretical point, balloon’s position and attitude information.

Fig. 8. Correlation between yaw angle and azimuth angle.

Fig. 9. New coordinates of ground target.

References

1. C. Lambert, M. Nahon, and D. Chalmers, “Study of a multi−tethered aerostat system: experimental observations andmodel validation”, AIAA J. Aircraft 43, 1182–1189 (2006).

2. A. Colozza and J.L. Dolce, “High−altitude, long−enduranceairships for coastal surveillance”, NASA Technical Report,NASA/TM−2005−213427 (2005).

3. J.P. Hirl, “Tethered telecommunications, broadcast, and mo−nitoring systems”, Lighter−than−Air Systems TechnologyConf., Palo Alto, Calif., 173–180 (1979).

4. G.M. Djuknic, J. Freidenfelds, and Y. Okunev, “Establishingwireless communications services via high−altitude aeronau−tical platforms: A concept whose time has come?”, IEEECommun. Mag. 35, 128–135 (1997).

5. T.C. Tozer and D. Grace, “High−altitude platforms for wire−less communications”, IEE Electron. Commun. Eng. J. 13,127–137 (2001).

6. “Jane's electro−optic systems 1999−2000”, Jane's Elec−tro−Optics Systems (Hardcover), Jane’s Information Group,5th edition, 1999.

7. M. Hewish, “The all−seeing eye above”, Jane's Int. DefenceRev. 35, 48–53 (2002).

8. W.M. Hu, T.N. Tan, L. Wang, and S. Maybank, “A surveyon visual surveillance of object motion and behaviours”,IEEE T. Syst., Man Cy. C 34, 334–352 (2004).

9. J. Jinfeng, “Research of infrared image target detection,identification and tracking technology”, Northwestern Poly−technical University, Xi’an, China, 2004.

10. W.H. Zhou, Y.W. Wang, X.Q. Han, and J. Yuan, “Vehicletethered aerostat optoelectronic monitoring platform systemfor Shanghai World EXPO”, Proc. SPIE, 7544, 75443R(2010).

11. J. Farrell and M. Barth, The Global Positioning System andInertial Navigation, New York, McGraw−Hill, 2003.

12. SUN Hongxing, “DGPS/INS integrated position and attitudedetermination and its application in MMS”, The School ofRemote Sensing Information Engineering, Wuhan Univer−sity, Wuhan, 2004.



Tracking technology for optoelectronic imaging platform of ...

Documents