J.N.T.U COLLEGE OF ENGINEERING KAKINADA presented by: V.DHARA M.TEJASWI II/IV B.Tech , MEC HAN CAL II/IV B.Tech , MEC HANCAL MAILID: MAIL ID: [email protected][email protected]CONTACT NO: 9248788146 CONTACT NO: 9247818348 VISION BASED SENSING AND CONTROL FOR SPACE ROBOTIC APPLCATOINS ABSTRACT: The following problems arise in the precise positioning of payloads by space manipulators: 1) The precis e measurement of the relati ve posit ion and motion of objects in the workspace of the robot; 2) The des ign of a co ntro l s yst em, whi ch is rob ust and performs well in spite of the effects of structural fle xibi lit y and osci lla tio ns typ ical ly ass ocia ted wit h space robots. Thi s pap er dis cus ses the sol uti on to the meas ur ement pr oble m by a vi si on syst em us ing pho tog rammat ic image processing to det ermine the motion of obje cts in real time. Perfo rmanc e characteristics are pre sented. The control problem is addressed by a new technique dealing effectively with t he challenge posed by the noncollocated sen sor/ act uato r conf igur atio n on the fle xibl e robo t st ru cture. Th e la bora to ry impl ementa ti on of th e meas ur ement and contro l conc ep ts is di sc us se d. Preliminary results validate the concepts. Key Terms —Artificial vision, control, measurement of motion, Photogrammetry, robotics. INTRODUCTION ROBOTIC systems will play an important role in reduc ing hazards and inc reas ing pr oduct ivity ofhum ans in space. A pr ime examp le is the Mo bil e Se rvici ng Sys te m (MSS) shown in Fi g. 1 whic h is pre se ntl y being de veloped by the Ca nad ian Space Agency for the assembly and external maintenance ofthe International Space Station (ISS) [1]. As the tasks performed by space robots become more complex, the need for more human-like characteristics emerges. As with humans, the sense of sight is essential to enabling efficient int er ac ti on wi th the env ir onment . Mo re important than the sense of sight per se is the ability to p rocess images in su ch a wa y as to enable more efficient, accurate and autonomous control of the robot. Fig. 1. Mobile Servicing System on the Internati onal Space Station. This paper addresses measurement and control pro blems asso ci at ed wit h t he precise positioning of large space robot manipulators like the Space Station Remote Manipulator System (SSRMS) shown in Fig. 1, whic h typically have a very high payl oad -t o-m ani pula tor mass ra ti o (e. g. 11 6 00 0 kg/1500 kg for SSRMS) and relatively low stiffness,
This document is posted to help you gain knowledge. Please leave a comment to let me know what you think about it! Share it to your friends and learn new things together.
ROBOTIC systems will play an important rolein reducing hazards and increasing productivity of
humans in space. A prime example is the Mobile
Servicing System (MSS) shown in Fig. 1 which is presently being developed by the Canadian Space
Agency for the assembly and external maintenance of
the International Space Station (ISS) [1]. As the tasks performed by space robots become more complex, the
need for more human-like characteristics emerges. A
with humans, the sense of sight is essential to enablinefficient interaction with the environment. Mo
important than the sense of sight per se is the ability
process images in such a way as to enable mo
efficient, accurate and autonomous control of the robo
Fig. 1. Mobile Servicing System on the Internation
Space Station.
This paper addresses measurement an
control problems associated with the preci
positioning of large space robot manipulators like thSpace Station Remote Manipulator System (SSRM
shown in Fig. 1, which typically have a very hig
payload-to-manipulator mass ratio (e.g. 116 0kg/1500 kg for SSRMS) and relatively low stiffnes
8/2/2019 1419 Robotics
http://slidepdf.com/reader/full/1419-robotics 2/6
resulting in highly time-variant dynamic behavior with
significant low frequency oscillations. A theoreticalconcept for the systematic design of an instrumentation
architecture for such systems was presented in [2]. This
paper discusses the experimental implementation andevaluation of this concept in a laboratory setting.
Section II discusses the measurement of the
manipulator payload motion, including thecontributions due to structural flexibility, relative to
other objects in the manipulator workspace using a
vision system. In Section III we extend the theoretical
concept of [2] to the case of partially noncollocatedsensor/actuator configurations on flexible structures
and discuss the design and performance of a control
system for the laboratory robot.
II. VISION-BASED MEASUREMENT SYSTEM
A. Experimental Set-Up
The objective of the experimental work is
to develop, evaluate and demonstrate concepts for vision-based sensing and control for space roboticapplications. The robot shown fig 2 is specifically
designed to emulate certain dynamic
characteristics of a space robot such as SSRAM.
The robot has three revolute joints and a very flexiblelink to which various payloads can be attached,
affecting considerably the dynamic behavior of the
system. The joints are actuated by dc motors withharmonic drives and are instrumented with tachometers
and encoders. One particular objective is to
demonstrate the rapid and precise positioning of payloads with respect to a berthing site in the
workspace which roughly measures 1.5x1.5x1.5 m3 .
The low fundamental frequencies and structuraldeformations of the flexible beam present significant
challenges. The vision system used in the laboratory
functionally equivalent to the Space Vision System
(SVS) successfully flown on the Space Shuttle in 1992
Fig. 2. Experimental robot.
and 1995 [3]. A next-generation SVS being develope
by the Canadian Space Agency will support robotassembly operations on the International Space Station
Fg 3 Experimental set up
The setup in the laboratory shown inFig. 3 is in principle similar to that for a space missio
A single video camera is used to image the rob
workspace, which includes dedicated visual targeThe relative position, orientation and velocity betwee
the robot “Payload” and its “Berth” to which it is to b
maneuvered, are determined by image processing
B. Measurement of Position and Orientation
The various points and associated coordinaframes used by the vision system are identified in Fi
4, which also defines abbreviations for long names. Th
relative position and orientation of the varioreference frames can be conveniently described b
homogeneous coordinate transformations, defined b
transformation matrices TЄR 4x4 [4]. The transformatio
T A
B
transforms the coordinates of a point expressed
reference frame A into the coordinates in frame B. Th
inverse transformation for going from B to A1−
= T T A
B
B
A and always exist.
Our goal is to determine the relativ
position and orientation between Berth and Payload order to provide a guidance command to the robot. Th
transformation can be expressed in terms of a chain
homogeneous transformations
8/2/2019 1419 Robotics
http://slidepdf.com/reader/full/1419-robotics 3/6
... arg
arg T T T T et PT
Payload
Cam
et PT
Berth
Cam
Berth
Payload = -----(1)
The transformation T Berth
Cam is given by
T T T T T T CamH
Cam
PTU
CamH
CamB
PTU Cam
O
Berth
Berth
Cam .... 01−= -----(2)
The transformation T Cam
et PT arg in (1) is determined
by photogrammetric image processing as discussed indetail below, while the other transformations on the
right-hand side of (1) must be “known” a priori in order
to determine the left-hand side. Transformations T Cam
et PT arg
, T et PT
Berth
arg, T CamH
Cam can be accurately measured “before
flight” and are not subject to significant variations “in
flight” because of the relatively small translationstypically associated with these transformations.
Although the transformations T
O
CamB and T
O
erthOB can also be measured “before flight,” thermal deformations and
other effects can lead to variations “in flight” whichcan be significant in comparison to the accuracy of the
SVS. Cameras are often mounted on pan-and-tilt units
(PTU’s) which provide a read-out of pan and tiltangles, but with insufficient accuracy for our purposes.
Hence one has to assume that there is significant
uncertainty associated with T PTU
CamH .The effect of all these
errors can be reduced considerably by calibration using
a stationary auxiliary Berthing Target. After
photogrammetric measurement of T Camet BT arg we can use
the following relationship to calibrate the estimate for
T Berth
Cam in (2):
Fig. 4. Reference frames used by vision system.
1arg1
arg .−−= T T T et BT
Berth
Cam
et BT
Berth
Cam -----(3)
Fig. 5. Photogrammetric relationships.
C. Photogrammetric Image Processing
Fig. 5 shows the basic relations between th
position of a target in the real world and its image o
the CCD of an ideal camera with focal length f e . It
well known [4] that the X,Y,Z coordinates of
particular target element j in real world (P frame) an
its y,z photo-coordinates in the image plane (I fram
are related as follows
)(
)()(
j P
j P
f
jCamI
X
Cam
Y
Cam
e
y= ------(4)
)(
)()(
j P
j P
f
jCamI
X
Cam
Z
Cam
e
z = -----(5)
Part of the target design is the placement
target elements with respect to each other. Thus thlocations of the target elements are known in t
payload target reference frame. Based on this a prioknowledge, the six unknown parameters of t
transformation T Cam
et PT arg can be uniquely determin
from (4) and (5) iff the target array has three or mo
elements, i.e. j=1…k,k ≥3. The solution of the
nonlinear equations requires iterative numeric
methods in most cases. The SVS applies smacorrections to (4) and (5) in order to compensate f
optical imperfections of the camera/lens assembl
which are determined in a one-time camera calibratio process. The centroid position of each target elemen
is interpolated to a sub-pixel level to further increa
the accuracy of the photogrammetric solution. Due their symmetries, circular target elements permit ve
accurate and robust centroid determination and a
therefore preferred target elements. However, the SV
is also able to track other target elements such corners, intersections of lines and ends of lines. As a
example for photogrammetric image processing, Fig.
shows the motion of one of the payload target elemen
8/2/2019 1419 Robotics
http://slidepdf.com/reader/full/1419-robotics 4/6
in the camera image as the payload moves through the
workspace. For the same case, Fig. 7 shows the timehistory of the payload motion in the berth frame,
computed by solving(4) and (5) and performing the
transformation (1) and (2).
D. Performance Characteristics
Very fast image processing is required in order tosupport the real-time control of a robotic system. The
SVS hosted on a 486 PC computer with C40 DSP
boards is capable of processing, at a video frame rate of
30 Hz, the photogrammetric solutions and two targetarrays with up to ten elements each, using one or two
cameras. The measurement accuracies achieved vary
considerably with the distance between camera andtarget, the size and the accuracy of the targets, and the
accuracy of the various calibration schemes. During
preliminary tests in the laboratory the distance camera– berth was 1.83 m and planar target arrays with four
circular dots of 42 mm diameter were used in ambient
lighting conditions (Fig. 2).
Fig6. Camera Image of target motion.
Fig 7. Payload motion calculated by vision system.
Fig. 8. Robot oscillations tracked by vision systems.
Fig. 8 shows the measurement of the open-loo
transient response of the robot with a fundament
frequency of oscillation of 1.1 Hz. The results confirthe ability of the SVS to track the Payload motion
spite of relatively high velocities and accelerations, an
to resolve the small components of the motion alon
the Z-axis and about the roll axis. When the payload moved through the workspace in a single-joint rob
motion, joint-angle encoder measurements can b
compared with SVS measurements to estimate tabsolute measurement accuracy achieved. Although th
geared encoders are considered very accurate, sm
residual vibrations in the robot flexible structure, whichave been identified in the power spectrum of the SV
measurements, introduce some “errors” into th
comparison. Fig. 9 shows the difference of th payload–berth range measurements by the SVS and t
encoder for 70 data points spread over the range o
roughly -0.4 m to +0.5 m, and a least-squares fit to thdata. All measurement differences are less than 10 m
over the entire range. A “systematic error” of about 1
mm/m is observed and attributed to impreci
calibration of the SVS. The “random errors,” whihave a standard deviation of 2.3 mm from th
regression line, are attributed in part to residu
vibrations of the robot and partly to measurement noiin the SVS. The latter could be reduced by filtering th
SVS measurements taken at 30 Hz. The rob
oscillations could be reduced by more carefexperimentation. In spite of their preliminary natur
these results provide confidence in the accuracy of th
photogrammetric technique and its suitability for rea
time robotics applications.
8/2/2019 1419 Robotics
http://slidepdf.com/reader/full/1419-robotics 5/6
Fig. 9. Differences between SVS and encoder
measurements.
I.SENSORS AND CONTROL
A.Hyperstability Condition:
The approach for defining the overallinstrumentation and control architecture for the flexible
robot experiment is based on the methodology
presented in [2]. This method based on hyperstability
theory [5] promises to overcome control systemcoordinate transformations for stability issues
associated with the structural flexibility, time variant
dynamics and nonlinearity of the robotic system in arobust manner, and does not require an accurate model
of the system dynamics. In a nutshell, the method
requires the selection of sensors and actuators such that
the “hyperstability condition” (6) is satisfied. Then arelatively simple control algorithm will robustly
stabilize the system in the bounded-input/bounded-output (BIBO) sense [2]
010
1
0
0 ,0,)()( t t dt t yt t
t
t
u ≥∀≥−≥∫ γ γ . -----(6)
In (6), the signals yi(t) from sensors were combined toa system output vector y(t) and the forces and torques
ui(t) applied by actuators were combined to a system
input vector u(t).Based on Hamilton’s extended principle and the
principle of conservation of energy, condition (6) can be linked to conditions involving the energy transferred
by the actuators of electromechanical systems such asthe SSRMS and the laboratory robot [2]. In the specific
case of the laboratory robot, it follows from the results
of [2] that the combination of tachometers and torquemotors in the joints satisfies condition (6). The
condition can be shown to permit sensor errors such as
errors in gain and linearity, saturation effects, noise, bias and dynamic low-pass characteristics.
Fig. 10. Extended sensing and control architecture.
B. Extension of Instrumentation and Contr
Concept
However, the measurement of the rob
end-point motion by the vision system does not satisthe sufficient condition (6) and in fact violat
necessary conditions on the placement of sensors o
flexible structures developed in [2]. It is well knowthat having “noncollocated” sensors and actuators on
flexible structure gives rise to a variety of problems f
active vibration control of the structure [6]. In order
address and solve these problems the concept of [2] extended by defining a set of m “performance sensor
whose measurement signals i z are combined to a vect
Z. This set includes all sensors necessary to achieve thcontrol objective but do not meet the hyperstabili
conditions on sensors presented above and in [2].In th
case of the laboratory robot the set of measuremen provided by the vision system constitute
performance measurements Z.
Fig. 10 shows a block diagram of the extendconcept in which the sensors providing measuremen
y which meet the hyperstability condition (6) are no
called “hyperstability sensors.” The control signvector u is generated by a sensor fusion and contr
operator C based on the measurements of t
hyperstability sensors y , the performance sensors
and the commanded trajectory or set point vector w(t)
U(t)=C{w(t),y(t),z(t)}. ---- (7)
may be composed of algebraic, integral, differential,
fuzzy logic operations on w,y,z . It can be shown that
will stabilize the system of Fig. 10 in the BIBO sensif C satisfies the following condition for an arbitra
finite constant 0≥γ :
}{01
1
0
,,, t t dt z ywC yt
t
T ≥∀∫ γ < -----(8)
A particular class of control laws satisfying (8) is give
by
8/2/2019 1419 Robotics
http://slidepdf.com/reader/full/1419-robotics 6/6
{ 0},,{.},,,{}{
.0},,{.},{
221
21
<+
≥=
z ywC y for z ywC wC
z ywC y for wC
T
T C ----
(9)
C1{w,y,z} in (9) is a feedforward control term, which
does not affect the closed-loop stability of the controlsystem as long as it is internally stable. Any knowledge
of the dynamic behavior of the robotic system can be
used in its construction.C2{w,y,z} in (9) is a feedback
control term, which can be, chosen freely to satisfy the performance objectives of the control system. It can be
shown that BIBO stability of the closed-loop system isguaranteed by the logic implied in (9).
C. Control Design for Visual Servoing
Equations (8) and (9) provide a generalframework for the design of a control system. We will
now see how it can be applied to the laboratory robot
by using conventional design approaches. For theclarity of the illustration we pick the simplest situation
of controlling the motion of a single joint. Theobjective is to achieve close tracking of the robot payload position Z(t) to a trajectory W(t) defined
relative to the berthing site in the workspace.
In the case of a scalar system a standardapproach to the feedback control problem is PID
control [4] as defined below in the frequency domain (s
is the Laplace operator,K P,K D,K I are gain constants)
Fig. 11. Simulated motion of laboratory robot.
C2{w(s),z(s)} =(Kp+K I/s + K Ds)(w(s)-z(s)) ----(10)
For feedforward control of robots, several
methods to determine torque profiles from trajectory
commands have been proposed in the robotics literature[4]. As one of our objectives is to demonstrate that our
method does not rely on accurate system models we
will use a simple lead filter as feedforward, which
incorporates only very rudimentary knowledge of thesystem, such as “which way” to move a joint, but not
“how far.” For the simple single-joint case, the lead
feedforward control term, with gain and tim
constants , is defined in the frequency domain follows:
C1{w(s)}=(K 1s/(1+K 2s)).w(s) ------(11).A particular design for the general control law C
now fully defined by (9)–(11).
Fig. 11 shows a preliminary simulation of th
robotic system with the flexible beam and the contr
system defined above in response to a step function the trajectory command. The diagram shows a smoo
motion of the robot tip and payload which rapid
converges to the new set point of 1 rad within abouts. In contrast, the joint motion has a significa
oscillatory component due to the flexibility of the lin
as indicated by the time history of the joint velocitOther simulation results have confirmed the robustne
of the approach to variations in payload mass and rob
dynamic characteristics. Hardware experimen
intended to confirm the simulations are in preparation
IV. CONCLUSIONS
Photogrammetric measurement and robcontrol techniques have been developed for rapid an
precise positioning of payloads with flexible spa
manipulators. Using dedicated visual targets, a visiosystem provides measurements of the motion of objec
in the robot workspace in real time at 30 Hz. T
ability of the vision system to track fast movinmanipulator motions with sufficient accuracy has bee
experimentally verified. A control technique w
developed which overcomes problems with partialnoncollocated sensor/actuator configurations o
flexible robots. Simulation results confirm the effica
and robustness of the control approach.
R EFERENCES
[1] C. P. Trudel, D. G. Hunter, and M. E. Stieber,“Control and operation of space manipulator systems
in NATO AGARD Lecture Series 193: Advanc
Guidance and Control Aspects in Robotics, 1993.
[2] M. E. Stieber, E. Petriu, and G. Vukovic“Systematic design of instrumentation architecture f