Paper Title (use style: paper title) · The Jacobian matrix provides this information for controller. In each time-step of control process Jacobian matrix and its inverse must be

userSticky Note1.F. Abadianzadeh, V. Derhami, and M. Rezaeian, Visual Servoing Control of Robot Manipulator in 3D Space Using Fuzzy Hybrid Controller”, in IEEE Fourth RSI/ISM International Conference on Robotics and Mechatronics (ICRoM), pp.61-65, 2016.

Visual Servoing Control of Robot Manipulator in 3D Space Using Fuzzy Hybrid Controller

Fatemeh Abadianzadeh Computer Engineering Dept.

Faculty of Eng., Yazd University Yazd, Iran

[email protected]

Vali Derhami Computer Engineering Dept.

Faculty of Eng., Yazd University Yazd, Iran

[email protected]

Mehdi Rezaeian

Computer Engineering Dept. Faculty of Eng., Yazd University

Yazd, Iran [email protected]

Abstract—In this paper a new visual servoing control for robot manipulator in 3D space is presented. In traditional visual servoing control, models of robot and camera are needed. Obtaining these models are time consuming and sometimes impossible. In addition, Jacobean matrix has an important role on visual servoing. Calculating this matrix and its inverse (if exists) in each step-time of control process is complicated. Here, we use intelligent methods to cope with the above challenges. A hybrid fuzzy controller is proposed to control robot manipulator. Visual inputs of the controller are provided by Kinect and the outputs are the rotation of joints motors. The first controller in based on fuzzy inverse model which approximates real inverse model of robot using gathered data. In order to increase accuracy a second fuzzy controller is used, which is designed by expert knowledge. The output of this controller suppresses the first controller output where the end-effector is in the predefined near-goal area. The proposed method is applied to control a real robot manipulator (ARM_6Ax18). Experimental results show that by using the proposed method in practice, the end-effector reaches from any random start position to the goal position with a good accuracy in robot workspace.

Keywords— visual servoing; fuzzy systems; fuzzy inverse model; robot control; fuzzy controller; robot maipulator.

I. INTRODUCTION

Visual servoing control is a method that uses visual information to control the motion of a robot. This information could be pixels in an image, lines or an area of an image, etc. The visual information may be provided by camera [1]. The camera can be mounted to the robot manipulator1 or can be fixed in the robot working environment2 [2]. Visual servoing has three main categories [1]:

Position-based visual servoing

Image-based visual servoing

Hybrid visual servoing

Position-based visual servoing uses features are extracted from the image to estimate position of robot or target, based on the camera or a world coordinates. The goal of this approach is to

1 Eye-in-hand

minimize distance between current and desired position. This distance called error signal [3]. In image-based visual servoing the error signal is defined as distance between image features in current and desired position [3]. The hybrid approach of visual servoing uses both image-based and position-based visual servoing properties and it also avoids their disadvantages [1].

This paper is focused on image-based visual servoing methods to control robot manipulator. In classical image-based methods robot-camera model is require. In other word in classical image-based visual servoing, it has been assumed that the model is known. The Jacobian matrix provides this information for controller. In each time-step of control process Jacobian matrix and its inverse must be computed before doing any action [4]. Since Jacobian matrix may not have inverse always, in [5] neural networks are used to approximate this matrix. Each row of Jacobian matrix (the relation between changes in one of the image feature and robot joints changes) is computed by a separate neural network. They also used a simple webcam to capture images and their controller control the robot manipulator in 2D space. In [6] camera parameters are unknown and it is approximated analytically (least square optimization approaches and etc.). In [7] learning methods are used to obtain these parameters. Camera-robot model can be estimate by learning methods. For example in [4] fuzzy technique is used to design a controller to control the robot manipulator in 2D space. An inverse fuzzy model is used to compute the joints velocity. The method is applied to a puma 560 robot [8] with eye-to-hand visual servoing configuration. In [9], [10], [11] reinforcement learning methods is used in order to control robot manipulator. In control process in order to increase accuracy depth information is needed. Since Kinect [12] provides RGB and depth information, these capabilities is encouraged researchers to use it. In [13] Kinect is used to get RGB and depth information and robot manipulator is controlled using this information.

The aim of this paper is control of ARM-6AX18 [14] robot manipulator (Fig. 1) to reach its end-effector from any start position to different desired goal position in robot workspace. All visual information inputs are provided by Kinect. A hybrid fuzzy

2 Eye-to-hand

Proceedings of the 4th International Conference on Robotics and Mechatronics October 26-28, 2016, Tehran, Iran

978-1-5090-3222-8/16/$31.00 ©2016 IEEE 61

Downloaded from http://iranpaper.irhttp://www.itrans24.com/landing1.html

controller is proposed to generate command signals of joints motors.

This paper is organized as follows. In Section II, classical image-based visual servoing, fuzzy model, and fuzzy inverse are explained briefly. The proposed method is introduced in section III. Experimental results are presented in section IV. Finally, section V conclusion and future works are given.

II. PRELIMINARIES

A. Classical Image-based Visual Servoing

Image-based visual servoing is based on modeling the transformation between image features velocity and joints velocity. This model is defined as follow [5]:

� = ��

Where � is image features velocity, � is the joints velocity and � is Jacobian matrix. If feature points position use to measure error, the error can be describe as (2) [5]:

�(�) = ��

Where �� , � denote current and goal state features vector. Finally the control signal is as follow [5]:

� = � = ��(�)

� is the control signal, K is a constant matrix and �� is a pseudo inverse Jacobian.

B. Fuzzy Modeling

Fuzzy modelling is an approach to declare expert knowledge in a verbal form with a set of if-then rules. It can also be designed using input-output data. In other word fuzzy model is an approximation of a system. For example if y=F(x) is an unknown system, fuzzy model try to construct y=f(x) using input-output data to approximate F(x) [4].

In this paper Sugeno fuzzy model is used. This model’s rules are as follow [4]:

�� = �� …�� = �� + �� =1,2, … , �

Where �� is � -th rule, [��, … , ��] are inputs, [��1, … , ��n] are fuzzy sets is defined on input space and �

� rule output. Equation

(5) is the output of system.

�� =∑ ��

∑ ��

Where �� is the firing strength of �-th rule:

�� = ∏ ��(��)�� = 1,2, … , �

And ��: � → [0,1] is membership function of each fuzzy

set ��is defined on input space.

Fuzzy rules can be extracted from data. If x, y are inputs and output of an unknown system respectively, a function like f(x, y) must be approximated. For this purpose, input data could be clustered. Parameters of antecedents of rules are obtained based on cluster properties. Consequence parameters are computed using least square optimization methods [15].

C. Fuzzy Inverse Control

Designing fuzzy inverse controller has two steps. First step is using fuzzy technique to model the inverse dynamic of the plant. The second step is to use this obtained fuzzy model in application to generate control signal of system. If the all states of the plant are measurable, then [15]:

�(� + 1) = �(�(�), �(�))

Where �(� + 1)the state at time k + 1, �(�)the state at time k, and u(k) is the control action at time k . So the inverse dynamic of the plant can be show as (8):

� = �(�(�), �(� + 1))

It means a unique input U can change the plant state from �(�) to �(� + 1) [16].

In order to obtain this inverse model, �(�) and �(� + 1)are considered as inputs of a fuzzy system and � as output. The result fuzzy system approximate the plant inverse.

III. PROPOSED METHOD

Here, control of a three-degree of freedom robot is studied. A hybrid controller is used for this purpose. A fuzzy inverse model is used to control the robot as a first controller. The second controller is a fuzzy model which is designed by expert knowledge. The controller diagram is shown in Fig. 2. For controlling process end-effector position is needed in each time-step which is provided by Kinect. The end-effector position is defined by three parameters x, y and z. Its coordinates in RGB image use as x and y and corresponding value of these coordinates in depth image is used as z. In addition, values of joints motors are used as system current state. The proposed inverse model is a zero-order Sugeno fuzzy model [17] with six inputs and three outputs (Fig. 3). [ �, �, �] are the distance between end-effector and the goal in RGB and depth image (error vector). Other inputs

Fig. 1. ARM-6AX18 robot manipulator

62


[m1, m2, m3] are current state of system which they are the three joints motors values. Outputs of system are changes in values of motors joints. In each step-time error vector and current state are given to the controller and it calculates the changes in each robot motors values.

A. Data Gathering

Fuzzy model can be made using data. In this paper the proposed fuzzy model is designed using data which is gathered in learning process. In order to data gathering the end-effector is placed in difference positions. Then its position in RGB and depth image, and robot motors values saved as training data. In data gathering process each motors values changed ten degree and saved the require data. After data gathering, the model train with this data. In this paper MATLAB fuzzy toolbox is used. Genfis3 function give a zero-order Sugeno using provided data. Data provided for this function is end-effector displacement from previous position in x, y, z, and the motors current values as inputs. The outputs are amount of changes in motors values.

B. Second Controller

In experiments using only inverse model has some drawbacks. Since system did not have inverse in some states and gathered data had noise, the final error is not acceptable. In order to overcome this problem, a second fuzzy controller is designed using expert knowledge. The first controller brought the end-effector to the near goal area. Then the second controller suppressed the first controller outputs. All of this controller properties are obtained using expert knowledge.

The second controller like the first one is a zero-order Sugeno fuzzy model. The inputs of this model are error vector and the outputs are the amount of changes in motors values. Since this controller work in a limited area, state of the system is omitted. Each input variable has three membership function are named negative, zero, and positive (Fig. 4). Since elbow motor is different from the others, its input range is different. The output of the system for base motor are [-15, 0, 15], for shoulder are [-20, 0, 20], and for elbow are [-60, 0, 60]. This system has 27 rules. Some of this rules are shown in Fig. 5.

IV. EXPERIMENTL RESULTS

The proposed method has been applied on ARM-6AX18 robot manipulator with XBOX-360 Kinect which is fixed in the workspace (Fig. 6.). The robot has one servo motor Dynamixel AX-18 for base, two of the same type for shoulder and one Dynamixel MX-28 for elbow. In each step of data gathering

Fig. 2. Hybrid fuzzy controller diagram

Fig. 3. The proposed fuzzy inverse model

Fig. 4. Membership function of input variables of fuzzy expert system

63


process one of motors value has been changed in ten degree and others are constant until all of workspace area in front of the robot is covered. In each step end-effector position in RGB and depth image saved. In order to find end-effector in image, it is covered by red color. After training fuzzy model it is tested in real position. In practice, robot manipulator is supposed to be in goal area if its error vector was between [-30, 0], [-30, 30] pixels in x, y coordinate in RGB image, respectively, and [-10, 30] in depth image. If it was in these ranges, this means that it reached to the goal without error. The goal is placed in different position in working environment and its position sets in controlling program. Controlling program is written in MATLAB. When only first controller is used final error is more than the specified goal area.

To reduce this final error, second controller is applied to system. Table 1 compares the final error when only fuzzy inverse controller vs. hybrid controller are used at the same five difference goal position. According to table 1 final error is reduced when hybrid controller is used and the end-effector reached to goal as is shown in Fig. 7. One of the experiments videos is available in [17].

In Fig. 8 end-effector trajectory is shown from start to goal position. It shows that the end-effector trajectory by using this model is good and it does not have many extra movement.

V. CONCLUSIONS AND FUTURE WORKS

In this paper, a hybrid fuzzy controller was proposed to control a real robot manipulator in 3D space without using robot model. The inputs are provided by Kinect and joints motors values. The first controller was a fuzzy inverse model of plant which approximate the real inverse model of the system. The second fuzzy controller is designed by encoding expert knowledge. In practice, the first controller is applied to plant. When end-effector

Fig. 5. Some rules of fuzzy expert system

Fig. 6. System setup

Table 1. Inverse vs. hybrid controller final error. Zero values mean that end-effector reached to the defined goal area.

Ex # controller Error in x

(Pixel)

Error in y

(Pixel)

Error in z

(mm)

1 inverse -50 -23 0

hybrid 0 0 0

2 inverse 0 34 0

hybrid 0 0 0

3 inverse 0 63 102

hybrid 0 0 0

4 inverse 56 49 -25

hybrid 0 0 0

5 inverse -3 82 125

hybrid 0 0 0

Fig. 7. End-effector position when it reached to goal area.

64


reached near goal, the second controller suppressed the first controller outputs. In practice this hybrid method has been reduced the final error and increase accuracy. Experimental results show that by using this method robot trajectory is good and the manipulator does not have extra movement.

This method is applicable when the environment is fixed. If some changes is applied in camera position or robot base position this method does not work properly. For future work we will try to make the controller adaptive to small changes using online learning methods.

REFERENCES

[1] F. Chaumette, S. Hutchinson, “Visual servo control, Part I: Basic approaches,” IEEE Robotics and Automation Magazine, pp.82-90, 2006.

[2] D. Kragic, and H. Christensen, “Survey on visual servoing for manipulation,” Computational Vision and Active Perception Laboratory, Fiskartorpsv 15 ,2002.

[3] S. Hutchinson, G. D. Hager, and P. I. Corke, “A Tutorial on Visual Servo Control,” IEEE Transactions on Robotics and Automation, vol. 12, pp. 651–670, 1996.

[4] P. Goncalves, L. Mendonca, J. Sousaand and J. Pinto, “Uncalibrated eye‐to‐hand visual servoing using inverse fuzzy models”, IEEE Transactions on Fuzzy Systems, pp. 341–353, 2008.

[5] F. Nadi, V. Derhami, and M. Rezaeian, “Visual Servoing Control of Robot Manipulator with Jacobian Matrix Estimation,” in 2nd International Conference on Robotics and Mechatronics, Tehran, Iran, 2014, pp. 405-409.

[6] J. Piepmeier and H. Lipkin, "Uncalibrated Eye-in-Hand Visual Servoing", The International Journal of Robotic Research, vol. 22, no. 10, pp. 805-819, 2003.

[7] I. Suh and T. Kim, "A visual servoing algorithm using fuzzy logics and fuzzy-neural networks", Mechatronics, vol. 10, no. 1-2, pp. 1-18, 2000.

[8] "Programmable universal machine for assembly", Wikipedia, 2016. [Online]. Available: https://en.wikipedia.org/wiki/Programmable_Universal_Machine_for_Assembly. [Accessed: 08 Jul 2016].

[9] C. Distante, A. Anglani, and F. Taurisano, “Target Reaching by Using Visual Information and Q -learning Controllers,” Autonomous Robots, vol. 9, pp. 41–50, 2000.

[10] Z. Miljkovic, M. Mitic, M. Lazarevic, and B. Babic, “Neural network Reinforcement Learning for visual control of robot manipulators,” Expert Systems with Applications., vol. 40, pp. 1721–1736, 2013.

[11] M. Sadeghzadeh, “Self-Learning Visual Servoing of Robot Manipulator Using Explanation-Based Fuzzy Neural Networks and Q-Learning,” Ph.D. dissertation, University of Guelph, 2014.

[12] "Kinect | Xbox 360", Xbox.com, 2016. [Online]. Available: http://www.xbox.com/en-US/xbox-360/accessories/kinect. [Accessed: 08 Jul 2016].

[13] M. Deisenroth, C. Rasmussen, and D. Fox, “Learning to Control a Low-Cost Manipulator Using Data-Efficient Reinforcement Learning,” in International Conference on Robotics: Science & Systems, 2011, pp. 57–64.

[14] "Robotic arms", Pishrobot.com, 2016. [Online]. Available: http://www.pishrobot.com/en/products/robotic_arms.htm. [Accessed: 08 Jul 2016].

[15] J. Jang, C. Sun and E. Mizutani, Neuro-fuzzy and soft computing. Upper Saddle River, NJ: Prentice Hall, 1997.

[16] K. Passino and S. Yurkovich, Fuzzy control. Menlo Park, Calif.: Addison-Wesley, 1998.

[17] "Fuzzy hybrid control of robot with camera", aparat, 2016. [Online]. Available: http://www.aparat.com/v/5UWeh. [Accessed: 08 Jul 2016].

Fig. 8. End-effector trajectory from start to goal position

65


Paper Title (use style: paper title) · The Jacobian matrix provides this information for controller. In each time-step of control process Jacobian matrix and its inverse must be

Documents