8/14/2019 Session: Modeling, Simulation and Optimization
1/31
8/14/2019 Session: Modeling, Simulation and Optimization
2/31
Session: Modeling, Simulation and Optimization
Applications and algorithms of non-
linear regression using least squares
Approximation based on Case Studies
S. Vignesh, T.K. Premannanth
Department of Chemical Engineering,
St. Josephs College of Engineering,
Chennai 600119.
By
8/14/2019 Session: Modeling, Simulation and Optimization
3/31
Abstract
Three data sets from literature were taken to investigate the importance of method of
least squares in approximation methods in engineering. The case studies such as
heat capacity data to quadratic equation in temperature, vapour liquid equilibrium
data to Wilson equation and fitting Gilliland Sherwood data were taken Multi Non
Linear Regression (MNLR) was successfully obtained using least square
approximation. The regression model obtained was subjected to Distributed Errors
Which was characterized by decrease of some global error measure with respect to
the whole approximation interval as the order of approximation increases. The best
Multiple Non Linear Regression model was evaluated with the small value of error.
Graphs to evaluate Goodness of Fit were drawn for the three data sets which werefound to be good.
8/14/2019 Session: Modeling, Simulation and Optimization
4/31
Introduction
The so-called method of least squares is a very important approximationmethod in engineering.
This method is perhaps the best known distributed error approximationmethod. Least square approximation is valuable in problems such asfitting equations to discrete data points and in analyzing measurementerrors.
The subject of least square analyses also plays central role in theapplication of the theory of statistics, which treats problems involvingrandom.
The subject of random variables and statistics is beyond the scope of ourpresentation, we will therefore use least squares in our casestudies.
Least squares are also useful for continuous approximations, such asdeveloping simple approximation to known functions.
In least squares, distributed error methods are characterized by adecrease of some global error measure with respect to the wholeapproximation interval as the order of approximation increases.
8/14/2019 Session: Modeling, Simulation and Optimization
5/31
Linear Regression Linear regression is solved by the method of least squares and the
error percentage was found out and Goodness of Fit graph wasplotted. The Least Square equations are as follows.
General equation:
y = a0 + a1x (2.1) where,
a0 = (yi / n) - a1 (xi/n) (2.2)
a1 = [xi yi (xi yi)/ n] / xi^2 (xi )^2 / n (2.3)
Where a0 and a1 are constants that are determined.
8/14/2019 Session: Modeling, Simulation and Optimization
6/31
Polynomial Regression
Non Linear Regression is solved by polynomial Regressionmethod (Second order). The Polynomial Regression equations areas follows:
General equation:
y = a0 + a1x + a2x^2 + ..+ anx^n (2.4)
(2.5)
On solving the above matrix, we can get the values of a0, a1 & a2.
8/14/2019 Session: Modeling, Simulation and Optimization
7/31
Case Studies
Three case studies has been
analyzed, they are:
Heat capacity data to
quadratic equation intemperature.
Vapour liquid equilibrium datato Wilson equation.
Fitting Gilliland-Sherwood
data.
8/14/2019 Session: Modeling, Simulation and Optimization
8/31
Case I
Heat Capacity Data to theQuadratic Equation
Here we have analyzed the heat capacity data for liquidmethylcyclohexane (C7H14) using the equation:-
Cp = a0 + a1T
Where Cp is the Heat capacity, T is the absolutetemperature and a0 and a1 are parameters which we havefound out by Linear Least Squares.
8/14/2019 Session: Modeling, Simulation and Optimization
9/31
T, K CP, KJ/KgK
150 1. 426
160 1. 447
170 1. 469
180 1. 492
190 1. 516
200 1. 541
210 1.567
220 1. 596
230 1. 627
240 1. 661
250 1. 696
260 1. 732
270 1. 770
280 1. 808
290 1. 848
300 1. 888
Data: TABLE I (a)
8/14/2019 Session: Modeling, Simulation and Optimization
10/31
On solving these datas by Least Square method, we got thevalues of a0 and a1 and it was found to be0.96 and 0.00297respectively.
On substituting the values of a0 and a1 in the equation 2.1, we gotthe predicted value, which seems to benear when compared to
the experimental values.
y = 0.96 + 0.00297x
On substituting the temperature values on the above equation, weget the predicted values
8/14/2019 Session: Modeling, Simulation and Optimization
11/31
PredictedValues Experimental values
1.405 1.426
1.434 1.447
1.457 1.469
1.489 1.492
1.509 1.516
1.538 1.541
1.558 1.567
1.587 1.596
1.611 1.627
1.658 1.661
1.684 1.696
1.725 1.732
1.768 1.770
1.795 1.808
1.821 1.848
1.867 1.888
TABLE I (b)
8/14/2019 Session: Modeling, Simulation and Optimization
12/31
Goodness of fit
0
0.2
0.4
0.6
0.8
1
1.2
1.4
1.6
1.8
2
EXP
1
EXP
2
EXP
3
EXP
4
EXP
5
EXP
6
EXP
7
EXP
8
EXP
9
EXP
10
EXP
11
EXP
12
EXP
13
EXP
14
EXP
15
EXPERIMENT
ALPREDICTED
The above graph shows the Goodness of Fit for the Heat Capacity Dataof Methylcyclohexane. The experimental andpredicted values showedabove shows the minimum percentage of error. The error percentagewould approximately lies between 2-5%.
Fig I (c)
8/14/2019 Session: Modeling, Simulation and Optimization
13/31
Case II
Vapour Liquid Equilibrium toWilson Equation
Vapour Liquid equilibrium data were taken from Heptane-Toluene binary system at 1 atm pressure.
Here we fitted activity coefficient data to Wilson Equation As it requiredNon-Linear regression, we have used
polynomial method and we havegot the predicted valueswhich are nearer to the experimental.
Here the equation we use is:
y = a0 + a1x + a2x^2 + ..+ anx^n
8/14/2019 Session: Modeling, Simulation and Optimization
14/31
Data:
xi yi1.000 0.0000
0.790 0.1259
0.596 0.1509
0.480 0.1392
0.390 0.1250
0.293 0.1111
0.220 0.0950
0.150 0.0707
0.065 0.0290
0.000 0.0000
TABLE II (a)
8/14/2019 Session: Modeling, Simulation and Optimization
15/31
On solving these data's by Polynomial method, we got the valuesof a0, a1 and a2 and it was found to be -0.00425, 0.575, -0.5568respectively.
On substituting these values on the equation 2.4, we got thepredicted values which were very nearer to the experimental
values.
y = -0.00425 + 0.575x -0.5568x^2
On substituting the xi values, we got the predicted values which
are tabulated as follows.
8/14/2019 Session: Modeling, Simulation and Optimization
16/31
Predicted Experimental
0.0012 0.000
0.1154 0.1259
0.1495 0.1509
0.1435 0.1392
0.1343 0.1250
0.1102 0.1111
0.0925 0.0950
0.0695 0.0707
0.0278 0.0290
0.0000 0.0000
TABLE II (b)
8/14/2019 Session: Modeling, Simulation and Optimization
17/31
Goodnessof fit
0
0.02
0.04
0.06
0.08
0.1
0.12
0.14
0.16
EXP 1EXP 2 EXP 3 EXP 4 EXP 5 EXP 6 EXP 7 EXP 8EXP 9
EXPERIMENTAL
PREDICTED
The above graph shows the Goodness of Fit for the Vapour LiquidEquilibrium data. The experimental andpredicted lines showed aboveshows the minimum percentage of error. The error percentage wasapproximately lies between 6 7%.
Fig II (c)
8/14/2019 Session: Modeling, Simulation and Optimization
18/31
Case III
Mass Transfer Data of Gilliland-Sherwood Equation
The Mass Transfer Data's were taken and analyzed by Gilliland-
Sherwood Equation As it required Non-Linear regression, we used the polynomial method
and we have got the predicted values which are nearer to theexperimental values.
Here the equation we use is:
y = a0 + a1x + a2x^2 + ..+ anx^n
8/14/2019 Session: Modeling, Simulation and Optimization
19/31
Data:
xi yi
43.7 0.60
24.2 1.80
51.6 1.87
32.3 1.86
26.1 2.16
92.8 2.17
TABLE III (a)
8/14/2019 Session: Modeling, Simulation and Optimization
20/31
On solving these datas by polynomial method, we got the valuesof a0, a1 and a2 and it was found to be 16.11, -0.7588 and 0.0053respectively.
On substituting these values on the equation 2.4, we got thepredicted values which were very nearer to the experimental
values.
y = 16.11 0.7588x + 0.0053x^2
On substituting the xi values, we got the predicted values whichare tabulated below.
8/14/2019 Session: Modeling, Simulation and Optimization
21/31
PREDICTED EXPERIMENTAL
0.48 0.60
1.62 1.80
1.69 1.87
1.70 1.86
1.92 2.16
1.95 2.17
TABLE III (b)
8/14/2019 Session: Modeling, Simulation and Optimization
22/31
Goodness of fit
0
0.5
1
1.5
2
2.5
EXP 1 EXP 2 EXP 3 EXP 4 EXP 5 EXP 6
EXPERIMENT
ALPREDICTED
The above graph shows the Goodness of Fit for the Mass TransferData from Gilliland-Sherwood equation. The experimental andpredicted lines showed above shows the minimum percentage oferror. The error percentage was approximately lies between 20 25%. The error was little high since the data was Non-Linear.
Fig III (c)
8/14/2019 Session: Modeling, Simulation and Optimization
23/31
Results and Discussion
The three case studies analyzed using least squares andpolynomial regression yield good results.
The goodness of fit graphs was plotted and error % wascalculated for all the three datas.
The error percentage was found to be approximately2to5% in the first case study, similarly 7to10% in thesecond and 20to25% in the third respectively.
The first two case studies error was obtained veryminimal and the best fit graph was plotted for theGilliland-Sherwood data it was little higher because thevalues are highly non-linear and it was difficult topolynomial apply regression to the result. Further studyhas to be made to minimize the error in the third casestudy.
8/14/2019 Session: Modeling, Simulation and Optimization
24/31
Discussion - Case I
In the first case study, the table I (a) shows the datataken for Performing regression.
Using those values and by manually calculating thenecessary terms, we have substituted those terms in the
equations 2.1, 2.2 and 2.3 and we have obtained thegeneral equation.
The table I (b) is the table containing the experimentaldata and predicted data. Using those values we haveplotted a graph I (c) which shows the goodness of fit.
From the graph we have concluded that the first casestudy has come out well with minimum error %. As wesee the graph we can see the two lines very closeindicating that the regression was successful with veryless error.
8/14/2019 Session: Modeling, Simulation and Optimization
25/31
Discussion - Case II
In the second case study, the table II (a) shows the data takenfor Performing regression.
Using those values and by manually calculating the necessaryterms, we have substituted those terms in the equations 2.4and 2.5 and we have obtained the general equation.
The table II (b) is the table containing the experimental dataand predicted data. Using those values we have plotted a graphII (c) which shows the goodness of fit.
From the graph we have concluded that the second case study
has come out well with minimum error %. As we see the graphwe can see the two lines very close indicating that theregression was successful with very less error.
8/14/2019 Session: Modeling, Simulation and Optimization
26/31
Discussion - Case III
In the third case study, the table III (a) shows the data taken forPerforming regression.
Using those values and by manually calculating the necessary terms, wehave substituted those terms in the equations 2.4 and 2.5 and we haveobtained the general equation.
The table III (b) is the table containing the experimental data and predicteddata. Using those values we have plotted a graph III (c) which shows thegoodness of fit.
From the graph we have concluded that the second case study has come outwell with minimum error %. As we see the graph we can see the two linesvery close indicating that the regression was successful with very less error.
The error % is high when compared to the first two cases because the datais non-linear.
8/14/2019 Session: Modeling, Simulation and Optimization
27/31
Algorithm
General algorithm for all the three cases: Step 1: The datas were taken from the case
studies. Step 2: Regression was applied to the data using
the formulas which are stated in the beginning. Step 3: The necessary values of Ao and A1 was
determined. Step 4: These values are substituted in the general
equations (2.1 for case study I and 2.4 forCase study II and III).
Step 5: Using that we have determined thepredicted values.
Step 6: A graph was drawn between theexperimental and the predicted data.
Step 7: The error percentage was calculated. Step 8: The graph plotted is the goodness of fit.
8/14/2019 Session: Modeling, Simulation and Optimization
28/31
Conclusion
The three sets of data's were analyzed and good results havebeen obtained in all three case studies.
The first 2 case studies have come perfectly with minimum errorand the goodness of fit graph is plotted and was found to begood.
For the third case study the error was little high when comparedto the other two, this is because the data is too non-linear.
Goodness of fit graph was plotted for the third case study andhas come out well.
For further minimizing the error for the Gilliland-Sherwood data,further studies have to be made.
On the whole the results i.e., the regression model obtained byusing Least Squares and Polynomial regression were successfulin prediction and the representation of the system was good.
8/14/2019 Session: Modeling, Simulation and Optimization
29/31
References1. Berezin I. S., and Zhidkov N. P., (1965), Computing Methods, Addision-Wesely,
Menlo Park, CA2. PERRY, R. H., Green, D. W., and Maloney, J. O. (1984), Chemical Engineers Hand
book3. Modeling and Analysis of Chemical Engineering Processes, Balu.K,
Padmanabhan.K, I.K. International Pvt. Ltd.4. Optimization Theory and Practice, Mohan C Joshi, Kannan M Moudgalya, Narosa
Publishing House.5. Neural Networks - A Comprehensive Foundation, Simon Haykin, Pearsoneducation second edition, 2004.
6. Neural Networks, Ananda Rao. M, Srinivas.J, Narosa Publishing House.7. Design and analysis of Experiments, Montgomery D C, 5th ed., John Wiley & sons,
New York, 2007.8. Experiment optimization in chemistry and chemical engineering, Akhnazarova S,
Kafarov V, MIR publishers, Moscow, 1982.
9. Experimental methods for engineers, Holman, McGraw Hill Publications.
8/14/2019 Session: Modeling, Simulation and Optimization
30/31
Our sincere thanks
Organizing committee - J.N.T.U College of Engineering,Anantapur
Judges and coordinator's for their best support.
Head of the departmentChemical Engineering
St. Josephs College of Engineering
8/14/2019 Session: Modeling, Simulation and Optimization
31/31