A new iterative linearization approach for solving nonlinear equations systems

Article History: Received 17 August 2018 Accepted 24 July 2019 Available 14 January 2020 Nonlinear equations arise frequently while modeling chemistry, physics, economy and engineering problems. In this paper, a new iterative approach for finding a solution of a nonlinear equations system (NLES) is presented by applying a linearization technique. The proposed approach is based on computational method that converts NLES into a linear equations system by using Taylor series expansion at the chosen arbitrary nonnegative initial point. Using the obtained solution of the linear equations system, a linear programming (LP) problem is constructed by considering the equations as constraints and minimizing the objective function constructed as the summation of balancing variables. At the end of the presented algorithm, the exact solution of the NLES is obtained. The performance of the proposed approach has been demonstrated by considering different numerical examples from literature.


Introduction
Numerical analysis and computers are intimately related with each other regarding to solve mathematical problems. With the development of computers, numerical methods have been increased for solving scientific and engineering problems. The numerical methods are used to find approximate solution of such problems because it is not possible to obtain exact solution by using algebraic processes. One of the most important issues for solving NLES in science and engineering is to find a solution that is frequently arising in optimization and computational mathematics. Because NLESs cannot be solved as easily as linear systems, iterative methods are improved as a new class of numerical solution methods.
Iterative method is a procedure repeated over and over again to find either the root of an equation or the solution of an NLES. In numerical methods, the sequence of approximate solutions converges to the root of the system. If the convergence rate of an iterative method is rapid, then a solution may be found in less iterations compared with other methods. As the iterations begin to have successive same values, this is an indication that the obtained solution is the exact solution of the NLES. However, when the obtained solution of the system does not converge, it is indicated that there is an error in the computations or there is no solution. Therefore, an NLES has finite or infinite number of solutions or no solution. There are numerous conventional methods to solve NLESs having algebraic and transcendental equations. One of the most popular and traditional numerical methods is Newton method which is widely used for finding roots of the NLES. This method is based on Taylor series expansion of a function, and converges rapidly to the exact solution of the NLES. It can be presented as an advantage that Newton method requires less iterations to reach the solution compared to other known methods.

*Corresponding Author
Another advantage of the Newton method is the framework is clear, and therefore it can be used to solve a variety of problems. On the contrary, due to the difficulty in computation of both Jacobian matrix and its inverse at each iteration. Using the Newton method would be time-consuming regarding to the size of the system. To avoid these impracticabilities, some developments and modifications are made to the Newton method, such as Quasi-Newton method, Dimension Reducing method, Modified Reducing Dimension method and Perturbed Dimension Reducing method.
Grapsa and Vrahatis [1] reviewed a class of methods for solving NLESs and optimization problems named Dimension Reducing methods. Frontini and Sormani [2] extended to p-dimensional case the modification of Newton method. This method is used to solve NLES and compared with Newton method and Halley-Chebyshev method. Babolian et al. [3] extended the Adomian decomposition method for solving the NLES. Nie [4] transformed the NLES into a constrained nonlinear optimization problem and used null space algorithm to solve the problem. Also, Nie [5] proposed a new approach by converting an NLES into a constrained nonlinear programming problem, and solved this problem by using a line search sequential quadratic programming approach. Jafari and Daftardar-Gejji [6] suggested a modification of Adomian decomposition method and demonstrated that series solution obtained converges faster than that of standard Adomian decomposition method. Darvishi and Barati [7] presented an iterative third-order Newton-type method based on Adomian decomposition method for solving NLESs. Golbabai and Javidi [8] considered homotopy perturbation method to construct an iterative method for solving the NLES, compared the results with that of the revised Adomian decomposition method in [6] obtained, and showed the accuracy and fast convergence of the proposed method. Biazar and Ghanbary [9] constructed a new iterative approach based on the concept of Jacobi method and presented the effectiveness of the proposed method as the number of equations and variables increases. Grosan and Abraham [10] proposed a novel approach transforming NLES to a multiobjective optimization problem and revealed that it deals with the large scale system of equations. Hosseini and Kafash [11] presented an algorithm based on Adomian decomposition convergence basis method for solving functional equations. Gu and Zhu [12] presented an effective filter algorithm for solving both the nonlinear systems of equalities and inequalities. They transformed the system into a nonlinear programming problem, and used the nonmonotone technique and the global line search strategy in the algorithm. Vahidi et al. [13] implemented the restarted Adomian decomposition method for solving the NLESs and showed that the proposed method converges to the exact solutions more rapidly than the Adomian decomposition method. Sharma and Gupta [14] presented two iterative methods for solving NLES. One of the methods is a third-order method having two-steps which are the Newton iteration and the weighted-Newton iteration, respectively. The other method is a fifth and sixth-order method having three-steps of which the first two steps are same as that of third-order method and third step is the weighted-Newton iteration again. Wang and Pu [15] proposed a nonmonotone filter trust region method to solve the NLES. The system is converted to a nonlinear programming problem in which some equations are treated as constraints whereas the others are taken as objective function. Zhang [16] reviewed some methods, especially iterative methods, of solving system of nonlinear equations in the technical report. Dhamacharoen [17] proposed a new hybrid method having less computations than others. This hybrid method is composed of two methods that are the Newton method and the Broyden method. The proposed method is compared with the Newton method and the Darvishi−Barati method [7], and it is seen that the number of computations is fewer than the compared ones even if it requires more iterations to reach the solution. Izadian et al. [18] proposed a new approach combining Newton method and Homotopy Analysis method to solve the algebraic and transcendental equations system. The main purpose of this combined approach is to accelerate the rate of convergence and to obtain the local convergence. Narang et al. [19] presented a fourth order two parameter Chebyshev-Halley like twopoint family for solving the nonlinear equations of large-scale systems. Saheya et al. [20] presented an improved Newton method based on iterative rational approximation model. Wang and Fan [21] presented two high computational efficient derivative-free iterative methods. The methods have low computational cost by reducing the number of lower-upper decomposition of matrix in each iteration. Xiao and Yin [22] presented a technique using the extended Newton iteration for increasing the order of convergence for iterative methods. They applied the proposed technique to several known methods and obtained new methods having higher order of convergence. Balaji et al. [23] solved the NLES by using the integrated restarted Adomian decomposition method and Adomian decomposition method. Madhu et al. [24] proposed a new method which is an improvement of double-step Newton method. It is two-step fifth-order method in which two functions and two first order Frechet derivatives are used. Sharma and Arora [25] proposed Newtonlike iterative methods of fifth and eighth-order of convergence to solve NLESs.
There are numerious traditional approaches such as Muller method and the Secant method for solving NLESs, however, these methods have many shortcomings. The methods are very sensitive to the choice of initial values and may show oscillatory behavior or even diverge in the case of closeness between the initial value chosen and the root of the system [26]. Moreover, most of these methods require continuously differentiable nonlinear equations. To avoid the negative aspects of the traditional methods, some approaches based on metaheuristic optimization methods such as Genetic Algorithm, Particle Swarm Optimization, Simulated Annealing have been presented. These methods are used with no assumptions about the function being optimized such as smoothness, convexity or differentiability. Dai et al. [27] mixed Genetic Algorithm and quasi-Newton method for solving NLES. Hirsch et al. [28] proposed a modified metaheuristic GRASP method in which all roots are found through the multiple minimizations of an objective function to find all real solutions of NLES. Pourjafari and Mojallali [26] proposed a novel optimization-based method finding all real and complex roots of a system.
In this paper, we introduce a new iterative approach to solve an NLES as an optimization problem. By means of the first order Taylor series expansion and by choosing an arbitrary nonnegative initial point, a system of linearized equations is solved at each iteration. New variables are obtained by adding balancing variables to the initial solution of the system of linearized equations, and then Maclaurin series expansion is used to linearize the NLES reconstructed by substituting these new variables in the system. At each iteration, a LP problem is constructed of which the linearized equations are considered as constraints whereas the objective function is the minimization of the summation of balancing variables. The iterative approach is processed until all ballancing variables are zero, and the optimal solution of the NLES is found.
The organization of the paper is as follows. In Section 2, some brief information is given. In section 3, the proposed approach is presented. In Section 4, some numerical examples and results are demonstrated and the paper ends with conclusion at Section 5.

Preliminaries
In this section, some definitions are given related with the proposed approach. In this paper, it is assumed that each equation in the NLES are continuously differentiable.
linearizes the function f in n variables.

Definition 5. [29] A set of vectors converges if
where k is the number of iterations. The vector x = (x 1 , ..., x n ) is the root of the function if it satisfies that |f i (x)| < ǫ, i = 1, ..., m where ǫ ≥ 0 is a given tolerance.

The proposed approach
A linearization method based on Taylor series expansion is adopted. Each nonlinear multi variable function of the NLES given in Definition 2.1 is considered as f i (x 1 , ..., x n ), (i = 1, ..., m) and A = (a 1 , ..., a n ) is a nonnegative chosen point. By using the linear terms of Taylor series generated at the point A as presented in Definition 2.4, each original nonlinear equation of the NLES is reduced to a linear equation. Because the higher order terms will be close to zero while x j is sufficiently close to a j , we omit them to obtain the approximation. Thus, by using the expansion, each nonlinear function f i in n variables is linearized and a linear equations system is obtained. Using the linear equations system obtained, the algorithm generated to solve NLES is presented below.
Step 1. Load an NLES having m equations in n variables such that (1) Step 2. Choose any initial arbitrary nonnegative point such that A = (a 1 , ..., a n ).
Step 3. Linearize each equation in (1) by generating Taylor series expansion at the chosen point A, and construct a linear equations system having m equations in n variables as follows Step 4. Solve the linearized equations system (2), and obtain a solution (x 1 , ...,x n ).
Step 7. Linearize the NLES obtained in Step 6 by generating Maclaurin series expansion.
Step 8. Construct a LP problem such that where the subscript L defines the linearization, and solve (4).
The flowchart of proposed approach is given in Figure 1.

Numerical experiments
Example 1 [7] Consider the following NLES: Linearize each equation in (5) by generating Taylor series expansion at arbitrary nonnegative point A (3,5). Thus, we have the following linearized equations system as The solution of linearized system (6) is (x 1 , x 2 ) = (4.7143, −0.8571). Then, introduce new variables respectively, and substitute these variables in the NLES (5). After linearizing the NLES (5) by generating Maclaurin series expansion, the following LP problem is constructed: (u j + v j ) Optimal solution of the LP problem (7) is found as (u 1 , v 1 , u 2 , v 2 ) = (0, 2.0383, 1.0191, 0), and it is used to determine new variables as x 1 = 2.6760 + u 1 − v 1 , x 2 = 0.1620 + u 2 − v 2 , respectively. This approach is applied recurrently until all balancing variables are found zero. The summarized results are given in Table 1.
Linearize each equation in (8) by generating Taylor series expansion at arbitrary nonnegative point A (1, 1, 1). The solution of linearized equations system is found as (x 1 , x 2 , x 3 ) = (0.8269, 0.7308, 0.4423). New variables are introduced as x 1 = 0.8269+u 1 −v 1 , x 2 = 0.7308+u 2 − v 2 and x 3 = 0.4423 + u 3 − v 3 , respectively. Constructed LP problems are solved until all balancing variables are found zero, and the desired solution is obtained after four iterations. The summarized results are given in Table 2.   Linearize each equation in (9) by generating Taylor series expansion at point A(0, π/2). The solution of linearized equations system is found as (x 1 , x 2 ) = (1, 0). New variables are introduced as x 1 = 1 + u 1 − v 1 and x 2 = 0 + u 2 − v 2 , respectively. The approach is processed and the solution of (9) is found that is illustrated in Figure 2. The summarized results are given in Table 3.

Conclusion
In this paper, a linearization approach is proposed to solve NLESs. Although our approach based on linearization using Taylor series involves more iterations than many other methods used in the literature, the fundamental of the approach is based on a very basic and important formation. Therefore, this proposed approach can be used to have less computational complexity and easier application and to obtain more accurate results. Numerical experiments are presented from the literature to demonstrate the ability and accuracy of the proposed approach for solving NLES.