Memetic computing through bio-inspired heuristics integration with sequential quadratic programming for nonlinear systems arising in different physical models

Abstract

Background

In this study, bio-inspired computing is exploited for solving system of nonlinear equations using variants of genetic algorithms (GAs) as a tool for global search method hybrid with sequential quadratic programming (SQP) for efficient local search. The fitness function is constructed by defining the error function for systems of nonlinear equations in mean square sense. The design parameters of mathematical models are trained by exploiting the competency of GAs and refinement are carried out by viable SQP algorithm.

Results

Twelve versions of the memetic approach GA-SQP are designed by taking a different set of reproduction routines in the optimization process. Performance of proposed variants is evaluated on six numerical problems comprising of system of nonlinear equations arising in the interval arithmetic benchmark model, kinematics, neurophysiology, combustion and chemical equilibrium. Comparative studies of the proposed results in terms of accuracy, convergence and complexity are performed with the help of statistical performance indices to establish the worth of the schemes.

Conclusions

Accuracy and convergence of the memetic computing GA-SQP is found better in each case of the simulation study and effectiveness of the scheme is further established through results of statistics based on different performance indices for accuracy and complexity.

Background

Introduction

Solving system of non-linear equations in particular based on large set are generally considered to be the most difficult and a challenging problem for the research community in the field of numerical computation. These systems arise frequently in a spectrum of applied mathematics and engineering applications including trajectory planning, kinematics, combustion theory and neurophysiology etc. (Grosan and Abraham 2008a, b; Morgan 1987). Currently, number of numerical methods have been developed to deal with nonlinear equations effectively, however, one of the simplest, oldest and widely used solvers for these problems is the Newton–Raphson method (NRM) (Ortega and Rheinboldt 1970; Kelley 2003). Similar to most of the numerical methods for solving system of non-linear equations, the performance of the NRM can be highly sensitive to the initial guess of the problem and generally fail with bad initial parameters. Therefore, normally, any global search methodology is used to determine the initial bias values which are then supplied to the NRM for solving viably the system of non-linear equations. Besides NRM, many other iterative methods for solving linear and nonlinear equations are reported in the literature with their own strengths, limitations and applicability domain on specific scenarios or environments. For instance, Kelley, Campbell, and Broyden’s classically provide different solvers for these equations (Kelley 1999; Campbell et al. 1996; Darvishi and Barati 2007). Moreover, the Jacobian-free Newton–Krylov method is applied broadly for non-linear equations arising in many applications in which an effective two sided bi-colouring method is used to get the lower triangular half of the sparse Jacobian matrix via automatic differentiation (Broyden 1971; Knoll and Keyes 2004; Saad and van der Vorst 2000). Recently, many researchers, including Jaffari and Gejji, Abbasbandy, Sharma et al., Vahidi et al. have given an updated version of methods to solve the nonlinear system of equation reliably and efficiently (Jafari and Gejji 2006; Abbasbandy 2005; Vahidi et al. 2012; Sharma and Guha 2013; Sharma and Gupta 2013, 2014; Sharma and Arora 2014).

Most of the existing literature available for solving nonlinear system of equation is based on iterative and recursive procedure, and working on these methods is usually dependent on values of initial guess or start point of the algorithms. On the other hand, these systems of equations have been used in modelling of many physical problems arising in a wide spectrum of fields (Morgan 1987; de Soares 2013). Therefore, design of numerical procedures that are accurate, reliable, robust, and efficient, has attracted the research community significantly. The aim of this study is to step further in this domain by exploring and exploiting the strength of soft computing framework (SCF) to determine the solution of systems of nonlinear equation without prior knowledge of biased initial guess or weights. The soft computing techniques based on genetic algorithms and swarming intelligence has been used extensively for different applications such as Van-der-Pol oscillatory systems (Khan et al. 2015), reliable feature selection for Arabic text summarization (Al-Zahrani et al. 2015), effective navigation of mobile robot in unknown environment(Algabri et al. 2014), robust feature selection and classification (Nekkaa and Boughaci 2015), fuel ignition model in combustion theory (Raja 2014), change detection mechanism in synthetic aperture radar images (Li et al. 2015), optimization of multirate quadrature mirror filter bank (Baicher 2012), integrated process planning and scheduling problems (Li et al. 2014), thin film flow of third grade fluids (Raja et al. 2014), Troesch’s problem (Raja 2014), second order system of boundary value problems (Arqub and Abo-Hammour 2014; Abu-Arqub et al. 2014), prediction of linear dynamical systems (Abo-Hammour et al. 2013), Jeffery-Hamel Flow in the presence of high magnetic field (Raja and Samar 2014), Painlevé equations (Raja et al. 2015), modeling of electrical conducting solids (Raja et al. 2016), nanofludics problems (Raja et al. 2016), Riccati fractional differential equations (FrDEs) (Raja et al. 2015), real time cross layer optimization (Elias et al. 2012) and Bagley-Torvik FrDEs (Raja et al. 2011). These are the motivating factors for the authors to explore in this domain. The objective of this study is to design memetic evolutionary techniques based on effective global search and efficient local search methodologies and then apply the proposed SCF for an accurate, effective and reliable solution of system of nonlinear equations.

The rest of the organization of the paper is as follows. In “Methods” section, proposed design methodology is presented for the solutions of nonlinear system of equation by formulation of fitness functions, stepwise working criteria and its learning mechanism. In “Results and discussion” section, the results of numerical experimentations of proposed schemes are presented for six benchmark problems, including application arising in combustion theory, neurophysiology and Kinetic modelling etc. along the comparison of the results in term of performance operators. In “Comparative studies” section, results of the proposed algorithms are compared using statistical performance indicators for both accuracy and complexity. Concluding remarks as well as future research directions are given in the last section.

Methods

In this section, design methodology is presented for finding the solution of a system of nonlinear equations. Our aim is to provide a platform for optimization of variables for the given system in order to find the accurate and precise solution. Genetic algorithms (GAs) is an optimization tool which can be used effectively for finding the solution of a given system of nonlinear equation without using the initial guess.

Formulation of fitness function

Generally, system of nonlinear equation is expressed as:

$${\mathbf{F}}(t) = 0,$$

(1)

or

$${\mathbf{F}}(t) = \left\{ {\begin{array}{*{20}l} {f_{1} (t_{1} ,t_{2} ,t_{3} , \ldots ,t_{n} ) = 0,} \hfill \\ {f_{2} (t_{1} ,t_{2} ,t_{3} , \ldots ,t_{n} ) = 0,} \hfill \\ \vdots \hfill \\ {f_{n} (t_{1} ,t_{2} ,t_{3} , \ldots ,t_{n} ) = 0} \hfill \\ \end{array} } \right..$$

(2)

To find out the precise solution of (1), the first step is to formulate the fitness or objective function on the basis of absolute value of the function or mean square error as:

$${\text{Minimize}}\,\,\left\{ {\,\varepsilon = \left| {{\mathbf{F}}(t)} \right|\,\,\,\,\,{\text{or}}\,\,\,\, = \left( {{\mathbf{F}}(t)} \right)^{2} } \right..$$

(3)

Fitness function (3) depends upon bounded or unbounded constraints on variables and these variables are used to define the given system of nonlinear equations.

The next step is to optimize the formulated fitness function (3) by using reliable optimization mechanisms such that for $\varepsilon \to 0$ then ${\mathbf{F}}(t) \to 0$. Eventually the correspondingly values of the vector $t = (t_{1} ,t_{2} ,t_{3} , \ldots ,t_{n} )$ are the solution set for the given system of nonlinear equation.

Learning methodology

From the last few decades’ mathematicians and researchers made serious efforts to produce quality of initial guesses to find the optimal final solutions. This was really tough, especially when the number of variables involving in the system of nonlinear equations exceeds a specific level. Genetic algorithms (GAs) are basically the modelling of the phenomenon of natural evolution (Miettinen 1999). The effectiveness and systematic operation of GA’s depends upon not only the selection of different constitutional operators, but also on the settings of algorithm, variable parameters and the design constraints.

GAs is considered to be one of the best optimization algorithm as compared to the others because of their multi-dimensionally operations to produce most feasible solutions. GAs are also effective for the problems for which the construction of fitness function is much complex (Johnson et al. 2014; Kociecki and Adeli 2014). In our daily life, most problems have a very large solution space, which is generally not dealt by the ordinary algorithms while GAs deals these situations efficiently and correctly. GAs is considered to be the most viable and accurate method for finding the numerical approximate solutions of the given system of nonlinear equations by determining the best fit from the extensive range of search space. In the present study, the memetic computing approach is developed based on variants of GAs (Raja et al. 2015) hybrid with sequential quadratic programming (SQP) technique to obtain unknown design variables of nonlinear system given in Eq. (1).

The different computing approaches based on variants of GA hybrid with SQP are evaluated for finding the solution of systems of nonlinear equations. The proposed hybrid computing schemes are listed in Table 1, while procedural overview is given graphically in Fig. 1. Necessary details of procedural steps for hybrid computing algorithms are given below:

Step 1:

Initialization Initialize the chromosome with number of elements equal to the number of variable in nonlinear system of equations as:

$$c = (t_{1} ,t_{2} ,t_{3} , \ldots ,t_{n} ),$$

where n represents variables in nonlinear system. Set of chromosome represents an initial population and it is given mathematically as:

$$P = (c_{1} ,c_{2} ,c_{3} , \ldots ,c_{m} ),$$

here m is the total number of chromosomes in the population P.

Initial assignments and declarations of GAs program and are set using MATLAB built-in functions of the optimization toolbox based on ‘ga’ and ‘gaoptimset’ routines. The fix parameter settings of all twelve variants of GAs are used such as 200 generations. The choice of these settings is made with care, after a lot of experimentation, and experience of operating optimization solvers.

Step 2:

Fitness calculation Evaluate the fitness values of each individual or chromosome of the population using a problem specific fitness function as defined in (3).

Step 3:

Termination criteria Terminate the updating process of the algorithm, if any of the following predefined conditions are fulfilled.

• Fitness ε values are less than or equal to 10⁻³⁵.

• Generations, i.e., 400 times step increment are made in the program execution.

• Limited values of any of the functions, tolerance (TolFun), constraints tolerance (TolCon) and stall generation limit (StallGenLimit) is achieved.

Go to step 5 in case of termination conditions are satisfied.

Step 4:

Reproduction Create the next generation of each variant of GAs based on a different set of combination for the reproduction mechanism using selection, crossover, and mutation routines as listed in Table 1. Go to step 2

Step 5:

Hybridization Global best individual of GAs variants is given to a local search method based on SQP algorithm, i.e., the memetic computing approach of learning, for further refinements in the results. SQP algorithm is implemented by invoking ‘fmincon’ function of MATLAB optimization toolbox for constraint problems as per following procedure:

• (a) Initialization Initial weights or start point of SQP algorithm is the global best chromosomes of GAs variants. The bounds, declarations and initial parameters are given in ‘optimset’ function such as number of iteration 500.

• (b) Fitness calculation Calculate the fitness value using the Eq. (3).

• (c) Termination criteria Terminate the cyclic process of updating in variables if any of the following predefine conditions fulfilled.

  • Total number of Iterations/cycles are executed.

  • Limited values for any TolFun, maximum function evaluations (MaxFunEvals), X-tolerance (TolX), and TolCon are achieved as given in ‘optimset’ function.

  Go to the step 6 in case of termination conditions satisfied.

• (d) Updating of variables: Updating of weights is made on each step increment as per SQP procedure and continues from step 5(b).

Step 6:

Storage Store the values of the weights, fitness, generations, MaxFunEvals and time taken for this run in case of all twelve hybrid schemes based on GA-SQP.

Step 7:

Statistical analysis Repeat the procedure for a sufficient large number of times from step 1 to step 6 to generate large data set for reliable and effective analysis of the performance of the algorithms.

Table 1 Proposed algorithms for system of nonlinear equations

Fig. 1

Graphical view of proposed design methodology

Results and discussion

In this section, results of proposed schemes for solving system of nonlinear equations are presented. Six different problems are taken for the study and proposed methods base on twelve variants of memetic computing using GAs and SQP algorithms are applied for these equations. Six different models of system of nonlinear equation are taken for numerical experimentation to check the effectiveness of the proposed design schemes. These models with governing mathematical relations are described in six problems.

Problem 1: generic nonlinear system of equations

In this case, a generic system of nonlinear equations based on four equations with four unknowns is taken and it is represented by the following mathematical relations (Grosan and Abraham 2008b)

$$\left\{ {\begin{array}{*{20}l} {f_{1} ({\mathbf{t}}) = f_{1} (t_{1} ,\,t_{2} ,\,t_{3} ,\,t_{4} ) = t_{1}^{2} + 2t_{2}^{2} + \cos (t_{3} ) - t_{4}^{2} = 0} \hfill \\ {f_{2} ({\mathbf{t}}) = f_{2} (t_{1} ,\,t_{2} ,\,t_{3} ,\,t_{4} ) = 3t_{1}^{2} + t_{2}^{2} + \sin^{2} (t_{3} ) - t_{4}^{2} = 0} \hfill \\ {f_{3} ({\mathbf{t}}) = f_{3} (t_{1} ,\,t_{2} ,\,t_{3} ,\,t_{4} ) = - 2t_{1}^{2} - t_{2}^{2} - \cos (t_{3} ) + t_{4}^{2} = 0} \hfill \\ {f_{4} ({\mathbf{t}}) = f_{4} (t_{1} ,\,t_{2} ,\,t_{3} ,\,t_{4} ) = - t_{1}^{2} - 2t_{2}^{2} \, - \cos^{2} (t_{3} ) + t_{4}^{2} = 0} \hfill \\ \end{array} } \right..$$

(4)

The exact solution for system (4) is (1, −1, 0, 2) for (t ₁, t ₂, t ₃, t ₄), respectively, and these solutions are used to verify the applicability of the proposed methods. The design approaches are applied to solve the system (4) as per procedure and settings given in “Results and discussion” section, however, the fitness function formulated in this case is given as:

$$\varepsilon = \frac{1}{4}\left( {\begin{array}{*{20}l} {\left( {t_{1}^{2} + 2t_{2}^{2} + \cos (t_{3} ) - t_{4}^{2} } \right)^{2} + \left( {3t_{1}^{2} + t_{2}^{2} + \sin^{2} (t_{3} ) - t_{4}^{2} } \right)^{2} } \hfill \\ {\quad +\, \left( { - 2t_{1}^{2} - t_{2}^{2} - \cos (t_{3} ) + t_{4}^{2} } \right)^{2} + \left( { - t_{1}^{2} - 2t_{2}^{2} \, - \cos^{2} (t_{3} ) + t_{4}^{2} } \right)^{2} } \hfill \\ \end{array} } \right).$$

(5)

Optimization of fitness function (5) is carried out with twelve variants of memetic computing algorithms and one set of solution obtained by GAs and GA-SQP algorithm is shown in Fig. 2 for each variant. While these adaptive parameters along with their fitness values are tabulated in Table 2.

Fig. 2

Trained parameters of GA and GA-SQP algorithms in cases of problem 1, while the figures (a–l), represent the results of variant 1–12 of proposed methodology, respectively

Table 2 Comparison of trained parameters along with their fitness for GA and GA-SQP algorithms in case of problem 1

The optimized variables given in Table 2 and Fig. 2 are used Eq. (4) in order to determine the values of $f_{1} ({\mathbf{t}})$, $f_{2} ({\mathbf{t}})$, $f_{3} ({\mathbf{t}})$ and, $f_{4} ({\mathbf{t}})$ and results are given in Table 3. The smaller is the values of fitness then better is the performance of the algorithm

Table 3 Comparison of the performance on the basis of absolute values of constitutional equations of problem 1

It is seen from Table 2 that the values of fitness ε for variants of GAs are around 10⁻⁰⁴ to 10⁻¹⁰ while these values for the hybrid approach GA-SQP variants are around 10⁻¹⁷. Moreover, it is seen from Table 3 that the values of, $f_{i} ({\mathbf{t}})$, i = 1–4, for GAs and GA-SQP algorithms lie 10⁻⁰² to 10⁻⁰⁵, and 10⁻⁰⁹, respectively. There is no noticeable difference between the performances of the memetic computing approaches; however, the best results are obtained with GA-SQP-8 algorithm. It is observed that generally highly accurate results are determined by memetic computing techniques than variants of GAs.

Problem 2: interval arithmetic benchmark model

In this case, the performance of the proposed methods is evaluated by taking a renewed problem of nonlinear systems named as an interval arithmetic benchmark model (IABM). The governing mathematical relations for IABM is given in the form of following nonlinear system of equations (Grosan and Abraham 2008b; Van Hentenryck et al. 1997; Hong and Stahl 1994):

$$\left\{ {\begin{array}{*{20}l} {f_{1} ({\mathbf{t}}) = t_{1} - 0.25428722 - 0.18324757t_{4} t_{3} t_{9} = 0} \hfill \\ {f_{2} ({\mathbf{t}}) = t_{2} - 0.37842197 - 0.16275449t_{1} t_{10} t_{6} = 0} \hfill \\ {f_{3} ({\mathbf{t}}) = t_{3} - 0.27162577 - 0.16955071t_{1} t_{2} t_{10} = 0} \hfill \\ {f_{4} ({\mathbf{t}}) = t_{4} - 0.19807914 - 0.15585316t_{7} t_{1} t_{6} = 0} \hfill \\ {f_{5} ({\mathbf{t}}) = t_{5} - 0.44166728 - 0.19950920t_{7} t_{6} t_{3} = 0} \hfill \\ {f_{6} ({\mathbf{t}}) = t_{6} - 0.14654113 - 0.18922793t_{8} t_{5} t_{10} = 0} \hfill \\ {f_{7} ({\mathbf{t}}) = t_{7} - 0.42937161 - 0.21180486t_{2} t_{5} t_{8} = 0} \hfill \\ {f_{8} ({\mathbf{t}}) = t_{8} - 0.07056438 - 0.17081208t_{1} t_{7} t_{6} = 0} \hfill \\ {f_{9} ({\mathbf{t}}) = t_{9} - 0.34504906 - 0.19612740t_{10} t_{6} t_{8} = 0} \hfill \\ {f_{10} ({\mathbf{t}}) = t_{10} - 0.42651102 - 0.21466544t_{4} t_{8} t_{1} = 0} \hfill \\ \end{array} } \right.,$$

(6)

where ${\mathbf{t}} = \left( {t_{1} , \, t_{2} , \ldots ,t_{10} } \right)$. The design memetic computing approaches are applied to solve the system (5) on a similar pattern as adopted in last problem; however, the fitness function formulated in this case is given as follows:

$$\varepsilon = \frac{1}{10}\left( {\left( {f_{1} ({\mathbf{t}})} \right)^{2} + \left( {f_{2} ({\mathbf{t}})} \right)^{2} + \left( {f_{3} ({\mathbf{t}})} \right)^{2} + \cdots \left( {f_{10} ({\mathbf{t}})} \right)^{2} } \right)$$

(7)

Optimization of fitness function (7) is carried out with the proposed scheme and results are shown graphically in Fig. 3 for variants of GAs and GA-SQP algorithms. The absolute values of constitutional equations are calculated and results are tabulated in Table 4 for each variant. It is seen that the maximum values of $f_{i} ({\mathbf{t}})$, i = 1–10, for GA and GA-SQP algorithms are 1.13 × 10⁻⁰² and 1.46 × 10⁻¹⁶, respectively. It is observed that generally the hybrid computing approaches give the results with higher precision from the rest.

Fig. 3

Trained parameters of GA and GA-SQP algorithms in case of problem 2, while the figures (a–l), represent the results of variant 1–12 of proposed methodology, respectively

Table 4 Comparison of trained parameters along with their fitness for GA and GA-SQP algorithms in case of problem 2

Problem 3: chemical equilibrium applications

Consider another potential example of a system of nonlinear equations arises in chemical equilibrium applications (CEA) (Grosan and Abraham 2008b; Meintjes and Morgan 1990) to evaluate the performance of the proposed schemes. Mathematical model of CEA is given below:

$$\left\{ {\begin{array}{*{20}l} {f_{1} ({\mathbf{t}}) = t_{1} t_{2} + t_{1} - 3t_{5} = 0} \hfill \\ {f_{2} ({\mathbf{t}}) = 2t_{1} t_{2} + t_{1} + t_{2} t_{3}^{2} + c{}_{8}t_{2} - ct_{5} + 2c_{10} t_{2}^{2} + c_{7} t_{2} t_{3} + c_{9} t_{2} t_{4} = 0} \hfill \\ {f_{3} ({\mathbf{t}}) = 2t_{2} t_{3}^{2} + 2c_{5} t_{3}^{2} - 8t_{5} + c_{6} t_{3} + c_{7} t_{2} t_{3} = 0} \hfill \\ {f_{4} ({\mathbf{t}}) = c_{9} t_{2} t_{4} + 2t_{4}^{2} - 4ct_{5} = 0} \hfill \\ {f_{5} ({\mathbf{t}}) = t_{1} (t_{2} + 1) + c_{10} t_{2}^{2} + t_{2} t_{3}^{2} + c_{8} t_{2} + c_{5} t_{3}^{2} + t_{4}^{2} - 1 + c_{6} t_{3} + c_{7} t_{2} t_{3} + c_{9} t_{2} t_{4} = 0} \hfill \\ \end{array} } \right.,$$

(8)

where $c = 10,$ $\,c_{5} = 0.193,$ $\,c_{6} = 0.000410,\,$ $c_{7} = 0.000545,$ $\,c_{8} = 0.000000449,$ $c_{9} = 0.0000340,$ and $c_{10} = 0.000000961$. The designed computing approaches are applied to solve the system (8) on a similar pattern as adopted in last problems, but the fitness function is formulated for this case as:

$$\varepsilon = \frac{1}{5}\left( {\left( {f_{1} ({\mathbf{t}})} \right)^{2} + \left( {f_{2} ({\mathbf{t}})} \right)^{2} + \left( {f_{3} ({\mathbf{t}})} \right)^{2} + (f_{4} ({\mathbf{t}}))^{2} + \left( {f_{5} ({\mathbf{t}})} \right)^{2} } \right)$$

(9)

An optimization problem based on a fitness function (9) is solved with twelve variants of memetic computing. The trained design variables are given in Table 5. These parameters are used to calculate the values of functions $f_{1} ({\mathbf{t}})$, $f_{2} ({\mathbf{t}})$, $f_{3} ({\mathbf{t}})$, $f_{4} ({\mathbf{t}})$ and $f_{5} ({\mathbf{t}})$. The results are given in Table 6. It is seen from Table 5 that the values of fitness ε for variants of GAs are of the order 10⁻⁰⁴ to 10⁻⁰⁵ while these values for variants of hybrid approach GA-SQPs are around 10⁻¹⁷. Additionally, it is seen from Table 6 that the values of $f_{i} ({\mathbf{t}})$, i = 1–5, for GA and GA-SQP algorithms lie around 10⁻⁰² to 10⁻⁰⁶, and 10⁻¹⁰, respectively. It is observed that the best results are obtained with GA-SQP-8 algorithm from the rest.

Table 5 Comparison of trained parameters along with their fitness for GA and GA-SQP algorithms in case of problem 3

Table 6 Comparison of the performance on the basis of absolute values of constitutional equations of problem 3

Problem 4: neurophysiology applications

In this case, the performance of the design scheme is examined on Bioinformatics problem based on neurophysiology applications (NPAs). Following a system of nonlinear equations representing NPA as (Grosan and Abraham 2008b; Verschelde et al. 1994):

$$\left\{ {\begin{array}{*{20}l} {f_{1} ({\mathbf{t}}) = t_{1}^{2} + t_{3}^{2} = 1} \hfill \\ {f_{2} ({\mathbf{t}}) = t_{2}^{2} + t_{4}^{2} = 1} \hfill \\ {f_{3} ({\mathbf{t}}) = t_{5} t_{3}^{2} + t_{6} t_{4}^{2} = c_{1} } \hfill \\ {f_{4} ({\mathbf{t}}) = t_{5} t_{1}^{3} + t_{6} t_{2}^{3} = c_{2} } \hfill \\ {f_{5} ({\mathbf{t}}) = t_{5} t_{1} t_{3}^{2} + t_{6} t_{4}^{2} t_{2} = c_{3} } \hfill \\ {f_{6} ({\mathbf{t}}) = t_{5} t_{1}^{2} t_{3} + t_{6} t_{2}^{2} t_{4} = c_{4} } \hfill \\ \end{array} } \right.,$$

(10)

where the values of constant c₁ to c₄ are chosen arbitrarily and ${\mathbf{t}} = \left( {t_{1} , \, t_{2} , \ldots ,t_{6} } \right)$. In model (10) the values of constant are taken zero. The design memetic computing approaches are applied to solve (10) by formulation of the fitness function in this case as:

$$\varepsilon = \frac{1}{6}\left( {\left( {f_{1} ({\mathbf{t}})} \right)^{2} + \left( {f_{2} ({\mathbf{t}})} \right)^{2} + \left( {f_{3} ({\mathbf{t}})} \right)^{2} + (f_{4} ({\mathbf{t}}))^{2} + \left( {f_{5} ({\mathbf{t}})} \right)^{2} + \left( {f_{6} ({\mathbf{t}})} \right)^{2} } \right).$$

(11)

Optimization of fitness function (11) is carried out with design variants of hybrid computing algorithms and results are tabulated in Table 7 along with the value of fitness and also in Table 8 for constitutional equations. It is seen that the values of fitness in case of GAs are around 10⁻⁰⁹ to 10⁻¹⁵ while these values for the hybrid approach GA-SQP variants are around 10⁻²⁴. Generally, observed that highly accurate results are determined by hybrid methodologies for this problem also.

Table 7 Comparison of trained parameters along with their fitness for GA and GA-SQP algorithms in case of problem 4

Table 8 Comparison of the performance on the basis of absolute values of constitutional equations of problem 4

Problem 5: combustion theory applications

Well-known problem of combustion theory with temperature around 3000 °C is taken in this case (Morgan 1987; Grosan and Abraham 2008b). Governing mathematical relations for the problem in term of system of nonlinear equations is given as follows:

$$\left\{ {\begin{array}{*{20}l} {f_{1} ({\mathbf{t}}) = t_{2} + 2t_{6} + t_{9} + 2t_{10} = 10^{ - 5} } \hfill \\ {f_{2} ({\mathbf{t}}) = t_{3} + t_{8} = 3 \times 10^{ - 5} } \hfill \\ {f_{3} ({\mathbf{t}}) = t_{1} + t_{3} + 2t_{5} + 2t_{8} + t_{9} + t_{10} = 5 \times 10^{ - 5} } \hfill \\ {f_{4} ({\mathbf{t}}) = t_{4} + 2t_{7} = 10^{ - 5} } \hfill \\ {f_{5} ({\mathbf{t}}) = 0.5140437\, \times 10^{ - 7} t_{5} = t_{1}^{2} } \hfill \\ {f_{6} ({\mathbf{t}}) = 0.1006932 \times 10^{ - 6} t_{6} = 2t_{2}^{2} } \hfill \\ {f_{7} ({\mathbf{t}}) = 0.7816278 \times 10^{ - 15} t_{7} = t_{4}^{2} } \hfill \\ {f_{8} ({\mathbf{t}}) = 0.1496236 \times 10^{ - 6} t_{8} = t_{1} t_{3} } \hfill \\ {f_{9} ({\mathbf{t}}) = 0.6194411 \times 10^{ - 7} t_{9} = t_{1} t_{2} } \hfill \\ {f_{10} ({\mathbf{t}}) = 0.2089296 \times 10^{ - 14} t_{10} = t_{1} t_{2}^{2} } \hfill \\ \end{array} } \right.$$

(12)

Here, the input vector ${\mathbf{t}} = \left( {t_{1} , \, t_{2} , \ldots ,t_{10} } \right)$. The design computing approaches are applied to this case by constructing a fitness function as given below:

$$\varepsilon = \frac{1}{10}\left( {\left( {f_{1} ({\mathbf{t}})} \right)^{2} + \left( {f_{2} ({\mathbf{t}})} \right)^{2} + \left( {f_{3} ({\mathbf{t}})} \right)^{2} + , \ldots , + \left( {f_{10} ({\mathbf{t}})} \right)^{2} } \right).$$

(13)

Optimization of fitness function (13) is carried out with the designed schemes and results are represented graphically in Fig. 4 for variants of GAs and GA-SQP algorithms. The absolute values of constitutional equations are calculated and results are tabulated in Table 9 for each variant. It is seen that the maximum values of $f_{i} ({\mathbf{t}})$, i = 1–10, for GA and GA-SQP algorithms are 1.00 × 10⁻⁰⁹ and 1.15 × 10⁻⁰⁸, respectively. It is observed that generally hybrid computing approaches outperformed the rest of techniques.

Fig. 4

Trained parameters of GA and GA-SQP algorithms in case of problem 5, while the figures (a–l), represent the results of variant 1–12 of proposed methodology, respectively

Table 9 Comparison of the performance on the basis of absolute values of constitutional equations of problem 5

Problem 6: economics modelling application

Another problem is selected for the study arising in econometric modelling applications (EMAs) extensively based on system of nonlinear equation of arbitrary dimensions (Morgan 1987; Grosan and Abraham 2008b). These problems are generally considered to be stiff to solve numerically and given in terms of mathematical relations as:

$$\left\{ {\begin{array}{*{20}l} {\left( {t_{k} + \sum\limits_{i = 1}^{n - k - 1} {t_{i} t_{i} + k} } \right)t_{n} - c_{k} = 0;\quad 1 \le k \le n - 1} \hfill \\ {\sum\limits_{i = 1}^{n - 1} {t_{i} + 1} = 0.} \hfill \\ \end{array} } \right.,$$

(14)

here the value of “c_k” can be randomly chosen. A special case of the problem (14) is taken using n = 5 and k = 1 to 4 for evaluation of the proposed algorithms and its governing nonlinear system of the equations are written as:

$$\left\{ {\begin{array}{*{20}l} {f_{1} ({\mathbf{t}}) = t_{1} + t_{2} + t_{3} + t_{4} = - 1} \hfill \\ {f_{2} ({\mathbf{t}}) = (t_{1} + t_{1} t_{2} + t_{2} t_{3} + t_{3} t_{4} )t_{5} = 1\quad\,for\,k = 1} \hfill \\ {f_{3} ({\mathbf{t}}) = (t_{2} + t_{1} t_{3} + t_{2} t_{4} )t_{5} = 1\quad\,for\,k = 2} \hfill \\ {f_{4} ({\mathbf{t}}) = (t_{3} + t_{1} t_{4} )t_{5} \, = \,1\quad\,for\,k\, = \,3} \hfill \\ {f_{5} ({\mathbf{t}}) = t_{4} t_{5} = 1\quad\,for\,k\, = 4} \hfill \\ \end{array} } \right.,$$

(15)

where ${\mathbf{t}} = \left( {t_{1} , \, t_{2} , \ldots ,t_{5} } \right)$. The design approaches are applied to solve the system (15) with the help of optimization of the fitness function as defined below:

$$\varepsilon = \frac{1}{5}\left( {\left( {f_{1} ({\mathbf{t}})} \right)^{2} + \left( {f_{2} ({\mathbf{t}})} \right)^{2} + \left( {f_{3} ({\mathbf{t}})} \right)^{2} \left( {f_{4} ({\mathbf{t}})} \right)^{2} + \left( {f_{5} ({\mathbf{t}})} \right)^{2} } \right).$$

(16)

Results of proposed adaptive algorithms are tabulated in Tables 10 and 11 and these results show that values of fitness ε for GAs lie around 10⁻⁰⁷ to 10⁻¹⁰ while these values for GA-SQP variants are around 10⁻³³. Generally, it is observed that the most accurate results are determined by hybrid computing platforms.

Table 10 Comparison of trained parameters along with their fitness for GA and GA-SQP algorithms in case of problem 6

Table 11 Comparison of the performance on the basis of absolute values of constitutional equations of problem 6

Comparative studies

In this section, comparative studies based on the results of statistical analysis are presented for variants of GA and GA-SQP for all the six systems of nonlinear equations. These analyses are used to draw reliable and constructive inferences on the performance of designed algorithms.

Statistical performance indicators

Statistical performance indicator base on mean and standard deviation (STD) are used to analyze the performance for each variant of hybrid technique GA-SQP for 100 independent runs of the algorithm to solve all six nonlinear equations. Results based on the data generated from these simulations are used to draw a constructive and effective inference for the performance of each algorithm.

Hundred independent runs for all 12 proposed schemes are performed to determine the solution of six problems based on nonlinear equations using the similar procedure as adopted in the last section. Results based on values of statistical indices are given in Table 12, while the values based on truncated data for 75 best runs, i.e., the run with minimum fitness values. Analysis based on truncated runs is given due to the fact that one single bad run of the algorithm spoiled all the results. It is observed that the values of mean for GA, GA-SQP variants lie in the range of 1.1 × 10⁻⁰² to 4.4 × 10⁻⁰⁷ and 1.2 × 10⁻⁰² to 1.9 × 10⁻³² respectively, while from Table 12 these respective values lie in the range of 1.0 × 10⁻⁰³ to 1.1 × 10⁻⁰⁹ and 1.2 × 10⁻⁰⁸ to 9.1 × 10⁻³². Small values of the mean along with its STD are generally observed from each hybrid computing mechanism.

Table 12 Comparison of results on the basis of statistical performance indices for truncated 75% independent runs with better fitness

The accuracy and convergence of the proposed design hybrid schemes are analyzed based on 100 independent executions of algorithms. Results on the basis of value of fitness ε against number of independent runs on semi-logarithmic scale are plotted for GA-1 and GA-SQP-1 for all six problems in Figs. 5 and 6, respectively, while the results for the variants of hybrid computing approaches in case of problem 2 are plotted in Fig. 7. Moreover, graphical illustrations of results in cases of different algorithms on different problems are given in Fig. 8. It is seen that the performance of magnetic versions is much superior to a variant of GAs. For first five problems, almost 100% convergent and accurate solutions are found by the hybrid schemes, however, very few, i.e., less than 5% runs, are observed to trap in premature convergence in case of problem 6. The most accurate results are found for problem 2, 4 and 6 by the design hybrid computing techniques.

Fig. 5

Plot of fitness against hundred independent runs for GA-1 algorithm in case of all six problems, while the figures (a–f) represent the results of problems 1–6, respectively

Fig. 6

Plot of fitness against hundred independent runs for GA-SQP-1 algorithm in case of all six problems, while the figures (a–f) represent the results of problems 1–6, respectively

Fig. 7

Plot of fitness against hundred independent runs for variants of GA-SQP algorithm in case of problem 2, while the figures (a–e), represent the results of variant 2–6, respectively

Fig. 8

Plot of fitness against hundred independent runs for variants of GA-SQP algorithm in case of all six problems, while the figures (a–f), represent the results of variant of problem 1–6, respectively

Results of proposed variants of GA-SQP are compared with simulated annealing (SA), pattern-search (PS), Nelder–Mead (NM) and Levenberg–Marquardt (LM) algorithm. Results of SA, PS, NM and LM methods are determined using built-in functions in Matlab environment for all six problems of nonlinear system of equations. The mean fitness values based on 20 runs of the SA, PS, NM and LM approaches lies in the range 10⁻⁰² to 10⁻⁰⁴, 10⁻⁰² to 10⁻⁰⁵, 10⁻⁰³ to 10⁻¹⁷ and 10⁻⁰² to 10⁻⁰³, respectively. It is clearly inferred that results of proposed GA-SQP algorithm is much superior from all four algorithm based on SA, PS, NM and LM methods in case of each problem.

Complexity analyses are conducted based on the values of three operators, i.e., mean execution time (MET), mean generations (MGENs), mean number of function evaluations (MFEs). Complexity operator MET, MGENs, and MFEs are defined as the average time consumed for training of weights, average generations executed and average number of functions executed for the algorithms, respectively. The values of MET MGENs, and MFEs based on 100 independent runs of each technique for all six case studies are tabulated in Table 13. It is seen from the results given in tables that there is no significant difference in value of complexity operators in case of all six problems., i.e., all the variant of GA the values of MET, MGEN and MFE operator are around 2.75 ± 0.75 s(s), 198 ± 2 and 6000 ± 30, respectively, while the memetic versions, the respective values three operators are around 25 ± 10 s(s), 600 ± 100 and 30,000 ± 9000. The shortest time taken from GA variants in GA-6, while in case of hybrid computing approach GA-SQP-12 take smallest time for learning of weights. All the calculations for the simulation studies are conducted on a Dell Inspiron mini laptop with a 1.33 GHz processor, 1.00 GB RAM, and running MATLAB version 2012b in Microsoft Windows XP environment.

Table 13 Complexity analyses bases on values of MET, MGen and MFE indices

To elaborate the performance of proposed schemes the analyses continues by defining the global operators based on mean fitness (MFit), truncated MFit (TMFit) global MET (GMET), global MGens (GMGens), and global MFEs (GMFEs). The term global stands for the average values calculated for a number of independent executions of algorithms. The values of global operators obtained from 100 independent runs of each algorithm are listed in Table 14 and these values are based on the overall performance of algorithms in case of all six problems. Small variations in the results are generally observed, however, on the basis of MFit values the performance of GA-1 is the best, while in memetic versions, the performance of GA-SQP-10 is superior from the rest. The values of global complexity operators GMET, GMGens and GMFEs are the best for GA-QSP-12, GA-SQP-06, and GA-QSP-01 respectively. The complexity of the memetic computing approaches is a bit more than that of GAs variants but this aspect can be overshadowed due to their dominance of the performance in terms of accuracy and convergence.

Table 14 Comparative studies bases on values of global performance indices

Conclusions and future research directions

Conclusions are listed as follows:

• Design of stochastic computational intelligence algorithms based on memetic computing using variants of GAs hybrid with SQP algorithm are developed in this study for solving systems of nonlinear equations arising in arithmetic benchmark, chemical equilibrium, neurophysiology, combustion theory, and economics models and proposed results established the accuracy, reliability and effectiveness.

• Accuracy and convergence of the memetic computing GA-SQP are found better than that of GAs in case of simulation studies performed for all six problems.

• Validation and verification of the performance of memetic algorithms are evaluated on the basis of statistical indicators in terms of the mean and standard deviations; which are calculated for 100 independent runs for all six nonlinear systems and small values of these indices show that proposed algorithms are consistent.

• The correctness of the proposed schemes are examined further based on values of global performance indices, i.e., MFit, GMET, GMGens and GMFEs, and results show that all given schemes provide viable but the performance of GA-SQP-4 is relatively better from the rest.

• Computational complexity analyses of the proposed design scheme GA-SQP in terms of MET, MGens and MFEs values show that there is no prominent variation found in complexity operators. However, the complexity of the memetic computing approaches is on the higher side as compared with GAs but this factor can be over-shadowed due to the superior performance of hybrid algorithms from the rest.

• Beside the provision of reliable and viable solutions of nonlinear systems of equations other valuable advantages of the proposed schemes are simplicity of the approach, easily understandable methodologies, implementation ease, readily extendable to different applications and availability of the solutions without prior known initial bias guess.

Future research opening bases on present study are listed below:

• Modern optimization solvers may play their significant role to enhance accuracy and convergence in solving nonlinear system of equations. Few recently introduced such schemes include fractional particle swarm optimization (PSO) algorithm, fractional Darwinian PSO, chaotic PSO, genetic programming, differential evolution, chaos optimization algorithms and gravitational search algorithms etc.

• One should explore to extend the application of these variants of memetic algorithms to solve stiff nonlinear differential equation, differential–algebraic systems and integral equations by transforming into a system of nonlinear equations with the help of discretization process.