Optimal control analysis of deterministic and stochastic epidemic model with media awareness programs

Article History: Received 29 November 2016 Accepted 30 July 2017 Available 01 November 2018 The present study considered the optimal control analysis of both deterministic differential equation modeling and stochastic differential equation modeling of infectious disease by taking effects of media awareness programs and treatment of infectives on the epidemic into account. Optimal media awareness strategy under the quadratic cost functional using Pontrygin’s Maximum Principle and Hamiltonian-Jacobi-Bellman equation are derived for both deterministic and stochastic optimal problem respectively. The Hamiltonian-Jacobi-Bellman equation is used to solve stochastic system, which is fully non-linear equation, however it ought to be pointed out that for stochastic optimality system it may be difficult to obtain the numerical results. For the analysis of the stochastic optimality system, the results of deterministic control problem are used to find an approximate numerical solution for the stochastic control problem. Outputs of the simulations shows that media awareness programs place important role in the minimization of infectious population with minimum cost.


Introduction
Epidemiology is the study of the spread of diseases with the objective to trace factors, which are responsible for or contribute to their occurrence.Mathematical modeling of the spread of infectious diseases continues to become an important tool in understanding the dynamics of diseases and in decision making processes regarding diseases intervention programs for disease in many countries.Controlling infectious diseases has been an increasingly complex issue in recent years.Media awareness program is an important strategy for the elimination of infectious diseases [1,2].The field of stochastic modeling of biological and ecological systems [3] is currently undergoing considerable development as of complex stochastic models by simulation methods are more feasible.Mathematicians have contributed a range of papers which can be found in the literature of probability theory and statistical physics characterizing the theoretical properties of a large variety of stochastic models.
Optimal control theory has found wide-ranging applications in biological and ecological problems.Specifically, there have been various studies of epidemiological models, where optimal control methods have been applied [4,5].Optimal control theory is a systematic approach to controller design where by the desired performance objectives are encoded in a cost function, which is subsequently optimized to determine the desired controller [6].There are two underlying and universal themes i.e., dynamic programming and filtering.Dynamic programming is one of the fundamental tool of optimal control, the other being Pontryagins principle.Dynamic programming is a means by which candidates optimal control can be verified optimally.The procedure is to find a suitable solution to dynamic programming equation (DPE), which denotes the optimal performance and to use it to compare the performance candidates control may be determined from Pontryagins Maximum Principle [7] and later developed by Fleming and Rishel [8] is successfully applied in a number of studies, to explore optimal control theory in some mathematical models for infectious diseases.Epidemic models are inevitably affected by environmental white noise, which is an important component in realism, because it can provide an additional degree of realism in comparison to their deterministic counterparts.Many stochastic model for epidemic populations have been developed in literature [9,10].For SDE models in epidemiology, optimal control has not been studied extensively.One of the reasons for this could very well be the difficulty with high dimensionality of the resulting partial differential equation (PDE) for the value function (See, Sulem and Tapiero [11]), for instance, a four-compartmental SIVR model such as in [12,13] could easily lead to a PDE having the time variable together with three state variables.In control problems, the aim of the study is to characterize the control variable on a finite time interval, which minimizes the number of infected individuals balanced against the cost of controlling the epidemic.
In the present study it is proposed and developed optimal control policies for deterministic and stochastic SIR epidemic model with awareness programs by media.The aim of this model is to depict how the provision of awareness modifies the contact structure and thereby affects the future course of an epidemic.In the absence of any pharmaceutical intervention, to control the spread of disease at the population level needs to change the individual activities, which in turn depends on information being provided to the individuals about the epidemic.If the susceptibles are aware about the preventive measures for emergent disease, they are likely to modify their activities.The study contracts on disease which spread through interaction between susceptible and infective, i.e. direct contact.Therefore to control the outbreak of any epidemic, it is informed to avoid contact, by which some can contract infection and minimize the possibility of contracting infection.In vision of this, it is assumed that when awareness is propagated by media about the disease, susceptible form a separate class within the population i.e., to avoid being in contact with other members of the population.Another important aspect of this study is to check whether size of the infectious population is directly proportional to awareness campaigns by media.The explicit inclusion of awareness campaigns by media in the modeling process are assumed to be proportional to the size of infectious individuals in the population.This study differs from other epidemic modeling by performing the stochastic optimal control analysis, which is rarely studied by researchers like [6,14] in the field of epidemics and by including the transmission of infection in two modes in the model, A.K. Misra [1], has discussed the epidemic model with media awareness and stability analysis for deterministic model by considering single transmission parameter β, in the present study, the transmission of infection is considered by two modes i.e. transmission between unaware susceptible and infectives and the transmission between hospitalized individuals and unaware susceptible denoted by by β 1 and β 2 respectively.It is assumed that the rate of contact of susceptible with infectives who are on treatment is much less than the infectives who are not on treatment (β 2 ≪ β 1 ).This is so because on hospitalization of infectives for treatment their contact with susceptible group of a population is reduced and may contribute little to the spread of infection.In the numerical analysis of the deterministic and corresponding stochastic model, it is discussed the comparison of deterministic and stochastic solution and also shown, how the control variable vary for different values of a parameters.The rest of the work is organized as follows: Section 2 deals with deterministic model framework and optimal control analysis, while in Section 3 formulation of stochastic model with constant controls and optimal control analysis is carried out.Section 4, consist of numerical simulations and discussion of results and principle findings of the paper are discussed in Section 5.

Deterministic Model
In this section, deterministic nonlinear SIR model is considered by taking media awareness and treatment into account.The variables and parameters of the model are described in Table 1 and Table 2 respectively.

Variables Explanation X(t)
The number of susceptible at time t; Y (t) The number of infectives at time t; X m (t) The number of aware susceptible at time t; T (t) The infectives who are on treatment at time t; Z(t) The recovered population at time t; M (t) The cumulative density of awareness programs driven by media in the region at time t; To model the situation considered a region with total population N (t) at any instant of time t.By taking into account the aforementioned considerations, the system of equations that capture the dynamics of the infectious disease is designed and the ordinary differential equations of the system (1) is as follows.The modification parameter due to treatment for recovery; σ 2 The modification parameter for disease induced death rate due to treatment; µ The rate at which awareness programs has being implemented; µ 0 The depletion rate of awareness programs due to infectiveness, social problems in population; γ 0 The loss rate of immunity of recovered individuals; φ The rate at which infective are hospitalized for treatment; where, X > 0, Y > 0, X m ≥ 0, T ≥ 0, Z ≥ 0 and M ≥ 0.
To show the existence of the feasible set of a system (1) which attracts all solutions initiation in the interior of positive orthant, it has to prove that the system (1) is dissipative, i.e., all solutions are uniformly bounded in a proper subset Ω ∈ ℜ 6 + .Let (X, Y, X m , T, Z, M ) ∈ ℜ 6 + be any solution with non-negative initial conditions.By adding first five equations of system (1) it is obtained After solving equation ( 2), we have where N (0) is the sum of initial values and above result that Q δ 1 is the upper bound of N it can deduced that lim t → ∞ M → µQ µ 0 δ 1 .Therefore the region of attraction is given by the set: and attracts all solutions initiation in the interior of positive orthant.

Deterministic optimal control problem
In this section it is formulated and solved for deterministic version of control problem.The control variable in the model system (1), where implementation rate of awareness campaigns (µ) is represented by a Lebesgue measurable function u(t), on a finite time interval [0, T f ].In the model u(t) represents the some part of susceptible population has media awareness at time t.Our aim is to obtain optimal media awareness programs u * (t), which minimizes the number of infectives and on the other hand cost of infection (treatment) during the infectious period [0, T f ].To investigate the optimal level of efforts that would be needed to control the disease, the objective function J is formed.Since objective of the spread of disease control is to decrease the infected individuals and infected individuals who are on treatment and increase the aware susceptible population.Hence the problem of minimizing the cost functional is, where, X > 0, Y > 0, X m ≥ 0, T ≥ 0, Z ≥ 0 and M ≥ 0. A, B and C are the positive weights.The term C 1 2 is the cost associated with u(t).An optimal control u * (t) is such that where control set is defined as u(t) is Lebesgue measurable} .

Existence of deterministic optimal control problem
The existence of optimal control can be proved by using the result from Fleming and Rishel [8] .
Theorem 1.For the optimal control problem ( 6) and ( 7) on a fixed interval [0, T f ], there exist an optimal control u * (t) ∈ U .
Proof.The boundedness of solution of system (7) asserts the existence of solution to control system using results by [15], therefore, set of controls and corresponding state variables are non-empty.The control set is closed and convex by definition.The solution of system (7) are bounded above by linear function in control and state.The integrand in cost functional, , where p depends upon the upper bound of Y (t), T (t) and X m (t) and q = C 1 .Hence the existence of an optimal control is established.

Characterization of optimal control
The Pontryagin's Maximum principle converts the problem of minimizing the cost functional subject to state variables into minimizing the Hamiltonian with respect to the controls at each time t.For the purpose of simplicity it is introduced the functions f 1 , f 2 , f 3 , f 4 , f 5 and f 6 , to right side expressions of equations (7).(11) where λ i for i = 1, 2...6 are adjoint functions associated with their respective state variables.
The necessary conditions that an optimal control problem must satisfy Hamiltonian H comes from the Pontryagins maximum principle [7].Given an optimal control and corresponding states, there exists adjoint variable λ i satisfying the following equations: with transversality conditions λ i (T ) = 0, for i = 1, 2...6.The transversality conditions are zero because the objective functional is independent of states at the final time.
The Hamiltonian is minimized with respect to u(t) at the optimal value u * (t) .Since differentiating H with respect to u and according to Pontrygins Maximum Principle, the unrestricted optimal control u * (t) satisfies ∂H ∂u = 0 at u(t) = u * (t).So it is given by Therefore we have the following theorem.
Due to a priori boundedness of the state and adjoint system functions and the resulting Lipschitz structure of the ODE's, it is obtained the uniqueness of the optimal control for small T. The state system coupled with the adjoint system, with the initial conditions, the transversality condition together with the above characterization of the control form the optimality system.

Stochastic Model
In this section a non-linear stochastic SIR type epidemic model is proposed by introducing a noise in system (7), and transformed the deterministic problem into a corresponding stochastic problem.
The noise can induce non-trivial effects in physical and biological systems.The presence of noise source modifies the behavior of corresponding deterministic evolution of the system to stochastic system.The real spread of infectious disease, due to variation in the environment and the weather will exhibit some kinds of random fluctuation in the infection and other variables.Here it is considered the perturbed transmission coefficients β 1 and β 2 in system (7), and hence the infection rate is replaced by where η(t) represents the Gaussian white noise with zero mean and unit co-variance and ǫ is a constant.The relation between the Wiener process W (t) and Gaussian white noise η(t) such that dW (t) = η(t)dt, then the stochastic version of the corresponding deterministic system (7) takes the following form: where, X > 0, Y > 0, X m ≥ 0, T ≥ 0, Z ≥ 0 and M ≥ 0.
In this process, it is assumed that W (t) is one dimensional real Wiener process defined on a filtered complete probability space (Ω, F, {F t } t≥0 , P ).For some n ∈ N , some x 0 ∈ ℜ n , and an n-dimensional Wiener process W (t), consider the general n-dimensional stochastic differential equation, A solution to the above equation is denoted by x(t, x 0 ).It is assumed that F (t, 0) = G(t, 0) = 0 ∀ t ≥ 0, so that the origin point is an equilibrium of ( 16).Let us denote by L the differential operator associated with the function displayed in (16), defined for a function Here trp denotes the transpose and Trc means trace of a matrix.In view of Ito's formula, if x(t) ∈ ℜ d , then dU (x, t) = LU (x, t)dt + V x (x, t)g(x, t)dW (t).

Existence and uniqueness of positive solutions
In this section, using Lyapunov analysis method (mentioned in refs.[16,17]), we show that the solution of system ( 15) is positive and global.
Proof.Since the coefficient of the equation are locally Lipschitz continuous for any given initial value (X(0), Y (0), X m (0), H(0), Z(0), M (0)) ∈ ℜ 6  + , there is a unique local solution , where τ e is the explosion time (see Ref. [18]).To show that this solution is global, we need to show that τ e = ∞ a.s.Let k 0 ≥ 0 be sufficiently large so that X(0), Y (0), X m (0), T (0), Z(0) and M (0) all lie within the interval [1/k 0 , k 0 ].For each integer k > k 0 , define the stopping time where throughout this section, we set inf ∅ = ∞(as usual ∅ denotes the empty set).According to the definition, τ k is increasing as k → ∞.
Set τ ∞ = lim k→∞ τ k , whence τ ∞ ≤ τ e a.s.If we can show that τ ∞ = ∞ a.s., then τ e = ∞ and {X(t), Y (t) X m (t), T (t), Z(t), M (t)} ∈ ℜ 6 + a.s.for all t ≥ 0. In other words, to complete the proof, all we need to show that τ ∞ = ∞ a.s.If this statement is false, then there exist a pair of constants τ > 0 and ǫ 1 ∈ (0, 1) such that Hence there is an integer k 1 ≥ k 0 such that For t ≤ τ k , we can see, for each k, and also and so, where LV : Therefore It is then follows ( 19) and ( 22) that So we must therefore have τ ∞ and hence the proof.

Stochastic optimal control problem
In this section stochastic version of the optimal control problem ( 1) is formulated and discussed.
For stochastic control theory refer [19] of Oksendal.Here the objective is to find an optimal media awareness programs u * (t) which minimizes the objective functional with an initial state x 0 is defined by Here the expectation is obtained on the initial condition of the state (at time t = 0) system is x 0 .For the deterministic problem of earlier, it is assumed that there is a fixed constant u(t) ≤ 1 with u(t) ≤ u (a.s.).The class of admissible control laws is (24) To solve this stochastic control problem, the performance criterion is defined as follows: where the expectation is conditional on the state of the system being a fixed value x at time t.The value function is define as It is determined a control law that minimizes the expected value J : A → ℜ + given by ( 26).Now the stochastic analogue of the optimal control problem is formulated, subsequent to which the solution formulae is presented.
P roblem: Given the system ( 16) and given A as in (24) with J as in ( 25), find the value function and an optimal control function An expression for the optimal media awareness program u * (t) is computed through the following theorem.
Theorem 4. A solution to the optimal media awareness program problem stated in problem ( 24) is of the form Proof.To determine u * (t) through the dynamic programming approach it is necessary to calculate LU (t) i.e. by (17): Applying the Hamilton-Jacobi-Bellman theory (see, for instance, [19]) To compute the equation (31) it requires to derive partial derivative of the below given expression with respect to u, and equating to zero.
This leads to the equation: In the following section numerical analysis of the results are discussed.

Numerical Simulations
The feasibility of analysis regarding deterministic optimality and stochastic optimality conditions are simulated numerically over t = 30 units of time.All parameter values in the computations are the same in both scenarios.The common parameter values used in the computations are An iterative scheme of fourth order Runge-Kutta method is used for solving the deterministic optimality system.This method of numerically integrated ordinary differential equations by using trial step at midpoint of an interval to eliminate lower order errors terms.The algorithm is the forward-backward scheme; starting with an initial guess for the optimal controls, the state variables are then solved forward in time from the dynamics of system (1) using a Runge-Kutta method of the fourth order.Then, those state variables and initial guess for the controls are used to solve the adjoint equation ( 12) backward in time with given final conditions (13), again employing a fourth order Runge-Kutta method.The controls are updated and used to solve the state and then the adjoint system.This iterative process terminates when current state, adjoint, and control values converge sufficiently (See, [4,5]).
Numerical simulation to the system comprising state system (15) compelled with the proxy adjoint system (12) with transverslity conditions and characterization of the control variable u * (t) in equation ( 29) are carried out using forward backward algorithm.The state system (15), i.e., stochastic differential equations were first simulated using forth order Range-Kutta method by introducing noise through Euler Maruyama method [20] and then adjoint system (12) are simulated backward in time with final conditions(See, Witbooi et al. [14]).In particular, we use as a proxy for λ 6 (Y + T ) in the calculation of u(t) in this case.We note that the presence of Y (t) makes u(t) into a stochastic variable even with the said proxy (in the stochastic case).
Figure 1 shows the time series plot to illustrates the variation of the number of individuals in each compartment of the population and number of awareness campaigns with respect to time (in weeks) and Figure 1  Figure 2 illustrates the stochastic model solutions using the same parameter values and initial conditions as that of deterministic model parameters used in the illustration of Figure 1 and the corresponding control profile u(t) for stochastic model.It is observed that stochastic model solution also depicts same scenario as that of deterministic model solution under the time dependent control u(t) and also control profile u(t) exhibits same state of affairs as that of deterministic control profile.An important point to note about our approximation is that it fully accommodates the stochasticity (embodied and concentrated in the factor Y of the expression u * (t)).To investigate how optimal control depends upon different parameters of the deterministic and stochastic model, control profile is plotted for different values of transmission rate β 1 and recovery rate γ in Figures 3 and 4 respectively.It is observed from Figure 3(a) that for higher value of transmission rate β 1 , to achieve the optimal scenario awareness campaign must be implemented with maximum rate up to to 28 units of time.
However for lower value of β 1 i.e., β 1 = 0.000001, and β 1 = 0.0000001, the optimal scenario can be obtained by implementing awareness campaigns with maximum rate only for initial 21 and 14 units of time, respectively.The stochastic control profile Figure 3(b) also depicts similar state of affairs, but the optimal scenario can be obtained by implementing awareness campaigns with maximum rate only for initial 7 and 8 units of time, for β 1 = 0.000001, and β 1 = 0.0000001 respectively, then onwards implementation of awareness campaigns will be guided by stochastic control profile.These course of remedies are observed for the reason that when the transmission of disease is slow, less people get affected and hence less awareness campaigns are needed to control the disease.Similarly optimal scenario will change from implementing awareness campaigns with maximum rate up to 28 units of time to 16 and 6 units of time for the change in recovery rate γ = 0.15 to γ = 0.8 and γ = 1 respectively (Figure 4(a)).Again stochastic control profile Figure 4(b) also depicts similar state of affairs, but the optimal scenario can be obtained by implementing awareness campaigns with maximum rate only for initial 6 and 7 units of time, for γ = 0.8 and γ = 1 respectively, then onwards implementation of awareness campaigns will be guided by stochastic control profile.Figure 5 shows the effect of transmission rate β 1 on infected population for the deterministic and stochastic models.Increase in the transmission rate β 1 leads increase in number of infections, and hence it requires to continue the implementation of awareness campaigns at maximum rate.Figure 7 shows the difference in the number of individuals in each state of a system (15) for different values of perturbation parameter ǫ.From Figure 7(a) it is observed that the number of unaware susceptible are decreasing as ǫ increases initially up to 12 units of time and then increasing till final time.In case of infectives as perturbation increases number of infections are increasing up to 8 units of time later it is decreasing till final time see Figure 7(b) and from Figure 7(c) it is clear that as perturbation increases, aware susceptible are increasing till final time.Figure 7(d),7(e),7(f) are varying in the same direction as that of infectives, as ǫ increases.To investigate how the optimal control varies depends upon the positive weight C 1 , it is plotted the control profile for different values of C 1 .It is observed from the Figure 8(a) that as the positive weight C 1 increased up to 5000 the optimal scenario is achieved in 25 units of time and when C 1 = 230 it is sufficient to implement control on awareness programs at maximum rate up to 29 units of time.For stochastic optimal control it is observed from Figure 8(b) that when C 1 = 5000 it is enough to implement optimality at maximum rate up to 22 units of time and for lower value of C 1 = 230 it is necessary to continues the implementation of awareness programs up to 27 units of time at maximum rate.This indicates that as the weight of control (awareness programs) increases, the disease can be controlled in a minimum time.

Conclusion
Media campaigns and epidemics are closely related to each other.The bases of this association is human behavioral responses.The present study considered the optimal control analysis of both deterministic differential equation modeling and stochastic differential equation modeling of infectious disease by taking effects of media awareness programs and treatment of infectives on the epidemic into account.Optimal media awareness strategy under the quadratic cost functional using Pontrygin's Maximum Principle and Hamiltonian-Jacobi-Bellman equation are derived for both deterministic and stochastic optimal control problem respectively.The Hamiltonian-Jacobi-Bellman equation is used to solve stochastic system, which is fully non-linear equation, however it ought to be pointed out that for stochastic optimality system, it may be difficult to obtain the numerical results.For the analysis of the stochastic optimality system, the results of deterministic control problem are used to find an approximate numerical solution for the stochastic control problem.Outputs of the simulations shows that media awareness programs place important role in the minimization of infectious population with minimum cost.The model analysis further shows that awareness programs through the media campaigning are helpful in decreasing the spread of infectious diseases by isolating a fraction of susceptible from infectives.Numerical simulation of stochastic optimal control problem enables to measure the feasibility of option followed.A formal approach to the numerical simulation of the stochastic optimal control problem is far more complex and labour intensive and our method is a workable approximate alternative.

Figure 1 .
Figure1shows the time series plot to illustrates the variation of the number of individuals in each compartment of the population and number of awareness campaigns with respect to time (in weeks) and Figure1(a) shows time series plot for the deterministic epidemic model under the time dependent control u(t) where as Figure1(b) representing the control profile of the same model.Further it is evident from the Figure1(b) that it is optimal to run awareness campaigns up to 29 units of time at maximum rate and lower down afterwords.

Figure 2 .
Figure 2. Simulation of Stochastic model solution (a) and control profile u(t)(b).

Figure 3 .
Figure 3. Simulation of Deterministic (a) and Stochastic (b) control profile for different values of β 1 .

Figure 4 .
Figure 4. Simulation of Deterministic (a) and Stochastic (b) control profile for different values of recovery rate γ.

Figure 6 (
Figure 6(a) shows the simulation of deterministic and stochastic cost function and cumulative density of awareness programs 6(b).From the Figures 6(a) and 6(b) it is clear that cost and awareness programs are proportional to each other, which implies that, as number of media awareness programs increases cost of control for epidemic is also increasing.

Figure 7 .
Figure 7. Simulation of solution of each states for different values of perturbation parameter ǫ.

Figure 8 .
Figure 8. Simulation of control profile for different values of C 1 .

Table 1 .
Description of variables of the model.