 Research
 Open Access
The shortest path problem in the stochastic networks with unstable topology
 Gholam H. Shirdel†^{1}Email author and
 Mohsen Abdolhosseinzadeh†^{1}
 Received: 24 February 2016
 Accepted: 31 August 2016
 Published: 13 September 2016
Abstract
The stochastic shortest path length is defined as the arrival probability from a given source node to a given destination node in the stochastic networks. We consider the topological changes and their effects on the arrival probability in directed acyclic networks. There is a stable topology which shows the physical connections of nodes; however, the communication between nodes does not stable and that is defined as the unstable topology where arcs may be congested. A discrete time Markov chain with an absorbing state is established in the network according to the unstable topological changes. Then, the arrival probability to the destination node from the source node in the network is computed as the multistep transition probability of the absorption in the final state of the established Markov chain. It is assumed to have some wait states, whenever there is a physical connection but it is not possible to communicate between nodes immediately. The proposed method is illustrated by different numerical examples, and the results can be used to anticipate the probable congestion along some critical arcs in the delay sensitive networks.
Keywords
 Stochastic network
 The stochastic shortest path
 Discrete time Markov chain
 Arrival probability
Mathematics Subject Classification
 Primary 90C59
 90C35; Secondary 90C40
Background
The deterministic shortest path problem has been studied extensively and applied in many fields of optimization; there are polynomial time algorithms to solve the deterministic shortest path problem (Dijkstra 1959; Bellman 1958; Orlin et al. 2010). However, paths in networks should be reliable to transmit flow from a source node to a destination node especially in delay sensitive networks. The best connection helps to avoid traffic congestion in networks. So, the arrival probability is used to evaluate the reliability of paths and it has been considered as an optimality index of the stochastic shortest path length (Bertsekas and Tsitsiklis 1991; Fan et al. 2005; Kulkarni 1986; Shirdel and Abdolhosseinzadeh 2016). The stochastic shortest path problem (SSP) is defined as the best path with stochastic optimality condition. Liu (2010) assumed the arc lengths to be uncertain variables. Pattanamekar et al. (2003) considered the individual travel time variance and the mean travel time forecasting error. Also, Hutson and Shier (2009) and Rasteiro and Anjo (2004) supposed two criteria: mean and variance of path length. Fan et al. (2005) assumed known conditional probabilities for link travel times that each link could be congested or uncongested. Wu et al. (2004) modeled a stochastic and timedependent network with discrete probability distributed arc weights. Peer and Sharma (2007) assumed two kinds of nodes, possible failure and always working. Ji (2005) solved three models of the shortest path by integrating stochastic simulation and genetic algorithm. The considered model in this paper is a directed acyclic stochastic network with known discrete distribution probabilities of leaving or waiting in nodes.
Our criterion to evaluation of the connections from the source node toward the destination node in the network is presented as the arrival probability, which is obtained by the established discrete time Markov chain (DTMC) in the network (Shirdel and Abdolhosseinzadeh 2016); then, the best possible connection is determined with the largest arrival probability. Liu (2010) converted his models into deterministic programming problems. Hutson and Shier (2009) and Rasteiro and Anjo (2004) obtained the maximum expected value of a utility function. Fan et al. (2005) applied a procedure for dynamic routing policies. Nie and Fan (2006) formulated the stochastic ontime arrival problem with dynamic programming, and Fan et al. (2005) minimized the expected travel time.
In this paper, the maximum arrival probability from a given source node to a given destination node is computed according to known discrete distribution probabilities of leaving or waiting in nodes, and a DTMC stochastic process is used to model the problem rather than dynamic programming or stochastic programming. Kulkarni (1986) developed a method based on a continuous time Markov chain (CTMC) to compute the distribution function of the shortest path length. Azaron and Modarres (2005) applied Kulkarni’s method to queuing networks. Thomas and White (2007) modeled the problem of constructing a minimum expected total cost route as a Markov decision process. They wanted to respond to dissipated congestion over time according to some known probability distribution.
The arrival probability gives overall information of the network conditions to transmit flow from the source node toward the destination node. Two conditions at any state of the established DTMC are assumed: departing from the current state to a new state, or waiting in the current state with expecting better conditions. There are several unstable connections between nodes. The leaving distribution probability from one node toward another node is known as the probability that their connected arc is uncongested. A DTMC with an absorbing state is established and the transition matrix is obtained. Then, the arrival probability from the source node toward the destination node is computed as the multistep transition probability from the initial state to the absorbing state in DTMC. The arrival probability introduced by Shirdel and Abdolhosseinzadeh (2016) is reviewed in this paper, and it is extended and the concepts and definitions are organized to find the stochastic shortest path.
This paper is organized as follow. In “The unstable topology of the network” section some definitions and assumptions of networks with unstable topology is introduced. The concept of the stochastic process and the established DTMC in the network is described in “The established discrete time Markov chain” section; also, the computations of the arrival probability and the stochastic shortest path are presented in “The established discrete time Markov chain” section. “Numerical results” section contains some numerical results of implementation of the proposed method on some networks with various topologies.
The unstable topology of the network
In this section, we introduce some definitions and assumptions of networks with unstable topology. Let network \(G=(N,A)\), with node set N and arc set A, be a directed acyclic network. Then, we can label its nodes in a topological order such that for any \((i,j)\in A, i<j\) (Ahuja et al. 1993). The physical topology for any \((i,j)\in A\) shows the possibility of communication between nodes \(i,j\in N\) in the network. In the transportation networks there are some physical connections between nodes, but we cannot traverse anymore toward the destination node because of probable congestion. If there are some facilities in the network G, but it is not possible to use them continuously, then G has unstable topology. So, for any arc \((i,j)\in A\) it is not mean there is a stable communication between nodes \(i,j\in N\) all the time (it could be probably congested). For any node i, it is supposed that the uniform distribution probabilities of leaving arcs (i, j) to be uncongested are known (Shirdel and Abdolhosseinzadeh 2016).
The stochastic variable of arc (i, j) according to the unstable topology is shown by \(x_{ij}\). If \(x_{ij}=1\), it is possible to traverse arc (i, j), and otherwise \(x_{ij}=0\). The probability that arc (i, j) to be uncongested is \(q_{ij}=Pr[x_{ij}=1]\), and it represents the uniform probability that node i is leaved toward node j (an adjacency node). Then, the wait probability in node i, is \(q_{ii}=1\sum _{\{j:(i,j)\in A\}}q_{ij}\), and it is the probability that leaving arcs by node i are congested.
Figure 1 shows the example network with its topological ordered nodes and it is the initial physical topology of the network. The numbers on arcs show the leaving probabilities \(q_{ij}\). Node 1 is the source node and node 4 is the destination node. It is not possible to traverse arc (2, 4) because it does not exist in the physical topology of the example network. However, the arcs in the physical topology could be congested according to the known distribution probabilities.
The established discrete time Markov chain
The state space of the established DTMC for the example network
States  \(S_1\)  \(S_2\)  \(S_3\)  \(S_4\)  \(S_5\) 

Nodes  \(\{1\}\)  \(\{1,2\}\)  \(\{1,3\}\)  \(\{1,2,3\}\)  \(\{ 1,2,3,4\}\) 
The final state contains the destination node of the network, where DTMC does not progress anymore, and it is called assumption i. The states of the established DTMC contain the traversed nodes of the network, those are reached from some nodes in a previous state, and it is called assumption ii. It is not allowed to return from the last traversed node; however, it is possible to wait in the current state. Clearly, a new state is revealed if a leaving arc \((i,j)\in A\) is traversed such that the current node i is contained in the current state and the new node j is contained in the new state, and it is called assumption iii. As previously said, the wait states are one of option 1 or option 2.
The transition and the wait probabilities

\(0\le p_{kl}\le 1\) for \(k=1,2,\ldots ,S\) and \(l=1,2,\ldots ,S\)

\(\sum _l{p_{kl}}=1\), for \(k=1,2,\ldots ,S\).
Theorem 1
If \(p_{kl}\) is klth element of matrix P, that \(k \ne l, l < S\) and \(S_{k}=\{v_0=1,v_1,\ldots ,v_m\}\) is the current state, then the transition probability from state \(S_{k}\) to state \(S_{l}\) is computed as follow
Proof
Since, it is not allowed to traverse from one state to the previous states (assumption ii), then necessarily \(p_{kl}=0\), for \(l\ <\ k\). Otherwise, suppose \(l\ >\ k\), during transition from the current state \(S_k\) to the new state \(S_l\), it should be reached just one node other than the nodes of the current state, so \(S_l\backslash S_k=1, v\in S_k\), and \(w\in S_l\backslash S_k\) are held by assumption ii and iii. Two components of \(p_{kl}\) formula should be computed.
In the last node \(v_m\) of the current state \(S_k\), it is possible to wait in \(v_m\) with probability \(q_{v_mv_m}\). Notice, it is not possible to wait in the other nodes \(v\in S_k\backslash \{v_m\}\) because it should be leaved to construct the current state, however it is not necessary for node \(v_m\) with the largest label (leaving \(v_m\) leads to a new node, and therefore results in a new state). If \(w\in S_l\backslash S_k\), then one or all of events \(E_{vw}\) (i.e. to traverse a connecting arc between a node of the current state and another node of the new state) can happen for \(\left( v,w\right) \in \Psi\), and the arrival probability of node \(w \in S_l\) from the current state \(S_k\) is equal to \(Pr[\bigcup _{\left( v,w\right) \in \Psi }{E_{vw}}]\). The collection probability should be computed because of deferent representations of the new state (for example see Fig. 2). Then, the nodes of the current state \(v\in S_k\backslash \{v_m\}\) (while waiting in \(v_m\)) should be prevented from reaching other nodes \(u\notin S_k\) and \(u\ne w\) (assumption iii), so arcs \(\left( v,u\right)\) are not allowed to traverse and they are excluded simultaneously, thus it is equal to \(\prod _{\left( v,w\right) \in \Psi }{(1\sum _{\begin{array}{c} \left( v,u\right) \in A \\ u\ne w,u\notin S_k \end{array}}{q_{vu}))}}\). The other possibility in node \(v_m\), that is leaving it toward the new node \(w\in S_l\backslash S_k\) with probability \(q_{v_mw}\). \(\square\)
Theorem 2 describes the transition probabilities to the absorbing state \(S_{S}\), and they are the last column of the transition matrix P.
Theorem 2
Proof
To compute the transition probabilities \(p_{kS}\), for \(k=1,2,\ldots ,S1\) it should be noticed the final state is the absorbing state \(S_{S}=\{1,2,3,\ldots ,N\}\) containing all nodes of the network, and the stochastic process does not progress any more (assumption i). So, it is sufficient to consider leaving arcs \((v,v_n)\) from \(v\in S_k\), the nodes of the current state, toward the destination node \(v_n\in S_{S}\). Then, one or all of events \(E_{vv_n}\) (i.e. to traverse a connecting arc between a node of the current state and the destination node of the absorbing state) can happen and the transition probability from the current state \(S_k\) to the absorbing state \(S_{S}\) is totally equal to \(Pr[\bigcup _{v\in S_k,(v,v_n)\in A}E_{vv_n}]\). The collection probability should be computed because of different representations of the states (for example see Fig. 2). \(\square\)
For state \(S_4\), transition probability \(p_{45}\) is obtained by \(P(E_{14}\cup E_{34}\cup E_{24})\), however \(q_{24}=0\) as seen in Fig. 1, then \(p_{45}=q_{14}+q_{34}q_{14}\times q_{34}\). The wait probabilities, those are the diagonal elements of the transition matrix P, are obtained by Theorem 3.
Theorem 3
Proof
The wait probabilities \(p_{kk}\) are the complement probabilities of the transition probabilities from the current state \(S_k\), for \(k=1,2,\ldots ,S1\), toward the all departure states \(S_j\), for \(j=k+1,k+2,\ldots ,S\). Then, we have \(p_{kk}=1\sum ^{S}_{j=k+1}p_{kj}\), for \(k=1,2,\ldots ,S1\), in other word, they are the diagonal elements of matrix P, those are computed for any row \(k=1,2,\ldots ,S1\) of the transition matrix (see Ibe 2009). The absorbing state \(S_{S}\) does not have any departure state, so \(p_{SS}=1\) as the transition matrix P. \(\square\)
The arrival probability
The arrival probability determines the overall reliability of connections in the network, and it shows the probability that they are not congested during the transmission of flow from the source node to the destination node in the network. The arrival probability is defined as multistep transition probability from the initial state \(S_1\) to the absorbing state \(S_{S}\) in the established DTMC. According to the assumptions i, ii and iii, the state space of DTMC is directed and acyclic (otherwise return to the previous states is allowed contradictively). Outdegree of any state is at least one (without loop wait transition arcs consideration), except the absorbing state \(S_{S}\), then for any state \(S_k\), there is one\multistep transition from the initial state to the absorbing state that traverses state \(S_k\). Consequently, the absorbing state is accessible from the initial state after some finite transitions. Let \(p_{kl}(r)=Pr[X_{m+r}=S_lX_m=S_k]\) denote the conditional probability that the process will be in state \(S_l\) after exactly r transitions, given that it is presently in state \(S_k\). So, if matrix P(r) is the transition matrix after exactly r transitions, it can be shown that \(P(r)=P^r\), and let \(p_{kl}(r)\) be klth element in matrix \(P^r\) (see Ibe 2009). Thus, the arrival probability after exactly r transitions is \(p_{1S}(r)=Pr[X_r=S_{S}X_0=S_1]\) and it is the 1Sth element in the matrix \(P^r\).
The stochastic shortest path
 a.
if \(j \notin N_\Pi\) and \((j,i) \in A_\Pi\) then \(q_{ji}:= 0\) and \(q_{jj}:= q_{jj}+q_{ji}\)
 b.
if \(j \in N_\Pi\) and \((i,j) \notin A_\Pi\) then \(q_{ji}:= 0\).
Numerical results
Some implementations of the proposed method on the networks with different topologies are presented in this section. The instances are directed acyclic networks and there is a path from each node to the destination node. The leaving probabilities of nodes are random numbers produced by the uniform distribution probability. Then, the arrival probability is computed for the established DTMC. All of the experiments are coded in MATLAB R2008a and they are performed on Dell Latitude E5500 (Intel(R) Core(TM) 2 Duo CPU 2.53 GHz, 1 GB memory). To avoid vague demonstration just the stochastic shortest path with the arrival probability computation results are shown by square and circle markers in the figures, respectively; whereas, dashed lines are the results for other paths.
We use two propositions inductively to be sure there will be a path from the source node to the destination node in its initial topology, and the created network is an acyclic network.
Proposition 1
If node k is the first node with larger index than source node 1 and \(in{\text{}}degree(k)=0\), let \(1 \le l < k\) is an arbitrary node, then by adding arc (l, k) there exists a path from source node 1 to node k.
Proposition 2
If node k is the first node with smaller index than destination node n and \(out{\text{}}degree(k)=0\), let \(k < l \le n\) is an arbitrary node, then by adding arc (k, l)there exists a path from node k to destination node n.
The leaving probabilities of arcs in network 1
(i, j)  \(q_{ij}\)  (i, j)  \(q_{ij}\)  (i, j)  \(q_{ij}\) 

(1,2)  0.0115  (2,5)  0.4622  (5,6)  0.1661 
(1,3)  0.2059  (2,7)  0.1531  (5,7)  0.3756 
(1,5)  0.0080  (3,5)  0.3232  (5,8)  0.1135 
(1,6)  0.7532  (3,8)  0.5544  (6,7)  0.3756 
(2,3)  0.0877  (4,5)  0.5714  (6,8)  0.4780 
(2,4)  0.1900  (4,6)  0.2184  (7,8)  0.8725 
The leaving probabilities of arcs in network 2 and network 3
Network 2  Network 3  

(i, j)  \(q_{ij}\)  (i, j)  \(q_{ij}\)  (i, j)  \(q_{ij}\)  (i, j)  \(q_{ij}\)  (i, j)  \(q_{ij}\) 
(1,2)  0.0397  (3,5)  0.3229  (5,8)  0.1930  (1,2)  0.9035  (5,6)  0.4611 
(1,4)  0.7819  (3,6)  0.4969  (5,9)  0.2432  (1,4)  0.0172  (5,8)  0.3773 
(1,5)  0.0883  (4,5)  0.0366  (6,8)  0.3481  (2,3)  0.3828  (6,9)  0.8620 
(2,3)  0.4905  (4,7)  0.1349  (6,9)  0.6039  (2,5)  0.5076  (7,8)  0.6601 
(2,4)  0.1362  (4,8)  0.7921  (7,8)  0.9531  (3,6)  0.6024  (8,9)  0.6725 
(2,5)  0.1199  (5,6)  0.3169  (8,9)  0.7373  (4,5)  0.2695  
(2,6)  0.1360  (5,7)  0.1909  (4,7)  0.7305 
The destination node of network 2 is accessible after at least four transitions, and it is done for network 3 after at least three transitions.
The leaving probabilities of arcs in network 4
(i, j)  \(q_{ij}\)  (i, j)  \(q_{ij}\)  (i, j)  \(q_{ij}\)  (i, j)  \(q_{ij}\)  (i, j)  \(q_{ij}\) 

(1,2)  0.0386  (2,3)  0.0540  (3,5)  0.7742  (4,8)  0.0420  (6,9)  0.4366 
(1,3)  0.6457  (2,4)  0.0384  (3,6)  0.0322  (4,9)  0.1813  (7,8)  0.4114 
(1,4)  0.0272  (2,5)  0.5310  (3,7)  0.0165  (5,6)  0.0977  (7,9)  0.3568 
(1,5)  0.0012  (2,6)  0.0338  (3,8)  0.1278  (5,7)  0.1253  (8,9)  0.6908 
(1,6)  0.0979  (2,7)  0.1042  (3,9)  0.0180  (5,8)  0.2494  
(1,7)  0.0568  (2,8)  0.0682  (4,5)  0.5820  (5,9)  0.4490  
(1,8)  0.0433  (2,9)  0.1101  (4,6)  0.0358  (6,7)  0.1300  
(1,9)  0.0443  (3,4)  0.0165  (4,7)  0.1545  (6,8)  0.2981 
The obtained arrival probability in a network determines the general situation of the network to transmit flow from a source node toward a destination node (Shirdel and Abdolhosseinzadeh 2016); however, the presented method precisely determines the path with the largest probability amongst all paths.
Conclusions
The arrival probability from a given source node to a given destination node was computed according to the probability of transition from the initial state to the absorbing state by multistep transition probability of the established discrete time Markov chain in the original network. The proposed method to obtain the arrival probability determines that the destination node is accessible for the first time. The stochastic shortest path was separately determined which has the largest arrival probability value. So, this method can be applied to rank paths of a network by considering their obtained arrival probabilities. Also, the proposed method evaluates the reliability of connections in the networks. So, it can be used in the shortest path problem with recourse, where locally should be decided which path is selected to traverse. The discrete nature of the proposed model could apply metaheuristic methods to reduce the computations. Also, the proposed method can be used for the stochastic problems as a policy evaluation index.
Notes
Declarations
Authors’ contributions
Both authors have equal contribution to this article. GS participated in the design and conceived of the study. MA worked out the algorithms and helped to draft the manuscript. Both authors read and approved the final manuscript.
Competing interests
The authors declare that they have no competing interests.
Open AccessThis article is distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made.
Authors’ Affiliations
References
 Ahuja RK, Magnanti TL, Orlin JB (1993) Network flows: theory, algorithms, and applications. PrenticeHall, Englewood CliffsMATHGoogle Scholar
 Azaron A, Modarres M (2005) Distribution function of the shortest path in networks of queues. OR Spectr 27:123–144MathSciNetView ArticleMATHGoogle Scholar
 Bellman R (1958) On a routing problem. Q Appl Math 16:87–90MATHGoogle Scholar
 Bertsekas DP, Tsitsiklis JN (1991) An analysis of stochastic shortest path problems. Math Oper Res 16:580–595MathSciNetView ArticleMATHGoogle Scholar
 Dijkstra EW (1959) A note on two problems in connexion with graphs. Numer Math 1:269–271MathSciNetView ArticleMATHGoogle Scholar
 Dréo J, Pétrowski A, Siarry P, Taillard E (2006) Metaheuristics for hard optimization. Springer, BerlinMATHGoogle Scholar
 Fan YY, Kalaba RE, Moore JE (2005) Arriving on time. J Optim Theory Appl 127:497–513MathSciNetView ArticleMATHGoogle Scholar
 Fan YY, Kalaba RE, Moore JE (2005) Shortest paths in stochastic networks with correlated link costs. Comput Math Appl 49:1549–1564MathSciNetView ArticleMATHGoogle Scholar
 Hutson KR, Shier DR (2009) Extended dominance and a stochastic shortest path problem. Comput Oper Res 36:584–596MathSciNetView ArticleMATHGoogle Scholar
 Ibe OC (2009) Markov processes for stochastic modeling. Academic Press, LondonMATHGoogle Scholar
 Ji X (2005) Models and algorithm for stochastic shortest path problem. Appl Math Comput 170:503–514MathSciNetMATHGoogle Scholar
 Kulkarni VG (1986) Shortest paths in networks with exponentially distributed arc lengths. Networks 16:255–274MathSciNetView ArticleMATHGoogle Scholar
 Liu W (2010) Uncertain programming models for shortest path problem with uncertain arc length. In: Proceedings of the first international conference on Uncertainty Theory. Urumchi, China, pp 148–153Google Scholar
 Nie Y, Fan Y (2006) Arrivingontime problem discrete algorithm that ensures convergence. Transp Res Rec 1964:193–200View ArticleGoogle Scholar
 Orlin JB, Madduri K, Subramani K, Williamson M (2010) A faster algorithm for the single source shortest path problem with few distinct positive lengths. J Discr Algorithms 8:189–198MathSciNetView ArticleMATHGoogle Scholar
 Pattanamekar P, Park D, Rillet LR, Lee J, Lee C (2003) Dynamic and stochastic shortest path in transportation networks with two components of travel time uncertainty. Transp Res Part C 11:331–354View ArticleGoogle Scholar
 Peer SK, Sharma DK (2007) Finding the shortest path in stochastic networks. Comput Math Appl 53:729–740MathSciNetView ArticleMATHGoogle Scholar
 Rasteiro DDML, Anjo AJB (2004) Optimal path in probabilistic networks. J Math Sci 120:974–987MathSciNetView ArticleMATHGoogle Scholar
 Ross SM (2006) Introduction to probability models. Academic Press, New YorkMATHGoogle Scholar
 Shirdel GH, Abdolhosseinzadeh M (2016) A genetic algorithm for the arrival probability in the stochastic networks. SpringerPlus, Berlin : 114MATHGoogle Scholar
 Thomas BW, White CC III (2007) The dynamic and stochastic shortest path problem with anticipation. Eur J Oper Res 176:836–854MathSciNetView ArticleMATHGoogle Scholar
 Wu Q, Hartly J, AlDabass D (2004) Timedependent stochastic shortest path(s) algorithms for a scheduled transportation network. Int J Simul 6:7–8Google Scholar