Jekyll2023-03-15T15:11:49+00:00https://konaeakira.github.io/feed.xmlKonaeAkira’s blogMy personal homepage and programming blogUsing the Shortest Path Faster Algorithm to find a negative cycle.2020-05-05T05:20:00+00:002020-05-05T05:20:00+00:00https://konaeakira.github.io/posts/using-the-shortest-path-faster-algorithm-to-find-negative-cycles<script type="text/javascript" src="https://cdn.mathjax.org/mathjax/latest/MathJax.js?config=TeX-AMS-MML_HTMLorMML"></script>
<p><a rel="license" href="http://creativecommons.org/licenses/by/4.0/"><img alt="Creative Commons License" style="border-width:0" src="https://i.creativecommons.org/l/by/4.0/80x15.png" /></a><br />This work is licensed under a <a rel="license" href="http://creativecommons.org/licenses/by/4.0/">Creative Commons Attribution 4.0 International License</a>.</p>
<h3 id="introduction">Introduction</h3>
<p>The Shortest Path Faster Algorithm (SPFA) is an improvement over the Bellman-Ford Algorithm. Both are used to calculate the shortest paths from a single source vertex in a weighted directed graph. The SPFA is almost always preferred over the Bellman-Ford Algorithm because of its speed. It has an (unproven) average runtime complexity of \(\mathcal{O}(m)\).</p>
<p>The Bellman-Ford Algorithm is also used to detect/find negative cycles, and the SPFA can do that, too! In this article I will show how to use the SPFA to find a negative cycle, and propose a method to terminate the algorithm early in the presence of a negative cycle.</p>
<h3 id="detecting-a-negative-cycle">Detecting a negative cycle</h3>
<p>The Bellman-Ford Algorithm exploits the fact that in a graph with no negative cycles, the length of any shortest path is at most \(n-1\). Therefore, we simply need to keep track of the length of the current shortest path for each vertex.</p>
<p>Below is the pseudocode for the SPFA with negative cycle detection.</p>
<ul>
<li><code class="language-plaintext highlighter-rouge">Queue</code> is a First-In-First-Out queue.</li>
<li><code class="language-plaintext highlighter-rouge">w(u, v)</code> is the weight of the edge <code class="language-plaintext highlighter-rouge">(u, v)</code>.</li>
<li><code class="language-plaintext highlighter-rouge">dis[u]</code> is the weight of the current shortest path from the source to <code class="language-plaintext highlighter-rouge">u</code>.</li>
<li><code class="language-plaintext highlighter-rouge">len[u]</code> is the length (in amount of edges) of the current shortest path from the source to <code class="language-plaintext highlighter-rouge">u</code>.</li>
</ul>
<p><strong>Note:</strong> I have modified the SPFA so that every vertex has a starting “shortest path” of \(0\). This has the same effect as creating an imaginary source vertex \(s\) and creating an edge with weight \(0\) from \(s\) to all other vertices. This ensures that the algorithm can detect a negative cycle regardless of the graph’s connectivity. Now the only purpose of the algorithm is to detect a negative cycle. If you still want to calculate the shortest paths from a source vertex in case there are no negative cycles, then <em>do not</em> make this modification.</p>
<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>function SPFA(G):
for v in V(G):
len[v] = 0
dis[v] = 0
Queue.push(v)
while !Queue.is_empty():
u = Queue.pop()
for (u, v) in E(G):
if dis[u] + w(u, v) < dis[v]:
len[v] = len[u] + 1
if len[v] == n:
return "negative cycle detected"
dis[v] = dis[u] + w(u, v)
if !Queue.contains(v):
Queue.push(v)
return "no negative cycle detected"
</code></pre></div></div>
<p>It is worth noting that although the SPFA is very fast when there is no negative cycle, it slows down significantly in the presence of one, especially in this version, because it has to keep calculating the shortest distance over and over again until a path reaches length \(n\). Further down in this article I will propose a way to halt the algorithm early.</p>
<h3 id="reconstructing-the-negative-cycle">Reconstructing the negative cycle</h3>
<p>Reconstructing the negative cycle can be done with a few modifications to the algorithm.</p>
<p>We will create a new array <code class="language-plaintext highlighter-rouge">pre</code>, where <code class="language-plaintext highlighter-rouge">pre[u]</code> is the direct predecessor of <code class="language-plaintext highlighter-rouge">u</code> in the current shortest path. We will have to update <code class="language-plaintext highlighter-rouge">pre[v]</code> whenever we have the condition <code class="language-plaintext highlighter-rouge">dis[u] + w(u, v) < dis[v]</code>.</p>
<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>// some code before
if dis[u] + w(u, v) < dis[v]:
pre[v] = u
len[v] = len[u] + 1
// some code after
</code></pre></div></div>
<p>Then we create a trace function to find the negative cycle. Here <code class="language-plaintext highlighter-rouge">Stack</code> is a First-In-Last-Out stack.</p>
<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>function Trace(pre[], v):
while !Stack.contains(v):
Stack.push(v)
v = pre[v]
cycle = [v]
while Stack.top() != v:
cycle.insert(Stack.pop())
cycle.insert(v)
return cycle
</code></pre></div></div>
<p><strong>Note:</strong> The vertex <code class="language-plaintext highlighter-rouge">v</code> in the <code class="language-plaintext highlighter-rouge">SPFA</code> function that triggers the negative cycle condition doesn’t neccesarily have to be part of a negative cycle. It may be a vertex that is reachable from a negative cycle.</p>
<p>The runtime complexity of the SPFA algorithm is at most \(\mathcal{O}(nm)\), and the trace function takes \(\mathcal{O}(n)\).</p>
<h3 id="early-termination">Early termination</h3>
<p>As mentioned before, the SPFA will be really slow in the presence of a negative cycle, as it has to keep computing until a path reaches length \(n\). In this section I will propose a way to terminate the algorithm early while keeping the worst-case runtime the same.</p>
<p>Let’s define the <em>dependency graph</em> \(D\) of \(G\) as the graph made of <code class="language-plaintext highlighter-rouge">(pre[u], u)</code> edges after the SPFA has terminated. Notice that in the case that \(G\) has no negative cycles, \(D\) is a Directed Acyclic Graph (DAG). Once there is a negative cycle, the dependency graph will also contain a cycle. This cycle in \(D\) also corresponds to the negative cycle in \(G\).</p>
<p><em>Proof:</em> <strong>[WIP]</strong></p>
<p>If we can detect when a cycle in \(D\) forms, we can terminate the SPFA early instead of waiting for some vertex to reach path length \(n\). A simple DFS on the induced graph \(D\) will do the trick. But since the DFS takes \(\mathcal{O}(n)\) (the induced graph \(D\) has \(n\) vertices and at most \(n\) edges), we will only do this step every \(n^{th}\) iteration of the inner loop.</p>
<p>Below is the pseudocode for the early-terminated SPFA.</p>
<ul>
<li><code class="language-plaintext highlighter-rouge">iter</code> is a counter for us to see how many iterations of the inner loop have passed.</li>
<li><code class="language-plaintext highlighter-rouge">detect_cycle(G, pre[])</code> is a function that returns <code class="language-plaintext highlighter-rouge">true</code> iff there is a cycle in the induced graph \(D\). There are many ways you can implement this to run in \(\mathcal{O}(n)\) and in this post I will not be discussing a specific implementation.</li>
</ul>
<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>function SPFA(G):
for v in V(G):
len[v] = 0
dis[v] = 0
Queue.push(v)
iter = 0
while !Queue.is_empty():
u = Queue.pop()
for (u, v) in E(G):
if dis[u] + w(u, v) < dis[v]:
pre[v] = u
dis[v] = dis[u] + w(u, v)
iter = iter + 1
if iter == n:
iter = 0
if detect_cycle(G, pre):
return "negative cycle detected"
dis[v] = dis[u] + w(u, v)
if !Queue.contains(v):
Queue.push(v)
if detect_cycle(G, pre):
return "negative cycle detected"
return "no negative cycle detected"
</code></pre></div></div>
<p>Worst-case time complexity: \(\mathcal{O}(nm)\).</p>
<p>Because the function <code class="language-plaintext highlighter-rouge">detect_cycle()</code> only runs once every \(n\) iterations, the worst-case runtime complexity stays the same. However, because this version terminates early, we can expect some pretty huge performance speedups as demonstrated in the next section.</p>
<h3 id="benchmarks">Benchmarks</h3>
<p>First I tested both variants (Path Length and Early Termination) on graphs with no negative cycles to demonstrate the awesomeness of the Shortest Path Faster Algorithm. For this, I used randomly-generated graphs with 1e5 vertices and 4e5 edges and a sample size of 100.</p>
<ul>
<li>Path length: 4 ms.</li>
<li>Early termination: 4 ms.</li>
</ul>
<p>Then, I tested both on graphs <em>with</em> negative cycles. Again, I used randomly generated graphs with 1e5 vertices and 4e5 edges and a sample size of 100.</p>
<ul>
<li>Path length: N/A.</li>
<li>Early termination: 4 ms.</li>
</ul>
<p>The path-length version really took a hit here (it wouldn’t finish within 3 minutes so I stopped benchmarking that variant for this test case), whereas the early-terminated version doesn’t seem to be affected at all.</p>
<p>Then I tested both variants on some smaller graphs so that the path-length variant can still produce a result within a reasonable time frame. This time, graphs have 1e4 vertices and 3e4 edges.</p>
<ul>
<li>Path length: 1737 ms.</li>
<li>Early termination: 0 ms.</li>
</ul>
<p>The <a href="/assets/code-snippets/cycle-detection-with-spfa.cpp">C++ Code</a> I used to benchmark the algorithm.</p>
<h3 id="conclusion">Conclusion</h3>
<p>It is clear (both from the theoretical runtime and from my experiments) that the early-terminated SPFA is a great algorithm to detect/find a negative cycle in a directed acyclic graph.</p>
<p>However, it still uses the Shortest Path Faster Algorithm as its backbone, and one can easily generate graphs for which the SPFA takes \(\mathcal{O}(nm)\) time.</p>Segmented SPFA: An improvement to the Shortest Path Faster Algorithm.2020-05-04T07:39:00+00:002020-05-04T07:39:00+00:00https://konaeakira.github.io/posts/segmented-spfa-an-improvement-to-the-shortest-path-faster-algorithm<script type="text/javascript" src="https://cdn.mathjax.org/mathjax/latest/MathJax.js?config=TeX-AMS-MML_HTMLorMML"></script>
<p><a rel="license" href="http://creativecommons.org/licenses/by/4.0/"><img alt="Creative Commons License" style="border-width:0" src="https://i.creativecommons.org/l/by/4.0/80x15.png" /></a><br />This work is licensed under a <a rel="license" href="http://creativecommons.org/licenses/by/4.0/">Creative Commons Attribution 4.0 International License</a>.</p>
<h3 id="summary">Summary</h3>
<p>The Shortest Path Faster Algorithm (SPFA) is an improvement of the Bellman-Ford algorithm. It computes the shortest paths from a single source vertex to all other vertices in a weighted directed graph. The SPFA has a worst-case runtime equal to that of the Bellman-Ford algorithm, but in practice it has been observed to be much faster.</p>
<p>In this post I will present the Segmented Shortest Path Faster Algorithm, which gives a significant speedup to the SPFA on ‘hard’ graphs with many Strongly Connected Components (SCC).</p>
<h3 id="the-algorithm">The algorithm</h3>
<p>Given a directed weighted graph with no negative cycles, the algorithm starts by partitioning the graph into SCCs. This forms a Directed Acyclic Graph (DAG) of SCCs. Then the algorithm uses the SPFA to find shortest paths in each SCC in topological order. In each iteration, the algorithm also relaxes vertices outside of the current SCC, but doesn’t add those vertices to the queue yet.</p>
<p>The SPFA can be slow in case a vertex is relaxed with a non-optimal value and it is used to relax other vertices. Then a second relaxtion has to ‘catch up’ and undo what the first relaxation propagated. The motivation for segmenting the graph is to limit ‘catching up’ to within the same SCC.</p>
<p>Below is the pseudocode for the Segmented SPFA. Here <code class="language-plaintext highlighter-rouge">Tarjan(G, s)</code> is Tarjan’s algorithm for finding SCCs.</p>
<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>procedure SegmentedSPFA(G, s):
TopoOrder = Tarjan(G, s)
for v in V(G) \ {s}:
d[v] = inf
d[s] = 0
for SCC in TopoOrder:
for v in SCC:
if d[v] != inf:
Queue.push(v)
while !Queue.is_empty():
u = Queue.pop()
for (u, v) in E(G):
if d[u] + w(u, v) < d[v]:
d[v] = d[u] + w(u, v)
if SCC.contains(v) and !Queue.contains(v):
Queue.push(v)
</code></pre></div></div>
<p>The runtime of the Segmented SPFA is the time it takes to find SCCs and order topologically (can be done in 1 pass of Tarjan’s algorithm), and running the SPFA for each of the SCCs. In the worst case (when there is only 1 SCC), the complexity can be \(\mathcal{O}(VE)\), which gives no improvement over the SFPA.</p>
<h3 id="proof-of-correctness">Proof of correctness</h3>
<p>We can prove the correctness of the algorithm via induction.</p>
<p>Let \(C_i\) (\(i \geq 2\)) be the component that the algorithm is currently working on. Note that \(C_i\) has to be reachable from the source, otherwise we can ignore this component. We assume that for all \(j \in \{1, ..., i - 1\}\) the algorithm has already computed the correct shortest path to vertices in \(C_j\). Then there must be at least \(1\) vertex in \(C_i\) for which the correct shortest path has been computed. Let’s call this vertex \(s_i\).</p>
<p><em>Proof:</em> Let \(x\) be an arbitrary vertex in \(C_i\). Let \(p(x)\) be a vertex so that it is a direct predecessor of \(x\) for at least one shortest path from the source to \(x\). Because of the topological ordering, \(p(x)\) is either in \(C_i\) or in some \(C_j\) with \(j \lt i\). Also, there cannot exist a cycle of the form \(\{x, p(x), p(p(x)), ..., x\}\), because this implies that there is a negative cycle. We cannot add \(\|C_i\|\) directed edges contained in \(C_i\) without creating a cycle, thus at least one edge must point out of \(C_i\), i.e. there exists a vertex \(x \in C_i\) so that \(p(x) \in C_j\) with \(j \lt i\).</p>
<p>Because the shortest path to \(s_i\) is finite, it is added to the queue for \(C_i\). The Shortest Path Faster Algorithm then takes care of the rest and computes the correct shortest paths for all vertices in \(C_i\) (It only needs one vertex with a correct shortest path to be in the queue. Other vertices slow down the algorithm by propagating suboptimal paths, but do not affect the correctness of the algorithm). It also relaxes edges going out from \(C_i\), completing the precondition for \(C_{i+1}\).</p>
<p>The base case is trivial, because the source is part of \(C_1\). And the shortest path to the source is \(0\), otherwise we can show that there exists a negative cycle.</p>
<h3 id="benchmarks">Benchmarks</h3>
<p>For randomly generated graphs, the SPFA is expected to run in \(\mathcal{O}(E)\) (unproven), so the Segmented SPFA gives no significant runtime improvement and can even be slower because it has to find all SCCs in the graph.</p>
<p>However, for graphs for which the SPFA takes longer (<a href="http://poj.org/showmessage?message_id=136458">zhougelin</a> shows us one way to construct such a graph), the runtime is dominated by the SPFA and we can really observe the speedup that the Segmented version gives. I will call such graphs ‘hard’ graphs.</p>
<p>You can download the <a href="/assets/code-snippets/segmented-spfa-benchmark.cpp">C++ code</a> I used to assess the performance of both algorithms. For each entry I used a sample size of 100.</p>
<p>First I tested both algorithms on <strong>easy graphs</strong>, with varying amounts of SCCs. These are connected graphs of 100000 vertices and 400000 edges.</p>
<table>
<tbody>
<tr>
<td>Algorithm</td>
<td>1 SCC</td>
<td>5 SCCs</td>
<td>20 SCCs</td>
<td>100 SCCs</td>
<td>500 SCCs</td>
</tr>
<tr>
<td>Segmented</td>
<td>41 ms</td>
<td>42 ms</td>
<td>41 ms</td>
<td>40 ms</td>
<td>39 ms</td>
</tr>
<tr>
<td>Vanilla</td>
<td>17 ms</td>
<td>19 ms</td>
<td>18 ms</td>
<td>13 ms</td>
<td>11 ms</td>
</tr>
</tbody>
</table>
<p>Unsurprisingly, the Segmented version is slower in all cases, because it computes all SCCs, whereas the vanilla version just zooms through the graph.</p>
<p>Then I tested both algorithms on <strong>hard graphs</strong> (test cases inspired by <a href="http://poj.org/showmessage?message_id=136458">zhougelin’s idea</a>, but designing the test cases so that they are still solveable with the SPFA instead of making it approach \(\mathcal{O}(VE)\)). Again, these are connected graphs of 100000 vertices and 400000 edges.</p>
<table>
<tbody>
<tr>
<td>Algorithm</td>
<td>1 SCC</td>
<td>5 SCCs</td>
<td>20 SCCs</td>
<td>100 SCCs</td>
<td>500 SCCs</td>
</tr>
<tr>
<td>Segmented</td>
<td>397 ms</td>
<td>377 ms</td>
<td>312 ms</td>
<td>128 ms</td>
<td>49 ms</td>
</tr>
<tr>
<td>Vanilla</td>
<td>377 ms</td>
<td>362 ms</td>
<td>293 ms</td>
<td>260 ms</td>
<td>271 ms</td>
</tr>
</tbody>
</table>
<p>This is where the Segmented version begins to show some potential. The runtime in these cases are dominated by the time spent on SPFA, not Tarjan’s algorithm.</p>
<p>What if I design even harder test cases? Putting the SPFA up against the Segmented version like this seems unfair, but I’ll do it just because. This time, the graphs have 10000 vertices and 40000 edges.</p>
<table>
<tbody>
<tr>
<td>Algorithm</td>
<td>1 SCC</td>
<td>5 SCCs</td>
<td>20 SCCs</td>
<td>100 SCCs</td>
<td>500 SCCs</td>
</tr>
<tr>
<td>Segmented</td>
<td>1085 ms</td>
<td>540 ms</td>
<td>162 ms</td>
<td>32 ms</td>
<td>7 ms</td>
</tr>
<tr>
<td>Vanilla</td>
<td>1056 ms</td>
<td>1104 ms</td>
<td>1110 ms</td>
<td>1107 ms</td>
<td>1099 ms</td>
</tr>
</tbody>
</table>
<h3 id="conclusion">Conclusion</h3>
<p>The Segmented Shortest Path Algorithm works best when the graph can be partitioned into lots of SCCs. Its speedup over the vanilla Shortest Path Algorithm is only noticable when the graph is ‘hard’, i.e. the runtime is dominated by the SPFA part of the algorithm.</p>
<p>For randomly generated graphs, it is still best to use the Shortest Path Faster Algorithm.</p>
<p>For hard graphs with few SCCs, it is still best to use the Shortest Path Faster Algorithm.</p>
<p><em>Further note: Segmenting can also be done for other path-finding algorithms like Dijkstra’s algorithm. I haven’t looked into that yet, but perhaps I will follow up on that in a future blog post.</em></p>