To understand why Simulated Annealing is such a powerful tool, it helps to look at the problem it solves better than most traditional optimization methods. Many algorithms try to find the best solution by always moving in the direction of improvement. At first, this approach sounds effective.<\/p>\n\n\n\n
However, it has a major limitation. The algorithm can easily get stuck in a local optimum. This is a solution that appears optimal within a small neighborhood but is far from the best possible overall solution.<\/p>\n\n\n\n
Escaping Local Optima Through Controlled Randomness<\/strong><\/h3>\n\n\n\nImagine climbing hills in thick fog, trying to reach the highest peak. A greedy algorithm will always move up when it can. It stops when there\u2019s no higher hill. If it ends up on a small hill, it believes it has reached the top.<\/p>\n\n\n\n
Simulated Annealing avoids this pitfall. At the start, it\u2019s bold and willing to move downhill. This temporary loss is part of the strategy, allowing the algorithm to explore other areas and potentially find the true peak.<\/p>\n\n\n\n![\"Visual](\"https:\/\/beon.tech\/blog\/wp-content\/uploads\/2025\/12\/image.png\")
<\/a><\/figure>\n\n\n\nSimulated Annealing procedure (blue (mid gray in print version) dot \u2013 initial solution, orange (light gray in print version) dot \u2013 local minimum, green (mid gray in print version) dot \u2013 global minimum, red (dark gray in print version) arrow \u2013 hill climbing (move accepted with certain probability)).<\/p>\n\n\n\n
This idea is driven by a fundamental principle: controlled randomness. Instead of behaving rigidly from the start, Simulated Annealing introduces a variable called temperature<\/strong>, that controls how willing the algorithm is to accept worse solutions<\/strong>.<\/p>\n\n\n\nAt the beginning of the process, the temperature is set high<\/strong>. This means the algorithm is deliberately tolerant of mistakes:<\/p>\n\n\n\n\n- It embraces experimentation, <\/li>\n\n\n\n
- Tries risky moves, <\/li>\n\n\n\n
- Explores distant possibilities and, <\/li>\n\n\n\n
- Accepts solutions that are technically worse than the current one. <\/li>\n<\/ul>\n\n\n\n
This may feel counterintuitive<\/em>, but that initial flexibility is exactly what allows Simulated Annealing to escape rigid thinking and avoid getting trapped in shallow local optima.<\/p>\n\n\n\nAs the temperature gradually decreases, this tolerance fades. The algorithm becomes more selective, accepting fewer and fewer worse moves, and slowly transitions from exploration into refinement<\/strong>. By the end, only genuine improvements are considered, guiding the solution into a stable and refined state. <\/p>\n\n\n\nThe Maths Behind Simulated Annealing Decisions<\/strong><\/h3>\n\n\n\nAnother reason SA works is the mathematical foundation behind those decisions. When a worse solution appears, the algorithm doesn\u2019t just flip a coin<\/strong>. Its chance of acceptance follows a precise probability function<\/strong> based on the difference between solutions and the current temperature. The poorer the new solution, the less likely it is to be accepted. The cooler the environment becomes, the stricter the acceptance rule turns<\/strong>. This balance between exploration and exploitation allows SA to stretch farther than many classical methods, without losing direction. Mathematically, the acceptance probability can be expressed as:<\/p>\n\n\n\nP(accept)=e-ET<\/p>\n\n\n\n
Where <\/p>\n\n\n\n
\n- E=Enew-Ecurrent, is the increase<\/strong> in cost (or energy)<\/li>\n\n\n\n
- T <\/em>is the temperature<\/strong><\/li>\n<\/ul>\n\n\n\n
This formula has deep meaning:<\/p>\n\n\n\n
\n- If the new solution is only slightly worse<\/strong> (E small), it is likely to be accepted.<\/li>\n\n\n\n
- If the new solution is much worse<\/strong> (E large), it is rarely accepted.<\/li>\n\n\n\n
- If the temperature <\/strong>T<\/em><\/strong> is high<\/strong>, even bad solutions are often accepted.<\/li>\n\n\n\n
- If <\/strong>T <\/em><\/strong>is low<\/strong>, almost no bad solutions are accepted.<\/li>\n<\/ul>\n\n\n\n
So temperature directly controls how adventurous the algorithm is. The temperature is gradually reduced according to a cooling schedule<\/strong>, typically something like: <\/p>\n\n\n\nTk+1\u200b=\u03b1Tk with 0<\u03b1<1<\/p>\n\n\n\n
This exponential decay slowly transforms the algorithm from an explorer into a refiner.<\/p>\n\n\n\n
\n- Early phase<\/strong>: high T, <\/em>high randomness \u2192 exploration.<\/li>\n\n\n\n
- Late phase<\/strong>: low T<\/em>, low randomness \u2192 convergence.<\/li>\n<\/ul>\n\n\n\n
This dynamic shift is what prevents premature convergence while still guaranteeing stabilization. SA generates a Markov Chain whose stationary distribution at temperature T <\/em>is <\/p>\n\n\n\n\u03c0(x) \u221d e-E(x)T, As T0<\/p>\n\n\n\n
This distribution becomes increasingly concentrated around the global minima of E(x). In other words, the algorithm becomes statistically biased toward optimal solutions as it cools<\/strong>.<\/p>\n\n\n\nFinally, Simulated Annealing works because it mirrors the real behavior of physical systems<\/strong>. In metallurgy, slow cooling lets atoms settle into highly ordered structures instead of chaotic ones. By imitating this process computationally, we benefit from millions of years of natural optimization.<\/p>\n\n\n\nIn practice, Simulated Annealing can explore enormous solution spaces and still find solutions close to the best possible, even when millions or billions of combinations exist. It transforms randomness into strategy, and uncertainty into structure. That\u2019s why, decades after its invention, it remains a favorite across diverse fields\u2014from engineering and logistics to AI and science.<\/p>\n\n\n\n
A Practical Example: Traveling Salesperson Problem<\/strong><\/h2>\n\n\n\nGiven N<\/em> cities (points), find the shortest tour that visits each city once and returns to start.<\/p>\n\n\n\n\n- State<\/strong> S<\/em>: a permutation (ordering) of cities<\/li>\n\n\n\n
- Energy<\/strong> E(S): total tour length<\/li>\n\n\n\n
- Neighbor move<\/strong>: a 2-opt swap<\/strong> (reverse a segment) \u2014 standard TSP move that\u2019s simple and strong.<\/li>\n<\/ul>\n\n\n\n
Representing the problem<\/strong><\/h3>\n\n\n\nCities<\/strong><\/p>\n\n\n\nWe store cities as 2D points: points = [(x1, y1), (x2, y2), …]. Each city is just a coordinate.<\/p>\n\n\n\n
A \u201ctour\u201d (a solution)<\/strong><\/p>\n\n\n\nA tour is a permutation of city indices: tour = [0, 5, 2, 1, 4, …]<\/p>\n\n\n\n
Meaning: visit city 0 \u2192 then city 5 \u2192 then city 2 \u2192 \u2026 and finally return to the start.<\/p>\n\n\n\n
This representation is super common because it\u2019s simple and it guarantees you visit each city exactly once.<\/p>\n\n\n\n
Measuring quality: the cost function<\/strong><\/h3>\n\n\n\nDistance between two cities<\/strong><\/p>\n\n\n\ndef distance(a, b):<\/p>\n\n\n\n
return math.hypot(a[0] – b[0], a[1] – b[1])<\/p>\n\n\n\n
math.hypot(dx, dy) <\/p>\n\n\n\n
Computes the euclidean distance between both cities through Pythagoras theorem.<\/p>\n\n\n\n
Total tour length (energy)<\/strong><\/p>\n\n\n\ndef tour_length(points, tour):<\/p>\n\n\n\n
total = 0.0<\/p>\n\n\n\n
n = len(tour)<\/p>\n\n\n\n
for i in range(n):<\/p>\n\n\n\n
total += distance(points[tour[i]], points[tour[(i + 1) % n]])<\/p>\n\n\n\n
return total<\/p>\n\n\n\n
Key detail:<\/strong> (i + 1) % n wraps around so the last city connects back to the first city, forming a loop. This tour_length is your energy E(S) in SA terms.<\/p>\n\n\n\nGenerating a neighbor: the 2-opt move<\/strong><\/h2>\n\n\n\nTo explore the search space, SA needs to propose \u201cnearby\u201d solutions.<\/p>\n\n\n\n
def two_opt_neighbor(tour):<\/p>\n\n\n\n
n = len(tour)<\/p>\n\n\n\n
i, j = sorted(random.sample(range(n), 2))<\/p>\n\n\n\n
new_tour = tour[:]<\/p>\n\n\n\n
new_tour[i:j+1] = reversed(new_tour[i:j+1])<\/p>\n\n\n\n
return new_tour<\/p>\n\n\n\n
What this does:<\/strong><\/p>\n\n\n\n\n- pick two random indices i < j<\/li>\n\n\n\n
- reverse the subsegment from i to j<\/li>\n<\/ul>\n\n\n\n
This is called 2-opt<\/strong> and it\u2019s great for TSP because it can remove crossings, makes structural changes and it\u2019s still cheap to compute.<\/p>\n\n\n\nExample:<\/strong><\/p>\n\n\n\nTour: [0, 1, 2, 3, 4, 5]
If i=2, j=4, reverse [2,3,4] \u2192 [4,3,2]<\/p>\n\n\n\n
New tour: [0, 1, 4, 3, 2, 5]<\/p>\n\n\n\n
The Simulated Annealing loop<\/strong><\/h2>\n\n\n\nThis is the heart of it:<\/p>\n\n\n\n
current = shuffled tour<\/p>\n\n\n\n
best = current<\/p>\n\n\n\n
curr_E = tour_length(current)<\/p>\n\n\n\n
best_E = curr_E<\/p>\n\n\n\n
T = T0<\/p>\n\n\n\n
Now repeat for many iterations the following steps:<\/p>\n\n\n\n
Step 1: propose a candidate solution<\/strong><\/h3>\n\n\n\ncandidate = two_opt_neighbor(current)<\/p>\n\n\n\n
cand_E = tour_length(candidate)<\/p>\n\n\n\n
dE = cand_E – curr_E<\/p>\n\n\n\n
dE <= 0 means the candidate is better<\/strong> (shorter tour), meanwhile dE > 0 means worse<\/strong>.<\/p>\n\n\n\nStep 2: accept or reject with Metropolis criterion<\/strong><\/h3>\n\n\n\nif dE <= 0 or random.random() < math.exp(-dE \/ T):<\/p>\n\n\n\n
current = candidate<\/p>\n\n\n\n
curr_E = cand_E<\/p>\n\n\n\n
If it\u2019s better: take it. If it\u2019s worse: maybe take it depending on temperature.
<\/p>\n\n\n\n
The probability of accepting worse moves is P(accept)=e-ET, as said at the very beginning of this article. So:<\/p>\n\n\n\n
\n- early (high T<\/em>): accepts worse moves often \u2192 exploration<\/li>\n\n\n\n
- late (low T<\/em>): almost never accepts worse moves \u2192 refinement<\/li>\n<\/ul>\n\n\n\n
Step 3: track the best-ever solution<\/strong><\/h3>\n\n\n\nif curr_E < best_E:<\/p>\n\n\n\n
best = current[:]<\/p>\n\n\n\n
best_E = curr_E<\/p>\n\n\n\n
Even if the search later wanders, we keep the best solution found so far.<\/p>\n\n\n\n
Step 4: cool down<\/strong><\/h3>\n\n\n\nT *= alpha, this makes the algorithm gradually more strict.<\/p>\n\n\n\n
Final Results<\/strong><\/h2>\n\n\n\n![\"Visuals](\"https:\/\/beon.tech\/blog\/wp-content\/uploads\/2025\/12\/image-4.png\")
<\/figure>\n\n\n\nThe four figures illustrate how Simulated Annealing transforms a random, inefficient solution into a structured and near-optimal one. The first plot shows the initial tour, which is a random ordering of cities. The path jumps chaotically across the plane, producing many long crossings and unnecessary detours. This results in a very large total distance (around 18.8 in this run). This is typical: a random solution contains no structure and therefore serves as a good baseline to demonstrate improvement. The temperature curve shows a rapid initial drop followed by a long, slow tail.<\/p>\n\n\n\n
\n- Early on, temperature is relatively high, so the algorithm frequently accepts worse moves.<\/li>\n\n\n\n
- This allows the search to move freely and escape bad local structures.<\/li>\n\n\n\n
- As temperature decreases, acceptance of worse moves becomes rarer, and the algorithm gradually stabilizes.<\/li>\n<\/ul>\n\n\n\n
By about 1000 iterations, the temperature is already close to zero, meaning the algorithm is effectively behaving like a greedy local optimizer from that point onward.<\/p>\n\n\n\n
The cost curve reveals the core dynamic of Simulated Annealing:<\/p>\n\n\n\n
\n- The steep drop<\/strong> at the beginning corresponds to rapid structural improvements in the tour.<\/strong><\/li>\n\n\n\n
- The small upward spikes<\/strong> represent moments where worse solutions were accepted temporarily<\/strong>.<\/li>\n\n\n\n
- The long flat region <\/strong>at the end shows convergence: the algorithm is no longer exploring and is only making<\/strong> very small refinements<\/strong>.<\/li>\n<\/ul>\n\n\n\n
This pattern \u2014 rapid improvement, noisy exploration, and slow stabilization \u2014 is exactly the behavior Simulated Annealing is designed to produce.<\/p>\n\n\n\n
The final tour has a total length of about 4.9, compared to the initial 18.8 \u2014 a dramatic improvement. The structure is visibly smoother: fewer crossings, shorter jumps, and a more natural geometric progression through the points. <\/strong>While not guaranteed to be globally optimal, the result is already very close to the best possible for this random instance as you can see in this Github code<\/a>.<\/p>\n\n\n\nConclusion: From Intuition to Intelligence<\/strong><\/h2>\n\n\n\nWhat makes Simulated Annealing especially valuable is its generality. It requires no gradients, no convexity assumptions, and no special structure beyond the ability to compare solutions and generate neighbors. This makes it applicable to a wide range of problems \u2014 from scheduling and routing to machine learning and game AI \u2014 where traditional optimization techniques struggle.<\/p>\n\n\n\n
In the end, Simulated Annealing teaches a broader lesson about problem solving: sometimes, progress does not come from always making the best possible move, but from allowing temporary setbacks in service of deeper exploration. By embracing uncertainty early and discipline later, the algorithm turns randomness into structure and complexity into tractability.<\/p>\n","protected":false},"excerpt":{"rendered":"
<\/a><\/figure>\n\n\n\nSimulated Annealing procedure (blue (mid gray in print version) dot \u2013 initial solution, orange (light gray in print version) dot \u2013 local minimum, green (mid gray in print version) dot \u2013 global minimum, red (dark gray in print version) arrow \u2013 hill climbing (move accepted with certain probability)).<\/p>\n\n\n\n
This idea is driven by a fundamental principle: controlled randomness. Instead of behaving rigidly from the start, Simulated Annealing introduces a variable called temperature<\/strong>, that controls how willing the algorithm is to accept worse solutions<\/strong>.<\/p>\n\n\n\n At the beginning of the process, the temperature is set high<\/strong>. This means the algorithm is deliberately tolerant of mistakes:<\/p>\n\n\n\n This may feel counterintuitive<\/em>, but that initial flexibility is exactly what allows Simulated Annealing to escape rigid thinking and avoid getting trapped in shallow local optima.<\/p>\n\n\n\n As the temperature gradually decreases, this tolerance fades. The algorithm becomes more selective, accepting fewer and fewer worse moves, and slowly transitions from exploration into refinement<\/strong>. By the end, only genuine improvements are considered, guiding the solution into a stable and refined state. <\/p>\n\n\n\n Another reason SA works is the mathematical foundation behind those decisions. When a worse solution appears, the algorithm doesn\u2019t just flip a coin<\/strong>. Its chance of acceptance follows a precise probability function<\/strong> based on the difference between solutions and the current temperature. The poorer the new solution, the less likely it is to be accepted. The cooler the environment becomes, the stricter the acceptance rule turns<\/strong>. This balance between exploration and exploitation allows SA to stretch farther than many classical methods, without losing direction. Mathematically, the acceptance probability can be expressed as:<\/p>\n\n\n\n P(accept)=e-ET<\/p>\n\n\n\n Where <\/p>\n\n\n\n This formula has deep meaning:<\/p>\n\n\n\n So temperature directly controls how adventurous the algorithm is. The temperature is gradually reduced according to a cooling schedule<\/strong>, typically something like: <\/p>\n\n\n\n Tk+1\u200b=\u03b1Tk with 0<\u03b1<1<\/p>\n\n\n\n This exponential decay slowly transforms the algorithm from an explorer into a refiner.<\/p>\n\n\n\n This dynamic shift is what prevents premature convergence while still guaranteeing stabilization. SA generates a Markov Chain whose stationary distribution at temperature T <\/em>is <\/p>\n\n\n\n \u03c0(x) \u221d e-E(x)T, As T0<\/p>\n\n\n\n This distribution becomes increasingly concentrated around the global minima of E(x). In other words, the algorithm becomes statistically biased toward optimal solutions as it cools<\/strong>.<\/p>\n\n\n\n Finally, Simulated Annealing works because it mirrors the real behavior of physical systems<\/strong>. In metallurgy, slow cooling lets atoms settle into highly ordered structures instead of chaotic ones. By imitating this process computationally, we benefit from millions of years of natural optimization.<\/p>\n\n\n\n In practice, Simulated Annealing can explore enormous solution spaces and still find solutions close to the best possible, even when millions or billions of combinations exist. It transforms randomness into strategy, and uncertainty into structure. That\u2019s why, decades after its invention, it remains a favorite across diverse fields\u2014from engineering and logistics to AI and science.<\/p>\n\n\n\n Given N<\/em> cities (points), find the shortest tour that visits each city once and returns to start.<\/p>\n\n\n\n Cities<\/strong><\/p>\n\n\n\n We store cities as 2D points: points = [(x1, y1), (x2, y2), …]. Each city is just a coordinate.<\/p>\n\n\n\n A \u201ctour\u201d (a solution)<\/strong><\/p>\n\n\n\n A tour is a permutation of city indices: tour = [0, 5, 2, 1, 4, …]<\/p>\n\n\n\n Meaning: visit city 0 \u2192 then city 5 \u2192 then city 2 \u2192 \u2026 and finally return to the start.<\/p>\n\n\n\n This representation is super common because it\u2019s simple and it guarantees you visit each city exactly once.<\/p>\n\n\n\n Distance between two cities<\/strong><\/p>\n\n\n\n def distance(a, b):<\/p>\n\n\n\n return math.hypot(a[0] – b[0], a[1] – b[1])<\/p>\n\n\n\n math.hypot(dx, dy) <\/p>\n\n\n\n Computes the euclidean distance between both cities through Pythagoras theorem.<\/p>\n\n\n\n Total tour length (energy)<\/strong><\/p>\n\n\n\n def tour_length(points, tour):<\/p>\n\n\n\n total = 0.0<\/p>\n\n\n\n n = len(tour)<\/p>\n\n\n\n for i in range(n):<\/p>\n\n\n\n total += distance(points[tour[i]], points[tour[(i + 1) % n]])<\/p>\n\n\n\n return total<\/p>\n\n\n\n Key detail:<\/strong> (i + 1) % n wraps around so the last city connects back to the first city, forming a loop. This tour_length is your energy E(S) in SA terms.<\/p>\n\n\n\n To explore the search space, SA needs to propose \u201cnearby\u201d solutions.<\/p>\n\n\n\n def two_opt_neighbor(tour):<\/p>\n\n\n\n n = len(tour)<\/p>\n\n\n\n i, j = sorted(random.sample(range(n), 2))<\/p>\n\n\n\n new_tour = tour[:]<\/p>\n\n\n\n new_tour[i:j+1] = reversed(new_tour[i:j+1])<\/p>\n\n\n\n return new_tour<\/p>\n\n\n\n What this does:<\/strong><\/p>\n\n\n\n This is called 2-opt<\/strong> and it\u2019s great for TSP because it can remove crossings, makes structural changes and it\u2019s still cheap to compute.<\/p>\n\n\n\n Example:<\/strong><\/p>\n\n\n\n Tour: [0, 1, 2, 3, 4, 5] New tour: [0, 1, 4, 3, 2, 5]<\/p>\n\n\n\n This is the heart of it:<\/p>\n\n\n\n current = shuffled tour<\/p>\n\n\n\n best = current<\/p>\n\n\n\n curr_E = tour_length(current)<\/p>\n\n\n\n best_E = curr_E<\/p>\n\n\n\n T = T0<\/p>\n\n\n\n Now repeat for many iterations the following steps:<\/p>\n\n\n\n candidate = two_opt_neighbor(current)<\/p>\n\n\n\n cand_E = tour_length(candidate)<\/p>\n\n\n\n dE = cand_E – curr_E<\/p>\n\n\n\n dE <= 0 means the candidate is better<\/strong> (shorter tour), meanwhile dE > 0 means worse<\/strong>.<\/p>\n\n\n\n if dE <= 0 or random.random() < math.exp(-dE \/ T):<\/p>\n\n\n\n current = candidate<\/p>\n\n\n\n curr_E = cand_E<\/p>\n\n\n\n If it\u2019s better: take it. If it\u2019s worse: maybe take it depending on temperature. The probability of accepting worse moves is P(accept)=e-ET, as said at the very beginning of this article. So:<\/p>\n\n\n\n if curr_E < best_E:<\/p>\n\n\n\n best = current[:]<\/p>\n\n\n\n best_E = curr_E<\/p>\n\n\n\n Even if the search later wanders, we keep the best solution found so far.<\/p>\n\n\n\n T *= alpha, this makes the algorithm gradually more strict.<\/p>\n\n\n\n The four figures illustrate how Simulated Annealing transforms a random, inefficient solution into a structured and near-optimal one. The first plot shows the initial tour, which is a random ordering of cities. The path jumps chaotically across the plane, producing many long crossings and unnecessary detours. This results in a very large total distance (around 18.8 in this run). This is typical: a random solution contains no structure and therefore serves as a good baseline to demonstrate improvement. The temperature curve shows a rapid initial drop followed by a long, slow tail.<\/p>\n\n\n\n By about 1000 iterations, the temperature is already close to zero, meaning the algorithm is effectively behaving like a greedy local optimizer from that point onward.<\/p>\n\n\n\n The cost curve reveals the core dynamic of Simulated Annealing:<\/p>\n\n\n\n This pattern \u2014 rapid improvement, noisy exploration, and slow stabilization \u2014 is exactly the behavior Simulated Annealing is designed to produce.<\/p>\n\n\n\n The final tour has a total length of about 4.9, compared to the initial 18.8 \u2014 a dramatic improvement. The structure is visibly smoother: fewer crossings, shorter jumps, and a more natural geometric progression through the points. <\/strong>While not guaranteed to be globally optimal, the result is already very close to the best possible for this random instance as you can see in this Github code<\/a>.<\/p>\n\n\n\n What makes Simulated Annealing especially valuable is its generality. It requires no gradients, no convexity assumptions, and no special structure beyond the ability to compare solutions and generate neighbors. This makes it applicable to a wide range of problems \u2014 from scheduling and routing to machine learning and game AI \u2014 where traditional optimization techniques struggle.<\/p>\n\n\n\n In the end, Simulated Annealing teaches a broader lesson about problem solving: sometimes, progress does not come from always making the best possible move, but from allowing temporary setbacks in service of deeper exploration. By embracing uncertainty early and discipline later, the algorithm turns randomness into structure and complexity into tractability.<\/p>\n","protected":false},"excerpt":{"rendered":"\n
The Maths Behind Simulated Annealing Decisions<\/strong><\/h3>\n\n\n\n
\n
\n
\n
A Practical Example: Traveling Salesperson Problem<\/strong><\/h2>\n\n\n\n
\n
Representing the problem<\/strong><\/h3>\n\n\n\n
Measuring quality: the cost function<\/strong><\/h3>\n\n\n\n
Generating a neighbor: the 2-opt move<\/strong><\/h2>\n\n\n\n
\n
If i=2, j=4, reverse [2,3,4] \u2192 [4,3,2]<\/p>\n\n\n\nThe Simulated Annealing loop<\/strong><\/h2>\n\n\n\n
Step 1: propose a candidate solution<\/strong><\/h3>\n\n\n\n
Step 2: accept or reject with Metropolis criterion<\/strong><\/h3>\n\n\n\n
<\/p>\n\n\n\n\n
Step 3: track the best-ever solution<\/strong><\/h3>\n\n\n\n
Step 4: cool down<\/strong><\/h3>\n\n\n\n
Final Results<\/strong><\/h2>\n\n\n\n
<\/figure>\n\n\n\n\n
\n
Conclusion: From Intuition to Intelligence<\/strong><\/h2>\n\n\n\n