# SIG-FPT 2026-08-07 — Session Notes

Participants:
  - Patrick Nast | UTC-7
  - Venkatesh Rao | UTC-8
  - PAtwater | UTC-8 | 31P
  - Ellen Kaye-Cheveldayoff (UTC -4)
  - Ergod | UTC+1
  - Giovanni Merlino | UTC+2
  - Jordan Peacock | UTC-5
  - Matthew Bright UTC-7

---

## 📖 The Reading
The session focused on **Ant Colony Optimization (ACO)** and a more recent **deep learning revival ("Deep ACO")** paper. Patrick assigned two-to-three readings: an introductory/review piece on ACO and related ideas, and a recent paper applying deep reinforcement learning to learn ACO heuristics. Specific titles/authors were not stated verbatim, though Giovanni noted one paper came from the original ACO/Ant Colony research line and emphasized a link between stochastic gradient descent and ACO.

## 🧭 Overview
This is part of the group's "summer of stigmergy" series within the formal protocol theory SIG. Having previously covered the Deneubourg pheromone-signaling model and termite-construction (sematectonic vs. marker, qualitative vs. quantitative) stigmergy, the group now moved from natural/foundational models to *engineered* systems that use stigmergy — specifically ACO. Discussion centered on how ACO recasts combinatorial optimization as ants laying pheromone on abstract graphs, why the classic approach underperformed and where it still shines (dynamic routing), and how the newer deep-learning approach learns the heuristic rather than hand-tuning it. Much of the conversation was participants still working to understand the papers, plus threads on locality, stability, and real-world applications.

## 💡 Key Points & Themes
- **From nature to engineered protocols (Patrick):** The natural progression is moving from foundational models (Deneubourg pheromones, termite construction) to engineered systems that borrow/extend them. ACO is the cleanest computer-science example — casting problems like TSP, network routing, or protein folding as graph-path problems where digital "ants" lay pheromone tracks.
- **The core "move" (Patrick):** Replace the physical traversal environment with an *abstract computational (graph) space*. If a problem can be framed as finding a path on a graph, ants can lay pheromone along edges; choosing an update rule that reinforces better solutions more strongly yields iterative improvement — the same feedback-loop logic as the physical case.
- **Historical arc (Patrick):** ACO was heavily published in the late 1990s–early 2000s, reached maturity, but never became state-of-the-art for the big combinatorial optimization problems. A key limitation is the **heuristic** — a hand-tuned, "fiddly" update rule specific to each problem.
- **The Deep ACO contribution (Patrick):** The revival ("neuro-combinatorial optimization") *learns* the heuristic parameter rather than hand-tuning it, making the technique more flexible and compelling in the deep-learning era.
- **What the second paper actually does (Venkatesh):** Initially confused, he clarified it is *not* just exploring parameters of a single heuristic form (e.g., evaporation rates). It explores the **space of functional forms** for choosing between next nodes in the construction graph — e.g., the simple "nearer cities are better" form (1/distance) vs. alternatives like 1/distance². Open question for him: how much of the value is computational advantage vs. intuition/interpretability advantage over traditional optimization.
- **Where stigmergy earns its keep (Patrick):** Little computational benefit on static/centralized combinatorial problems (classical algorithms do fine). The exception is **dynamic routing** — nonstationary environments, distributed, partial-information settings — which is also the environment ants evolved to thrive in, making the fit natural. Connects to multi-agent reinforcement learning problems.
- **Quasi-static framing (Venkatesh):** The setup is neither fully static nor dynamic but "quasi-static" — nothing changes during epochs of partial-solution construction, then an update happens (to pheromones, or in Deep ACO, movement in the heuristic space). Analogous to **dual control problems** in control theory, switching between observation and control modes.
- **Exploration vs. exploitation (Venkatesh):** Drawing on his PhD distributed-scheduling work (related to multi-armed bandit problems), he contrasted a two-phase approach (explore, learn weights, then exploit) with a fully adaptive/real-time approach. Fully dynamic simultaneous identification and control causes **thrashing/crashing**, a problem "AI people are slowly rediscovering." Done right — connecting signals correctly across space and time — you can improve performance *and* solve stability.
- **Locality and calibrated nonlocality (Venkatesh):** Stigmergy is uniquely hard here because it's local in space and time by design (ants sense only locally; compute processes see only nearby memory and a few speculative steps). His intuition: make stigmergy more powerful by "contaminating" mostly-local information with a small, strategic amount of global/nonlocal information — e.g., ~90% local sensing plus an occasional variable from a distant ant or a solution from several steps ago ("calibrated amounts of nonlocal entanglement in space-time"). He lacks a formalization for the ant case.
- **Applications angle (PAtwater):** Interested less in benchmark performance and more in messy real-world use — e.g., **mutual aid after a disaster**, routing resources from where they are to where they're needed. The value is in applying these concepts to real-world situations, not just theoretical benchmarks.
- **Getting out of local optima (Ellen):** Read the second paper as claiming an advantage in escaping local optimization traps to find better global solutions. It adds an extra learning step trained across many types of optimization problems; she wasn't sure what the net payoff is or whether it's just an extra required step. She got the sense the authors were frustrated with features of existing ant simulations and wanted to improve them.
- **Structural weakness (Ergod):** Evaporation means the system eventually *forgets* good solutions; without bolting on global credit systems, pure stigmergy struggles. But he reframed evaporation as a *feature* for dynamic settings — it lets the system discard stale solutions and react to environmental change. Framed static setups as optimization theory and dynamic setups as control theory; sees stigmergy's appeal as "compiling" these methods into substrates.
- **Under-cited but promising link (Giovanni):** Focused on the second/third papers; found it academically strange that one paper (from the original ACO author's line) is not widely cited despite making a strong point about the **link between stochastic gradient descent and ACO**. Sees this as a generalization insight and a potential "rabbit hole" worth investigating — who has exploited this SGD–ACO link.

## 🔀 Questions & Disagreements
- **What is the second paper actually doing?** Multiple participants (Venkatesh, Ellen, Matthew) said their main task was still just understanding the paper. Venkatesh corrected an initial misreading (parameter space vs. functional-form space).
- **What's the real advantage of ACO/stigmergy framing?** Ergod raised the recurring question of why writing problems in stigmergic language is any better than standard solutions. Consensus leaned toward: little advantage on static/centralized problems, real advantage in dynamic/nonstationary/distributed settings.
- **Is the extra learning step in Deep ACO worth it?** Ellen was unclear whether the cross-problem learning step delivers a durable payoff or is just extra overhead each time.
- **Computational vs. intuition advantage** (Venkatesh's open question to his LLM): unresolved.
- **How to formalize calibrated nonlocality** (Venkatesh): admitted intuition without formalization for the ant case.
- No sharp disagreements; mostly complementary framings and shared uncertainty.

## 🔗 References Mentioned
- **Deneubourg model** of pheromone signaling (foundation from earlier sessions).
- **Ant Colony Optimization (ACO)** and the traveling salesman problem, network routing, protein folding.
- **Neuro-combinatorial optimization / Deep ACO** (deep reinforcement learning applied to learning heuristics).
- **Stochastic gradient descent (SGD)** — its link to ACO, per Giovanni.
- **Control theory** — dual control, identification and control, exploration vs. exploitation (Venkatesh).
- **Multi-armed bandit problem** (Venkatesh's PhD distributed-scheduling work).
- **Sematectonic vs. marker; qualitative vs. quantitative stigmergy** (from prior weeks).
- **Dan Schmidt** — building the ant simulator for the group (not present; noted by Venkatesh).

## ✅ Action Items & Next Time
- **Venkatesh** to send a new recurring calendar invite for a **simulator/modeling meeting**, Fridays at **11:00 Pacific** (one hour after this call), starting **next Friday**, over the next ~4–5 weeks.
- **Dan Schmidt** to submit his existing ant-simulator code to a **shared repo** Venkatesh set up; group can then contribute.
- Cadence: coding/simulator work every other week (with a short slot after this call on alternating weeks), full-hour theory sessions on the alternate weeks; coordination in the main channel.
- **Workshop:** two-day workshop on **September 21–22** (~5–6 weeks out). Goal: use the theory readings as inspiration to design a simulator with an easily programmable surface for investigating interesting questions during the workshop.
- **Ongoing thread of interest:** figuring out the right "language" for encoding stigmergic features (evaporation, etc.) into agents so they can be observed in the simulator (Ergod, Patrick).
- **Follow-up research idea:** investigate the SGD–ACO link and who has exploited it (Giovanni).
- Patrick to do periodic mid-session check-ins to keep the recording bot alive (per Venkatesh's note).

## ⭐ Memorable Quotes
- **Venkatesh:** "One way you make dynamics possible without destabilizing things is to basically contaminate local information with global information in a very strategic way."
- **PAtwater:** "Part of the things that's interesting about this is not just... getting the best algorithm on these theoretical benchmarks, but then how you could apply it in messy real world situations."
- **Ergod:** "The structural weakness to this paradigm is that we have evaporation. So eventually the system forgets good solutions."