The Attractor Problem: Why AI Systems Collapse Into Themselves

Every AI system I’ve built, at some point, starts repeating itself.

Not in a “this bug is easy to fix” way. In a “it found a comfortable shape and now it will not leave” way. Chatbots start answering every message with the same three-beat structure. Agents get stuck in loops that technically progress but semantically don’t. Story generators cycle the same metaphors for pages. Companion bots give you variants of “that’s a really interesting point” until you want to throw the laptop off the balcony.

I spent a lot of months treating this as a prompt engineering problem. It isn’t. It’s a physics problem. Once you see it as physics, you can actually do something about it. Before that, you’re just rewriting prompts forever.

This post explains what’s actually going on, how to spot it in your own systems, and the one mindset shift that changed how I build AI forever.

What an attractor is (in plain English)

Imagine you drop a marble into a bowl. It rolls around. Eventually, it settles at the lowest point. That lowest point is an attractor. The marble tends toward it, from every starting position.

Physics is full of attractors. A pendulum swings, loses energy, settles straight down. A planet falls into orbit. A hot coffee cools to room temperature. In every case, the system has stable states it prefers, and it tends to end up in them no matter where it started.

AI systems have attractors too. They’re not made of marbles, they’re made of probability distributions over next tokens. But the logic is identical. The system has “comfortable” output patterns. Over time, without something actively pushing against the slope, it rolls into the comfortable pattern and stays there.

You experience this as your chatbot becoming boring, your agent getting stuck, your story losing variation. What’s actually happening is the system found a low-entropy stable state and settled into it.

Why this happens to every LLM-based system eventually

Three forces pull your AI toward attractors, and they all compound.

Force 1: Training-era priors. Your base model has seen trillions of tokens and learned which response patterns are “safe,” “helpful,” “professional.” Those patterns have strong probability mass. Left alone, every generation gravitates toward them. That’s why 80% of first-draft AI content reads the same across systems. Different wrappers, same deep attractor.

Force 2: Self-reinforcement through context. The model reads its own past outputs as evidence of what comes next. If the first few messages in a conversation had a certain rhythm, subsequent messages lean into that rhythm. If the agent solved step one with a particular tool call, it’ll reach for the same tool for step two. Patterns perpetuate themselves through the context window.

Force 3: Narrow input distribution. In a real application, you don’t see maximally diverse user inputs. You see a user who’s writing about one project, one problem, one mood. That narrow input pulls the model into a narrow output region. Every turn makes the narrowing worse.

All three forces push the system toward a shrinking set of “comfortable” outputs. Without a counter-force, the system collapses into its attractor. This is the attractor problem.

How to tell it’s happening to yours

Four symptoms. If you see any of them, you have attractor collapse.

• The same structure keeps appearing. Your bot starts every response with an empathy statement, or ends every response with a question. Every code suggestion has the same shape. Every story beat follows the same rhythm.
• The vocabulary narrows. Words that appeared on day one don’t appear anymore. The system has a shrinking palette.
• Surprise disappears. You stop being surprised by outputs. Everything feels predictable. When the system responds, you already knew roughly what it would say.
• User frustration rises. The clearest signal. If users start asking “can you say that differently” or “why do you keep saying that,” you’re deep in the attractor.

If you have logs, you can measure it. Embedding distance between consecutive outputs trends toward zero. N-gram repetition across outputs climbs. Response length stabilizes in a narrow band. All of these are quantitative signatures of an attractor.

Why “just use a better model” doesn’t fix it

Bigger models have richer output distributions. You’d think this would make attractors weaker. It doesn’t, quite.

Bigger models have more attractors, at more scales, some of them quite sophisticated. A small model might collapse into “Let me help you with that.” A large model might collapse into a sophisticated four-paragraph essay structure that feels smart but is the same essay every time. The attractors are prettier. They’re still attractors.

Upgrading to a bigger model can mask attractor collapse for a while. The outputs get richer. The variety feels higher. But run the same system for long enough and the larger model settles into its own stable patterns. They’re more elegant. They’re still patterns. If you measure repetition across outputs over time, the larger model dips into attractor behavior on roughly the same timeline as the smaller one. It just sounds smarter while stuck.

The bet of “bigger model will fix it” is the same bet as “better prompt will fix it.” Both assume the attractor is about model capability. It isn’t. It’s about output-space geometry, and that geometry has the same basic shape across scales.

What actually works

The attractor doesn’t go away. You can’t eliminate it. What you can do is keep the system from settling into it. Three approaches, in order of how well they work in practice.

1. External perturbation. Something outside the model that forcibly varies the context. For the game companion bot I wrote about in another post, the game screen was doing this. Every turn, the environment had changed, so the context changed, so the output had to adapt. The system never settled because the input never stabilized. This is the cheapest, strongest anti-attractor force you can give your AI.

2. Temperature scheduling. Most people run generation at a fixed temperature. If you increase temperature when your system is repeating itself and lower it when it’s being coherent, you can punch through local attractors before they lock. This is a light-weight intervention that works surprisingly well for conversational systems.

3. Structural variety enforcement. If your system has any structured output (conversation history, agent plan, generated document), add explicit diversity constraints. “This response cannot start with the same word as the last three responses.” “This tool call must differ from the previous two calls.” These feel clunky but they’re load-bearing.

What doesn’t work, or works less reliably:

• Longer system prompts. Addressed in another post. They don’t overcome the priors that create the attractor.
• Memory of past outputs. You’d think logging past outputs and telling the model to vary would help. It does, a little, but the model still weights toward its priors harder than it weights your “be varied” instruction.
• Fine-tuning. Can shift the attractors but not eliminate them. You end up with different attractors in different places.

Detecting attractor collapse in your own system

For anyone building something and wanting to actually measure this, here’s a minimal Python sketch. This works with any LLM API, logs the last N outputs, and flags when the system has collapsed into repetition.

from collections import deque
from dataclasses import dataclass

@dataclass
class AttractorMonitor:
    window_size: int = 10
    ngram_size: int = 4
    collapse_threshold: float = 0.35  # fraction of repeated n-grams

    def __post_init__(self):
        self.outputs = deque(maxlen=self.window_size)

    def ngrams(self, text: str) -> set[tuple]:
        tokens = text.split()
        return {tuple(tokens[i:i+self.ngram_size])
                for i in range(len(tokens) - self.ngram_size + 1)}

    def add(self, output: str) -> dict:
        new_ngrams = self.ngrams(output)
        self.outputs.append(new_ngrams)

        if len(self.outputs) < 3:
            return {"collapsed": False, "repetition_ratio": 0.0}

        all_ngrams = set().union(*self.outputs)
        if not all_ngrams:
            return {"collapsed": False, "repetition_ratio": 0.0}

        # Count n-grams appearing in more than one output
        counts = {}
        for ng in all_ngrams:
            counts[ng] = sum(1 for out in self.outputs if ng in out)
        repeated = sum(1 for c in counts.values() if c > 1)
        ratio = repeated / len(all_ngrams)

        return {
            "collapsed": ratio > self.collapse_threshold,
            "repetition_ratio": ratio,
        }

Drop this into any generation loop, call monitor.add(response) after each generation, and trigger your intervention of choice when collapsed flips to True. Bump the temperature, inject noise, force a tool call, whatever fits your system.

The threshold is tuneable. 0.35 is a starting point I’ve found reasonable for chat-style outputs. Raise it if you’re getting false positives, lower it if collapse is slipping through undetected.

The deeper view

AI systems look like software. They’re not, quite. They’re dynamical systems that happen to run on software. Every property you care about (identity stability, output variety, coherent reasoning over time) is a dynamical property, not a code property.

That reframing is the single most useful thing I’ve picked up in the last year of building AI. If you treat your chatbot like software, you debug it like software. You rewrite code. You tune prompts. You swap models. You add rules.

If you treat it like a dynamical system, you look at it differently. You ask: what forces are acting on this thing? Where are the stable states? How do I shape the landscape so the system settles where I want it to settle, instead of where its priors want it to settle?

That shift is worth more than any prompt engineering course you could take. Once you see AI as physics, you stop fighting yesterday’s battles and start building systems that work because the dynamics are right, not because the prompt is long enough.

Closing thought

Attractors are not a bug. They’re a property of the system. You cannot remove them. You can only keep moving so you never settle into one.

This applies to AI. It also applies to people. There’s something unsettlingly useful about noticing when you’ve started repeating yourself, in your work or your conversations, and asking: what’s the external perturbation I’m missing?

Bots and people both benefit from a screen that keeps changing.