Technology · 6 min read
How Recommendation Algorithms Shape What We Want
You think the algorithm knows you—but what if it's been writing you all along? The recommendation engine isn't a mirror. It's a pen.

Six months ago, a friend texted me a photo of her workshop. Chisels, mallets, a half-finished dovetail joint. She'd never expressed any interest in woodworking before. When I asked how this happened, she laughed and said, "I genuinely don't know. YouTube just kept showing me videos, and at some point I bought a workbench." She paused, then added something that stuck: "I can't tell if I discovered this about myself, or if it was decided for me."
That question is no longer a philosophical curiosity. It is, increasingly, the central problem of living with algorithmic media.
How Recommendation Engines Actually Work
We tend to imagine recommendation systems as sophisticated listeners—patient, attentive, building an ever-more-accurate portrait of who we are. The reality is structurally different, and the difference matters.
The actual loop works like this: the algorithm observes your behavior, serves you content, records how you respond, updates its model of you, then serves new content shaped by that updated model. Repeat, indefinitely. What looks like learning is actually a tightening feedback spiral.
Here is the critical flaw buried inside that loop: the training data is not independent of the system. Every click, pause, and scroll that the algorithm interprets as a signal of your preferences was itself a response to something it chose to show you first. You didn't arrive with preferences and get measured. You arrived, got shown something, responded, and that response became "data about you."
There is no ground truth to measure against. The system isn't discovering a pre-existing taste—it's optimizing for engagement with content it selected, then treating that engagement as evidence of authentic desire. The map is drawing the territory.
Preference Laundering and the Ghost Self
Researchers have a useful term for what happens next: preference laundering. Behaviors that were generated by algorithmic nudges get recycled back into the system as evidence of genuine desire, with their manufactured origin stripped away. The algorithm doesn't record that it served you three consecutive videos about minimalist living before you started clicking on them. It only records that you clicked.
The empirical evidence here is striking. A 2019 study by researchers at the University of Massachusetts examining YouTube's recommendation architecture found that users' viewing behavior diverged significantly from their stated preferences within weeks of platform use—the algorithm was reliably pulling people toward more extreme or emotionally intense content than they reported wanting. A 2021 Spotify internal study, later reported by The Wall Street Journal, found that the platform's recommendation systems were meaningfully shaping users' musical identity, not merely reflecting it—particularly for younger users with less-formed tastes. TikTok's "For You" page accelerates this dynamic further still: its reinforcement model is aggressive enough that researchers at Mozilla have documented users' feeds transforming dramatically within 30 minutes of interaction.
The philosophical stake is real. If your preferences are substantially shaped by what you were shown, then the "self" the algorithm claims to model is partly a fiction it co-authored. The portrait looks like you because it was built from your behavior. But your behavior was staged on a set the algorithm designed.
Why This Is Different From Old Influence
Every generation has had its preferences shaped by outside forces. Advertising, peer pressure, editorial curation, the books your parents kept on their shelves—none of us arrives at our tastes from nowhere. Cultural critics have always understood this. So is algorithmic recommendation really different?
Yes. The differences are speed, invisibility, and personalization at scale—and together they constitute a qualitative shift, not just a quantitative one.
A magazine editor influences millions of readers the same way. That uniformity is, paradoxically, a protection: you can compare notes with your neighbor, notice the influence, push back collectively. An algorithm sculpts each user individually. Your feed and mine are different enough that the influence is nearly impossible to name, share, or resist as a social act. There is no common text to critique.
Then there is the compounding effect. Because each session's behavioral data feeds the next session's recommendations, small early nudges accumulate. Researchers call this preference drift—the gradual, invisible migration of your interests along a trajectory set partly by early algorithmic choices you had no reason to scrutinize. A slight initial tilt toward, say, skeptical-of-mainstream-medicine content can compound, across months, into something its host would not have predicted or chosen.
The Absent Check
What is structurally missing from every major recommender architecture is a moment of pause—a mechanism that asks whether your behavior reflects what you would have wanted without the system's intervention.
Technologists distinguish between autonomy-preserving and autonomy-undermining systems. An autonomy-preserving recommender would, at minimum, hold some representation of your stated goals that exists independently of your behavioral data, and periodically ask whether its recommendations are serving those goals or substituting for them. No major platform currently does this. Current architectures are not designed to ask what you want—they're designed to maximize what you'll do, which is a different question with a different answer.
This opens uncomfortable legal and ethical territory. If platforms are meaningfully shaping users' identities—and the research suggests they are—who bears responsibility for preference drift? Can a platform be liable for the person you became while using it? These are not yet answerable questions in any legal system, but the EU's Digital Services Act, which took full effect in 2024, represents the first serious regulatory attempt to address them, requiring large platforms to disclose how their recommender systems work and to offer users a version that isn't based on behavioral profiling.
What a Less Colonizing System Could Look Like
The design alternatives exist, at least in prototype. Researchers have proposed diversity-injecting algorithms that deliberately introduce content outside your established cluster to prevent spiral narrowing. Others have suggested serendipity scores—metrics that reward recommendations for being genuinely surprising while still relevant, rather than merely confirming existing patterns. Perhaps most promising are systems where users can input preference declarations that sit outside behavioral data—a statement of intent that the algorithm must weigh against what your clicks suggest.
Some platforms have experimented at the margins. Spotify's distinction between "discovery" modes and "mood" modes is a crude version of this: an acknowledgment that you might want different things from the system at different times. Several podcast apps have introduced genuine shuffle features, explicitly designed to break recommendation logic. These are early and imperfect, but they signal that the current design is a choice, not an inevitability.
The economic obstacle is blunt, however. Platforms monetize engagement, and engagement is maximized by the very feedback loop that constructs preferences. Reforming that loop without external pressure is structurally self-defeating—it would require platforms to voluntarily accept lower engagement metrics in the name of a user autonomy that doesn't show up on their dashboards. Regulatory pressure, not corporate goodwill, is the more realistic lever.
Learning to Read Your Own Appetites Skeptically
None of this is an argument to delete your apps. It is an argument to become a more skeptical reader of your own appetites—to treat the question "do I actually want this?" with some of the same rigor you'd apply to any other claim you couldn't fully verify.
A practical starting point: periodically seek out content through deliberate search rather than recommendation. The act of formulating a query is itself an exercise in knowing what you want, rather than discovering what you've been nudged toward. Introduce "off-model" inputs occasionally—something that genuinely surprises the system, chosen by you, from outside its predictions. This isn't about purity. It's about maintaining some territory in your attention that the algorithm didn't plant.
The deeper reflection is this: knowing yourself has always required effort, because the self is not a static thing waiting to be found. But in an age of algorithmic authorship, it requires an additional step—asking how much of what you experience as your taste, your curiosity, your identity, was quietly written for you, sentence by sentence, in the small hours of a thousand scrolling sessions.
You might have chosen the same story anyway. But you deserve to have been asked.
