The problem with collaborative filtering

Dear Amazon, I bought a toilet seat because I needed one. Necessity, not desire. I do not collect them. I am not a toilet seat addict. No matter how temptingly you email me, I’m not going to think, oh go on then, just one more toilet seat, I’ll treat myself. — Jac Rayner (@GirlFromBlupo)

This X user’s complaints form part of a broader trend of disgruntlement. Customers now have far greater expectations when it comes to the experience of buying a product: it must not only be what they want, but also what they don’t know they want, offered at the right time and in the right way.

One of the foundational components of recommenders like Amazon’s is collaborative filtering. Historical approaches included user-based collaborative filtering, which assumes that you’ll accept offers that ‘customers like you’ generally accept. This method takes features like demographics as the basis for recommendations, which naturally misses the nuance of an individual’s character, life experience and current context. Item-to-item collaborative filtering bases recommendations on similarities between products. For example, if you buy a kettle, you might be recommended a toaser as they are both forms of kitchenware. Amazon isn’t alone – companies like Netflix and Youtube have applied similar approaches. Collaborative filtering may have signified a step forward in sales strategies at the dawn of e-commerce, but the times have changed.

One prediction we have for 2026 is that personalization will move towards contextualization. Personalization learns about an individual. Contextualization learns how the individual changes from one context to the next.

ecosystem.Ai contextualizes every interaction by:

 

Aligning with behavior

Collaborative filtering misses the point of personalization. It makes the flawed assumption that desire follows similarity. In reality, a purchase is made through the combination of need, intrigue, and current context. Using behavior as the basis for recommendations through modules like Interaction Science and Intelligent Sales changes the game completely. These Modules allow you to learn from customer actions through the lens of human-centric AI and create a segment-of-one view of every customer at scale.

Continuous learning

Modules like Dynamic Experimentation allow your systems to learn from behavior in real-time, and effectively balance explore/exploit approaches with mutli-armed bandit algorithms.  your systems can continuously update prior assumptions to get a more holistic view of your customer. This is essential in an era where relevance is paramount and attention spans are transient. 

Recommenders built with dynamic models 

Collaborative filtering relies heavily on historical, static data – historical searches, purchases, and other activity. This can lead to the alienation of new or less frequent customers and leads to cases where models fixate on the sparse data. Recommenders built with dynamic models account for cases where data is sparse, taking real-time interactions into account while assuming that much is still unknown and available for exploration. 

Conclusion 

Effective recommendations in the current age need to move beyond static assumptions and match the complexity of human decision making. Recommendation systems that can consider context, act in real time and work from sparse data are essential for gaining relevance from early on and sustaining it. This requires a combination of behavioral algorithms and real-time intervention, with the technology that allows it to scale to every customer.