What Are the 2 Types of Recommendation Engines?

Almost 75% “of online consumers get frustrated with websites when content appears that has nothing to do with their interests” (Janrain). Content might be king, but personalized content is the trusted court vizier (and is next in line to the throne). It’s no secret: People want to feel known and understood by companies. They want to feel like human beings—not like numbers on an Excel sheet. The best way to achieve this warm and fuzzy feeling (which people are right to expect) is to give them personalized content that speaks to their interests and values.

That’s where recommendation engines come in. With roots in machine learning over 20 years old, these systems accept input, use one of two internal methodologies (which we’ll cover below) to search its database for similar items, and automatically yield an output that makes it fast and easy for people to explore more relevant content. Recommendation engines have shaped the modern digital landscape. Blue-blood brands like Netflix, Amazon, and Google are all but equated with their famous, sometimes uncanny ability to match people with different ideas, products, and media that never stray too far from their personal taste.

But how does it work? How does an automated system ‘know’ what new, supplementary things it should offer subscribers and shoppers? Two methodologies make recommendation engines possible: content-based filtering and collaborative filtering (CF). Each method has pros and cons, and neither method is perfect—which is why many powerhouse brands use a ‘hybrid’ engine that combines the two. But each method is good for reaching a certain end; it just depends on the goals, needs, and capabilities of the company using the engine.

Content-Based Filtering: Keywords in Edgewise

Content-based filtering is the simpler of the two methods. It’s based on the principle that related things share keywords, which the recommendation engine can search for in the database en route to finding other content with the same keywords. This stands in sharp contrast to collaborative filtering (explored in the next section), in which things in the database become connected or seen as ‘similar’ by the engine based on the past behavior of other users.

Powered not by community but by keywords, content-based filtering avoids the “cold start problem” endemic to CF engines. A content-based engine doesn’t need to gather data to start recommending; all it needs to know is how the company has pre-classified each item. Moreover, if keywords are well researched and creative, the engine can reliably produce relevant content that customers will appreciate. There is little risk the engine will recommend something illogical or embarrassing due to changes in incoming data or changes in customer behavior (which can be caused by external market forces or cultural trends). Content-based filters are static and steady, and recommendations won’t change unless keywords are changed.

This lack of dynamism, however, is also the downside of this approach. A recommendation engine that can’t be influenced by the actions of the people who use it is can be more easily managed. However, by default, it produces less timely, accurate, and vibrant recommendations. With this method, everything comes back to keywords. A company can research and whiteboard its keywords from here to Kalamazoo, trying to account for every possible link one item could have to others; but it is always more likely that active users will create new and more interesting connections with their actual behavior.

Let’s imagine, for example, there is a radio app used by a subscriber named David, who indicates he likes the band Joy Division. If the app’s recommendation engine uses content-based filtering, it might tell him to also listen to The Cure, since the bands share keywords on being a) English b) active in the eighties, and c) classifiable as ‘post-punk.’ Since Joy Division and The Cure do have a lot in common, David would likely be fine with the recommendation, viewing it as compatible with his musical taste. However, because the bands are so similar, it is also likely that David, as a Joy Division fan, was already familiar with The Cure, a highly influential post-punk band. So, while David may deem the recommended content relevant, he probably won’t think it terribly novel or exciting. And in a saturated market, David’s case multiplied by a couple hundred or thousand users might mean curtains for the app: unimpressive content leads to fewer customers, fewer high ratings, and fewer advocates.

Collaborative Filtering: Finding Hidden Gems

Of the two methods that can fuel a recommendation engine, collaborative filtering is the more complex. While content-based filters use prescribed keywords to find relevance between things, collaborative filters let the community at large determine item-to-item relevance. It does this by tracking the user base’s past behavior to find patterns. In other words, a CF-powered engine doesn’t start off ‘knowing’ which products, services, or media in the database are similar to which others. Instead it begins with a blank slate, and that slate becomes filled with x’s, o’s, and arrows of connection as more people use the engine, their searches and purchases ‘teaching’ it how content can tie together.

This process points to CF’s two main assets: predictive capability and digital foot printing. Users, not marketers or BDRs or programmers, affect what content is recommended to other users after the original query. And unlike content-based filtering, which gives static and unchanging recommendations (unless the keywords are changed), CF uses a dynamic forecasting model that draws on constantly updating data. It uses this data to, in essence, guess customers’ goals, preferences, and overall psychology as accurately as possible. As more users interact with more content, more connections form automatically between more things. And while bigger data needs stronger algorithms to control it, it also means sharper targeting—which is why CF is the method more likely to reveal hidden content gems.

Let’s continue with our above example and music lover David. When David used a radio app powered by content-based filtering, as a Joy Division fan he was recommended to test-drive The Cure—which, while a technically relevant group, was an obvious, high-level suggestion. But what if David were to download an app that used collaborative filtering? Then, he could receive content no less targeted but much more able to surprise and delight him. He could have a bespoke digital journey with content new enough to interest him yet similar enough to feel comfortable. The engine might recommend he try bands like, say, The Raincoats, This Heat, or Swell Maps—all of which resemble Joy Division (English, ‘80s, post-punk) but none of which achieved the fame of The Cure. If enough app users acted on content around both Joy Division and those bands, the engine would recommend those bands to David, too—who now gets to experience something fresh and different.

This is not to say CF is all pros and no cons. As mentioned, it can inflict the cold start problem on recommendation engines: Since it needs data on past user behavior to find relevant content, it can make bad recommendations if not enough people have searched, purchased, evaluated, and interacted. (And poorly targeted recommendations are worse than none at all.) But with sufficient data, CF is a technique to be reckoned with. Diverse recommendations that teach people more about the interests they already have are the best kind of content, because customers remember and appreciate being exposed to unique and esoteric ideas. This builds trust between customers and companies, and encourages people to convert, re-up later, and evangelize. Even if David ends up disliking all three weirdo subculture band suggestions (I say this with love; post-punk forever!), horizon-broadening ‘misses’ can do more good for the B2C relationship than boring, self-evident ‘hits.’

Which Method Is Better?

Recommendation engines are a necessary arrow in the quiver of modern marketing and sales. What is the best approach to building one? Content-based and collaborative filtering both have pros and cons; but the tradeoff from former to latter is straightforward.

An engine powered by content-based filtering runs on predefined keywords, which enables companies to start automating right from the get-go, without waiting on a community to create data patterns. With good keywords, this level of personalization can be effective. However, it does prevent unique and unexpected connections from forming freely between items and ideas.

Meanwhile, a CF-powered engine needs more time to actualize; its predictive analysis is based on user behavior that must first take place. It also calls for closer monitoring, with new data always streaming in. But more data means sharper, more engaging, and more diverse recommendations—which is the key to converting savvy customers who expect deep personalization.