Do recommendation engines really work?

Not for me.

They must, after all, a whole bunch of really, really smart people at really successful companies are pouring millions of dollars into them. Many of the smartest data scientists out there are working on them. It’s inconceivable that the benefit could really be nonexistent, right?

I’m going to look at this as a user, not as a data scientist. I have only the vaguest idea of what is going on behind the scenes at Amazon (which I use frequently), Netflix (which I used to use occasionally but dumped), or Hulu, Spotify, Pandora and whatever other platforms are trying to recommend stuff to people, most of which I’ve never used and never will.

You might guess from this that I’m somewhat of an outlier, and that might be the core problem. Most of my buying on Amazon is books, and mostly for myself. I don’t do much if any gift shopping, and my “stuff” buying tends to be somewhat eclectic. Other than books, I’m a virtual non-consumer of media. So while Amazon has a fair amount of history about my reading (plus whatever else they are privy to based on my wanderings around the internet) they may be somewhat less able to peg me than they are other people. Also, my book buying tends to be rather varied. Part of it is anchored in my professional interests, to be sure, but otherwise I move around between various topics of non-fiction, a few different sci-fi genres and the occasional other types of fiction.

Still, it’s hard to ignore how bad their recommendations often seem to be. While I recognize that the plural of anecdote is not “data,” I’m going to engage in the anecdotal for this post, if for no other reason than in this “shopping season” I’m deluged with more than the normal amount of promotional material, much of it coming from very smart companies, and almost all of it useless or worse.

And sorry for picking on my Amazon friends. You’re just an easy target.


I suppose it’s possible that “deals in books” is a general category, not tailored to my interests. In which case, that’s a really, really dumb thing to put right at the top of the recommendations section. And if it is tailored to me, I have to wonder why kids’ books are in there at all. By now, Amazon should have figured out that I don’t have any and in fact don’t even have any in the family to shop for. We’re at that point in the generations. There’s just nobody in the right age group for Highlights at this point. (OK, technically there is, but they’re in a different country and they’re being raised in another language.)

Recommendation engine grade: Zero. Point. Zero.

New for me?

Amazon seems to be very narrowly-focused..

Well, here’s a “meh” groups if I’ve ever seen one. Another self-indulgent jerk talking about god. Yeah, I’ve read a few of those. Some years back I struggled somewhere between my Jewish heritage and my core belief that almost all this stuff is bullshit. I’ve long since stopped caring about discussing the existence of somebody else’s imaginary friend. My take on it is mostly that of James Lindsey, specifically that there are more important things to talk about. Haven’t ordered anything about politics in a while, so not sure where those came from. Taleb really needed to quit after The Black Swan. His musings as a philosopher whose primary idea seems to be that the wisdom of the ages is right and anybody questioning it is imprudent, are tedious and boring. Pinker, meh. Read him before.

This, I suspect is where recommendation engines fail me. The whole premise of most of them seems to be “you want more of exactly what you’ve had before.” More complex patterns such as “you want things that are perhaps related, but by different authors, or expressing different points of view” are beyond what they are capable of doing, or at least beyond what they are currently designed to do. Whether it’s books or movies or anything else “something related but different” tends to be my primary pattern. I tire of endless serials by the same authors spinning new and more tired episodes with the same characters and settings.

Maybe this is where I fail the algorithm. A quick examination of sci-fi bestsellers suggests that “22nd episode of the same characters” is exactly what many people want. Occasionally serials work for me. Kim Stanley Robinson’s Mars trilogy, for example. But usually that’s only when the author is trying to tell a story that extends beyond the reasonable length of a single book. And to be honest, it’ll be a long time before I’m likely to read Robinson again. I’d rather try somebody new. It’s odd, given how sophisticated algos can be, that nobody has come up with one that can recognize this kind of “preference for variety.”

Recommendaton engine grade: B-. Reasonable in an amateurish way, but I’d have to be really out of ideas to buy any of these.

More to consider?

Seriously, I bought one book and now all you have to recommend are copycats?

This one’s just plain dumb. I recently got a small telescope and in order to make best use of it ordered a single book (Turn Left at Orion) recommended on a variety of online astronomy forums. I also got star and moon maps. Now they think I want five more intro to astronomy books? It might be one thing if I had shown interest but not bought anything, but I did buy something. I bought exactly what I looked for. Why would any sane engine believe that I would want more than one “intro” book to anything? I suspect what’s going on here is recency effect. I hadn’t shopped for anything in a while, then a handful of astronomy things. Stuff I ordered a few weeks or months ago was de-prioritized in the algorithm so that all they had left was intro-level stargazing and telescope use.

That’s a fault of whoever came up with the algorithm. Many of us shop in “spurts” of related items. The algorithm seems to get fooled by that.

Recommendation engine grade: C-. Even more amateurish and there’s nothing here I would buy because I’ve already bought a similar title and you know I bought a similar title and how many of the same thing do you really think I want???

So in conclusion

I have a tough time coming up with a good idea when yet another possible client asks for opinions on yet another recommendation engine. I don’t want to question the demands too much. After all, Amazon and Netflix are full of really smart people so there must be something to these things, but I have a tough time seeing how results like the ones shown to me are a good investment of company funds. Or maybe I just have a tough time relating to people who fail to value trying something different and are happy to be sold more of the same.

In any case, I’d love to see a recommendation engine that can recognize “preference for variety” as a pattern, in addition to “we’ll show you more of the stuff you bought last week.” You might actually get me to spend more. Or maybe they just realize that people like me will never spend enough to be worth catering to? I suppose that’s possible too.

If there’s something I’m missing, let me know. Don’t bother with “you don’t get marketing,” because I take that as a given.

Leave a Reply

Your email address will not be published. Required fields are marked *