If Netflix uses machine learning why are the recommendations so bad

A fine post by Tren Griffin this morning about Netflix’ Reed Hastings. A very important quote:

“We are investing heavily in [machine learning] because we want it to be: you turn on Netflix and there’s a row, there’s like four choices, and you just want to watch them all. To get to that consistent view is where we are targeting.”

If this is true, and I believe that Netflix (and Amazon) are using machine learning, why are the recommendations always terrible?

I’ve seen a stat that shows people spend more than 20 minutes daily looking for things to watch. Anecdotally no one has ever told me they liked the Netflix (or Amazon) searching or browsing ability. And of course if you add in a mix of people e.g. Mom, Dad, Kid 1, Kid 2, then the matrix of options and bad choices appears infinite.

A few thoughts on why these brilliant minds and powerful machines are failing:

  1. The tools being used seem to believe that because you liked or bought one thing, you probably want another. How often do you buy something from Amazon, e.g. a broom, only for the rest of the Internet to scream “you probably need a second, maybe third, maybe fourth broom!!!” And you know what? I never do. If I bought something, it was fairly unique. And I won’t need it again for another many years. The same goes for movies – if I watched a RomCom, odds are good I will watch something different next time.
  2. Sample size is small, cost of making a choice is high. I might watch one or two shows a week on Netflix. I simply don’t have 4+ hours available. So each movie I select is a big deal – it can’t suck. Contrast this with YouTube which has a ton of content and the “buy in” is very small. If I select a bad 8-minute clip it won’t matter much. And I can select 10 of these things (and thereby provide 10 signals to Google) vs. 1 to Netflix.
  3. For some strange reason Netflix doesn’t seem to take abandonment into account. “Hey, you watched Vikings, I bet you want some more Norse-y stuff”. Well no, I started ‘The Vikings’ and found it unwatchable.

I’m sure Netflix and Amazon are both working hard on this – I wonder sometimes if they ever leave the office and just watch how people really interact with content. Book recommendations are even worse if that’s possible so this isn’t a unique issue.

Remember When Netflix Wanted To Rent DVDs On A Different Website? Yeah, That Was A Fun Week | TechCrunch

Remember When Netflix Wanted To Rent DVDs On A Different Website? Yeah, That Was A Fun Week | TechCrunch

Fwd: An Explanation and Some Reflections

Bold move by Netflix. I like bold moves, kudos for that.

But it’s still a disaster. What Mr. Hastings simply isn’t allowing for is
that streaming just isn’t there yet. The choices are too few. Sure, the
website is awful to use, the mobile players are incomprehensible, but the
real flaw is the selection.

But I could overlook that because I too, like Hastings believe this is the
future. But sometimes the future is awkward and needs a little kick. And
having that old-school DVD around was a safety blanket. And it worked, I
basically paid $3/month to have the same DVD sitting around my house in case
I ever wanted it. Netflix has now asked me to examine my safety blanket and
I’ve found it wanting.

Begin forwarded message:

From: “Reed Hastings, Co-Founder and CEO of Netflix” info@netflix.com
Date: September 18, 2011 23:33:41 PDT
To: paul@bricin.net
Subject: An Explanation and Some Reflections

   Dear Paul,

I messed up. I owe you an explanation.

It is clear from the feedback over the past two months that many members
felt we lacked respect and humility in the way we announced the separation
of DVD and streaming and the price changes. That was certainly not our
intent, and I offer my sincere apology. Let me explain what we are doing.

For the past five years, my greatest fear at Netflix has been that we
wouldn’t make the leap from success in DVDs to success in streaming. Most
companies that are great at something – like AOL dialup or Borders
bookstores – do not become great at new things people want (streaming for
us). So we moved quickly into streaming, but I should have personally given
you a full explanation of why we are splitting the services and thereby
increasing prices. It wouldn’t have changed the price increase, but it would
have been the right thing to do.

So here is what we are doing and why.

Many members love our DVD service, as I do, because nearly every movie ever
made is published on DVD. DVD is a great option for those who want the huge
and comprehensive selection of movies.

I also love our streaming service because it is integrated into my TV, and I
can watch anytime I want. The benefits of our streaming service are really
quite different from the benefits of DVD by mail. We need to focus on rapid
improvement as streaming technology and the market evolves, without
maintaining compatibility with our DVD by mail service.

So we realized that streaming and DVD by mail are really becoming two
different businesses, with very different cost structures, that need to be
marketed differently, and we need to let each grow and operate

It’s hard to write this after over 10 years of mailing DVDs with pride, but
we think it is necessary: In a few weeks, we will rename our DVD by mail
service to “Qwikster”. We chose the name Qwikster because it refers to quick
delivery. We will keep the name “Netflix” for streaming.

Qwikster will be the same website and DVD service that everyone is used to.
It is just a new name, and DVD members will go to qwikster.com to access
their DVD queues and choose movies. One improvement we will make at launch
is to add a video games upgrade option, similar to our upgrade option for
Blu-ray, for those who want to rent Wii, PS3 and Xbox 360 games. Members
have been asking for video games for many years, but now that DVD by mail
has its own team, we are finally getting it done. Other improvements will
follow. A negative of the renaming and separation is that the Qwikster.comand
Netflix.com websites will not be integrated.

There are no pricing changes (we’re done with that!). If you subscribe to
both services you will have two entries on your credit card statement, one
for Qwikster and one for Netflix. The total will be the same as your current
charges. We will let you know in a few weeks when the Qwikster.com website
is up and ready.

For me the Netflix red envelope has always been a source of joy. The new
envelope is still that lovely red, but now it will have a Qwikster logo. I
know that logo will grow on me over time, but still, it is hard. I imagine
it will be similar for many of you.

I want to acknowledge and thank you for sticking with us, and to apologize
again to those members, both current and former, who felt we treated them

Both the Qwikster and Netflix teams will work hard to regain your trust. We
know it will not be overnight. Actions speak louder than words. But words
help people to understand actions.

Respectfully yours,

-Reed Hastings, Co-Founder and CEO, Netflix

p.s. I have a slightly longer explanation along with a video posted on our
where you can also post comments.

This message was mailed to [paul@bricin.net] by Netflix.
SRC: 1578.0.US.en-US
Use of the Netflix service and website constitutes acceptance of our Terms
of Usehttp://www.netflix.com/TermsOfUse?lnktrk=EMP&g=E8F6754583318814919E8A6DFEF1A8C70AC0315E&lkid=terms_footerand
© 2011 Netflix, Inc. 100 Winchester Circle, Los Gatos, CA 95032, U.S.A.

Down on Netflix

For the first time in months the family decided to watch a movie. We have a Netflix subscription and while I am peripherally aware of the recent changes Netflix made with regard to streaming and DVDs I hadn’t thought about it much.

Last night though we decided to watch a movie and ran through their new releases. And it was a wasteland of dreck. Nothing much new. So we went to the cupboard and checked out some old favorite the kids haven’t seen like “Say Anything” or “Ferris Bueller’s Day Off”. Nope.

So we checked out iTunes and yep… All there. So we paid Apple another $4.99 and this morning I am working out the math and thinking “I ditched cable TV and don’t miss it, why not ditch Netflix too?” I can hit RedBox on the way home from work if we want a movie. And since we rarely watch movies that would be a savings. And even if it wasn’t a financial savings it would be one less recurring charge that I just don’t need to deal with or think about.