Don't corrupt your objective function / by Mike Hudack

I recently switched to Apple Music from Spotify.

The nominal reason for the switch was 4:44. Jay-Z's newest album wasn't available on Spotify. And it wasn't just 4:44 that wasn't available. He pulled his entire catalog. Jay-Z engaged in a classic windowing strategy in which the album was first made available to Sprint and Tidal customers, and then on three streaming platforms that aren't Spotify: Apple Music, Google Music and Amazon Music. 

It's meaningful that Spotify wasn't in on the second window. I think it's because they're not willing to pay. 

Spotify's unwillingness to pay goes beyond losing out on content like 4:44. I fear that their pursuit of margin in advance of a forced IPO is hurting customers through algorithmic distortion.

A while ago Spotify created a series of mostly excellent data-driven products like Discover Weekly that promised to revolutionize the way we discover music. They worked. By using what must be an incredibly rich library of music listening data Spotify was able to create incredibly powerful discovery products that "just worked." I'm sure I wasn't the only one who would regularly just play my Discover Weekly instead of doing work to find albums and playlists to listen to. 

Over time Spotify's algorithmic offerings expanded. I started listening to Daily Mixes. Spotify advertises Daily Mixes as "[T]he music you love, minus the effort."

They continue:

Daily Mix is the perfect line-up of tracks ready to play at the touch of a button. Based on the different styles of music you regularly listen to, each mix is loaded with artists you love, plus a sprinkling of new discoveries that fit the vibe too. What’s more, it grows with you. As your music taste evolves, so does your Daily Mix.

Daily Mix is a great product. It has just one problem: it doesn't include all the music you love. 

Every song Spotify streams costs them money. Some songs cost more money than others. The most expensive songs come from the likes of Jay-Z. The cheapest songs come from fake artists created by Spotify (Spotify denies they create fake artists, but they do). 

If you're Spotify and you want to become profitable and you have to IPO because you raised a billion dollars under punitive terms that punish you for every day you don't IPO and your margins are terrible because everyone just wants to listen to Jay-Z what do you do? 

Well, to start with, you don't carry 4:44. But then you take the fabulous data products that you created -- Discover Weekly and Daily Mixes -- and you distort them. You change the objective function from being one that purely optimizes for customer value to one that attempts to balance customer value and profit. 

I can imagine the conversation, and it probably all seems reasonable and good. People love these playlists. They also love generic piano music. What if we played a little more generic piano music and a little less Arthur Rubinstein? No one would really notice

Then the test gets designed and run. It's a 28 day test in which a control group continues to receive the traditional playlists with the traditional customer-optimized objective function while an exposed group receives a playlist which includes "royalty cost" as a feature in the algorithm.

I imagine that the test shows no or very little difference in engagement between test and control and no measurable effect on retention, free or paid. The conclusion that's reached is that it's okay to include Spotify economics as a feature in the algorithm. 

Months pass. The impending doom of a looming IPO combined with crushing unit economics continue to weigh heavily on the shoulders of Spotify executives. The objective function gets tuned to place ever more value on Spotify's economics. The knob keeps turning. The tests keep showing no significant impact on retention. 

But then 4:44 comes out. A few people hop over to Apple Music or Google Play or Amazon to listen to it, because they like Jay-Z and heard 4:44 is good. Then, if you're me, something funny happens when you jump over to listen to 4:44. You start listening to expensive music again. 

Apple Music doesn't care if I listen to Arthur Rubinstein or Relaxing Piano Music. So I listen to Arthur Rubinstein. His Chopin Nocturnes are better. And DAMN. is good. I didn't realize it, but it'd been a really long time since I listened to The White Stripes. And there you are, Pusha T. 

Apple Music has their own version of Discover Weekly now. It's called "My New Music Mix." Mine has a really nice mix of up-and-coming acts and brands you know. Their "My Favorites Mix" has Tom Waits and the Ramones and Bob Dylan and CCR and Radiohead and Kendrick and Jay-Z. 

There is no fear of major labels at Apple Music. And this means that the experience is better. I'm listening to better music now. Indie music is getting surfaced because the algorithms that power Apple Music believe I'll like it, not because it's cheaper for Apple to stream. 

I'm not going back to Spotify. When I pick up an Android phone again I'll do it with Google Play Music. Spotify has treated me like a lobster in a pot of water. The heat kept getting turned up, slowly, and I barely noticed. But now that I've tasted an alternative streaming service I realize how mistreated I've been. 

This should be a lesson to anyone building algorithms of this sort. Don't optimize for yourself. Optimize for your customer. Figure out your own economics and your business separately. The moment that you cross that line your algorithm becomes corrupt.

You'll try to measure the degree to which you're hurting your customers, but your measurement will be wrong. When people do bad things for customers they never run the retention tests long enough to see the true impact of their changes. They say it's because they want to move fast, but it's really because they don't want to know the truth. If they wanted to know the truth they'd have a long-term holdout. They almost never do. 

Also: you should consider exclusive content. It's a smart marketing tool. Exclusive content causes customers to keep sampling your product, and allows you to iterate towards something good while getting multiple cracks at customers. Apple Music wasn't that great when it launched, but its content strategy has kept me coming back periodically for treats like 4:44. I kept doing that until I stuck around. 

But there's a broader point here. You should never let your interests diverge from the interests of your customers in the first place. Your focus in building software should be in improving customer experience, not increasing profit. Profit is a lagging indicator of customer happiness, not the other way around.