Computers are stupid. It may seem like a strange thing to say about a machine that can do calculus, translate between a hundred languages, and teleport information to the other side of the planet in the blink of an eye, but fundamentally, computers are very, very stupid, for one key reason: they only ever do exactly what they are told. They are electronic Amelia Bedelias, and when a computer is told to find and display the most popular Facebook posts a user has made, that is exactly what they will do. However much those posts might hurt to see.
For most of us, Facebook’s “Year In Review” app has been at best a fun way to occupy a few minutes, and at worst, a minor annoyance to have to scroll past as we read through our News Feeds. For Eric Meyer, though, who lost his six-year-old daughter Rebecca in 2014, seeing his child’s smiling face with the caption, “Eric, see what your year looked like!” was an unnecessary reminder of what he had gone through in the past year.
Posts like the above one Meyer posted on Twitter appear in a user’s newsfeed without any prompting or interest in year-reviewing on that user’s part: Facebook just selects a picture from one of the user’s posts that got a lot of comments or likes and posts it with the “Here’s what your year looked like!” caption to encourage you to create and share your own version of the slideshow. Even if, for example, your most-commented on post was your daughter’s obituary.
As BoingBoing reports, Meyer took to his blog to describe the jolt he felt upon seeing his little girl’s face suddenly pop up in his Facebook feed. Meyer, who is a web designer himself, was careful not to blame the team who created the Year in Review app, instead blaming what he calls “inadvertent algorithmic cruelty” – the rote routine of computers to do what they’re told. He even took to his blog again after receiving a personal apology from Jonathan Gheller, the product manager on the app team, to explain that he wasn’t personally angry with the design team, and that this event wasn’t anything that could be blamed on ‘kids these days’ because of the average Facebook employee’s youth, but rather to a more pervasive cause:
First off, by what right do we assume that young programmers have never known hurt, fear, or pain? How many of them grew up abused, at home or school or church or all three? How many of them suffered through death, divorce, heartbreak, betrayal? Do you know what they’ve been through? No, you do not. So maybe dial back your condescension toward their lived experiences.
Second, failure to consider worst-case scenarios is not a special disease of young, inexperienced programmers. It is everywhere.