I’m done buying physical books – except when I go to The Strand in the East Village, where I seem to never leave without buying something.
Vogue: The Editor’s Eye is my latest purchase. I have enough photo books but I liked that this one is purportedly about the influence of the fashion editors on the magazine’s photography. As a photographer, I’ve come to realize that editing is as important as the actual on-camera skills. Maybe even moreso in this day of near infinite film rolls.
For every hour I spend taking pictures, I probably spend five to ten hours picking out photos, organizing, and then publishing them. Even on my Flickr account, where I dump photos for archival purposes, I probably upload fewer than 1 out of 50 photos that I actually take.
Anyway, The Editor’s Eye is still worth it for the photography alone. The book covers eight fashion editors and devotes a few pages of text to their biographies. But the process of editing – it being more a confluence of personality, experience, and temperment as much as it is skill – is hard to capture in words and the book doesn’t shed any particular light on the work. It’s not a surprise, then, that fashion editors were uncredited until Anna Wintour took over in 1988.
The above 1950 photo by Clifford Coffin of Ernest Hemingway and Jean Patchett is one of my favorites in the book. The fashion editor then, Babs Simpson, said she was sent to Havana to “catch” Hemingway, without even a single phone number to reach him at. She went to the Ambos Mundos, one of his favorite drinking spots, and got his phone number from the manager.
Simpson described the circumstances of the difficult photo session to writer Dodie Kazanjian:
We wanted to start early in the morning before it got hot. When we got there at eight o’clock, as agreed, nobody was in sight. Two champagne glasses were bubbling away, though, and finally Hemingway appeared, with a very young and good looking Basque priest.
Hemingway took a great shine to Jean, of course, and I got on very well with the Basque priest…but it got worse and worse as the day went on, because Hemingway and the priest got drunker and drunker…Isn’t [Hemingway] revolting? They wanted us to go and see the pelota or something or other with them afterward. They wanted to spend their lives with us. So we got the first plane we could out of there.
I’m currently reading “Heads in Beds” by Jacob Tomsky, a purported insider’s account of the hotel business. Besides being pretty entertaining, it’s full of interesting facts (it costs $30 to $40 to turn over the average hotel room), helpful advice (speak out the employee’s name when making a request you don’t want said employee to forget), and unpleasant anecdotes, such as what happened to a pro athlete’s bottle of cologne when the athlete stiffed his bellman.
The worth of a bellman is a recurring topic in the book. Even in the age of wheeled suitcases, a good bellhop can still make a living. While the wages are low – the median salary is $20,880, according to current Labor Department statistics – a Manhattan bellhop who’s a “real hustling bullshit artist” might make “well over a hundred thousand dollars” annually, Tomsky claims, from an endless stream of ones and fives. And a guest who is too cheap to spare a couple bucks? Tomsky writes: “He shouldn’t use his toothbrush that night (or ever again, really).”
So when Tomsky, then a recent college graduate and college loan debtor, is offered a promotion to be a bellman or a manager, it’s not a straightforward decision. The bellman position pays much better for fewer hours. But his general manager frames the tradeoff in a memorable way:
(emphasis mine)
“I trust you, Tommy. I’m going to offer you a choice. You’re done with the front desk. I heard you’ve started to loosen up down there, started in with the jokes.â€
“Oh, well, I hope I haven’t—â€
“Not to worry. It’s natural. You’ve outgrown the position. So I’d like to offer you two opportunities. Whichever one you want is yours. As you are aware, there is a bellman position recently available. Extremely recently. It’s yours if you want it. You are fantastic with the guests. Or.â€
“Or?â€
“Housekeeping manager. Management, Tommy. Take over the evening position down there. You’d be in charge of turndown, scheduling, purchasing, and a thousand other things. A staff of 150.
…
“Let’s talk money. Housekeeping means ten-hour shifts or more, on salary. When you break it down hourly, you will make less than you are making now. You’ll have to purchase your own suits. The work is physically demanding, the staff is large and can be difficult. It’s a very challenging position. Bellman? You’ll double your money immediately and keep the eight-hour shifts. Zero responsibility.â€
“You think I should take the bellman position?â€
“Do that, and you’ll never be anything else in your life. Hate to say it, but it’s true. I’ve seen it my whole career: Show me a twenty-year-old kid getting his first job as a bellman, and I’ll show you a seventy-year-old bellman who started fifty years ago. You grow accustomed to that pay grade, and taking a step forward will always mean cutting your money in half. No one takes that step.â€
“Housekeeping,†I said.
Not a bad lesson, for the hotel business or any business.
One of my least fulfilling reporting experiences involved a meeting between Ukrainian community leaders and local school district officials. There had been a brawl at a middle school and the Ukrainians, being the minority in that area, suspected their kids were racially targeted.
Anyway, the difficulty was that they didn’t speak English very well and many others at the meeting – the district officials and me, included – didn’t speak Ukrainian. So we had translators. The meeting had taken some time to plan and parents were nervous about their children’s safety, so everyone hoped for some kind of resolution. But after the meeting kicked off with long speeches by both sides – made doubly long by having to listen to the sterile translation that came afterwards, it seemed obvious that even a meaningful, impassioned debate was futile, nevermind reaching a satisfying resolution. The district officials told me afterwards, of course, that this was a good use of time, while the Ukrainian community members didn’t think anything came out of it.
When you think about it in theory, having a translator seems like a “good enough” solution. But when you’re sitting there, even though you and your other side share the same thought processes and general human understanding (independent of spoken language), a translator cannot:
Connect statements to the body language and emotion nuance of the original speaker
Mediate the kind of free-flowing give-and-take that is necessary for fulfilling debates
Make up for the time delay between the original statement and spoken translation
And of course, there’s the problem of knowing if the translation is even correct, as we know from Suntory Time:
The fact is, the whole world should not learn to code anymore than all of us should learn to be a space shuttle engine designer or a lawyer. While I understand the need for more people to get interested in computer science and to fill our ranks with people who can meet the skills of the 21st century, going out there and telling everyone that coding is as easy as putting a bit of syntax down into an IDE and hitting compile is not the way. We need passionate people who are creative and want to learn to DESIGN software in addition to coding.
There are a couple problems with this argument. The first is making programming sound as time-intensive and domain-specific as space shuttle engineering or law. The second is making programming sound like an all-for-nothing proposition: don’t get into programming unless you can do it really well, or else your program will kill people.
(Not that that hasn’t happened…but the more relevant statement would be: don’t code people-killing programs unless you know what you’re doing.)
The reason to learn how code works is to have a direct involvement in the technology and data we rely on. Just as we learn to read and write at a basic level so that our time isn’t spent waiting for someone – who may be much better and specialized at it – to read the news or road signs or write love letters for us.
In the newspaper world, it’s assumed that every reporter knows how to write. That’s not even remotely close to reality. In the golden, fat days, a reporter could expect three to four layers of editors to go through his/her work before it got printed (today, readers just have to deal with a lot more typos and errors). But even then, you couldn’t get by as being a fantastic reporter and storyteller who was also illiterate. Think about it: you don’t need to know how to read or write to go around asking questions or dictating what you remember hearing of a blockbuster scoop. But imagine the editors who get to spend their day transcribing and retelling you what they’re actually putting down on paper.
More importantly, think about what you, the illiterate reporter loses out on. You don’t get hands-on involvement on how the facts and quotes are arranged, because you have to wait for it to be done (and then read back to you) before you have a say. And there’s a lot that you just don’t know that you don’t know. Like how a headline, by its size, color, and length, can change everything about how a reader perceives your story. Or how a story that’s in print has a much different, detached effect than what you’ve experienced with the spoken word (similarly, writing copy for broadcast is its own separate art compared to newspaper writing).
But you don’t know that because you know that there’s a story, of some sort, is on the paper and people seem to be reading it so what’s the big deal?
This is the gap of understanding for anyone who isn’t digitally literate. It’s not about knowing how to build a website or assemble your own computer. It’s not even that you need to know a lot, just as you don’t have to be able to recite the entire works of Shakespeare to justify traditional literacy. With digital literacy, it’s about knowing enough to know what’s possible with computers, data, and technology, to control more of how it affects you. Or, as Donald Rumsfeld puts it, having fewer unknown unknowns.
How much is “knowing enough”? There’s a lot that’s been written about it in better places, Code.org probably isn’t a bad place to start. Besides learning to program, which is not the only thing to do or even the ultimate end goal, the two practical steps I can suggest are:
Learn how to use a spreadsheet – not just for numbers, but for taking notes
Also this week, Jeremy Ashkenas announced the first release of Literate CoffeeScript, a format that combines CoffeeScript with the easy-to-read-and-turn-to-HTML Markdown syntax. If you’re an experienced coder, this seems like nothing more than block-extended comments that are found in any language.
However, in practice, the power is in the details. I know that I avoid documenting code – even though we spend far more time re-reading code than actually writing it – because I just have never committed to some kind of system of making code comments easy to read (should I use asterisks to emphasize something? Or all caps? Or double space things?). With Markdown, you have the power of HTML to document your code. And more importantly, it’s intuitive and easy, which makes you more likely to write comments in the first place.
On a related note: if you’re a programmer who doesn’t understand why non-programmers need to be digitally literate: imagine having to write code in another language. Not programming language, but human language, such as in Chinese characters, without actually knowing that language. Sure, you could conceive an amazing program. But writing it yourself, with the help of an interpreter who types out what you say? Sounds even more laborious and mind-numbing than the worst programming project imaginable.
My former colleague, Charlie Ornstein over at ProPublica, wrote a thought-provoking, emotional piece on the costs of end-of-life care. As a health care reporter (one of the best in the business; he was a Pulitzer Prize recipient at the LA Times), he has written a lot about how end-of-life care is often prolonged beyond reason – account for as much as 25% of Medicare payments in the last year of a patient’s life. But when his mother was dying, he writes, “none of my years of reporting had prepared me for this moment, this decision.”
My father, sister and I sat in the near-empty Chinese restaurant, picking at our plates, unable to avoid the question that we’d gathered to discuss: When was it time to let Mom die?
It had been a grueling day at the hospital, watching — praying — for any sign that my mother would emerge from her coma. Three days earlier she’d been admitted for nausea; she had a nasty cough and was having trouble keeping food down. But while a nurse tried to insert a nasogastric tube, her heart stopped. She required CPR for nine minutes. Even before I flew into town, a ventilator was breathing for her, and intravenous medication was keeping her blood pressure steady. Hour after hour, my father, my sister and I tried talking to her, playing her favorite songs, encouraging her to squeeze our hands or open her eyes.
It’s an eloquent description of the thankless job of journalism under a corrupt state and the motivations for those who still pursue it despite the danger and hardships. It is also personal, addressing the man whom Wickramatunga considers a long-time friend, and who Wickramatunga believes will be ultimately responsible for his death: President Mahinda Rajapaksa.
No other profession calls on its practitioners to lay down their lives for their art save the armed forces and, in Sri Lanka, journalism. In the course of the past few years, the independent media have increasingly come under attack. Electronic and print-media institutions have been burnt, bombed, sealed and coerced. Countless journalists have been harassed, threatened and killed. It has been my honor to belong to all those categories and now especially the last.
I have been in the business of journalism a good long time. Indeed, 2009 will be The Sunday Leader’s 15th year. Many things have changed in Sri Lanka during that time, and it does not need me to tell you that the greater part of that change has been for the worse. We find ourselves in the midst of a civil war ruthlessly prosecuted by protagonists whose bloodlust knows no bounds. Terror, whether perpetrated by terrorists or the state, has become the order of the day. Indeed, murder has become the primary tool whereby the state seeks to control the organs of liberty. Today it is the journalists, tomorrow it will be the judges. For neither group have the risks ever been higher or the stakes lower.
Why then do we do it? I often wonder that. After all, I too am a husband, and the father of three wonderful children. I too have responsibilities and obligations that transcend my profession, be it the law or journalism. Is it worth the risk? Many people tell me it is not. Friends tell me to revert to the bar, and goodness knows it offers a better and safer livelihood. Others, including political leaders on both sides, have at various times sought to induce me to take to politics, going so far as to offer me ministries of my choice. Diplomats, recognizing the risk journalists face in Sri Lanka, have offered me safe passage and the right of residence in their countries. Whatever else I may have been stuck for, I have not been stuck for choice.
But there is a calling that is yet above high office, fame, lucre and security. It is the call of conscience.
The Sunday Leader has been a controversial newspaper because we say it like we see it: whether it be a spade, a thief or a murderer, we call it by that name. We do not hide behind euphemism. The investigative articles we print are supported by documentary evidence thanks to the public-spiritedness of citizens who at great risk to themselves pass on this material to us. We have exposed scandal after scandal, and never once in these 15 years has anyone proved us wrong or successfully prosecuted us.
…
Every newspaper has its angle, and we do not hide the fact that we have ours. Our commitment is to see Sri Lanka as a transparent, secular, liberal democracy. Think about those words, for they each has profound meaning.
Transparent because government must be openly accountable to the people and never abuse their trust. Secular because in a multi-ethnic and multi-cultural society such as ours, secularism offers the only common ground by which we might all be united. Liberal because we recognize that all human beings are created different, and we need to accept others for what they are and not what we would like them to be. And democratic… well, if you need me to explain why that is important, you’d best stop buying this paper.
…
Many people suspect that The Sunday Leader has a political agenda: it does not. If we appear more critical of the government than of the opposition it is only because we believe that – pray excuse cricketing argot – there is no point in bowling to the fielding side. Remember that for the few years of our existence in which the UNP was in office, we proved to be the biggest thorn in its flesh, exposing excess and corruption wherever it occurred. Indeed, the steady stream of embarrassing exposes we published may well have served to precipitate the downfall of that government.
…
It is well known that I was on two occasions brutally assaulted, while on another my house was sprayed with machine-gun fire. Despite the government’s sanctimonious assurances, there was never a serious police inquiry into the perpetrators of these attacks, and the attackers were never apprehended. In all these cases, I have reason to believe the attacks were inspired by the government. When finally I am killed, it will be the government that kills me.
The irony in this is that, unknown to most of the public, Mahinda and I have been friends for more than a quarter century. Indeed, I suspect that I am one of the few people remaining who routinely addresses him by his first name and uses the familiar Sinhala address oya when talking to him. Although I do not attend the meetings he periodically holds for newspaper editors, hardly a month passes when we do not meet, privately or with a few close friends present, late at night at President’s House. There we swap yarns, discuss politics and joke about the good old days. A few remarks to him would therefore be in order here.
Mahinda, when you finally fought your way to the SLFP presidential nomination in 2005, nowhere were you welcomed more warmly than in this column. Indeed, we broke with a decade of tradition by referring to you throughout by your first name. So well known were your commitments to human rights and liberal values that we ushered you in like a breath of fresh air. Then, through an act of folly, you got yourself involved in the Helping Hambantota scandal. It was after a lot of soul-searching that we broke the story, at the same time urging you to return the money. By the time you did so several weeks later, a great blow had been struck to your reputation. It is one you are still trying to live down.
You have told me yourself that you were not greedy for the presidency. You did not have to hanker after it: it fell into your lap. You have told me that your sons are your greatest joy, and that you love spending time with them, leaving your brothers to operate the machinery of state. Now, it is clear to all who will see that that machinery has operated so well that my sons and daughter do not themselves have a father.
In the wake of my death I know you will make all the usual sanctimonious noises and call upon the police to hold a swift and thorough inquiry. But like all the inquiries you have ordered in the past, nothing will come of this one, too. For truth be told, we both know who will be behind my death, but dare not call his name. Not just my life, but yours too, depends on it.
…
As for me, I have the satisfaction of knowing that I walked tall and bowed to no man. And I have not travelled this journey alone. Fellow journalists in other branches of the media walked with me: most of them are now dead, imprisoned without trial or exiled in far-off lands. Others walk in the shadow of death that your Presidency has cast on the freedoms for which you once fought so hard. You will never be allowed to forget that my death took place under your watch. As anguished as I know you will be, I also know that you will have no choice but to protect my killers: you will see to it that the guilty one is never convicted. You have no choice. I feel sorry for you
…
That The Sunday Leader will continue fighting the good fight, too, is written. For I did not fight this fight alone. Many more of us have to be – and will be – killed before The Leader is laid to rest. I hope my assassination will be seen not as a defeat of freedom but an inspiration for those who survive to step up their efforts. Indeed, I hope that it will help galvanise forces that will usher in a new era of human liberty in our beloved motherland. I also hope it will open the eyes of your President to the fact that however many are slaughtered in the name of patriotism, the human spirit will endure and flourish. Not all the Rajapakses combined can kill that.
In September 2012, a majority stake in the Sunday Leader was bought by one of President Rajapksa’s allies. The Leader’s editor was reportedly sacked for continuing to publish critical articles of the Sri Lankan government. The newspaper‘s tagline, though, still remains: “Unbowed and Unafraid”
It’s kind of fun to read through the New York Times’ historical archive and read what the tech coverage back in 1851 – a recent gem posted on HN concerned the controversy behind Thomas Edison’s 141-question job interview (‘Where do we get borax from?‘).
I was looking up some computer history and was wondering when the word “computer” made its first appearance in the Grey Lady. The NYT’s archive search allows for some fuzziness, so “computer” brings up numerous articles with “compute” and “computed” – as in, to calculate.
The short blurb appears to be in the Classifieds section. It reads:
A COMPUTER WANTED
WASHINGTON, May 1. – A civil service examination will be held May 18 in Washington, and, if necessary, in other cities, to secure eligibles for the position of computer in the Nautical Almanac Office, where two vacancies exist – one at $1,000, the other at $1,400.
The examination will include the subjects of algebra, geometry, trigonometry, and astronomy. Application blanks may be obtained of the United States Civil Service Commission.
Again, the “to calculate” meaning is implied here, with the applicant expected to help compile the used by the U.S. Navy for celestial navigation. The almanac office is still in existence (as a part of the U.S. Naval Observatory) and you can still purchase your own copy of the almanac, which includes “the following data tabulated at hourly intervals to a precision of 0.1 arcminute: the Greenwich hour angle and declination of the Sun, Moon, and navigational planets; the Greenwich hour angle of Aries” and more.
—
It wouldn’t be surprising, though, to see “computer” as machine referred to in the 19th century. The concept of a calculating machine dates back to the English mathematician and engineer, Charles Babbage, though his “Difference engine” was never built during his lifetime.
In 1812 [Charles Babbage] was sitting in his rooms in the Analytical Society looking at a table of logarithms, which he knew to be full of mistakes, when the idea occurred to him of computing all tabular functions by machinery. The French government had produced several tables by a new method. Three or four of their mathematicians decided how to compute the tables, half a dozen more broke down the operations into simple stages, and the work itself, which was restricted to addition and subtraction, was done by eighty [human] computers who knew only these two arithmetical processes. Here, for the first time, mass production was applied to arithmetic, and Babbage was seized by the idea that the labours of the unskilled computers could be taken over completely by machinery which would be quicker and more reliable.
When was “Charles Babbage” first mentioned in the Times? Again, the fuzzy searching feature makes it hard to narrow down articles that contain only “Charles Babbage”. The earliest I found was this “Scientific Gossip” column, dated Feb. 6, 1881, which mentions his son, Henry P. Babbage, who “has recalled to public attention the numerical system of signals proposed by his father…a system which is applicable to light and sound.” According to that method:
Every light house would have its own number, which would be continually repeated, either by light or sound, as long as necessary. Thus, if the light-house was numbered 73, there would be during foggy weather seven blasts at short intervals, then a pause: then three blasts, and a longer pause; after which the same would be repeated as long as the fog lasted. The number of the light-house could be given in 30 seconds.
As the light-houses on either side would be arranged with numbers not having the same digits, (say, for example, 25 and 48,) the counting of one digit would in most cases indicate the light-house and the counting of the second would afford a check and give positive assurance of the correctness of the observation if it was found to tally with the number found on the chart.
In this system nothing else has to be done beyond counting the number of the blasts; no special training is required, and a glance at the chart tells everything wanted. Surely this is within the intellect of almost any sailor of any nation.
If you read the rest of that Scientific Gossip column, it seems that the audience of 1881 had more patience than the audience of today for verbose technical explanations.
Another random first-mention-in-the-Times search: Nintendo, found on Oct. 8, 1955. It’s a brief note on how the “Nintendo Playing Card Company” and the approximate translation of “Nintendo” is “a corporation whose fortune or prosperity should be left to the mercy of heaven.”
The article notes, “evidently heaven has smiled on the Nintendo Company, because it has been using the mark since 1887.”
Actually, there is some method to this madness. As part of the process of updating the Ruby book, I realized I needed to spin off some of the larger, non-Ruby related topics. So, at some point, there will be mini-books about HTML and SQL. Regular expressions, as I keep telling people who want to deal with data, are incredibly important, even if you think you never want to learn programming. Hopefully this mini-book will make a strong case for learning regexes.
The second motive is I’ve been looking for a html/text-to-pdf workflow. So this is my experiment with Leanpub, which promises to turn a set of Markdown files into PDF/mobi/etc, while handling the selling process. I don’t expect to sell any copies of the BBoRegexes, but I hope to get a lot of insight about the mechanics behind Leanpub and if it presents a viable way for me to publish my other projects.
Edward Tufte speaks at Aaron Swartz’s memorial on Jan. 19, 2013, in Cooper Union, NYC.
There were many powerful speakers during yesterday’s memorial for Aaron Swartz in Cooper Union. Among them was data-visualization pioneer Edward Tufte, whom I hadn’t known had ever collaborated with Aaron. But given their similar orbits in democracy and data, it is not at all surprising that they were friends.
Tufte’s message included a surprising anecdote about his own hacking career. You can watch the video (parts 1, 2)
I’ve transcribed Tufte’s words here for posterity. It’s an interesting piece of hacker history and a touching defense of Aaron Swartz and others who are “marvelously and vigorously different.”
Note: A commenter has pointed me to a full version of Tufte’s talk. I’ll update with his full comments when I have time.
Parts of Democracy Now’s video feed appears to have cut out, as part of Tufte’s opening words are missing in the available feed, but here he is talking about how he and Aaron came to know each other:
[Aaron] tells the story of how he had a choice between taking a final exam or coming to hear me talk. As it turned out, he did both. We then would meet over the years for a long talk every now and then and my responsibility was to provide him with a reading list, a reading list for life
And then about two years ago, Quinn [Norton] and Aaron came to Connecticut, and he told me about the 4 and a half million downloads of scholarly articles. And my first question was:
“Why isn’t MIT celebrating this?”
The archival video feed cut off here and I can’t find the missing excerpt on Democracy Now’s website. From what I remember from being at the memorial, in this part, Tufte related how he told Aaron, half-seriously, that Aaron’s main fault was downloading millions of articles when only a few thousand were worth downloading.
Tufte then said that Aaron asked him if he knew Bill Bowen, the former president of Princeton University and the founder of JSTOR, the digital library that Aaron was accused of illegally accessing.
…[Bowen] then became president of the Mellon Foundation and he had retired from the Mellon foundation. But he was asked by he foundation to handle the problem of JSTOR and Aaron.
So I wrote Bill Bowen an email about it. And I said first that Aaron is a treasure. And then I told a personal story about how I had done some illegal hacking as a student and had been caught at it and what happened.
In 1962, my housemate and I invented the first blue box. That’s a device that allows for free, undetectable, unbillable long-distance telephone calls.
And we got this up. And played around with it and at the end of our research came when we completed was what we thought was the longest long distance phone call ever made, which was from Palo Alto to New York time of day, via Hawaii.
Well, during our experimentation, AT&T, on the second day it turned out, had tapped our phone. But it wasn’t until about six months later when I got a call from a gentleman, A.J. Dodge, a senior security person at AT&T. And I said, “I know what you’re calling about”.
And so we met. And he said what we’re doing is a crime…But I knew it wasn’t serious because he actually cared about the kind of engineering stuff and complained that the tone signals we were generating were not up to standard. Because they recorded them and played them back into the network to see what numbers we were trying to reach and they couldn’t break through some of the noise of our signal.
He asked why we went off the air after about three months…And I said, well, we regarded it as an engineering problem and we made the longest long-distance telephone call…and that was it.
And so the the deal was, as I explained to my email to Bill Bowen, was that we wouldn’t try to sell this…we wouldn’t do any more of it, and that we would turn our equipment over to AT&T. And so they got a complete vacuum oscillator kit for making long distance phone calls.
But I was grateful for A.J. Dodge and, I must say, even AT&T, that they decided not to wreck my life.
And so I told Bill Bowen that he had a great opportunity here to not wreck somebody’s life. And of course he thankfully did the right thing.
Aaron’s unique quality was that he was marvelously and vigorously different.
There is a scarcity of that.
Perhaps we can be all a little more different too.
(Thanks to commenter Daniel for pointing out this Youtube full version)
After listening to Tufte talk, I Googled for any reference of him and Swartz. Here’s Swartz, “a loyal fan,” asking Tufte about the font he uses. And here’s Swartz describing a Tufte lecture.
The past two days have been emotionally draining for me. It shows how much Aaron Swartz accomplished in his young life that even though I never met him in person, I was as crushed by the news of his suicide as if a close friend had died.
I had always thought that if anyone could fight to change a system for good, Aaron was one of the best equipped for our generation. His death sucked a lot of hope out, frankly. But the letter today from MIT president Rafael Reif restored a little of it again. MIT had taken a beating from Aaron’s family and never would I have expected an institution to be so contrite and so concerned about a non-student accused of committing a campus crime. The incident took place before Reif’s term but he shows no desire to wash his hands of it, appointing a strong defender of Internet rights to conduct a “thorough analysis” into how MIT contributed to this tragedy:
To the members of the MIT community:
Yesterday we received the shocking and terrible news that on Friday in New York, Aaron Swartz, a gifted young man well known and admired by many in the MIT community, took his own life. With this tragedy, his family and his friends suffered an inexpressible loss, and we offer our most profound condolences. Even for those of us who did not know Aaron, the trail of his brief life shines with his brilliant creativity and idealism.
Although Aaron had no formal affiliation with MIT, I am writing to you now because he was beloved by many members of our community and because MIT played a role in the legal struggles that began for him in 2011.
I want to express very clearly that I and all of us at MIT are extremely saddened by the death of this promising young man who touched the lives of so many. It pains me to think that MIT played any role in a series of events that have ended in tragedy.
I will not attempt to summarize here the complex events of the past two years. Now is a time for everyone involved to reflect on their actions, and that includes all of us at MIT. I have asked Professor Hal Abelson to lead a thorough analysis of MIT’s involvement from the time that we first perceived unusual activity on our network in fall 2010 up to the present. I have asked that this analysis describe the options MIT had and the decisions MIT made, in order to understand and to learn from the actions MIT took. I will share the report with the MIT community when I receive it.
I hope we will all reach out to those members of our community we know who may have been affected by Aaron’s death. As always, MIT Medical is available to provide expert counseling, but there is no substitute for personal understanding and support.
My point is not that infinite scroll is stupid. It may be great on your website. But we should have done a better job of understanding the people using our website” – Dan McKinley, Principal Engineer at Etsy
A few weeks ago, Etsy engineer Dan McKinley gave a talk on “Design for Continuous Experimentation.” It’s an interesting, humorous presentation on large scale A/B testing and I fully recommend you check it out (slides and video here).
The moral was that A/B testing – much like the code it tests – must be done in a modularized fashion. The “fail” case he gave was when Etsy spent months developing and testing infinite scroll to their search listings, only to find that it had a negative impact on engagement.
McKinley said Etsy’s main lesson from this was that their A/B testing strategy was too monolithic:
Spend a ton of time building the infinite scroll feature and release it.
Verify that people love it, then find a Brooklyn warehouse to throw a celebration party.
“Seeing more items faster is presumed to be a better experience”, McKinley said. But the A/B tests showed various negative effects of the feature, including fewer clicks on the results and fewer items “favorited” from the infinite results page. And curiously, while users didn’t buy fewer items overall, “they just stopped using search to find these items.”
So basically infinite scroll failed in every major way. But not only was Etsy’s team wrong in assuming that users would benefit from infinite sroll, McKinley said, they were wrong in automatically accepting the two underlying assumptions behind infinite scroll:
Users want more results per page.
Users want faster results.
(Assumption #2 was tested by artificially slowing down search for a cohort of users. These users did not necessarily react positively to slower results. But their engagement level was not statistically significantly lessened.)
The point of McKinley’s talk was that instead of having the goal of “test infinite scroll,” Etsy realized it needed to test each assumption separately, and this going forward is their game plan (the success case McKinley gives is the revamp of Etsy’s search box).
Even if user engagement isn’t positively influenced by more results per page or by faster results, why does the combination of both have a decidedly negative impact?
The decision to try infinite scroll was partially influenced by Google apparent success with “instant search results,” according to McKinley. And it’s a feature that is prominent among popular Tumblr themes, Pinterest, and of course, at Facebook and Twitter, so presumably their A/B testing has yielded good results.
But why not at Etsy? Or at Amazon, which sticks with 16 results per search page? Users are notoriously fickle about interface changes. But if the search algorithm still brings up the best results at the top, then a user who has only 16 options before clicking through has no better advantage than the user who has 16 plus an “infinite” number of lesser results, if the latter doesn’t have to do any additional work to get them.
McKinley said he didn’t know why infinite scroll didn’t succeed for Etsy. There wasn’t, as far as they could tell, a technical fault (i.e. infinite scroll breaking in a specific browser). It was just a bad thing and the reason why wasn’t clear.
Now the actual merits of infinite scroll itself is still a controversial feature – even if there aren’t technical issues, which there almost always are. But in Etsy’s use case, it seems that at worst there should have been no effect, not negative effect. The most jarring problem of infinite scroll is that there’s no footer, which presumably the average Etsy user doesn’t need when making viewing/purchasing decisions.
Maybe it’s not that users consciously dislike infinite scroll. But in practice, maybe they lose a sense of orientation? Some users may have the habit of going further and further into search results – “playing the field” so to speak – before they realize that pages 1 and 2 are the best options they have. With pagination, it’s fairly easy to get back to those pages.
But if these users don’t have the habit of bookmarking/favoriting/writing-down-the-names of items as they scroll through, perhaps the deluge of infinite scroll makes it more likely for users to forget where they once were and what once caught their eye? Or maybe it’s a lack of developing the habits needed to actually act: when users are forced to deal with the loading time of each page, they learn to do their favoriting/click-throughs/impulse-purchasing on those pages before moving on? These users, when allowed to scroll for more items than they were used to, might simply lose the habit of click-to-favorite/browse/buy – but never develop the habit of scrolling back up. Or they’re just overwhelmed from the information overload and don’t feel like taking action any longer.
Infinite scroll may be pleasant for browsing, but does it lead to inaction? It’s an interface issue that is likely less a technical question than a psychological. I wonder what other online retail interfaces have found?
On Hacker News, user oconnore makes a great observation here, in that least some users, when returning to the search page after backing up from a product page, would find that they had lost their place in the infinite search stream (probably the biggest problem with infinite scroll implementations). I didn’t mention this in the original version of this post, but McKinley opened his talk with an anecdote about another poor assumption: power users (who worked at Etsy and made the suggestion) often opened up search results in new windows because they wanted to do side-by-side comparisons. But when Etsy made that the default behavior, testing found that most users did not appreciate it.
So putting 2 and 2 together: Perhaps many Etsy users have the habit of clicking-through a search result and then backing up to the search page. When the infinite-scroll didn’t properly mark their place, they *really* got lost and the search experience would obviously not be a good one. Seems as good as explanation as any, and so maybe it really is a technical issue after all.
Another update: HN user gfodor, who worked on the project, said the back-button problem may have applied to some IE users. But as McKinley says in his talk, the negative search effect occurred across all browsers, including ones that did back-up correctly. McKinley does go into good detail about how Etsy determines control group composition (sellers vs. non-sellers, for example, are a much different user type), but as this blog post is near-infinitely long I suggest again you check out his excellent talk on his blog.
