Author Archives: Dan Nguyen

  

Apple banned Pulitzer-Prize winning cartoonist’s app because it “ridicules public figures”

Pulitzer-Prize winning cartoonist Mark Fiore, who won this year’s Pulitzer for his editorial cartoons, says he tried to make an iPhone app but it was just too politically hot for Apple’s guidelines. From Nieman Labs (h/t Poynter):

In December, Apple rejected his iPhone app, NewsToons, because, as Apple put it, his satire “ridicules public figures,” a violation of the iPhone Developer Program License Agreement, which bars any apps whose content in “Apple’s reasonable judgement may be found objectionable, for example, materials that may be considered obscene, pornographic, or defamatory.”
…
Apple attached screenshots of the offending material, including an image depicting the White House gate crashers interrupting an Obama speech. Two other grabs include images referencing torture, Balloon Boy, and various political issues.


Here’s an “offending image” of Fiore’s.

  

Pultizer Prize at ProPublica

Pulitzer Prize

It’s been a huge last few days for ProPublica. My colleagues Jesse Eisinger and Jake Bernstein unveiled the result of 7+ months of reporting, a much anticipated collaboration with “This American Life” on how the hedge fund Magnetar Capital helped prolong the housing bubble by betting against risky investments that it advocated for. Also, our story on private jet owners hiding in public airspace, uncovered by Michael Grabell (after our lawyers’ successful litigation), was one of our most viewed, thanks to it getting top play by USA Today and Yahoo.

Those both alone would’ve made it one of ProPublica’s most prominent weeks, but then Sheri Fink won the Pulitzer for Investigative Reporting for her massive investigation, published in the NYT magazine, on how a hospital’s doctors, post-Katrina, reportedly put patients to death under the guise of mercy and grace under chaos. Sheri’s win is extremely gratifying, because her subject had a lot of things going against it: Katrina was a four-year-old painful, chaotic memory that most Americans wanted to forget. And for N.O. residents, it seemed that the overwhelming sentiment was for the doctors and other authorities who did what they could. Anna Pou, the doctor at the center of Sheri’s story, had been exonerated (and the prosecutor who went after her was removed). And after Sheri’s story, no new charges have been made against her.

The story itself is a long-read. In addition to the factors above going against it, it also doesn’t deliver an immediate payoff for the ADD-afflicted reader. It’s only until the end that you can appreciate the light that Sheri shed on a universally important, yet opaque topic: who deserves life in a time of crisis? I think Sheri’s story, and subsequent follow-ups related to swine flu preparations, raised the alarm that not even our medical professionals are on the same page, and moved the ball in such a way that her findings would shock even the most cynical skeptics of the medical profession.

Also, congrats to my colleagues Charles Ornstein and Tracy Weber for being finalists in the Public Service category for their exposure of California’s broken nursing board. For them to even be considered for that prize, considering they won it recently before in the same area (lax oversight of medical care) is a testament to how thorough their work was again, and how much impact their stories had (Gov. Schwarzenegger immediately sacked or forced out a majority of the board afterwards).

I think our office felt confident our work was as good as any Pulitzer contender and it wouldn’t be a shock to win, even though we would be the first online-only organization (and possibly the youngest, at two years old) to do it. The drama was less about whether if we would win but which one of our reporters would win. For example, T. Christian Miller and his work on defense contractors was, in my mind, as deserving as any. Like Sheri, he shed light, in an exhaustive, dogged fashion, on a subject that most people would rather not care about: the treatment of civilians who are injured in warzones while working as contractors. With the bad rep of Blackwater, it’s proof of T’s herculean reporting and writing efforts that he got lawmakers to make some real moves into an easily overlooked (for political reasons) but essential area of our national security (in terms of prizes though, T already brought home the Selden Ring).

And of course, all those stories above would’ve had a harder hill to climb without the collaboration of all our great editors and research staff. And in my own department, Krista Kjellman and Jeff Larson put in just as much dedication and deliberation to further illuminate the stories in their online presentation (and in the process, often provided research and work important to the stories themselves).

Congrats to the other Pulitzer winners. I haven’t had time to look through all their work. I did put WaPo’s Gene Weingarten’s winning feature on the hellish punishment of parents who left their children to die in overheated cars on my iPad’s Instapaper. I got about a fourth-way through before I had to put it away so I wouldn’t be crying in the subway car.

  

Coding for Journalists 101 : A four-part series

nico.cavallotto

Photo by Nico Cavallotto on Flickr

Update, January 2012: Everything…yes, everything, is superseded by my free online book, The Bastards Book of Ruby, which is a much more complete walkthrough of basic programming principles with far more practical and up-to-date examples and projects than what you’ll find here.

I’m only keeping this old walkthrough up as a historical reference. I’m sure the code is so ugly that I’m not going to even try re-reading it.

So check it out: The Bastards Book of Ruby

-Dan

—

Update, Dec. 30, 2010: I published a series of data collection and cleaning guides for ProPublica, to describe what I did for our Dollars for Docs project. There is a guide for Pfizer which supersedes the one I originally posted here.

So a little while ago, I set out to write some tutorials that would guide the non-coding-but-computer-savvy journalist through enough programming fundamentals so that he/she could write a web scraper to collect data from public websites. A “little while” turned out to be more than a month-and-a-half. I actually wrote most of it in a week and then forgot about. The timeliness of the fourth lesson, which shows how to help Pfizer in its mission to more transparent, compelled me to just publish them in incomplete form. There’s probably inconsistencies in the writing and some of the code examples, but the final code sections at the end of each tutorial do seem to execute as expected.

As the tutorials are aimed at people who aren’t experienced programming, the code is pretty verbose, pedantic, and in some cases, a little inefficient. It was my attempt to think how to make the code most readable, and I’m very welcome to editing changes.

DISCLAIMER: The code, data files, and results are meant for reference and example only. You use it at your own risk.

  

Coding for Journalists 104: Pfizer’s Doctor Payments; Making a Better List

Update (12/30): So about an eon later, I’ve updated this by writing a guide for ProPublica. Heed that one. This one will remain in its obsolete state.

Update (4/28): Replaced the code and result files. Still haven’t written out a thorough explainer of what’s going on here.

Update (4/19): After revisiting this script, I see that it fails to capture some of the payments to doctors associated with entities. I’m going to rework this script and post and update soon.

So the world’s largest drug maker, Pfizer, decided to tell everyone which doctors they’ve been giving money to to speak and consult on its behalf in the latter half of 2009. These doctors are the same ones who, from time to time, recommend the use of Pfizer products.

From the NYT:

Pfizer, the world’s largest drug maker, said Wednesday that it paid about $20 million to 4,500 doctors and other medical professionals for consulting and speaking on its behalf in the last six months of 2009, its first public accounting of payments to the people who decide which drugs to recommend. Pfizer also paid $15.3 million to 250 academic medical centers and other research groups for clinical trials in the same period.

A spokeswoman for Pfizer, Kristen E. Neese, said most of the disclosures were required by an integrity agreement that the company signed in August to settle a federal investigation into the illegal promotion of drugs for off-label uses.

So, not an entirely altruistic release of information. But it’s out there nonetheless. You can view their list here. Jump to my results here

Not bad at first glance. However, on further examination, it’s clear that the list is nearly useless unless you intend to click through all 480 pages manually, or, if you have a doctor in mind and you only care about that one doctor’s relationship. As a journalist, you probably have other questions. Such as:

  • Which doctor received the most?
  • What was the largest kind of expenditure?
  • Were there any unusually large single-item payments?

None of these questions are answerable unless you have the list in a spreadsheet. As I mentioned in earlier lessons…there are cases when the information is freely available, but the provider hasn’t made it easy to analyze. Technically, they are fulfilling their requirement to be “transparent.”

I’ll give them the benefit of the doubt that they truly want this list to be as accessible and visible as possible…I tried emailing them to ask for the list as a single spreadsheet, but the email function was broken. So, let’s just write some code to save them some work and to get our answers a little quicker.
Continue reading →

  

Coding for Journalists 103: Who’s been in jail before: Cross-checking the jail log with the court system; Use Ruby’s mechanize to fill out a form

This is part of a four-part series on web-scraping for journalists. As of Apr. 5, 2010, it was a published a bit incomplete because I wanted to post a timely solution to the recent Pfizer doctor payments list release, but the code at the bottom of each tutorial should execute properly. The code examples are meant for reference and I make no claims to the accuracy of the results. Contact dan@danwin.com if you have any questions, or leave a comment below.

DISCLAIMER: The code, data files, and results are meant for reference and example only. You use it at your own risk.

In particular, with lesson 3, I skipped basically any explanation to the code. I hope to get around to it later.

Going to Court

In the last lesson, we learned how to write a script that would record who was in jail at a given hour. This could yield some interesting stories for a crime reporter, including spates of arrests for notable crimes and inmates who are held with $1,000,000 bail for relatively minor crimes. However, an even more interesting angle would be to check the inmates’ prior records, to get a glimpse of the recidivism rate, for example.

Sacramento Superior Court allows users to search by not just names, but by the unique ID number given to inmates by Sacramento-area jurisdictions. This makes it pretty easy to link current inmates to court records.


However, the techniques we used in past lessons to automate the data collection won’t work here. As you can see in the above picture, you have to fill out a form. That’s not something any of the code we’ve written previously will do. Luckily, that’s where Ruby’s mechanize comes in.

Continue reading →

  

Coding for Journalists 101: Go from knowing nothing to scraping Web pages. In an hour. Hopefully.

UPDATE (12/1/2011): Ever since writing this guide, I’ve wanted to put together a site that is focused both on teaching the basics of programming and showing examples of practical code. I finally got around to making it: The Bastards Book of Ruby.

I’ve since learned that trying to teach the fundamentals of programming in one blog post is completely dumb. Also, I hope I’m a better coder now than I was a year and a half ago when I first wrote this guide. Check it out and let me know what you think:

http://ruby.bastardsbook.com

Someone asked in this online chat for journalists: I want to program/code, but where does a non-programmer journalist begin?

My colleague Jeff Larson gave what I believe is the most practical and professionally-useful answer: web-scraping (jump to my summary of web-scraping here, or read this more authorative source).

This is my attempt to walk someone through the most basic computer science theory so that he/she can begin collecting data in an automated way off of web pages, which I think is one of the most useful (and time-saving) tools available to today’s journalist. And thanks to the countless hours of work by generous coders, the tools are already there to make this within the grasp of a beginning programmer.

You just have to know where the tools are and how to pick them up.

Click here for this page’s table of contents. Or jump to the the theory lesson. Or to the programming exercise. Or, if you already know what a function and variable is, and have Ruby installed, go straight to two of my walkthroughs of building a real-world journalistic-minded web scraper: Scraping a jail site, and scraping Pfizer’s doctor payment list.

Or, read on for some more exposition:

Continue reading →

  

The WikiLeaks Hellfire Video vs. Video Games

The WikiLeaks release of classified U.S. military video depicting American helicopters gunning down Iraqis (which appeared to include children and two Reuters staff) was easily a milestone of modern journalism. Even though Reuters had reported the story aggressively, the deaths of Namir Noor-Eldeen and Saeed Chmagh were easily forgotten amid the war’s constant newscycle in 2007.

The video below, combined with boots-on-the-ground reporting by WikiLeaks, has an unmatched power to shock, awe, and sicken:

A side-angle to all this is how chillingly-similar the released video is to today’s video games. This was a point Jane Mayer touched on in her excellent New Yorker piece on Obama’s increased usage of Predator drones:

Using joysticks that resemble video-game controls, the reachback operators—who don’t need conventional flight training—sit next to intelligence officers and watch, on large flat-screen monitors, a live video feed from the drone’s camera. From their suburban redoubt, they can turn the plane, zoom in on the landscape below, and decide whether to lock onto a target. A stream of additional “signal” intelligence, sent to Langley by the National Security Agency,* provides electronic means of corroborating that a target has been correctly identified. The White House has delegated trigger authority to C.I.A. officials, including the head of the Counter-Terrorist Center, whose identity remains veiled from the public because the agency has placed him under cover.

People who have seen an air strike live on a monitor described it as both awe-inspiring and horrifying. “You could see these little figures scurrying, and the explosion going off, and when the smoke cleared there was just rubble and charred stuff,” a former C.I.A. officer who was based in Afghanistan after September 11th says of one attack. (He watched the carnage on a small monitor in the field.) Human beings running for cover are such a common sight that they have inspired a slang term: “squirters.”

Read more: http://www.newyorker.com/reporting/2009/10/26/091026fa_fact_mayer#ixzz0kIeSEK1h

The striking similarity inspired some soul-searching from this Redditor:

After watching the wikileaks video I found myself thinking back to the aerial segments of Modern Warfare and MW2. I’m not sure I’d want to play them again; the anonymity of the people you’re shooting seems a little too true to life for me.

Modern Warfare and MW2 are part of the highly-successful Call of Duty first-person shooter video games. One of the segments has the players manning an AC-130 Spectre gunship to wipe out the enemy:

Some interesting comments from Redditors on that angle:

a_culther0 22 points 9 hours ago[-]
I always believed the main point of those levels in the game was to illustrate that certain things in Modern War can be achieved with the push of a button. The AC-130 level in COD4 has essentially 0 difficulty; which in my eyes makes an excellent statement on its own.
permalinkparentreportreply

awills 15 points 9 hours ago[-]
This is also how I read this scene. It’s actually the most realistic depiction of war in the entire game, because what you’re doing is remarkably similar to what it would be like in real life, just aiming at tiny targets and destroying them. Interestingly, it was also the most distancing from the actual results of your actions.

bumrushtheshow 35 points 9 hours ago[-]
…The grainy TV footage and shooting at tiny people made me question more than usual what the hell I was doing. I was blowing up “bad” people who looked exactly like the “good” people. I was clearly in the “bad” people’s country, with only vague justifications for why I was there blowing the place up.
I’ve seen some awful footage from Apache gun cams on Youtube. Ones where maimed “bad guys” crawl out of a burning truck, while hillbillies say “that one’s still moving, hit ‘im again” left me literally feeling nauseous. I thought of these throughout the AC-130 level in COD4.

  

iPad: First Impressions

iPad at Fat Cat

As if chess at a bar wasn't geeky enough. I think we raised the bar at West Village's Fat Cat.

So I plunked $580 for the 16GB wireless iPad (that includes tax and the Apple case), plus about $60-$80 more on apps. I already have a laptop, a blackberry, and a netbook…but I justified this luxury purchase because I do think that tablet computing (whether or not the iPad leads it) will be the next boom in computer usage, and I’d like to at least be aware of it.

And I like doodling and watching movies in bed.

That said, I didn’t have a strong response for everyone who asked “How is this different than a bigger iPhone?” There’s not much difference, actually, but I think we have yet to see what touch-computing can offer, and it’s a decidedly different experience than a traditional laptop. I’ve owned a MSI Wind for a year now and barely use it, unless I need to pack a laptop when I’m carrying camera equipment around. I’ve already played more games and watched more movies in the past two days on the iPad than I have on my netbook, it just seems better suited for it. I think the lack of a attached keyboard makes for a fundamentally-different experience as least as significant as the iPod Touch (and to some extent, the iPhone) over every other mp3 player before it.

Brushes pic

One of my first attempts on Brushes

I’m not much of an artist, but I’ve always wanted to just sketch for fun without the hassle of buying and maintaining art supplies. A mouse and Illustrator just doesn’t do it for me, nor would a tablet connected to a laptop. Brushes has been one of my favorite apps so far.

I’ve mostly stopped playing video games. I downloaded a few of the marquee titles, including Real Racing, and barely touched them after a few minutes. But the more social, multiplayer games, like Flight Control and the various board games, were really entertaining when the bar we were at, Fat Cat, didn’t have the games we wanted.

As far as productivity…I haven’t used it at all for anything meaningful. When I was sitting around in my living room with both the laptop and iPad, I still, out of habit, switched to my laptop to do even just regular browsing. Typing is a wrist-killer…and touch-navigating the web is still cumbersome.

Didn’t do much reading, but I like that one of the free apps allows you to download classics like Alice in Wonderland for free, with pretty decent, readable text. I still do most of my New York Times reading on my Blackberry as I’m waiting in line or at the subway.

I think I’ll keep the iPad for now…Selling it while people still think it’s cool is still a possibility…but I see a lot of potential in it so far. But I wholeheartedly agree with Kotaku’s Mike Fahey: I feel like an asshole for owning an iPad and don’t feel comfortable using it in public, yet. That kind of reduces the device’s utility…For now, I’m keeping the plastic wrap on it that it came with.

Some other notes:

Cons:

  • I synced up my iPad just now for the first time with my laptop. I took awhile to figure out how to transfer photos from my laptop to the iPad (using the not-so-visible Photos tab in iTunes, and then having to create a special folder on my computer with duplicate copies of photos). And while it was doing that, it decided to delete all the apps from the iPad. The file management on the iPad, as it has been with the iPod, is fucking stupid, and possibly the worst part of any iProduct. I stopped using Sony products because of their proprietary – and generally inferior – formats (a $200 voice recorder I bought years ago is useless because Sony no longer produces/updates the software to access its files). I hope Apple doesn’t go the same path.
  • Apps were generally buggy. Netflix crashed many times.
  • Still haven’t figured out how to comfortably type.
  • Takes awhile to charge up the battery.
  • It’s hard to find a good Chess or Cards game…either they have hotseat-multiplayer or computer AI, rarely both.
  • Yeah, it is a bit heavy to not be resting on your lap.

Pros

  • Touch-interface is as solid as it is on the iPod Touch.
  • Being able to lockdown the screen rotation is great.
  • Lots of decent free apps. My favorite so far are craigsphone (craigslist on the pad), the NYT editor’s choice, Netflix, and Free Books
  • The launch games have been pretty good, including Flight Control, Minigore, Real Racing
  • Netflix streaming on my nightstand is great. Finally, I’ll finish 30 Rock.