Tag Archives: journalism

UPDATE (12/1/2011): Ever since writing this guide, I’ve wanted to put together a site that is focused both on teaching the basics of programming and showing examples of practical code. I finally got around to making it: The Bastards Book of Ruby.

I’ve since learned that trying to teach the fundamentals of programming in one blog post is completely dumb. Also, I hope I’m a better coder now than I was a year and a half ago when I first wrote this guide. Check it out and let me know what you think:

http://ruby.bastardsbook.com

Someone asked in this online chat for journalists: I want to program/code, but where does a non-programmer journalist begin?

My colleague Jeff Larson gave what I believe is the most practical and professionally-useful answer: web-scraping (jump to my summary of web-scraping here, or read this more authorative source).

This is my attempt to walk someone through the most basic computer science theory so that he/she can begin collecting data in an automated way off of web pages, which I think is one of the most useful (and time-saving) tools available to today’s journalist. And thanks to the countless hours of work by generous coders, the tools are already there to make this within the grasp of a beginning programmer.

You just have to know where the tools are and how to pick them up.

Click here for this page’s table of contents. Or jump to the the theory lesson. Or to the programming exercise. Or, if you already know what a function and variable is, and have Ruby installed, go straight to two of my walkthroughs of building a real-world journalistic-minded web scraper: Scraping a jail site, and scraping Pfizer’s doctor payment list.

Or, read on for some more exposition:

Continue reading

On Friday, golfer Tiger Woods held a TV appearance to talk about life after marital problems. At around 2:30 p.m., I screen capped some of the websites for some of the largest news organizations and aggregators. Today, I looked at the screen-caps, cropped them to the top 1600 pixels, and marked in green the areas of the pages devoted to Woods coverage (or related coverage, such as “Slideshow: Top 10 Adultery Confessions).

Continue reading

CareerCast.com released a list of 200 jobs ranked by such factors as stress level, pay, work environment, and hiring outlook. Read their methodology here. The WSJ made it into a sortable multipage list but I took the liberty of making a single-page version with bar graphs showing the starting, mid, and top salaries.

At first glance…seems like it’s great to be a geek, with the top 6 jobs steeped in the mathematics and science (exception being historian…which is a geekiness of its own sort).

But going down the list…say, all the way to position #11, and your BS meter should be going off. Apparently, philosopher is the 11th best job, with very low physical demands and stress, a “very good” hiring outlook, and a median income of $60,000.

Really? A comment on this physicsforums thread sums up my a priori assumption: “I have no factual information but I guess your career choices would be either getting a faculty position at some university or flipping burgers.

Continue reading

Get rich in the temp nursing business

Get rich in the temp nursing business

On Sunday, my ProPublica colleagues Tracy Weber and Charles Ornstein, in conjunction with the Los Angeles Times, put out a story examining the lack of standards in the temp nursing agency, a dangerous situation considering California’s desperate shortage of nursing staff.

Emboldened by a chronic nursing shortage and scant regulation, the firms vie for their share of a free-wheeling, $4-billion industry. Some have become havens for nurses who hopscotch from place to place to avoid the consequences of their misconduct. (see related story: A ‘Crazy’ Way for an Industry to Operate)

A joint investigation with the Los Angeles Times found dozens of instances in which staffing agencies skimped on background checks or ignored warnings from hospitals about sub-par nurses on their payrolls. Some hired nurses sight unseen, without even conducting an interview.

The gist of the problem: California lacks virtually any kind of tracking of errant temp nurses. This nurse, for example, was accused of stealing drugs from at least six hospitals, suffered a drug-induced seizure on the job, and had his Minnesota nursing license suspended before California got around to filing an accusation against him. Two years later, after a few more reported incidents of drug theft, the California registered nursing board finally revoked his license when he didn’t make his hearing on time.

Charlie and Tracy have been covering this story even before they joined ProPublica; LATimers Maloy Moore and Doug Smith contributed a massive amount of the essential research and data-analysis. This temp nurses chapter is just another consequence of what appears to be awful records-keeping and sloth by the various oversight bodies.

My own contribution to the coverage was small, the most notable aspect of which was this Ruby on Rails site I built to catalogue the sanctioned nurses, a relatively minor task compared to actually collecting and parsing the data (i.e. reading through all the PDF files for the buried information). . It was pretty simple, allowing users at a glance to see the numbers of disciplined nurses by various categories, including year and type of discipline. I was a little skeptical of doing it at first, just because the CA nursing board does have a searchable and functional database of its own.

Theoretically (well, if it weren’t the case that the records themselves are often incomplete, so that criminal nurses come up with a clean sheet), any member of the public could look up their own nurses’ records and avoid the bad ones. But the meat of the Charlie’s and Tracy’s is the numbers: 1,254 days on average to discipline a nurse (compared to 173 for Texas). 1,706 days before one nurse, who was kicked out of a drug-recovery program and considered a threat to public safety, had even an accusation filed against her. Our site makes it evident that hard numbers, not just heartbreaking anecdotes,  argue against California’s regulatory status quo.

A screenshot from our sanctioned nurses database

A screenshot from our sanctioned nurses database

The reporters on this story put in months of time manually tabulating the data to come up with the thrust of their stories. Sadly, all of these numbers and statistical conclusions were probably right under the nursing board’s nose. The regulators apparently track dates and types of accusations and disciplines for each nurse. A few simple database queries would’ve quickly uncovered the glaring delays and bottlenecks in the system (e.g. (SELECT AVG(TO_DAYS(`date_discipline`)-TO_DAYS(`date_initial_complaint`)) as average_delay from `disciplinary_actions`).

A day after Charlie and Tracy’s initial story in July 2009, Gov. Schwarzenegger sacked a majority of the registered nursing board and new regulations include making public the restrictions on a nurse’s license. Read ProPublica’s complete coverage on California’s flawed oversight of health-care workers here.