No magic Internet database for everything

Investigative reporting is often a tedious, time-consuming process. But it has gotten much easier in recent years as more and more information is being made available online.

This week’s special report, Aid Debate, shows how the Internet offers a wealth of information, but also how there is no such thing as a magic database that has perfect information on any single topic.

The U.S. Department of Agriculture could make available with relative ease a searchable online database showing who takes federal farm subsidy payments, and how much. But for reasons that aren’t entirely clear (the agency says it’s a cost issue, but it’s just as likely a political one), USDA chooses not to release that information online.

So a public policy advocacy group, the Environmental Working Group, does the horribly painstaking work of requesting that information via the Freedom of Information Act and compiling it in an easily searchable form.

But there were some Wyoming farm subsidy data I found in a separate online database, FedSpending.org, managed by OMBWatch, that didn’t show up in the EWG database. That’s because, even though the USDA oversees farm subsidy programs, payments are funneled through a complex and Byzantine series of interlocking and overlapping offices and agencies.

It’s virtually impossible to find, cross-reference and present all that information. Even the federal government’s efforts at “total information awareness” for homeland security were a high-profile venture that riled privacy advocates and produced mixed results, at best, after more than $200 million spent.

One Wyoming farm subsidy recipient I spoke to said he had received an additional $30,000 or more that my research had failed to uncover. After searching several more places, I still couldn’t verify that amount, so I didn’t include it in the piece. But the point is that no single database, even one covering a relatively circumscribed and specific topic like farm subsidies, contains complete or perfect information.

But there’s no reason such public information shouldn’t be more available, more accurate and more complete. It’s our data, we paid for it and we have a right to search, review and reference it however we like.

Here’s hoping the next decade brings more advances on that front than we saw during the past one.

— Ruffin Prevost, WyoFile managing editor