No magic Internet database for everything

Investigative reporting is often a tedious, time-consuming process. But it has gotten much easier in recent years as more and more information is being made available online.

This week’s special report, Aid Debate, shows how the Internet offers a wealth of information, but also how there is no such thing as a magic database that has perfect information on any single topic.

The U.S. Department of Agriculture could make available with relative ease a searchable online database showing who takes federal farm subsidy payments, and how much. But for reasons that aren’t entirely clear (the agency says it’s a cost issue, but it’s just as likely a political one), USDA chooses not to release that information online.

The federal government failed to achieve "total information awareness," despite spending $200 million on the effort.

So a public policy advocacy group, the Environmental Working Group, does the horribly painstaking work of requesting that information via the Freedom of Information Act and compiling it in an easily searchable form.

But there were some Wyoming farm subsidy data I found in a separate online database,, managed by OMBWatch, that didn’t show up in the EWG database. That’s because, even though the USDA oversees farm subsidy programs, payments are funneled through a complex and Byzantine series of interlocking and overlapping offices and agencies.

It’s virtually impossible to find, cross-reference and present all that information. Even the federal government’s efforts at “total information awareness” for homeland security were a high-profile venture that riled privacy advocates and produced mixed results, at best, after more than $200 million spent.

One Wyoming farm subsidy recipient I spoke to said he had received an additional $30,000 or more that my research had failed to uncover. After searching several more places, I still couldn’t verify that amount, so I didn’t include it in the piece. But the point is that no single database, even one covering a relatively circumscribed and specific topic like farm subsidies, contains complete or perfect information.

But there’s no reason such public information shouldn’t be more available, more accurate and more complete. It’s our data, we paid for it and we have a right to search, review and reference it however we like.

Here’s hoping the next decade brings more advances on that front than we saw during the past one.

— Ruffin Prevost, WyoFile managing editor

Join the Conversation

1 Comment

Want to join the discussion? Fantastic, here are the ground rules: * Provide your full name — no pseudonyms. WyoFile stands behind everything we publish and expects commenters to do the same. * No personal attacks, profanity, discriminatory language or threats. Keep it clean, civil and on topic. *WyoFile does not fact check every comment but, when noticed, submissions containing clear misinformation, demonstrably false statements of fact or links to sites trafficking in such will not be posted. *Individual commenters are limited to three comments per story, including replies.

Your email address will not be published. Required fields are marked *

  1. Interesting, how much in Federal subsidies, Farm Aid, etc. did you turn up for our Congressional delegation? I’d like to check it against my info and figures.