No magic Internet database for everything

Investigative reporting is often a tedious, time-consuming process. But it has gotten much easier in recent years as more and more information is being made available online.

This week’s special report, Aid Debate, shows how the Internet offers a wealth of information, but also how there is no such thing as a magic database that has perfect information on any single topic.

The U.S. Department of Agriculture could make available with relative ease a searchable online database showing who takes federal farm subsidy payments, and how much. But for reasons that aren’t entirely clear (the agency says it’s a cost issue, but it’s just as likely a political one), USDA chooses not to release that information online.

The federal government failed to achieve "total information awareness," despite spending $200 million on the effort.

So a public policy advocacy group, the Environmental Working Group, does the horribly painstaking work of requesting that information via the Freedom of Information Act and compiling it in an easily searchable form.

But there were some Wyoming farm subsidy data I found in a separate online database, FedSpending.org, managed by OMBWatch, that didn’t show up in the EWG database. That’s because, even though the USDA oversees farm subsidy programs, payments are funneled through a complex and Byzantine series of interlocking and overlapping offices and agencies.

It’s virtually impossible to find, cross-reference and present all that information. Even the federal government’s efforts at “total information awareness” for homeland security were a high-profile venture that riled privacy advocates and produced mixed results, at best, after more than $200 million spent.

One Wyoming farm subsidy recipient I spoke to said he had received an additional $30,000 or more that my research had failed to uncover. After searching several more places, I still couldn’t verify that amount, so I didn’t include it in the piece. But the point is that no single database, even one covering a relatively circumscribed and specific topic like farm subsidies, contains complete or perfect information.

But there’s no reason such public information shouldn’t be more available, more accurate and more complete. It’s our data, we paid for it and we have a right to search, review and reference it however we like.

Here’s hoping the next decade brings more advances on that front than we saw during the past one.

— Ruffin Prevost, WyoFile managing editor

Join the Conversation

1 Comment

WyoFile's goal is to provide readers with information and ideas that foster constructive conversations about the issues and opportunities our communities face. One small piece of how we do that is by offering a space below each story for readers to share perspectives, experiences and insights. For this to work, we need your help.

What we're looking for: 

  • Your real name — first and last. 
  • Direct responses to the article. Tell us how your experience relates to the story.
  • The truth. Share factual information that adds context to the reporting.
  • Thoughtful answers to questions raised by the reporting or other commenters.
  • Tips that could advance our reporting on the topic.
  • No more than three comments per story, including replies. 

What we block from our comments section, when we see it:

  • Pseudonyms. WyoFile stands behind everything we publish, and we expect commenters to do the same by using their real name.
  • Comments that are not directly relevant to the article. 
  • Demonstrably false claims, what-about-isms, references to debunked lines of rhetoric, professional political talking points or links to sites trafficking in misinformation.
  • Personal attacks, profanity, discriminatory language or threats.
  • Arguments with other commenters.

Other important things to know: 

  • Appearing in WyoFile’s comments section is a privilege, not a right or entitlement. 
  • We’re a small team and our first priority is reporting. Depending on what’s going on, comments may be moderated 24 to 48 hours from when they’re submitted — or even later. If you comment in the evening or on the weekend, please be patient. We’ll get to it when we’re back in the office.
  • We’re not interested in managing squeaky wheels, and even if we wanted to, we don't have time to address every single commenter’s grievance. 
  • Try as we might, we will make mistakes. We’ll fail to catch aliases, mistakenly allow folks to exceed the comment limit and occasionally miss false statements. If that’s going to upset you, it’s probably best to just stick with our journalism and avoid the comments section.
  • We don’t mediate disputes between commenters. If you have concerns about another commenter, please don’t bring them to us.

The bottom line:

If you repeatedly push the boundaries, make unreasonable demands, get caught lying or generally cause trouble, we will stop approving your comments — maybe forever. Such moderation decisions are not negotiable or subject to explanation. If civil and constructive conversation is not your goal, then our comments section is not for you. 

Your email address will not be published. Required fields are marked *

  1. Interesting, how much in Federal subsidies, Farm Aid, etc. did you turn up for our Congressional delegation? I’d like to check it against my info and figures.

    Reply