May 29, 2012
One of the metrics the Workshop looks at when studying local news sites is their reach. How many people read the news they create? We believe reach is important because we believe that news sites serve a critical public purpose, and that readership may, albeit imperfectly, reflect the value a site holds in the eyes of its communities.
Having said that, though, while we believe understanding the reach of sites is important, it’s an imperfect item to measure at best.
In the first NEW News report, we relied on self-reported data as one criteria in our rankings algorithm. The good thing about relying upon self-reported data is that it allowed us to have data on every operator in the study, including smaller sites whose traffic was not captured by Alexa or Google. The bad thing is that it was self-reported, which generated a lot of questions about whether we were all measuring the same metrics in the same way or, in some cases, whether the data was accurate.
In the subsequent NEW News study, we have instead relied upon measures that are available to everyone, which leads to a different set of challenges.
- Public estimates of site traffic, from Quantcast, Alexa or Compete.com, often drastically undercount traffic to smaller sites. Because they are based on a small subset of overall website traffic, they tend to have accurate numbers for larger sites and less accurate numbers for smaller sites. In addition, some sites code their sites for better Quantcast data, while others do not allow their Quantcast data to be displayed to the public.
- Public proxies of site traffic, such as the number of RSS subscribers in Google Reader, or the number of Facebook likes or Twitter followers, represent the actions of a subset of visitors. They may be more likely to be regular readers. But it’s still not everyone, and we don’t have a consistent way to say that a site with, perhaps, 1,000 Facebook likes actually has 10,000 monthly unique visitors.
At this point, we intend to pull analytics or traffic ranks from Quantcast, Alexa and Compete.com, as well as the number of RSS subscribers in Google Reader, Facebook likes, and followers of an organization or editor’s Twitter account, and use those numbers to create a scoring system for reach. (For consideration, the Facebook and Twitter accounts must be readily available from the organization’s home page.) For sites such as Compete.com that present data over time, we will likely create an average of the prior six months of traffic.
What do you think of our ranking options? What would you add or delete, and what would you be concerned about? Please share your comments in the comment field below.
Online sites we should include in our research? Comment here.
Interested in commenting on overall project? Comment here.