What do web stats mean, anyway?
September 17th, 2007 | Published in Nonprofit Tech, Nonprofits | 10 Comments
There is an interesting discussion happening between Holly Ross, soon to be ED of NTEN, and Allan Benamer, about web statistics, and whether or not nonprofits should be “transparent” and publish their web statistics. Allan’s argument is that because NTEN is in a leadership position in the field, it should lead in showing transparency by publishing its web stats. And, he thinks that NTEN should be responsive to him, as a member, in asking to publish web stats. Holly’s argument is, basically that web stats don’t equal accountability.
The question I want to ask is, what do web stats really mean, anyway? For organizations, web stats are useful indicators of how many people are being reached by their message, the geographical spread of the visitors and whether or not a specific campaign was successful in driving traffic or creating actions (like donations, or letters, etc.) It is an internal assessment tool which helps organizations figure out what parts of their online strategy are working, and what parts might need tweaking.
As some sort of measure of accountability, raw web statistics (this site got x visits and y pageviews in t timeframe) mean zilch. Nothing. Nada. Just because organization 1 gets 45,000 unique visits per month, and organization 2 gets 3,000 means nothing in relationship to the impact that organization has in the world, or in relationship to how it uses its resources. Organization 1 could be spending all of its money on its web presence, and none on its mission, organization 2 could be doing just the opposite. And the mission of the organization matters too. Even for NTEN, which is extremely web-heavy in its mission, raw visit numbers will mean nothing related to how well it is doing its job.
The idea that web stats = some measure of nonprofit accountability is a result of a mindset that suggests that web presence should be the central part of a nonprofit’s communications strategy, and that raw numbers of visits has some relationship to how well a nonprofit works. For some organizations with some limited kinds of missions, this may be the case. But for the vast majority of nonprofits out there, web strategy is a small part of their communications strategy, and the numbers of people that visit their site bears little or no relationship to how well they do their work, or what they do. And, I actually hope that doesn’t change. I don’t think we want homeless shelters, food pantries, mental health organizations, etc., to care a whole lot about how many hits they got in comparison to similar (or different) organizations. I don’t want to start a race to the top of the Nonprofit 25 - where organizations start spending more time worrying about their position on that list, and less time feeding people.
Allan says:
Granted, web site stats will not tell me anything about how many hungry people a nonprofit feeds. How odd is it then to teach Google Analytics to nonprofit techies but then say that site statistics had nothing to do with a nonprofit’s mission? Why bother having a web site at all? Properly used, web sites are more than just a payment solution for credit card bearing donors. They can be used for a nonprofit’s mission and that is why nonprofits should exercise transparency on web site analytics.
How does giving resources to nonprofits to help them understand how to use web stats to do internal assessment of web strategy inconsistent with choosing not to publish raw web stats? Asking NTEN to show leadership by publishing web stats is to suggest that NTEN would think that publishing web stats is a useful measure of nonprofit accountability. Holly doesn’t think so, and I don’t either.
Tags:googleanalytics nptech nten webstatistics
September 18th, 2007 at 12:00 am (#)
I’m frankly very puzzled by your argument, Michelle. Your site, zenofnptech.org, is already on Quantcast. It’s been there since 9/7/2007, two days after my call for getting people on Quantcast.
And in the interest of site analytics fairness, both nosi.net and zenofnptech.org’s site stats are available at:
http://siteanalytics.compete.com/zenofnptech.org+nosi.net?metric=uv
I think the notion that you can actually control your site stats seems to be a bit outmoded though. As you can see, either you get to define your site stats or it will be defined for you. Obviously, with zenofnptech.org, you’ve decided to take control of the site stats for yourself. I don’t quite understand why you won’t give the same advice to Holly.
It’s a terrible line of argument to assume that a focus on web site statistics and feeding people are incompatible. For one thing, feeding people is not what NTEN does. Secondly, I certainly saw that over the five years I was the IT Director at the Coalition for the Homeless that our clients were increasingly interested in initiating and maintaining electronic communications with our case workers. It’s fairly easy to extrapolate that even with the poorest of clients, we’re seeing web adoption especially as traditional sources of information for job and apartment listings leave print media. You cannot use the digital divide as a fig leaf over open site statistics anymore. That dog won’t hunt.
I believe that a web site will eventually become a major hub of communications even for direct service organizations. And in your case, as the head of a nonprofit open source organization, I think you’re going to find that not publishing site statistics will cause you more grief than you think.
September 18th, 2007 at 5:48 am (#)
Sure, I put Zen and the Art of Nonprofit Technology on Quantcast - I was curious to see what would happen. It wasn’t meant as a statement about what I think about whether or not web stats are a useful metric of much. Besides, the sole mission of this blog is … well, a blog. :-)
As far as NOSI is concerned, NOSI might indeed be one of those small minority of nonprofits where the web site metrics do in fact have some measure of relationship to our mission. And it doesn’t bother me that it’s on quantcast.
I’m not suggesting that nonprofits should *ignore* site statistics. And I’m not suggesting that it’s necessarily possible, especially as site stats become more public info, that organizations can, or should control stats. I’m just saying that for the most part, stats don’t mean much at all about mission, and NTEN should not be taken to task for not showing “leadership” in this arena - since in my opinion, this particular form of leadership isn’t necessarily useful.
September 18th, 2007 at 7:32 am (#)
My take on the discussion so far is that there is a real difference in opinion on the importance of a web site to an organization. Allan is convinced, rightly or wrongly that an organization’s web site will become the hub of their activity; that their mission will be focused through the web site and that all their constituents will make use of the web site as a vehicle for interacting with the organization. As such their web stats are meaningful to those who want to make an informed decision as to which organization they invest their time and buy into.
Allan assumes two points here. Firstly that the more visits a site has the better job they are doing. If a website is the hub or an organization’s activity (a big if) then does it necessarily follow that just because the masses go there the organization best serves my needs?
The second point is the assumption that the web site will become the hub of an organization’s mission. In many cases this may happen. Certain organizations will, if they have not already, use the internet for this purpose. But in many areas there will always be scope for offline delivery of a non-profit’s message. There will always be constituents who need the offline presence, just in the same way there are still shops and banks on the high street.
Even if Allan’s vision does become reality some time in the future, why rush to present your figures now. This is not the reality now. By opening up one set of figures you are giving the world a skewed interpretation of the success of a mission. This could ultimately be detrimental to the very people you are trying to help.
David
September 18th, 2007 at 9:54 pm (#)
I strongly recommend that all nonprofits track site stats to help evaluate the effectiveness of their online communications, but unless there is a specific, clear benefit to releasing a particular site stat, I don’t see the reason why a nonprofit should be compelled to share all the data. If they want to brag about an accomplishment (a certain number of page views or visitors to a particular campaign page, for example), that’s fine.
To me, it’s more akin to personnel performance reviews. One could argue that a board’s performance review of its executive director is an indication of organizational performance, but you don’t see that kind of internal data shared (I know that’s a legal issue, but it’s the same kind of data in my mind.) The same people that want all your site stats made public probably want to know the salaries and benefit packages of the employees too, and they aren’t entitled to that either.
September 18th, 2007 at 11:50 pm (#)
For an added prespective on web stats try out some of the features from GoStats. Notably, you may find some of the advanced stats features help a great deal in understanding more about the web stats and what is really happening with your site.
September 19th, 2007 at 8:21 am (#)
What a great conversation; fun to have but for the fact that it misses the point. Webstats are, of course, important, both as an internal metric to watch, analyze, and improve, and for that matter, to put out there, in conjunction with lots of other data. No one disputes that. Our 990 is publicly accessible; you know how many people are in our online community at groups.nten.org; and we are committed to being accountable, so no skin off our back to publish most other data as well. What both Holly and Michelle are saying is that webstats ALONE do not say much, and that Allan’s argument is a red herring.
We here at NTEN have pretty extensive metrics that we track every week in a dashboard that most nonprofits can only dream about. We are data-heavy, analyze everything we do in terms of numbers and impact, and look at what makes the most sense for us to do in that context — where do we put our still-small resources most effectively to a. be of real value to the field and our members whom we serve first and foremost, and b. to help us be a sustainable and healthy organization.
Webstats alone do not tell the story and to look at them without some knowledge about what we do and how we do it, is silly, as even Allan would concede. Other metrics we look at include: What are our sign-ups for our newsletter? What content does the best? How many people sign up for which webinars? How many people are in our online community and where do they participate (and where not?) What are our sign-ups for new members, for renewal membership? Where do they come from and what messages and services do they connect to? How many people are reading our materials in their RSS readers? How many people take advantage of discounts that we offer? What do they say in surveys that we conduct? What are our registrations for conferences? What are the trends on all of the above? Where do we under-perform and can do better? What topics resonate with members most?
These are just some of the data we look at to assess the needs, health, and growth of our community, and more importantly, whether we are serving our members well.
Webstats are a piece of that — are we communicating our message well on our site and in other collateral and messaging — that we look at closely and always try to improve on. Looking at them in isolation without all the other data and the trends and information they entail makes no sense. Our dashboard rocks and we would be happy to share it widely — as we think that people SHOULD indeed be tracking all of their data as vigorously.
There is one additional point I want to make: On some topics that may not have wide resonance yet, there is, by definition little traffic.
I can tell you that if I publish a “How to Use Google Analytics” screencast (as we did) it will get a ton more traffic than a thoughtful opinion piece on why vendors should have (like this one: http://www.wesabe.com/page/security) a “data bill of rights” for customers. The latter is not as applied, not as immediate, and in the short-term not as usable to our members. It WILL get less traffic, though likely very important traffic from other bloggers and organizations who will pick up the idea (that would be called ‘organizing’). That, however, WILL not be reflected in our webstats for a while, if at all.
But, in the scheme of things, the latter topic is far more important in the long run to the more than 1 million nonprofits out there than how to use a web tool. Webstats alone only tell a tiny fraction of that story, if at all.
So, Michelle and Holly are right to argue for context. Organizing a community and serving it well has a lot of dimensions poorly reflected in webstats alone. Allan, for some reason, is out to pick a fight that is completely misleading (though surely driving his traffic up). But then again, demagoguery always is.
September 21st, 2007 at 1:23 pm (#)
well said Katrin. I signed up for quanticast just for the heck of out - mostly idle curiosity. But I agree with your:
“So, Michelle and Holly are right to argue for context. Organizing a community and serving it well has a lot of dimensions poorly reflected in webstats alone. Allan, for some reason, is out to pick a fight that is completely misleading (though surely driving his traffic up). But then again, demagoguery always is.”
I think that metrics to answer specific questions related to outcomes or improvement are useful. It think top 25 visitor lists or hit lists, particularly in a nonprofit community are pretty useless without context. So, my blog has three times as many monthly visitors than Allans (hmm.. I don’t pick fights, either). So, what does that mean?
Nothing!
September 21st, 2007 at 6:32 pm (#)
So here’s my thinking:
1. You can’t control web site stats. They start conversations you can’t stop, just like this one. If you don’t see the analogy to blogging or any adoption of social media, then you’re blind.
2. You can “withhold” web site stats but Compete, Alexa, GetClicky and a host of other sites will not let you do so.
3. This leaves you with a choice, either you embrace the openness of web site statistics or stick your head in the sand.
I could easily take the “website stats are private” argument and easily overlay them with every nonprofit exec’s resistance to adoption of social media. I’m surprised you’re all so very invested in that style of argumentation, even to the point of calling me a demagogue. It’s quite disappointing.
And this argument: “there are better stats somewhere out there, but we’re not going to show you” is really just not an appropriate argument to make. This is the nonprofit sector, we have nothing to hide but the good work we do. Transparency isn’t just something you say you have, it’s something that should extend through every strand of your nonprofit’s DNA.
Just be aware that when you demand openness from Blackbaud, Convio or any other of the CRM software vendors — that you don’t put yourself in a position where they are still more transparent than you are. It’s called leading by example, try it, you might like it.
September 22nd, 2007 at 7:52 pm (#)
@Allan:
Leading by example is great. I actually deeply believe in that. The question is - what example are you setting? Nobody said anything about trying to control or hide webstats - I don’t understand why you keep beating that dead horse.
The point is - the example you’re suggesting isn’t about greater openness - it’s providing a useless statistic out of context. No one in this conversation has their head in the Web 2.0 sand.
Holly’s latest post makes the whole thing quite clear.
September 26th, 2007 at 11:31 am (#)
Web stats are intended to be an internal control and analysis tool. The marketing people use them to improve strategies, the tech people use them to find broken stuff, the management selectively uses them to build external support, etc.
If made publicly available, who would actually go looking for them? Donors? (”Gee, I don’t think I’ll donate to that org unless they’ve got at least 5 million web hits a year.”) Never happen. Other nonprofits, nonprofit webmasters/marketing staff in particular are the only people I can imagine going after these numbers. And that essentially would boil down to a big old pissing contest. (”Yes! We got twice as many visitors this month as (insert-competitive-org-name-here)!”) The people who care about an organization’s web site statistics are the people in that organization.
On a related note, I think a benchmarking study might be a useful exercise for nonprofits. It’s difficult to tell what your web stats mean when you have no comparative data. I think an anonymous survey of NPOs, gathering and analyzing select critical web stats, might be an informative read.