Send subscriber count in user agent of feed crawler
The feed fetchers/crawlers of most web-based rss readers send the number of subscribed users in the UserAgent header when fetching the feed. This allows authors to track the overall subscriber count for their blogs. Please add such statistics to your crawler's UserAgent header.
Waldemar Kornewald
shared this idea
Done. Format: Mozilla/5.0 (compatible; theoldreader.com; 1 subscribers; feed-id=74898c0fee35cf8e1e9ba77f)
-
Jay Artz
commented
This is an important change. Need to be sure that publishers have access to information about their subscribers.
-
Garrett Guillotte
commented
Marco Arment called out The Old Reader on this today: http://www.marco.org/2013/07/10/post-google-reader-subscriber-stats
'Notably, AOL Reader, Digg Reader, and The Old Reader currently do not report subscribers. Please help me pressure them to add a subscriber count to their User-Agent strings — it’s important for publishers to know how many people are invisibly subscribed behind one-to-many crawlers like these.
The standard format is simply including “N subscribers” somewhere in the User-Agent.'
-
Waldemar Kornewald
commented
By the way, Google Reader also submits the feed ID. This is useful when your feed has moved (e.g., site.com/feed => FeedBurner) and you've setup redirects so all requests point to the same final URL. In our case, our blog gets multiple requests from Feedfetcher-Google with different feed IDs (one for site.com/feed, one for FeedBurner) and different subscriber counts. So, the total Google Reader subscriber count is the sum of both feed IDs' subscriber counts.
-
Waldemar Kornewald
commented
Many feed crawlers have a User-Agent like this:
FeedHQ/0.1.183 (https://github.com/feedhq/feedhq; 12 subscribers; https://github.com/feedhq/feedhq/wiki/User-Agent)
Feedfetcher-Google; (+http://www.google.com/feedfetcher.html; 1288 subscribers; feed-id=11050813093243668501)
livedoor FeedFetcher/0.01 (http://reader.livedoor.com/; 1 subscriber)
NewsBlur Feed Fetcher - -6 subscribers - http://www.newsblur.com (Mozilla/5.0 (Macintosh; Intel Mac OS X 10_7_3) AppleWebKit/536.2.3 (KHTML, like Gecko) Version/5.2)
There is no official specification, but the most common format seems to be "...; 12345 subscribers". Could you please add that to your User-Agent header, too?