If you prefer to see the charts in action before going through the details, visit HN Charts.
In my last hack, I created a process to visualize support/oppose statistics for Wikipedia black out discussion on Jimbo Wales user talk page. Next time I saw a Hacker News poll in the front page, an idea struck my mind – why not create a process to visualize data from the Hacker News polls? Having worked with HN Search API and the charting process, I thought it would be fairly easy for me to create this service. The project turned out to be more complicated but at the mean time more fun than I had originally anticipated.
Polls are currently in beta in Hacker News and I think for that reason poll data are not accessible from HN Search API. I wrote a quick e-mail to PG with the following details:
It appears that the Poll stats are not associated with the parent item
in the HN Search API:
- Parent item query does not include the Poll stats in the response (e.g. http://api.thriftdb.com/api.hnsearch.com/items/3420203-c2ed1 )
- Comments query for the parent item does not include the Poll stats either (e.g. http://api.thriftdb.com/api.hnsearch.com/items/_search?q=3420203-c2ed1&pretty_print=true)
I can imagine the volume of e-mails PG receives and was not anticipating a response but still sent an e-mail, just in case. I haven’t received a response yet.
That didn’t stop me from moving ahead. Afterall, I scraped the content in my previous hack using YQL. But then I found that HN is blocking YQL and Yahoo Pipes from accessing the stories in Hacker News. I ended up using another service for scraping that I will not disclose here to make it easier for HN to keep up with careless scrapers. I am caching data and using HN API / RSS feeds whenever possible; resorting to scrapes only when absolutely needed.
Once I built the visualization page for individual polls, I wanted to make it easier to visualize interesting polls. To do this, I added two new features i) a list of frontpage polls to visualize (using RSS) and ii) a list of best Hacker News polls (using HN Search API)
The end result is HN Charts, a service to visualize Hacker News polls.
The site is designed using Twitter Bootstrap and developed in PHP. The charts are rendered using the Google Visualization API. The charts pages are cached for ten minutes so poll results may be upto ten minutes old.
I have also placed social share buttons on the individual chart pages so that you can share your favorite polls.
Understanding the efforts of the HN community to prevent Eternal September, I decided not to place any links in the poll visualization pages to the parent poll. Your feedback will be highly appreciated.