Background

Open chrome to google news homepage, receiving constant email newsletters, multiple browser tabs always open, too many news aggregator apps to keep track of - does this sound familiar? This was my problem.

I love reading and browsing the internet, and I am eager to subscribe to new blogs and sites I find interesting and could learn from. But despite the numerous news aggregators or similar services I used, not a single one supports ALL the different sites I wanted to subscribe to. And as nice as mobile friendly email newsletters are, it would be much nicer to have them standardized and viewable in a single easy to read dashboard.

Enter my search for an open sourced, customizable, and embeddable, RSS feed reader.

Since creating this site I have wanted to use it as a central repository for public facing notes, and information I could easily access from anywhere. And if I am already here regularly looking for my notes, why not create the homepage as a go to place for my news and internet updates?

Objective: Create a simple and good looking dashboard to display RSS content from any source.

Features I wanted:

  • completely customizable
  • use Bootstrap 4 cards
  • support any RSS feeds
  • mobile friendly

TL;DR

Skip down to the Development section to simply steal my code.

Requirements

Due to this site being hosted on GitHub Pages, no server-side processing is supported. Therefore, any additional processing, like getting RSS feeds, parsing and printing will need to be in client-side scripts.

This means a pure JavaScript solution is needed, with no external libraries (e.g. node.js)

Research

Things I learned:

  • an RSS feed is simply XML data
  • natively requesting data from another domain is prohibited
  • not all RSS data is the same

Data Type

Despite all being XML, there are many forms of what feeds look like (e.g. Atom, RSS, Feedburner etc.) and they are all slightly different in naming convention and structure.

Looking into parsing XML in pure JavaScript looks like it could lead to more problems.

The simplest data to process would be JSON, especially when datasources have different keys and tree structures. It would be nice to be able to standardize all feeds into a similar JSON object.

Cross-Domain Origin Problem

The same-domain policy disallows any cross-domain requests.

Common solutions to this problem is to add a proxy to the request, or use an external API or extension which is allowed to make cross-domain requests.

Some examples of the proxy method are shown in this stackoverflow answer.

The jQuery docs from 2015 show an example using the YQL API in an Ajax call. So it seems like this may be a recommended solution.

Inconsistent Data

During the parsing of feeds process, I found out the JSON dictionaries were really not consistent. Some elements were values, some arrays, and some more objects. Therefore in the final function there are many checks for identifying the type of an object first (e.g. author) and then finding the necessary information based on it (e.g. author, author.content, or author.name etc.).

Solution

I decided to utilize an external API to parse the XML and return standardized JSON objects. This allows for consistent data to work with and removes a lot of work.

The dashboard will be built using Bootstrap 4 Cards formatted with the .card-columns class to be able to dynamically add as many feeds as I want.

Since I knew I wanted to include a Google News feed, because they already aggregate so well, I made sure to follow what guidelines I could find and standardize based on these rules (e.g. citations, links to original, etc.).

Dependencies

Unfortunately the Google Feed API has been deprecated as of December, 2016. This seems to have been a good solution and many people are looking for an alternative.

The dependency used is the Yahoo Query Language API.

Development

The solution I developed was inspired by this code snippet.

Container

Create a container where the feed cards will live using Bootstrap class card-columns. During the build function below, a Bootstrap card will be built for each feed and appended to the DOM inside the #feeds div.

<div id="feeds" class="card-columns">
  <!-- feeds get appended here -->
</div>

Build Function

The Gist below contains the worker function to build a card for each feed and append it to the DOM.

I reuse a few maker functions when building card header, block, and footer. The basic logic if for each feed url is:

  • make query, get JSON
  • dynamically get content
  • build a string of html elements that represent the card
  • append the card string to the container
  • repeat

Feeds List

Create a dynamic array of the RSS feeds you want to follow. Some example feeds are presented below.

feeds = ['https://news.google.com/news?topic=tc&output=rss', // google news - technology
         'http://feeds.feedburner.com/GDBcode', // google developer blog
         'http://feeds.feedburner.com/blogspot/gJZg', // google research blog
]

Get Feeds

A function to loop thru each feed and run the build function.

function get_feeds() {
  $('#feeds').empty();
  feeds.forEach(function (url, i) {
    buildFeedCard(url, '#feeds', i);
  });
};

Run on Page Load

Call the function on page load.

window.onload = function() {
  get_feeds();
};

Refresh Button

Alternatively to on page load, we can create a refresh button to manually re-pull the feeds and build the cards.

<div class="d-flex justify-content-end"><button type="button" class="btn btn-primary btn-sm" onclick="get_feeds()">Refresh</button></div>

Implementation

The final implementation looks something like the following image. You can check out my source code to see how exactly I manage the scripts on this site’s footer.

Custom RSS Reader