Ruby gem for stock / finance data

A post on the yahoo finance / stock gem had been overdue for a long time. Last week I added the xml support for the result formatting and published a new version of the gem. So I thought its a good time to write a bit about how I got started with it, implementation details, usage, etc.

Problem

My mother has a portfolio of stocks and it became a bit difficult for her to keep an eye on their performance either by using online or print medium. So she beautifully used her powers of delegation and asked me to check them regularly :). As usual, first thought, use google or yahoo finance for this kinda stuff but soon I wanted to generate more data for the stocks and I found that they lack them. I said, what the hell and built a rails application that would go to a stock exchange site and scrape the pages. Then I thought, an app like this could be useful for other people like me but the problem was, the site site I was using for scraping was country specific. I decided to make it a bit more general purpose before putting on the github and started using ScrAPI gem (ScrAPI uses CSS selectors for scraping) to scrape data from a site that has stocks from more than just one country. ScrAPI was working fine but wasn’t consistent in some cases. So I started looking for finance APIs, obviously google has a finance API but you need an account with google finance to use it. I think that’s a bit inconvenient for user’s to first have a google finance account to use a library though they may already have but that’s still a hassle. So started looking elsewhere and found yahoo finance. Best thing about yahoo finance is you don’t need any account to use it. And yahoo provides data as CSVs and its easy to use YQL (Yahoo Query Language). Hence, I decided to use Yahoo finance for the data purposes and write a ruby gem that can be used in any ruby or rails application. On another note, I met a couple of awesome yahoo guys at barcamplondon7 who showed me some cool APIs; Developer Apps like, open tables, YAP, blueprint etc,. Check them out here

This is how I got started on nas-yahoo_stock gem.

Implementation

The tests for the gem code are written using rspec 1.2.2 and it has cucumber features for integration testing.

There are different interface classes (based on their responsibilities) that are derived from the base interface class. For example, the quote interface provides quotes for multiple companies in a single request with variable parameters, symbol interface submits request to a yahoo page and scrapes data using some regex rules, etc.

Then we have different result classes to easily present result data in the format we want them. This has been achieved by implementing the Strategy Pattern to switch easily between different result formats. Benefit: a single result interface for the YahooStock::Quote / YahooStock::History / YahooStock::ScripSymbol classes for getting and presenting data into different formats (the examples are given in the Usage section).

I kept one thing into consideration while building the gem was to send requests to yahoo only when it is absolutely necessary. For instance, the gem sends a request to yahoo only when the results method is called on an object like YahooStock::Quote / YahooStock::History / YahooStock::ScripSymbol and not during any other operation. Which is fine but what about calling results multiple times for the same data and format or to generate different formats for the same data? An easy solution for that is to implement the Observer Pattern. The observers in nas-yahoo_stock gem keep an eye on the data requested whenever the results method is called on the quote, symbol or history objects and if the data requested currently is different from the previous request then only the gem hits yahoo else it returns the previous result without hitting yahoo.

The source code for the gem is available on github -> nas-yahoo_stock

Usage

To use the library, start by:

gem sources -a http://gemcutter.org
sudo gem install nas-yahoo_stock

Once the gem is installed, we can go to the irb and

require 'rubygems'
require 'yahoo_stock'

Before finding the history or quote information about a company stock we will need the stock symbol for that company. YahooStock::ScripSymbol class gives us all the possible stock symbols for a company. For instance, if we want to find the stock symbol for Yahoo.
symbol = YahooStock::ScripSymbol.new(‘Yahoo’)
To get the results,
symbol.results.output
the above method call give us a string of all the values.
If we want to store the result in a file and come back to it later, then we can
symbol.results.store('file_name')
The results can be easily presented into different formats like
1. to return an array instead of a string use:
symbol.results(:to_array).output
2. to return a hash, use:
symbol.results(:to_hash).output
3. to return xml, use:
quote.results(:to_xml).output

Once we have the stock symbol, then the quote or history information about that stock can be retrieved easily. Lets get to the history part first.

To get a stock’s history, the YahooStock::History class needs to be initialized with a hash of stock symbol, start and the end date for which we need the stock data

history = YahooStock::History.new(:stock_symbol => ‘yhoo’, :start_date => Date.today-20, :end_date => Date.today -2)
Then we can call all the same methods on the history object that we have called on the symbol object to get results, like
#to get history as an array
history.results(:to_array).output
#to return a hash, use:
history.results(:to_hash).output
#to return xml, use:
history.results(:to_xml).output
#to store results in a file, use:
history.results.store('filename')

To get quote information about a stock
1. Initialize quote object
quote = YahooStock::Quote.new(:stock_symbols => [‘YHOO’, ‘GOOG’])

If we call the results method on the quote object then it will give us only two values for the above two stocks which are included by default on object initialization. But a number of parameters can be passed to the quote object to get information on and these parameters can be viewed by sending valid_parameters message to the quote object.
quote.valid_parameters

To view the current parameters being used in the quote object
quote.current_parameters

The parameters are also categorised in realtime, standard and extended categories. To use parameters for these categories just pass one of the category names to the quote object and it will use those parameters, for e.g.
quote.realtime

To use all parameters just use:
quote.use_all_parameters

To view the current stock symbols used
quote.current_symbols

To add more stocks to the list
quote.add_symbols(‘MSFT’, ‘AAPL’)

To remove stocks from list
quote.remove_symbols(‘MSFT’, ‘AAPL’)

Then we can call all the same methods on the quote object that we have called on the symbol and history objects to get results, like
#to get quote as an array
quote.results(:to_array).output
#to return a hash, use:
quote.results(:to_hash).output
#to return xml, use:
quote.results(:to_xml).output
#to store results in a file, use:
quote.results.store('filename')

If you need the results in some other format apart from the formats provided by the gem then you can simply pass in a block to the result object like
quote.results{YourSpecialFormat.method_name}

Advertisements

Sphred.com – off internet

Sphred.com is off internet as I am not able to devote much time due to my other commitments.

But I still have all the data and code, so if someone wants to rerun it then email me or I just may put it on github when I have sometime.