Cobweb Scripts Data

Home Programs Tools Data About

thinkorswim End of Day Option Data

Background

FREE end of day option data is hard to come by. In a world where stock data is given out like candy, option data remains hidden behind paywalls. Before I started this project, I scoured the internet to see if anyone had already compiled free option data. While I did find some (they are listed at the bottom of the page under Alternative Sources), I found that even the best sources were limited to recent times. There was only one place that retail traders could get option data that stretched to over 20 years ago: thinkorswim.

There was, however, one problem: data can only be exported one day at a time. Going back to the internet, people on subreddits had come to the same conclusion: the process was too slow, with some posts talking about how it took days to export one ticker’s end of day option data. Despite that, I attempted the process myself. I tried reading network data, but from what I could tell, the data was encrypted until it reached the thinkorswim application. I only have a basic, conceptual idea of network security, so I did not find it feasible to figure out how to perform a man-in-the-middle interception to easily grab the data. This left me with automating the download process through thinkorswim itself. At a cursory glance, you would assume an auto clicker or a simple macro would be enough to export the data. This turned out not to be the case. There are so many caveats and exceptions regarding thinkorswim’s interface that it effectively acts as a deterrent against this sort of GUI automation. Eventually, after several months, I created a proof of concept that was robust enough to export data, which leads us to here.

Broker Data Weakness

When talking about option data, a point comes up about the quality and thinkorswim data is no exception. For groups, such as hedge funds, looking at option data is related to seeking alpha. They are attempting to use the data to enhance a model or build a new model to take advantage of what they believe to be unrealized value. But why is broker data considered low quality for this purpose? Well data must be usable and representative of real-life circumstances. End of day option data from brokers usually represent the final snapshot of the last trades before the closing bell; however, data for modeling option prices may need to be some time before the closing bell to represent real life liquidity and prices. On top of that, brokers do not particularly care to capture the data itself as accurately as possible. You may have data that has not been properly adjusted, data that has missing trades, data that may just be flat out inaccurate, or dirty and unprocessed.

Despite all these weaknesses, I believe retail traders still deserve a chance to glance over something before opening their wallets. Even if it is not the best, at least they have an opportunity to peek before taking the next step.

thinkorswim Specific Weaknesses

Sources:

  1. Yahoo Finance - META May 30, 2012
  2. companiesmarketcap.com - NVDA
  3. Yahoo Finance - NVDA January 2, 2003

thinkorswim Data Format and Specification

I did not include anything thinkorswim considers a holiday or weekend. In other words, all data provided here is considered a trading day, by thinkorswim’s standards.

The follow data was extracted in the csv file (they may not be filled in every file):

Feedback

Please feel free to get in touch with me using one of the accounts on the About page. I am open to requests!

Dataset

Links to the data, the range exported (format of [MM/DD/YYYY]), and notes about the data are available below. All tickers follow thinkorswim notation.

Indices

ETFs

Stocks

Alternative Sources