Archive for July, 2015

Tangent: fitness

Friday, July 31st, 2015

About 4 months ago, I finally got my FitBit Charge HR and started using it to look at my fitness. I say “look at” because it has been just that – momentary looks, with no sort of history. I don’t like the way the FitBit web site presents the data – it confuses me and tries to make things too “candy-coated” – so I had to figure out a way to track trends myself. Oh, and before we go any further – this is not a review paid for by FitBit. This is just me telling you why I think having a FitBit and using it is a good idea – I’m getting zero benefit to writing this aside from the finger exercise involved in actually typing.

Fortunately, FitBit is really awesome about giving individuals access to their data through the API they have set up, and they’re also awesome about providing individuals access to the Partner API which allows access to intraday data. That was one of the major reasons I went with a FitBit instead of an Up3 from Jawbone – Jawbone says they allow access to your data, but in my testing of it, I couldn’t find a programmatic API and even the “data download” area of their web site only gave me data from a year ago, not current data.

So, I got myself access to the Partner API from FitBit, and started pulling down my personal data daily. I’ve only been doing this for about 7 days so far, so I don’t have very much in the way of trends yet, but it’s already started helping me understand some things about my habits. Since I’ve found it so useful, I figured I’d share what I’ve done in hopes that someone else will find it useful as well.

First things first – get yourself a FitBit. I chose the Charge HR because I wanted the intraday heart rate measurements, but I didn’t see the benefit to the location data the Surge provides. In hindsight, I probably could have made use of it, but it’s not something that I feel adds sufficient value to my analysis for the price differential. Once you have the FitBit – whatever model you end up getting – use it! No sense spending money on something that’s going to sit in your kitchen junk drawer.

Now that you have your FitBit, you need to open the door to downloading your data. This can get a bit confusing – it took me several tries to figure it all out – but stick with me here. Step one, register an application at https://dev.fitbit.com/apps/. I gave my app a name of “Personal” – the name doesn’t matter too much, it’s just something you have to put in. For this method, the OAuth 1.0 Application Type should be “Browser” and the OAuth 2.0 Application Type should be “client”. I used “http://localhost/callback” as the callback URL – this field has to be filled in, but for what we’re doing here, it doesn’t matter much what you put there. Once you’ve done that, send an email to “api@fitbit.com”  and request access to the Partner API for intraday data. Be sure to include the app’s client ID as given to you after registering the app ont he dev site. Please note – they are very supportive of personal use, but don’t try to slide a commercial application that you’ll be selling in claiming that you want access for personal use. That’s just bad form. It may take them a while to get to your request depending on volume – it took about 3 weeks for me to get Partner API access after my inital email.

Now that you have access, you need to set up the authentication key. FitBit has decent documentation for doing this on their site at https://wiki.fitibit.com/display/API, but this is where it got confusing for me. I’m only going to cover the OAuth 2.0 authentication bits, since that’s what you need for heart rate measurements and it’s a superset of what OAuth 1.0 gets you. Please note that as of when I write this article, OAuth 2.0 at FitBit is in beta state, so it might break without warning. Buyer beware, caveat emptor, and all that. We’ll be looking at the “Authorization Code Grant Flow” at https://wiki.fitbit.com/display/API/OAuth+2.0.

The instructions tell us to “redirect the user to FitBit’s authorization page”. This really confused me, since I hadn’t directed myself anywhere yet – ultimately, it means I have to poke a FitBit URL with a well-known set of URL parameters, which include the application’s client ID as given to you by the “Manage My Apps” page (https://dev.fitbit.com/apps). The easiest way to do this for now is to type the following into the location bar of your web browser: https://www.fitbit.com/oauth2/authorize?scope=activity+heartrate+location+nutrition+profile+settings+sleep+social+weight&response_type=code&client_id=${ID_HERE}

Replace the ${ID_HERE} with your app client ID. This page will try to redirect you to your callback URL, which if you use the values above won’t exist, so you’ll end up seeing a URL in your location bar with a “code=” part to it. Save the long string after the “code=” – this is the part you need for the next step.

Next, FitBit tells us the applciation needs to “exchange the authorization code for an access token”. This must be completed within 10 minutes, or the code we got expires and we have to start over. For this, the response will be in JSON so I used an interactive Python session. Here’s what I did:

$ python
>>> import requests
>>> import base64
>>> import urllib
>>> clientid='XXXXXX'
>>> secret='YYYYYYYYY'
>>> code='ZZZZZZZZZZZZZZZZZZZZZ'
>>> authStr = "Basic "+base64.b64encode("%s:%s" % (clientid, secret))
>>> authHdr = {'Authorization' : authStr}
>>> body=urllib.quote("clientid=%s&grant_type=authorization_code&code=%s" % (clientid, code))
>>> req = requests.post('https://api.fitbit.com/oauth2/token', headers=authHdr, data=body)

You’re probably asking, “So what does all this mess mean?” Well, it becomes a little more clear when you replace the XXX’s with the client ID from the FitBit API page and the YYY’s with the applciation secret from the same page. Then replace the ZZZ’s witht he code you got from your browser above.

Once this is done, dump the result of the request with:

>>> req.json()

This will show you the JSON notation for the request response. The important parts are the “access_token” and the “refresh_token” strings, so we’ll want to save those in another variable:

>>> access = req.json()['access_token']
>>> refresh = req.json()['refresh_token']

Now we want to save those two items to a file locally, since we’ll need both pieces of information in the future. The easiest way to do so:

>>> tok = {}
>>> tok['access_token'] = access
>>> tok['refresh_token'] = refresh
>>> with open ('.fitbitAuthFile', 'w') as fh:
...   json.dump(tok, fh)
...
>>>

Exit the interactive Python interpreter and confirm the “.fitbitAuthFile” file contains the access_token and refresh_token we just wrote to it. If it doesn’t, you’ll probably need to start the process over by going back to the web page to get a new code. If it does, congratulations, you’ve finished the hard part!

The actual retrieval of the data is both much simpler and much more complex. Simpler because we only have to read in the token information, test if it’s expired or not and if so refresh it, then ask for the data we want. More complex because this is where processing the data comes in to play. I’m saving data to spreadsheets through the openpyxl Python module. I haven’t finished developing the script or the spreadsheets, but you can download it in its current state from http://www/ncphotography.com/fitbitcollect.py. You’ll need to make some changes to insert the relevant values into places I’ve put generic all-caps strings, and please do keep in mind this was intended for a Linux (specifically, Fedora 21) system, not Windows. I don’t intend to make any changes to accommodate a Windows system either – I’m a linux systems administrator by trade and I don’t get along with Windows. If there’s enough interest, I’ll update it in the future and/or upload the weight tracking spreadsheet template I use.