Shimmer is an application that makes it easy to pull health data from popular third-party APIs like Runkeeper and Fitbit. It converts that data into an Open mHealth compliant format, letting your application work with clean and clinically meaningful data.

We currently support the following APIs

• Fitbit
• Google Fit
• Jawbone UP
• Misfit
• RunKeeper
• Withings

And the following APIs are in the works

• FatSecret
• Ginsberg
• iHealth
• Strava

This README should have everything you need to get started. If you have any questions, feel free to open an issue, email us, post on our form, or visit our website.

• Overview
  • Shims
  • Resource server
  • Console
• Installation
  • Option 1. Download and run Docker images
  • Option 2. Build the code and run it natively or in Docker
• Setting up your credentials
• Authorizing access to a third-party user account
  • Authorize access from the console
  • Authorize access programmatically
• Reading data
  • Read data using the console
  • Read data programmatically
• Supported APIs and endpoints
• Contributing

Shimmer is made up of different components - individual shims, a resource server, and a console - which are each described below.

A shim is a library that can communicate with a specific third-party API, e.g. Withings. It handles the process of authenticating with the API, requesting data from it, and mapping that data into an Open mHealth compliant data format.

A shim generates data points, which are self-contained pieces of data that not only contain the health data of interest, but also include header information such as date of creation, acquisition provenance, and data source. This metadata helps describe the data and where it came from. The library is called a shim because such clean and clinically significant data in not provided natively by the third-party API.

The resource server exposes an API to retrieve data points. The server handles API requests by delegating them to the correct shim. As more and more shims are developed and added to the resource server, it becomes capable of providing data points from more and more third-party APIs. The resource server also manages third-party access tokens on behalf of shims.

The console provides a simple web interface that helps users interact with the resource server. It can set configuration parameters, trigger authentication flows, and request data using date pickers and drop downs.

There are two ways to install Shimmer.

1. You can download and run pre-built Docker images.
2. You can build all the code from source and run it natively or in Docker.

If you don't have Docker, Docker Compose, and Docker Machine installed, download Docker Toolbox and follow the installation instructions for your platform. If you don't have a running Docker machine, follow these instructions to deploy one locally, or these instructions to deploy to the cloud on any of these cloud platforms.

Once you have a running Docker host, in a terminal

1. Clone this Git repository.
2. Run docker-machine ls to find the name and IP address of your active Docker host.
3. Run eval "$(docker-machine env host)" to prepare environment variables, replacing host with the name of your Docker host.
4. Run the ./update-compose-files.sh script.

• This step should be removed once Compose 1.5 is released.

1. Download and start the containers by running

• docker-compose up -d
• If you want to see logs and keep the containers in the foreground, omit the -d.
• This will download up to 1 GB of Docker images if you don't already have them, the bulk of which are MongoDB, nginx and OpenJDK base images.
• It can take up to a minute for the containers to start up. You can check their progress using docker-compose logs if you started with -d.

1. Visit http://<your-docker-host-ip>:8083 in a browser.

If you prefer to build the code yourself,

1. You must have a Java 8 or higher JDK installed. You can use either OpenJDK or the Oracle JDK.
2. You technically don't need to run the console, but it makes your life easier. If you're building the console,
3. You need Node.js.
4. You need Xcode Command Line Tools if you're on a Mac.
5. To run the code natively,
6. You need a running MongoDB instance.
7. To run the code in Docker,
8. You need Docker, Docker Compose, and Docker Machine, available in Docker Toolbox.
9. You need a running Docker host. If you don't have a running Docker host, follow these instructions to deploy one locally, or these instructions to deploy to the cloud on any of these cloud platforms.

If you want to build and run the code natively, in a terminal

1. Clone this Git repository.
2. Run the ./run-natively.sh script and follow the instructions.
3. When the script blocks with the message Started Application, the components are running.

• Press Ctrl-C to stop them.

1. Visit http://localhost:8083 in a browser.

If you want to build and run the code in Docker, in a terminal

1. Clone this Git repository.
2. Run docker-machine ls to find the name of your active Docker host.
3. Run eval "$(docker-machine env host)" to prepare environment variables, replacing host with the name of your Docker host.
4. Run the ./run-dockerized.sh script and follow the instructions.

• The containers should now be running on your Docker host and expose port 8083.
• It can take up to a minute for the containers to start up.

1. Visit http://<your-docker-host>:8083 in a browser.

If you can't run the Bash scripts on your system, open them and take a look at the commands they run. The important commands are marked with a "#CMD" comment.

You need to obtain client credentials for any shim you'd like to use. These credentials are typically an OAuth client ID and client secret, and you can generate them from the developer website of the third-party API. Visit the following links to register your application and obtain authentication credentials for each of the shims you want to use.

• Fitbit
• Google Fit (application management portal)
• Jawbone UP
• Misfit
• RunKeeper (application management portal)
• Withings

If any of the links are incorrect or out of date, please submit an issue to let us know.

Once credentials are obtained for a particular API, navigate to the settings tab of the console and fill them in.

If you didn't build the console, uncomment and replace the corresponding clientId and clientSecret placeholders in the application.yaml file with your new credentials and rebuild.

The data produced by a third-party API belongs to some user account registered on the third-party system. To allow a shim to read that data, you'll need to initiate an authorization process. This process lets the user account holder explicitly grant the shim access to their data.

To initiate the authorization process from the console,

1. Type in an arbitrary user handle. This handle can be anything, it's just your way of referring to third-party API users.
2. Press Find and the console will show you a Connect button for each API with configured authentication credentials.
3. Click Connect and a pop-up will open.
4. Follow the authorization prompts. You should see an AUTHORIZE JSON response.
5. Close the pop-up.

To initiate the authorization process programmatically,

1. Make a GET request to http://<host>:8083/authorize/{shim}?username={userId}

• The shim path parameter should be one of the names listed below, e.g. fitbit.
• The username query parameter can be set to any unique identifier you'd like to use to identify the user.

1. Find the authorizationUrl value in the returned JSON response and redirect your user to this URL. Your user will land on the third-party website where they can login and authorize access to their third-party user account.
2. Once authorized, they will be redirected to http://<host>:8083/authorize/{shim_name}/callback along with an approval response.

A shim can produce JSON data that is either normalized to Open mHealth schemas or in the raw format produced by the third-party API. Raw data is passed through from the third-party API. Normalized data conforms to Open mHealth schemas.

The following is an example of a normalized step count data point retrieved from Jawbone:

{
    "header": {
        "id": "243c773b-8936-407e-9c23-270d0ea49cc4",
        "creation_date_time": "2015-09-10T12:43:39.138-06:00",
        "acquisition_provenance": {
            "source_name": "Jawbone UP API",
            "modality": "sensed",
            "source_updated_date_time": "2015-09-10T18:43:39Z"
        },
        "schema_id": {
            "namespace": "omh",
            "name": "step-count",
            "version": "1.0"
        }
    },
    "body": {
        "effective_time_frame": {
            "time_interval": {
                "start_date_time": "2015-08-06T05:11:09-07:00",
                "end_date_time": "2015-08-06T23:00:36-06:00"
            }
        },
        "step_count": 7939
    }
}

To pull data from a third-party API using the console,

1. Click the name of the connected third-party API.
2. Fill in the date range you're interested in.
3. Press the Raw button for raw data, or the Normalized button for data that has been converted to an Open mHealth compliant data format.

To pull data from a third-party API programmatically, make requests in the format

http://<host>:8083/data/{shim}/{endPoint}?username={userId}&dateStart=yyyy-MM-dd&dateEnd=yyyy-MM-dd&normalize={true|false}

The URL can be broken down as follows

• The shim and username path variables are the same as above.
• The endPoint path variable corresponds to the type of data to retrieve. There's a table of these below.
• The normalize parameter controls whether the shim returns data in a raw third-party API format (false) or in an Open mHealth compliant format (true).

N.B. This API will be changing significantly in the near future to provide greater consistency across Open mHealth applications and to improve expressivity and ease of use. The data points it returns will not be affected, only the URLs used to request data and perhaps some book-keeping information at the top level of the response.

The following is a table of the currently supported shims, their endpoints, and the Open mHealth compliant data that each endpoint can produce. The values in the shim and endPoint columns are the values for the parameters of the same names used in programmatic access of the API.

The currently supported shims are:

¹ The Fitbit API does not provide time zone information for the data points it returns. Furthermore, it is not possible to infer the time zone from any of the information provided. Because Open mHealth schemas require timestamps to have a time zone, we need to assign a time zone to timestamps. We set the time zone of all timestamps to UTC for consistency, even if the data may not have occurred in that time zone. This means that unless the event actually occurred in UTC, the timestamps will be incorrect. Please consider this when working with data normalized into OmH schemas that are retrieved from the Fitbit shim. We will fix this as soon as Fitbit makes changes to their API to provide time zone information.

² Body weight, body mass index, and heart rate mappers have not been tested on real data from Jawbone devices. They have been tested on example data provided in Jawbone API documentation. Please help us out by testing Shimmer with real-world data of one of these types from a Jawbone device and letting us know whether or not it works correctly.

³ Uses the daily activity summary when partner access is disabled (default) and uses intraday activity when partner access is enabled. See the YAML configuration file for details. Intraday activity requests are limited to 24 hours worth of data per request.

⁴ Sleep data has not been tested using real data directly from a device. It has been tested with example data provided in the Withings API documentation. Please help us out by testing real-world Withings sleep data with Shimmer and letting us know whether or not it works correctly.

The list of supported third-party APIs will grow over time as more shims are added. If you'd like to contribute a shim to work with your API or a third-party API, or contribute any other code,

1. Open an issue to let us know what you're going to work on.
2. This lets us give you feedback early and lets us put you in touch with people who can help.
3. Fork this repository.
4. Create your feature branch from the develop branch.
5. Commit and push your changes to your fork.
6. Create a pull request.