.NET Core & the Google Analytics Real Time Data API

At Wehkamp, we use Google Analytics extensively to measure how many users are active. It is hard to correlate active users with your other statistics that "live" in Prometheus. In this blog I'll show how you can use the Google Real Time API to produce statistics for active users, page views and screen views.

A big shout-out to our Technical Web Analyst Byrge Leeuwangh for the collaboration on the setup.

1. Google setup

First, we need to set up a new project in Google Cloud Console and add a service account to it. Next we'll add the account to our Google Analytics view.

Google Cloud Console project

To access the Google Analytics API, we need to create a Google Cloud Console project first. Make sure you have a dev account with Google Cloud Console. The following steps will create a new project:

  1. Go to Google Cloud Console
  2. Create a new project
  3. Make sure the new project is selected in the top menu bar
  4. Menu > APIs & Services
  5. Click Enable APIs and Services button
  6. Search for the Google Analytics API and click the Enable button.

Add a service account

Now, let's create a service account that will be used in your application. A service account is a non-personal account that is attached to your project. You create one like this:

  1. Menu -> IAM & Admin > Service accounts
  2. Click Create Service Account button
  3. Enter a Service Account Name
  4. Click the Create button
  5. Click the Done button
  6. Copy the email address for later use
  7. Click on the created service account
  8. Click on the Keys tab
  9. Click on Add Key > Create new key
  10. Click on Create. This will download a JSON file that we'll use later on.

Add service account to Google Analytics

So, we've created a service account and downloaded the keys and copied the email address. Now, we need to link the service account to the Google Analytics view:

  1. Go to Google Analytics
  2. Navigate to the view you want to connect to your app.
  3. Click on the Admin button. This will open the settings showing the Account, Property and View properties.
  4. Click the View User Management item in the View panel.
  5. Click the big plus button and select Add users.
  6. Paste the email address of the service account you've copied.
  7. Make sure Read & Analyze is checked.
  8. Click the Add button.
  9. Close the User Management panel.
  10. Click the View Settings item in the View panel.
  11. Copy the View ID value for later use.

2. Notes on the Google Analytics API

We are interested in recording the following metrics:

NamePurpose
rt:activeUsersThe number of active users in the view.
rt:pageViewsThe number of pages that are being viewed. This only applies to web sites.
rt:screenViewsThe number of screen that are being viewed. This only applies to our Android and iOS app.
The metrics of the Google API.

Did you know you can try this out online? The docs have a nice 'Try this API' feature. Check the Real Time Data: get API docs.

Quotas

Beware of the Google Analytics API quotas:

What are the maximum quotas for Real Time Reporting API requests?
50,000 requests per project per day and 10 queries per second (QPS) per IP address.

Google Analytics Core Reporting API FAQs

In our case, we are doing 4 calls to 2 views. So if we do one run of calls every 15 seconds, we do:

  • 4*4=16 calls per minute
  • 16*60=960 calls per hour
  • 960*24=23040 calls per day, which is fine.

Consider creating multiple projects (per environment), so you don't hit your account limits (as we did). If you ever hit the quota; it will reset at midnight Pacific Time (PT).

3. .NET Core Implementation

Let's keep it simple and build a .NET Console App. Our implementation is based on this NuGet package from Google:

  • Install-Package Google.Apis.Analytics.v3 -Version 1.51.0.1679
  • dotnet add package Google.Apis.Analytics.v3 --version 1.51.0.1679
  • <PackageReference Include="Google.Apis.Analytics.v3" Version="1.51.0.1679" />

Replace the 98765432 number of the viewId variable with your own number. Copy the downloaded service account JSON file to the same directory as your application, rename it to credentials.json and make sure it is copied to the output directory.

using Google.Apis.Analytics.v3;
using Google.Apis.Auth.OAuth2;
using Google.Apis.Services;
using System;
using System.Threading.Tasks;

class Program
{
    async static Task Main(string[] args)
    {
        var credentialFilePath = "credentials.json";
        var credential = GoogleCredential.FromFile(credentialFilePath);
        var service = new AnalyticsService(new BaseClientService.Initializer()
        {
            HttpClientInitializer = credential.CreateScoped(
                AnalyticsService.Scope.AnalyticsReadonly
            ),
            ApplicationName = "google-analytics-real-time-api-in-net-core",
        });


        var viewId = "ga:98765432";
        var metric = "rt:activeUsers";
        var request = service.Data.Realtime.Get(viewId, metric);
        var response = await request.ExecuteAsync();
        var activeUsers = 0;

        if (response.Rows != null)
        {
            activeUsers = Convert.ToInt32(response.Rows[0][0]);
        }

        Console.WriteLine($"{activeUsers} active users.");
    }
}

Final thoughts

We've built a .NET Core API with a background service that polls the Google Analytics Real Time Data API. We're using Scrutor and decoration to convert the results to Prometheus metrics. We're plotting the metrics with Grafana. This will give us better insights during SRE.

And... we've added the Grafana dashboard to our DevOps Slack bot:

We're using our DevOps Slack bot to query Grafana views with the results of the Google Analytics Real Time Data API.
expand_less