At Wehkamp, we use Google Analytics extensively to measure how many users are active. It is hard to correlate active users with your other statistics that "live" in Prometheus. In this blog I'll show how you can use the Google Real Time API to produce statistics for active users, page views and screen views.
A big shout-out to our Technical Web Analyst Byrge Leeuwangh for the collaboration on the setup.
1. Google setup
First, we need to set up a new project in Google Cloud Console and add a service account to it. Next we'll add the account to our Google Analytics view.
Google Cloud Console project
To access the Google Analytics API, we need to create a Google Cloud Console project first. Make sure you have a dev account with Google Cloud Console. The following steps will create a new project:
- Go to Google Cloud Console
- Create a new project
- Make sure the new project is selected in the top menu bar
- Menu > APIs & Services
- Click Enable APIs and Services button
- Search for the Google Analytics API and click the Enable button.
Add a service account
Now, let's create a service account that will be used in your application. A service account is a non-personal account that is attached to your project. You create one like this:
- Menu -> IAM & Admin > Service accounts
- Click Create Service Account button
- Enter a Service Account Name
- Click the Create button
- Click the Done button
- Copy the email address for later use
- Click on the created service account
- Click on the Keys tab
- Click on Add Key > Create new key
- Click on Create. This will download a JSON file that we'll use later on.
Add service account to Google Analytics
So, we've created a service account and downloaded the keys and copied the email address. Now, we need to link the service account to the Google Analytics view:
- Go to Google Analytics
- Navigate to the view you want to connect to your app.
- Click on the Admin button. This will open the settings showing the Account, Property and View properties.
- Click the View User Management item in the View panel.
- Click the big plus button and select Add users.
- Paste the email address of the service account you've copied.
- Make sure Read & Analyze is checked.
- Click the Add button.
- Close the User Management panel.
- Click the View Settings item in the View panel.
- Copy the View ID value for later use.
2. Notes on the Google Analytics API
We are interested in recording the following metrics:
Name | Purpose |
rt:activeUsers | The number of active users in the view. |
rt:pageViews | The number of pages that are being viewed. This only applies to web sites. |
rt:screenViews | The number of screen that are being viewed. This only applies to our Android and iOS app. |
Did you know you can try this out online? The docs have a nice 'Try this API' feature. Check the Real Time Data: get API docs.
Quotas
Beware of the Google Analytics API quotas:
What are the maximum quotas for Real Time Reporting API requests?
Google Analytics Core Reporting API FAQs
50,000 requests per project per day and 10 queries per second (QPS) per IP address.
In our case, we are doing 4
calls to 2
views. So if we do one run of calls every 15
seconds, we do:
4*4=16
calls per minute16*60=960
calls per hour960*24=23040
calls per day, which is fine.
Consider creating multiple projects (per environment), so you don't hit your account limits (as we did). If you ever hit the quota; it will reset at midnight Pacific Time (PT).
3. .NET Core Implementation
Let's keep it simple and build a .NET Console App. Our implementation is based on this NuGet package from Google:
- Install-Package Google.Apis.Analytics.v3 -Version 1.51.0.1679
- dotnet add package Google.Apis.Analytics.v3 --version 1.51.0.1679
- <PackageReference Include="Google.Apis.Analytics.v3" Version="1.51.0.1679" />
Replace the 98765432
number of the viewId
variable with your own number. Copy the downloaded service account JSON file to the same directory as your application, rename it to credentials.json
and make sure it is copied to the output directory.
using Google.Apis.Analytics.v3;
using Google.Apis.Auth.OAuth2;
using Google.Apis.Services;
using System;
using System.Threading.Tasks;
class Program
{
async static Task Main(string[] args)
{
var credentialFilePath = "credentials.json";
var credential = GoogleCredential.FromFile(credentialFilePath);
var service = new AnalyticsService(new BaseClientService.Initializer()
{
HttpClientInitializer = credential.CreateScoped(
AnalyticsService.Scope.AnalyticsReadonly
),
ApplicationName = "google-analytics-real-time-api-in-net-core",
});
var viewId = "ga:98765432";
var metric = "rt:activeUsers";
var request = service.Data.Realtime.Get(viewId, metric);
var response = await request.ExecuteAsync();
var activeUsers = 0;
if (response.Rows != null)
{
activeUsers = Convert.ToInt32(response.Rows[0][0]);
}
Console.WriteLine($"{activeUsers} active users.");
}
}
Final thoughts
We've built a .NET Core API with a background service that polls the Google Analytics Real Time Data API. We're using Scrutor and decoration to convert the results to Prometheus metrics. We're plotting the metrics with Grafana. This will give us better insights during SRE.
And... we've added the Grafana dashboard to our DevOps Slack bot: