# Get keywords for images from the Google Cloud Vision API with C#
**Date:** 2016-06-21
**Author:** Kees C. Bakker
**Categories:** .NET Framework / C#
**Original:** https://keestalkstech.com/get-keywords-for-images-from-the-google-cloud-vision-api-with-c/
---
This blog will explore how to get keywords for images using the [Google Cloud Vision API](https://cloud.google.com/vision). In order to use this API you'll need a Google Console Project. I'll be using a JSON server-to-server token. If you're not sure how to set this up, please consult the [Quickstart](https://cloud.google.com/vision/docs/quickstart).
According to Google the project has a lot to offer:
> Google Cloud Vision API enables developers to **understand the content of an image** by encapsulating **powerful machine learning models** in an easy to use REST API. It quickly **classifies images** into thousands of categories (e.g., "sailboat", "lion", "Eiffel Tower"), **detects individual objects and faces within images**, and finds and reads printed words contained within images. You can build metadata on your image catalog, moderate offensive content, or enable new marketing scenarios through image sentiment analysis. **Analyze images uploaded in the request** or integrate with your image storage on Google Cloud Storage.
## Nuget
Never build yourself what others have been sweating to build for you! Google has provided a fine Nuget package for the API.
```shell
dotnet add package Google.Apis.Vision.v1 --version 1.49.0.2142
```
## Credentials + Service
The Google API uses many types of credentials. I'm using the server-to-server credential in a JSON format. The following code will create those credentials. *Don't forget to provide the scope, otherwise the credentials will fail! *
```cs
///
/// Creates the credentials.
///
/// The path to credential file.
/// The credentials.
public static GoogleCredential CreateCredentials(string path)
{
GoogleCredential credential;
using (var stream = new FileStream(path, FileMode.Open, FileAccess.Read))
{
var c = GoogleCredential.FromStream(stream);
credential = c.CreateScoped(VisionService.Scope.CloudPlatform);
}
return credential;
}
```
Now we can create the service using the credentials:
```cs
///
/// Creates the service.
///
/// Name of the application.
/// The credentials.
/// The service.
public static VisionService CreateService(
string applicationName,
IConfigurableHttpClientInitializer credentials)
{
var service = new VisionService(
new BaseClientService.Initializer()
{
ApplicationName = applicationName,
HttpClientInitializer = credentials
}
);
return service;
}
```
## Features
The Cloud Vision API returns keywords based on features. You can use the following feature constants:
## Prepare image request
First, let's create an `AnnotateImageRequest` that represent the data for a single file.
```cs
///
/// Creates the annotation image request.
///
/// The path.
/// The feature types.
/// The request.
private static AnnotateImageRequest CreateAnnotationImageRequest(
string path,
string[] featureTypes)
{
if (!File.Exists(path))
{
throw new FileNotFoundException("Not found.", path);
}
var request = new AnnotateImageRequest();
request.Image = new Image();
var bytes = File.ReadAllBytes(path);
request.Image.Content = Convert.ToBase64String(bytes);
request.Features = new List();
foreach(var featureType in featureTypes)
{
request.Features.Add(new Feature() { Type = featureType });
}
return request;
}
```
Note that you'll need to new up any property you need to use on the `AnnotationImageRequest`. The API will not create collections like `Features`.
## AnnotateAsync - single file
Now we can create an extension method that will extend the `VisionService` with a simple method to execute an annotation request for a single file. It ties the service and the `AnnotateImageRequest` together.
```cs
///
/// Annotates the file asynchronously.
///
/// The service.
/// The file.
/// The features.
/// The annotation response.
public static async Task AnnotateAsync(
this VisionService service,
FileInfo file,
params string[] features)
{
var request = new BatchAnnotateImagesRequest();
request.Requests = new List();
request.Requests.Add(CreateAnnotationImageRequest(file.FullName, features));
var result = await service.Images.Annotate(request).ExecuteAsync();
if (result?.Responses?.Count > 0)
{
return result.Responses[0];
}
return null;
}
```
The API supports batched requests. Implementation should not be hard, you'll just need to add more requests (`request.Request.Add`). I left it out of this tutorial.
## Proof of concept
All the components are ready. I've created a directory with some files from [the excellent Unsplash project](https://unsplash.com) and I ran the following program:
```cs
//find the files
var ext = new HashSet(StringComparer.OrdinalIgnoreCase) { ".png", ".jpg", ".gif" };
var dir = @"C:\temp\Examples";
var files =
Directory
.GetFiles(dir, "*.*", SearchOption.AllDirectories)
.Where(f => ext.Contains(Path.GetExtension(f)))
.Select(f => new FileInfo(f))
.ToArray();
//create service
var credentails = GoogleVisionApi.CreateCredentials("user_credentials.json");
var service = GoogleVisionApi.CreateService("MyApplication", credentails);
//process each file
foreach (var file in files)
{
string f = file.FullName;
Console.WriteLine("Reading " + f + ":");
var task = service.AnnotateAsync(file, "LABEL_DETECTION");
var result = task.Result;
var keywords = result?.LabelAnnotations?.Select(s => s.Description).ToArray();
var words = String.Join(", ", keywords);
Console.WriteLine(words);
f += ".keywords.txt";
File.WriteAllText(f, words);
}
```
## Results
This will produce the following results:
.thumbnail-results td { width:33%; }
## Final thoughts
The keywords are fine, but I had expected more from the Cloud Vision API. I'm missing keywords like: sun, puppy, strawberry, pots, window-sill. Maybe the service will improve. I can't wait to experiment with other services. Hopefully the code will help your project.