This blog will explore how to get keywords for images using the Google Cloud Vision API. In order to use this API you’ll need a Google Console Project. I’ll be using a JSON server-to-server token. If you’re not sure how to set this up, please consult the¬†Quickstart.

According to Google the project has a lot to offer:

Google Cloud Vision API enables developers to understand the content of an image by encapsulating powerful machine learning models in an easy to use REST API. It quickly¬†classifies images into thousands of categories (e.g., “sailboat”, “lion”, “Eiffel Tower”), detects individual objects and faces within images, and finds and reads printed words contained within images. You can build metadata on your image catalog, moderate offensive content, or enable new marketing scenarios through image sentiment analysis. Analyze images uploaded in the request or integrate with your image storage on Google Cloud Storage.

Nuget

Never build yourself what others have been sweating to build for you! Google has provided a fine Nuget package for the API.

Install-Package Google.Apis.Vision.v1»

Credentials + Service

The Google API uses many types of credentials. I’m using the server-to-server credential in a JSON format. The following code will create those credentials.¬†Don’t forget to provide the scope, otherwise the credentials will fail!¬†

Now we can create the service using the credentials:

Features

The Cloud Vision API returns keywords based on features. You can use the following feature constants:

  • LABEL_DETECTION
    Add labels based on image content (see Label Detection Tutorial)
  • TEXT_DETECTION
    Perform Optical Character Recognition (OCR) on text within the image
  • SAFE_SEARCH_DETECTION
    Determine image safe search properties on the image
  • FACE_DETECTION
    Detect faces within the image (see Face Detection Tutorial)
  • LANDMARK_DETECTION
    Detect geographic landmarks within the image
  • LOGO_DETECTION
    Detect company logos within the image
  • IMAGE_PROPERTIES
    Compute a set of properties about the image (such as the image’s dominant colors)

Prepare image request

First, let’s¬†create an¬†AnnotateImageRequest that represent the data for a single file.

Note that you’ll need to new up any property¬†you need to use on the¬†AnnotationImageRequest. The API will not create collections like¬†Features.

AnnotateAsync – single file

Now we can create an extension method that will extend the VisionService with a simple method to execute an annotation request for a single file. It ties the service and the AnnotateImageRequest together.

The API supports batched requests. Implementation should not be hard, you’ll just need to add more¬†requests (request.Request.Add). I left it out of this tutorial.

Proof of concept

All the components are ready. I’ve created a directory with some files from the excellent Unsplash project and I ran the following program:

Results

This will produce the following results:

Af0sF2OS5S5gatqrKzVP_Silhoutte photo-1414408134277-dde21bd7c338 photo-1433217684272-723f82dbf72d
light, silhouette, hand, lighting, sunrise plant, tree, leaf, branch shore, beach, sea, walkway, coast
photo-1440756427076-7480f7eab5bd photo-1447015993193-3f72d500c3fb photo-1447684808650-354ae64db5b8
vacation, sun tanning, photo shoot bird, nature, animal, wildlife, vertebrate pet, dog, mammal, animal, vertebrate
photo-1464465474479-26aa7f69b834 xIsiRLngSRWN02yA2BbK_submission-photo-7 YFdIoUsRJCAehcoUnQaS_Straw
plant, flower, flowering plant, floristry, floral design  crowd, city, road, street, vehicle dish, food, meal, produce, breakfast

Final thoughts

The keywords are fine, but I had expected more from the Cloud Vision API. I’m missing keywords like: sun, puppy, strawberry, pots, window-sill. Maybe the service will improve. I can’t wait to experiment with other services. Hopefully the code will help your project.