Vehicle Component Recognition is a game about optical character recognition. You will be given a series of images (via a URL) from which you need to identify either the position of an object in the image, or a piece of text from the image.

Using Microsoft's Computer Vision API, you must analyse the image you are given and then process the API response to find the labels and values you are looking for. In the Game Style "Numberplate Identifier", you are aiming to work out what the numberplate of the car is. In the Game Style "Component Recognition", you must return coordinates of a rectangle in which the specified feature (e.g. "wing mirror") can be found.

Microsoft Computer Vision API

You will need a subscription key to Microsoft's Computer Vision API to play this game. See Signing Up For Azure for more information.

Strategy

The template code gives you a structure of how you can analyse the Microsoft API response. Currently, it just finds the text on the image and guesses which text looks like a numberplate based on heuristics like whether it contains digits, or how long the text is. Can you make this more sophisticated?

The template code has not implemented any code for the mode "Component Recognition". Initially, just consider how to find a bounding box corresponding to the car's numberplate. How can you use code that has already been written for the other method, or code of a similar format, in order to find the location of the numberplate rather than its text?

Programmer's Reference

Making a move

The calculate_move() function is the equivalent of your main function. This is where you need to make your changes and from where you will control the game. The function:

1. Receives information about the game in a Python dictionary called 'gamestate'.

2. Returns a game move.

The gamestate

In order to work out what move you want to make, the calculate_move() function is given as input the gamestate. The gamestate is a Python dictionary where all of the game information is held. Examples of accessing data in the gamestate are:

gamestate["URL"] # the URL where the current image is held
gamestate["MyScore"] # the number of points you have scored

The gamestate contains the following keys:

An example gamestate:

{
        'Mode': 1,
        'IsMover': True,
        'URL': 'https://s3.eu-west-2.amazonaws.com/aigaming/VehicleComponentRecognition/Games/a87e3324-be85-464e-9d8c-c381ac0c6629/image_1.jpg',
        'Information': ['numberplate'],
        'MaxRounds': 3,
        'RoundNumber': 0,
        'MyPoints': 0,
        'OppPoints': 0,
        'EndTime': 1567763780146,
        'GameStatus': 'RUNNING',
        'InvalidMove': None,
        'OpponentId': 'housebot-practise'
}

Processing the form

You need a Microsoft API subscription key to play this game. The template code you are given takes you through how to get a JSON object analysis of the image by making a POST request with your key and the image URL. To understand the format of the object, take a look at the comments in the template code under the header "API return value". To help you process some of its information, helper functions are given in the template code.

Making a valid move

Every time it is your turn, the game engine will run your calculate_move() function, which needs to return a valid move. In "Numberplate Identifier" (mode 1), a valid move is a dictionary {'numberplate': s}, where s is a string (containing what you think the numberplate is). In "Component Recognition" (mode 2), a valid move is a dictionary of key-value pairs 'category': b, where 'category' is a category from gamestate["Information"] and b is a bounding box). A bounding box b = [[x1,y1],[x2,y2]] is a rectangle from the top-left corner coordinates (x1,y1) to the bottom-right corner coordinates (x2,y2).