Vehicle Component Recognition

Search…
Vehicle Component Recognition is a game about optical character recognition. You will be given a series of images (via a URL) from which you need to identify either the position of an object in the image, or a piece of text from the image.
Using Microsoft's Computer Vision API, you must analyse the image you are given and then process the API response to find the labels and values you are looking for. In the Game Style "Numberplate Identifier", you are aiming to work out what the numberplate of the car is. In the Game Style "Component Recognition", you must return coordinates of a rectangle in which the specified feature (e.g. "wing mirror") can be found.
Microsoft Computer Vision API
You will need a subscription key to Microsoft's Computer Vision API to play this game. See Signing Up For Azure for more information.
Strategy
The template code gives you a structure of how you can analyse the Microsoft API response. Currently, it just finds the text on the image and guesses which text looks like a numberplate based on heuristics like whether it contains digits, or how long the text is. Can you make this more sophisticated?
The template code has not implemented any code for the mode "Component Recognition". Initially, just consider how to find a bounding box corresponding to the car's numberplate. How can you use code that has already been written for the other method, or code of a similar format, in order to find the location of the numberplate rather than its text?
Programmer's Reference
Making a move
The calculate_move() function is the equivalent of your main function. This is where you need to make your changes and from where you will control the game. The function:
1.Receives information about the game in a Python dictionary called 'gamestate'.
2.Returns a game move.
The gamestate
In order to work out what move you want to make, the calculate_move() function is given as input the gamestate. The gamestate is a Python dictionary where all of the game information is held. Examples of accessing data in the gamestate are:
gamestate["URL"] # the URL where the current image is held
gamestate["MyScore"] # the number of points you have scored
The gamestate contains the following keys:
Key
Description
Mode
An integer containing the last two digits of the Game Style. That is, mode is 1 for "Numberplate Identifier" and 2 for "Component Recognition".
RoundNumber
The round number you are on, 0-indexed so that the first round is Round 0. (That is, it is the number of images you have already submitted answers to.)
MaxRounds
The maximum number of images in this game that you can submit answers to.
URL
A URL to the image which you are to identify features of. (Images are given to you one at a time, and you cannot get the next one until submitting your answers to the current one.) This URL will always end in "image_k.png" where it is the k-th image in the game (that is, k is one more than RoundNumber).
Information
A list of strings containing the information that you are to identify on the image. In "Numberplate Identifier" (mode 1), this list will be ['numberplate']. In "Component Recognition" (mode 2), this will be the list of components of the car that you are to submit bounding boxes to (e.g. ['wing mirror', 'numberplate']).
IsMover
A Boolean value indicating whether it is your turn to move. It will always be true when the calculate_move() function has been called.
EndTime
The epoch time, in milliseconds, that the game will end at, unless both players finish submitting images before this time. (Epoch time is the amount of time that has passed since midnight on 1 January 1970, which can be accessed in Python with time.time().)
GameStatus
A string that will have value "RUNNING" if the game is in progress or a reason the game has ended otherwise.
MyScore
Your current score—the number of correct key-value pairs you have submitted so far in the current game.
OppScore
Your opponent’s score—the number of correct key-value pairs they have submitted so far in the current game.
An example gamestate:
{
        'Mode': 1,
        'IsMover': True,
        'URL': 'https://s3.eu-west-2.amazonaws.com/aigaming/VehicleComponentRecognition/Games/a87e3324-be85-464e-9d8c-c381ac0c6629/image_1.jpg',
        'Information': ['numberplate'],
        'MaxRounds': 3,
        'RoundNumber': 0,
        'MyPoints': 0,
        'OppPoints': 0,
        'EndTime': 1567763780146,
        'GameStatus': 'RUNNING',
        'InvalidMove': None,
        'OpponentId': 'housebot-practise'
}
Processing the form
You need a Microsoft API subscription key to play this game. The template code you are given takes you through how to get a JSON object analysis of the image by making a POST request with your key and the image URL. To understand the format of the object, take a look at the comments in the template code under the header "API return value". To help you process some of its information, helper functions are given in the template code.
Making a valid move
Every time it is your turn, the game engine will run your calculate_move() function, which needs to return a valid move. In "Numberplate Identifier" (mode 1), a valid move is a dictionary {'numberplate': s}, where s is a string (containing what you think the numberplate is). In "Component Recognition" (mode 2), a valid move is a dictionary of key-value pairs 'category': b, where 'category' is a category from gamestate["Information"] and b is a bounding box).

A bounding box b = [[x1,y1],[x2,y2]] is a rectangle from the top-left corner coordinates (x1,y1) to the bottom-right corner coordinates (x2,y2).
Game Help - Previous
Twist Cube
Next - Game Help
Warehouse Logistics
Last modified 3yr ago
Copy link
Outline
Microsoft Computer Vision API
Strategy
Programmer's Reference
Making a move
The gamestate
Processing the form
Making a valid move