Microsoft Match Game

SG
Last updated 2 days ago

Play our updated version of the pairs matching game, but in our version, no images match at all.

Introduction

Match Game is our version of the popular card game where you have to find two cards with the same image to make a match. In our Match Game, no two tiles are the same. You have to match up the tiles that contain the same subject matter. We are going to be using the A.I. power of Azure Cognitive Services to analyse the images on each tile and to determine what subject matter each image contains.

Objective

You start with a set of tiles face down and you get to turn over two tiles at a time in order to match tiles in pairs, but, unlike the usual version of the matching game, our tiles are not strictly the same.

You are not looking for two identical images. You have to use Microsoft’s API to understand exactly what the content of each tile image is. Then we need to find another tile that also has that content on it.

You can see in the image below that an image of an elephant is matched with an image of a different elephant and that two different images of The Sydney Opera House from different angles are still recognised as the Sydney Opera House and so match each other. You can also see that each tile belongs to a category, either Animals, Landmarks or Words.

This isn’t just pairing up exact matches, this is knowing what the subject of every tile is and matching the tiles that have the same subject.

Programmer's Reference

See the programmer's reference for details about the gamestate and making a valid move for this game

Azure Cognitive Services

In order to analyse an image and identify what the subject of the image is, you will need to use Azure Cognitive Services to analyse each image. The data returned will contain a wealth of information that tells us what the subject of the image is.

Perhaps it’s a picture of The Taj Mahal. If it is, we have to match it against another Taj Mahal picture, but, there won’t be another tile with the same picture of the Taj Mahal, there will be a tile that has a different picture of The Taj Mahal, perhaps from a different angle.

We can’t just compare image files pixel by pixel to play the game. We have to understand what is in each image.

We do this for three different categories of tiles, either

  • Famous Landmarks from around the world.

  • Animals

  • Words

Signing up for Azure Cognitive Services

If you are a student, you can sign up for a student account on Azure that gives you $100 of credit. You can then create a Computer Vision API Key that will let you make 20 API calls per second.

Alternatively, you can get a free trial account on Microsoft's Cognitive API that lets you make 20 API calls every minute to a maximum of 5,000 calls every month.

You can follow our guide to signing up for a Microsoft free trial at this link.

Once you have a Microsoft Account, you will need to create resources within it and obtain API keys for one of Microsoft's Cognitive Services, Computer Vision, to play this game.

A suggested solution with pseudocode

We have written a possible approach to working through this problem. The pseudocode below suggests one way that you might go about structuring your code to solve this challenge.

In summary, the approach is:

We create a list variable that we can use to store information about the tiles we have seen. We use this to store the subject or text of each tile's image and whether we have analysed the tile yet or matched the tile yet.

  • Analyse any new tiles and remember them

  • Check to see if any of the tiles we know about match each other

  • If they do:

    • return a matching pair as our move

  • Else:

    • choose two tiles that we have not yet seen for our next move

Writing code to play the game

Template code

When you first load the Match Game game type in the Online Code Editor, we give you some template code that plays a very simple version of the game and does not require Computer Vision API keys.

We have also made another code template available that uses the Azure Cognitive Services. You can find this template in the Online Code Editor under the New button. Select this option when creating a new file to use our Microsoft API example template code.

Use our template code with examples of how to call the Computer Vision API

The calculate_move() function

The calculate_move() function is the equivalent of your main function. This is where you need to make your changes and from where you will control the game.

  • Called each time you need to submit a move to the game.

  • Receives information about the game in a Python dictionary that describes the current game state.

  • Must return a game move.

Understanding the game state

Your calculate_move() function will be passed the current state of the game, the 'gamestate'.

The gamestate is where all of the game information is held. It is a Python dictionary. The following shows an example of the gamestate information that you will receive for each move of the game and some examples of how to access data within it.

For the Match Game, the most important fields are Board, UpturnedTiles, AnimalList and CategoryList.

Fields that you are unlikely to need in this game are IsMover, ResponseDeadline, GameStatus, GameId and OpponentId.

An example of the gamestate JSON

{
"Board": [0, 0, 0, 0, 0, 0, 1, 0, 1, 0, 1, 1, 0, 0, 1, 1, 1, 0, 1, 0],
"MyPoints": 4,
"OppPoints": 2,
"UpturnedTiles": [{
"Index": 1,
"Tile": "https://s3.eu-west-2.amazonaws.com/aigaming/Pairs/Games/d46c704d-2f2b-4b38-b609-614da027f5ec/d2e5788f-a27d-4f05-81e5-549f31921a48.jpg"
}, {
"Index": 19,
"Tile": "https://s3.eu-west-2.amazonaws.com/aigaming/Pairs/Games/d46c704d-2f2b-4b38-b609-614da027f5ec/4b87f941-f03e-4196-a7c8-680dbde17272.jpg"
}],
"CategoryList": ["Words", "Landmarks", "Animals"],
"OpponentMatches": 2,
"TileBacks": [
"https://s3.eu-west-2.amazonaws.com/aigaming/Pairs/Games/d46c704d-2f2b-4b38-b609-614da027f5ec/2b3a03ba-8329-40be-82be-5604bfde365c.png",
"https://s3.eu-west-2.amazonaws.com/aigaming/Pairs/Games/d46c704d-2f2b-4b38-b609-614da027f5ec/2b3a03ba-8329-40be-82be-5604bfde365c.png",
"https://s3.eu-west-2.amazonaws.com/aigaming/Pairs/Games/d46c704d-2f2b-4b38-b609-614da027f5ec/2b3a03ba-8329-40be-82be-5604bfde365c.png",
"https://s3.eu-west-2.amazonaws.com/aigaming/Pairs/Games/d46c704d-2f2b-4b38-b609-614da027f5ec/2b3a03ba-8329-40be-82be-5604bfde365c.png",
"https://s3.eu-west-2.amazonaws.com/aigaming/Pairs/Games/d46c704d-2f2b-4b38-b609-614da027f5ec/1ce5aaed-e01a-49c1-bbd7-909b3fbd6c70.png",
"https://s3.eu-west-2.amazonaws.com/aigaming/Pairs/Games/d46c704d-2f2b-4b38-b609-614da027f5ec/2b3a03ba-8329-40be-82be-5604bfde365c.png",
"https://s3.eu-west-2.amazonaws.com/aigaming/Pairs/Games/d46c704d-2f2b-4b38-b609-614da027f5ec/1ce5aaed-e01a-49c1-bbd7-909b3fbd6c70.png",
"https://s3.eu-west-2.amazonaws.com/aigaming/Pairs/Games/d46c704d-2f2b-4b38-b609-614da027f5ec/2b3a03ba-8329-40be-82be-5604bfde365c.png",
"https://s3.eu-west-2.amazonaws.com/aigaming/Pairs/Games/d46c704d-2f2b-4b38-b609-614da027f5ec/50184310-a18a-43e3-b8e5-242aafe115e6.png",
"https://s3.eu-west-2.amazonaws.com/aigaming/Pairs/Games/d46c704d-2f2b-4b38-b609-614da027f5ec/2b3a03ba-8329-40be-82be-5604bfde365c.png",
"https://s3.eu-west-2.amazonaws.com/aigaming/Pairs/Games/d46c704d-2f2b-4b38-b609-614da027f5ec/50184310-a18a-43e3-b8e5-242aafe115e6.png",
"https://s3.eu-west-2.amazonaws.com/aigaming/Pairs/Games/d46c704d-2f2b-4b38-b609-614da027f5ec/1ce5aaed-e01a-49c1-bbd7-909b3fbd6c70.png",
"https://s3.eu-west-2.amazonaws.com/aigaming/Pairs/Games/d46c704d-2f2b-4b38-b609-614da027f5ec/1ce5aaed-e01a-49c1-bbd7-909b3fbd6c70.png",
"https://s3.eu-west-2.amazonaws.com/aigaming/Pairs/Games/d46c704d-2f2b-4b38-b609-614da027f5ec/2b3a03ba-8329-40be-82be-5604bfde365c.png",
"https://s3.eu-west-2.amazonaws.com/aigaming/Pairs/Games/d46c704d-2f2b-4b38-b609-614da027f5ec/1ce5aaed-e01a-49c1-bbd7-909b3fbd6c70.png",
"https://s3.eu-west-2.amazonaws.com/aigaming/Pairs/Games/d46c704d-2f2b-4b38-b609-614da027f5ec/1ce5aaed-e01a-49c1-bbd7-909b3fbd6c70.png",
"https://s3.eu-west-2.amazonaws.com/aigaming/Pairs/Games/d46c704d-2f2b-4b38-b609-614da027f5ec/1ce5aaed-e01a-49c1-bbd7-909b3fbd6c70.png",
"https://s3.eu-west-2.amazonaws.com/aigaming/Pairs/Games/d46c704d-2f2b-4b38-b609-614da027f5ec/2b3a03ba-8329-40be-82be-5604bfde365c.png",
"https://s3.eu-west-2.amazonaws.com/aigaming/Pairs/Games/d46c704d-2f2b-4b38-b609-614da027f5ec/1ce5aaed-e01a-49c1-bbd7-909b3fbd6c70.png",
"https://s3.eu-west-2.amazonaws.com/aigaming/Pairs/Games/d46c704d-2f2b-4b38-b609-614da027f5ec/2b3a03ba-8329-40be-82be-5604bfde365c.png"
],
"IsMover": true,
"ResponseDeadline": 1542124987192,
"GameStatus": "RUNNING",
"AnimalList": ["cat", "dog", "horse", "lion", "turtle", "deer", "zebra", "rhinoceros", "fox", "cow", "chicken", "sheep", "antelope", "ape", "bear", "cheetah", "donkey", "eagle", "echidna", "elephant", "flamingo", "giraffe", "goat", "hawk", "heron", "hummingbird", "kangaroo", "koala", "leopard", "meerkat", "ostrich", "owl", "parrot", "penguin", "phasianid", "rodent", "salamander", "seal", "snake", "squirrel", "tiger", "vulture", "wolf"],
"Bonus": "Landmarks",
"Multiplier": 1,
"GameId": 2386873,
"OpponentId": "housebot-practise"
}

Example to access the game state fields

Examples of accessing data in the gamestate would be:

gameState["Board"][0]
gameState["UpturnedTiles"][0]["Tile"]
gameState["UpturnedTiles"][0]["Index"]

The gamestate fields explained

The following list gives a description of what each element in the gameState represents:

  • Board - A list representing each tile in the game and whether that tile has been successfully matched or not. A "1" means the tile has been matched and "0" means the tile has not yet been matched.

    • An example of the Board data could be 'Board':[0, 0, 1, 0, 0, 1, 1, 0, 1, 0, 0, 0, 1, 0, 0, 0]

    • You could use the Board data to determine how many sets need to be matched by calling len(gamestate["Board"])

    • You would access a specific item in the list using gamestate["Board"][7]. This would give you the 8th item in the list.

  • UpturnedTiles - A list of dictionaries detailing the upturned tiles for this move.

    • The upturned tiles will be the tiles you submitted as your previous move. If your previous move was Tiles: [1,10] then you will receive the image url for tile 1 and for tile 10 in the UpturnedTiles field.

    • If there are upturned tiles, there will always be two upturned tiles.

    • Each upturned tile states the Index of the tile, which is its unique location, and a URL to the image that is on the tile. It is this image that you need to analyse and match.

    • An example of accessing an upturned tiles URL would be gamestate["UpturnedTiles"][0]["Tile"]

    • An example of the UpturnedTiles data is

'UpturnedTiles':[
{
'Index':1,
'Tile':'https://s3.eu-west-2.amazonaws.com/aigaming/Pairs/Games/fa9a2691-0974-4378-b4cd-95f20861d52c/677abb2c-c2e7-4355-ae21-390ff31e6c75.jpg'
},
{
'Index':10,
'Tile':'https://s3.eu-west-2.amazonaws.com/aigaming/Pairs/Games/fa9a2691-0974-4378-b4cd-95f20861d52c/b2b0e5f3-3102-4e1b-a1cb-a5c6ca1fb7a3.jpg'
}
]
  • TileBacks - A list of strings containing the URLs of the images for the backs of the tiles.

  • AnimalList - A list of strings that details all possible animals that will be displayed in the game.

    • To identify that the tile image you have analysed is a picture of an animal, you can search through the tags returned from the Microsoft API to see if any of the tags contain an animal name in the AnimalList.

    • You can check to see if the name of an animal exists in the AnimalList using if "elephant" in gamestate["AnimalList"]:

  • CategoryList - A list of strings that details all possible categories that tiles will belong to in the game.

    • Each tile belongs to one of the categories

    • If you choose tiles that are in the same category, they have a much better chance of matching than if you choose any unmatched tile from any category.

  • MyPoints - Your current points score for this game.

  • OpponentPoints - Your opponents current points score for this game.

  • OpponentMatches - The number of matched tile sets that your opponent has successfully identified.

    • You can use this value to identify how quickly your opponent is matching tiles

  • Bonus - A String containing the name of a category. If your move matches tiles in this bonus category, the score you receive will be multiplied by the Multiplier amount.

    • For example, this Bonus field might be set to "Landmarks". If you match a pair of Landmark tiles while the Bonus field is set to Landmarks, the score you receive for this move will be multiplied by the Multiplier field value

    • If either you or your opponent match a pair that is in the bonus category, the bonus category will be randomly set to another category in the next move.

  • Multiplier - The current score multiplier that is affected by matching tiles that are in the Bonus Category.

    • Consecutively matching tiles in the bonus category doubles the multiplier.

    • Not matching tiles in the bonus category resets the multiplier.

  • GameId - An integer representing the unique game id for the current game

    • You are unlikely to need the GameId for this game type

  • OpponentId - A string containing the name of your opponent

    • You are unlikely to need the OpponentId for this game type

  • ResponseDeadline - The epoch time, in milliseconds, that a successful move has to be sent and received by to prevent you from timing out.

    • There is a time limit to how long you have to calculate your move. If you exceed this time limit your game will be terminated and your opponent will be awarded as the winner.

    • It is unlikely that you will need to check this time as timeouts are set generously to allow you time to calculate your move, however, if you see yourself timing out a lot, you may need to limit yourself using this value.

  • GameStatus - A string that will have value "RUNNING" if the game is in progress or a reason the game has ended otherwise.

    • You are unlikely to need the GameStatus for this game type

  • IsMover - In this turned based game, this will always be true.

Making a valid move

The point of the calculate_move() function is for you to return the move you want to make in the game. In the Match Game, a move is the pair of tiles that you want to turn over to see if they match.

You should determine which tiles to turn over by remembering the tiles you have already seen and trying to turn over a pair of tiles that have the same subject.

To submit your move, you return a dictionary with the Key Pair of "Tiles" and then a list of the tiles that you want to turn over. For example:

return {"Tiles": [10, 14]}

Flowchart & Pseudocode

The pseudocode below expands on this and matches the template code that we give you for this game:

* Do we have any upturned tiles?
(We won't have any upturned tiles on the very first
move of the game and we won't have any upturned
tiles if our last move was a match.)
* If we do have some upturned tiles:
> Use the MS API to analyse the contents of the tiles
and record what is on them in our list of
remembered tiles
* Search our list of remembered tiles to see if we
have a record of any matching tiles
* If we do have matching tiles:
> Set our move for this go to be the matching tiles
* Else, if we do not have any matching tiles:
> Create an up to date list of all the unmatched tiles
by checking the Board for all tiles still marked as 0
> Create a list of unmatched tiles that we have not
yet analysed with MS API
> If there are some tiles that we have not yet analysed:
- Choose these tiles as our move so that we can
analyse them in our next move.
> Else, if we have analysed all of the tiles:
- This is a catchall scenario in case we cannot
match tiles using the MS API
- Choose two random tiles from the unmatched tiles for
our next move
* return our move

Finding the subject from the landmark Computer Vision API call

Using the search for an animal function as an example

Our template code gives you an example of calling the Computer Vision API and searching the response to see if an animal has been identified in the image. It does this in the check_for_animal() function where it loops through all of the entries in the "tags" section of the API response to see if one of the "tags" has a "name" with a value that matched one of the animals in our list of known animals.

The game state contains a list of all the animals that can be matched in the game:

'AnimalList':[
'cat', 'dog', 'horse', 'lion', 'turtle', 'deer', 'zebra', 'rhinoceros', 'fox', 'cow', 'chicken', 'sheep', 'antelope', 'ape', 'bear', 'cheetah', 'donkey', 'eagle', 'echidna', 'elephant', 'flamingo', 'giraffe', 'goat', 'hawk', 'heron', 'hummingbird', 'kangaroo', 'koala', 'leopard', 'meerkat', 'ostrich', 'owl', 'parrot', 'penguin', 'phasianid', 'rodent', 'salamander', 'seal', 'snake', 'squirrel', 'tiger', 'vulture', 'wolf'
]

The response from the Microsoft Cognitive API contains a section with "tags" that list what the API thinks is in the image.

"tags": [
{
"name": "grass",
"confidence": 0.9996933937072754
},
{
"name": "fox",
"confidence": 0.9955691695213318,
"hint": "animal"
},
{
"name": "outdoor",
"confidence": 0.991620659828186
},
{
"name": "animal",
"confidence": 0.9838736653327942
},
{
"name": "mammal",
"confidence": 0.9827039837837219,
"hint": "animal"
}
],

Here we can see that one of the "tags" in the response is the name of an animal, "fox". We can also see that fox is one of the strings in our game state list of known animals.

This means that if we were to query the Computer Vision API with this image of a fox and check all of the tags returned, we would see that one of these tags contained the name "fox". This would mean that we can be very confident that this is an Animal category tile and that it is a picture of a fox.

This is exactly what our check_for_animal() function does:

def check_for_animal(analysis, animal_list):
# Initialise our subject to None
subject = None
# For every tag in the returned tags, in descending confidence order
for tag in sorted(analysis["tags"], key=lambda x: x['confidence'], reverse=True):
# If the tag has a name and that name is one of the animals in our list
if "name" in tag and tag["name"] in animal_list:
# Record the name of the animal that is the subject of the tile
# (We store the subject in lowercase to make comparisons easier)
subject = tag["name"].lower()
# Print out the animal we have found here ----------------->
print("Animal: {}".format(subject))
# Exit the for loop
break
# Return the subject
return subject

We can use this code as a template to search for a Landmark in the response from the Computer Vision API.

Best steps to improve your code

These are the best steps to take in order to improve your code:

  1. Implement Landmark matching.

  2. Implement text recognition and word matching

  3. Read the tile backs to match tiles from the same category

  4. Check if you can match tiles in the bonus category

  5. Wait before matching any tiles to match tiles in consecutive Bonus Categories.

Searching for Landmark information in the Computer Vision API response

We can adapt a copy of the working function from our code template that searches for animals, to search the Computer Vision API response for Landmark information.

We have provided a placeholder function with comments in the template code. The comments include instructions about printing the response from the Computer Vision API to understand where to find the landmark subject e.g. "Big Ben". Note that unlike the Animals, we do not restrict the responses to a known set, any landmark might be returned.

def check_for_landmark(analysis):
# TODO: We strongly recommend copying and pasting the result of the
# Microsoft API into a JSON formatter (e.g. https://jsonlint.com/) to better
# understand what the API is returning and how you will access the landmark
# information that you need.
#
# Do this by copying and pasting the results (that are displayed in column
# 3 of the Editor) of the print(analysis) statement and pasting them into
# the Input box at https://jsonlint.com.
#
# Here is an example of identifying the landmark information in the
# Microsoft API response:
# analysis["categories"][0]["detail"]["landmarks"][0]["name"].lower()
# Initialise our subject to None
subject = None
print("Microsoft API Response: {}".format(analysis))
# For every category in the returned categories
# If the category has a "detail" field, and the detail contains a
# "landmarks" field
# Record the name of that landmarks field as the subject of the tile
# (We usually store the subject in lowercase to make comparisons easier)
# Print the name of the landmark we have found here ----------------->
# Exit the for loop
# Return the subject
return subject

In the example we recommend printing out the response from the Computer Vision API call and copying and pasting it into a JSON formatter like jsonlint.com so that you can have a good look at it. This will let you see all of the data that the API returns for an image and figure out where the relevant Landmark data is.

We've given you the location of the Landmark name data in the first comment of the function:

# Computer Vision API response:
# analysis["categories"][0]["detail"]["landmarks"][0]["name"].lower()

This refers to this section of the API response:

{
'categories': [{
'name': 'outdoor_',
'score': 0.00390625,
'detail': {
'landmarks': [{
'name': 'Big Ben',
'confidence': 0.9999606609344482
}]
}
}, {
'name': 'outdoor_house',
'score': 0.8203125,
'detail': {
'landmarks': [{
'name': 'Big Ben',
'confidence': 0.9999606609344482
}]
}
}],

Other suggestions for improving the template solution

The best approach to improving the template solution to play a more competitive game would be to develop the code in the following order:

  • Identify and match Landmarks

  • Identify and match Words

  • Read the backs of the tiles to identify tile categories

    • Instead of choosing random tiles to match, if you can read all of the tile backs, you will know what category the tile belongs to you and you can choose cards in the same category to improve your odds of matching.

  • Check the bonus category

    • Before you select matching cards, check the Bonus Category to see if you can make a match in this category for more points

  • Maximise the bonus category strategy

    • Deliberately avoid making any matches until you know what all of the tile images are, then try to match only tiles that are in the current bonus category to make the most of the points multiplier.

The following suggestions highlight further areas where you could implement improvements to the template code. Each improvement should increase your bot's chances of scoring the most points.

Identifying tile categories

The backs of the tiles are provided in the gamestate and indicate the category of content on the tile, ANIMAL, LANDMARK or WORD. You can use the Text Analytics API to read the back of the tile and determine the category to improve your match guesses.

Bonus points

Each round has a "bonus category". If in that round you match two tiles of that category, you will receive bonus points. The bonus starts at 1 point for a total of 2, but each sequential match in the required bonus category doubles the points of the previous match. So you can earn 2, 4, 8, 16 points etc. A strategy might be to determine where matches are but not guess them until later in the game and then ensure where possible you match tiles of the bonus category.

More information