mscroggs.co.uk
mscroggs.co.uk
Click here to win prizes by solving the mscroggs.co.uk puzzle Advent calendar.
Click here to win prizes by solving the mscroggs.co.uk puzzle Advent calendar.

subscribe

Blog

MENACE: Machine Educable Noughts And Crosses Engine

 2015-08-27 
In 1961, Donald Michie built MENACE (Machine Educable Noughts And Crosses Engine), a machine capable of learning to be a better player of Noughts and Crosses (or Tic-Tac-Toe if you're American). As computers were less widely available at the time, MENACE was built from from 304 matchboxes.
Taken from Trial and error by Donald Michie [2]
The original MENACE.
To save you from the long task of building a copy of MENACE, I have written a JavaScript version of MENACE, which you can play against here.

How to play against MENACE

To reduce the number of matchboxes required to build it, MENACE aways plays first. Each possible game position which MENACE could face is drawn on a matchbox. A range of coloured beads are placed in each box. Each colour corresponds to a possible move which MENACE could make from that position.
To make a move using MENACE, the box with the current board position must be found. The operator then shakes the box and opens it. MENACE plays in the position corresponding to the colour of the bead at the front of the box.
For example, in this game, the first matchbox is opened to reveal a red bead at its front. This means that MENACE (O) plays in the corner. The human player (X) then plays in the centre. To make its next move, MENACE's operator finds the matchbox with the current position on, then opens it. This time it gives a blue bead which means MENACE plays in the bottom middle.
The human player then plays bottom right. Again MENACE's operator finds the box for the current position, it gives an orange bead and MENACE plays in the left middle. Finally the human player wins by playing top right.
MENACE has been beaten, but all is not lost. MENACE can now learn from its mistakes to stop the happening again.

How MENACE learns

MENACE lost the game above, so the beads that were chosen are removed from the boxes. This means that MENACE will be less likely to pick the same colours again and has learned. If MENACE had won, three beads of the chosen colour would have been added to each box, encouraging MENACE to do the same again. If a game is a draw, one bead is added to each box.
Initially, MENACE begins with four beads of each colour in the first move box, three in the third move boxes, two in the fifth move boxes and one in the final move boxes. Removing one bead from each box on losing means that later moves are more heavily discouraged. This helps MENACE learn more quickly, as the later moves are more likely to have led to the loss.
After a few games have been played, it is possible that some boxes may end up empty. If one of these boxes is to be used, then MENACE resigns. When playing against skilled players, it is possible that the first move box runs out of beads. In this case, MENACE should be reset with more beads in the earlier boxes to give it more time to learn before it starts resigning.

How MENACE performs

In Donald Michie's original tournament against MENACE, which lasted 220 games and 16 hours, MENACE drew consistently after 20 games.
Taken from Trial and error by Donald Michie [2]
Graph showing MENACE's performance in the original tournament. Edit: Added the redrawn graph on the left.
After a while, Michie tried playing some more unusual games. For a while he was able to defeat MENACE, but MENACE quickly learnt to stop losing. You can read more about the original MENACE in A matchbox game learning-machine by Martin Gardner [1] and Trial and error by Donald Michie [2].
You may like to experiment with different tactics against MENACE yourself.

Play against MENACE

I have written a JavaScript implemenation of MENACE for you to play against. The source code for this implementation is available on GitHub.
When playing this version of MENACE, the contents of the matchboxes are shown on the right hand side of the page. The numbers shown on the boxes show how many beads corresponding to that move remain in the box. The red numbers show which beads have been picked in the current game.
The initial numbers of beads in the boxes and the incentives can be adjusted by clicking Adjust MENACE's settings above the matchboxes. My version of MENACE starts with more beads in each box than the original MENACE to prevent the early boxes from running out of beads, causing MENACE to resign.
Additionally, next to the board, you can set MENACE to play against random, or a player 2 version of MENACE.
Edit: After hearing me do a lightning talk about MENACE at CCC, Oliver Child built a copy of MENACE. Here are some pictures he sent me:
Edit: Oliver has written about MENACE and the version he built in issue 03 of Chalkdust Magazine.
Edit: Inspired by Oliver, I have built my own MENACE. I took it to the MathsJam Conference 2016. It looks like this:

A matchbox game learning-machine by Martin Gardner. Scientific American, March 1962. [link]
Trial and error by Donald Michie. Penguin Science Survey, 1961.

Similar posts

Building MENACEs for other games
MENACE at Manchester Science Festival
Visualising MENACE's learning
MENACE in fiction

Comments

Comments in green were written by me. Comments in blue were not written by me.
"When playing against skilled players, it is possible that the first move box runs out of beads. In this case, MENACE should be reset with more beads in the earlier boxes to give it more time to learn before it starts resigning."

If someone were doing this, you could do this automatically to avoid the perception or temptation of the operator to help it along. Instead of "oh, it's dead, let's repopulate the boxes", you could just make it part of the inter-game cleanup, like a garbage collection routine. After all the bead deleting/adding whatever, but before the next game starts, look at all the boxes, make sure that each box contains at least one of each color. Now this weakens the learning algorithm moderately, but it guarantees that it will never get stuck.
(anonymous)
   ×1              Reply
@(anonymous): Yes, those boxes are for O being MENACE and MENACE playing first
Matthew
×1                 Reply
@Matthew: Thank you for such a quick response. Just to let you know that that link did not work after .../tree/master/output, but I managed to search around for the right files :). In these files MENACE plays the Nought right? and the user plays the Cross?
(anonymous)
                 Reply
@Finlay: You can find them at https://github.com/mscroggs/MENACE-pdf.... The files boxes0.pdf to boxes3.pdf are the boxes for a MENACE that plays first.
Matthew
   ×1   ×1           Reply
Where can I find out all the Game states? I want to program MENACE for my computing coursework but for that I will need the Game states or matchboxes. Any help will be much appreciated.
Finlay
                 Reply
 Add a Comment 


I will only use your email address to reply to your comment (if a reply is needed).

Allowed HTML tags: <br> <a> <small> <b> <i> <s> <sup> <sub> <u> <spoiler> <ul> <ol> <li>
To prove you are not a spam bot, please type "x" then "-" then "a" then "x" then "i" then "s" in the box below (case sensitive):

Archive

Show me a random blog post
 2020 

Nov 2020

Christmas (2020) is coming!

Jul 2020

Happy πτ-e-ϕ-16 Approximation Day!

May 2020

A surprising fact about quadrilaterals
Interesting tautologies

Mar 2020

Log-scaled axes

Feb 2020

PhD thesis, chapter ∞
PhD thesis, chapter 5
PhD thesis, chapter 4
PhD thesis, chapter 3
Inverting a matrix
PhD thesis, chapter 2

Jan 2020

PhD thesis, chapter 1
Gaussian elimination
Matrix multiplication
Christmas (2019) is over
 2019 
▼ show ▼
 2018 
▼ show ▼
 2017 
▼ show ▼
 2016 
▼ show ▼
 2015 
▼ show ▼
 2014 
▼ show ▼
 2013 
▼ show ▼
 2012 
▼ show ▼

Tags

bodmas coins craft flexagons reddit finite element method advent calendar plastic ratio golden ratio puzzles squares preconditioning estimation sorting ucl binary draughts speed curvature folding paper manchester captain scarlet frobel christmas data visualisation twitter golden spiral programming polynomials gerry anderson latex menace stickers tmip machine learning palindromes inverse matrices light weather station dataset graph theory go approximation logs sobolev spaces hannah fry folding tube maps matrices chalkdust magazine european cup numerical analysis hexapawn tennis video games realhats ternary cambridge matrix multiplication hats propositional calculus interpolation the aperiodical logic javascript inline code python pi pi approximation day graphs london quadrilaterals martin gardner royal baby noughts and crosses countdown triangles matrix of minors pythagoras error bars pizza cutting christmas card mathslogicbot game show probability braiding mathsteroids news people maths sport map projections national lottery london underground data phd oeis sound manchester science festival probability simultaneous equations big internet math-off trigonometry nine men's morris asteroids matt parker pac-man accuracy dragon curves chess chebyshev matrix of cofactors game of life a gamut of games rugby geometry rhombicuboctahedron harriss spiral exponential growth bempp radio 4 wool raspberry pi dates weak imposition platonic solids wave scattering electromagnetic field php determinants convergence royal institution games geogebra bubble bobble computational complexity talking maths in public arithmetic books misleading statistics mathsjam football final fantasy signorini conditions cross stitch statistics boundary element methods gaussian elimination world cup fractals reuleaux polygons

Archive

Show me a random blog post
▼ show ▼
© Matthew Scroggs 2012–2020