Following the end of the first round of the World Cup group stage let’s see how the model is doing and how the predictions have changed.
For more details about my World Cup prediction model see my previous post.
A hugely important thing for any form of model is validation. How well is our model doing? One way to investigate this is by looking at the Brier Score of the model. The Brier score is used to measure the accuracy of probabilistic predictions. Fundamentally it is the mean square error of the forecast and varies from 0 (perfect prediction) to 2. So the lower the score the better. For each result a Brier score is calculated and then an overall score is given to the models by taking the mean value.
As a baseline a simple prediction of 0.333 for each result (win/draw/loss) gives you a Brier score of 0.666. So we should always be looking to do better than that. Likely the best model of this World Cup will be that of the bookies, specifically the average of a range of bookies rather than taking any one organisation). The odds from a large selection of bookies for each match can be found on oddsportal.com.
The Bookies Brier score after the first round of games is 0.580. In comparison our model has a score of 0.563. Slightly better than the bookies which is great (but we’ve only played 16 of 64 matches, plenty of time for that to change!). At least we can have some confidence in our model.
So using the updated Elo ratings for each team (following the first round matches) and simulating the remaining matches in the tournament the updated probabilities for a team the win to the World Cup are:
where the changes between this and our original prediction are shown below:
This biggest change is associated with Colombia, dropping from 6th favourite (with ~5% chance) to 16th favourite with about a 1% chance. This is all associated with their shock 2-1 defeat to Japan earlier today. Germany’s loss to Mexico drops them from 2nd favourite to 6th. Even though Brazil drew with Switzerland, the model still makes them the favourite for the tournament.