<?xml version="1.0" encoding="utf-8"?><feed xmlns="http://www.w3.org/2005/Atom" xml:lang="en"><generator uri="https://jekyllrb.com/" version="4.4.1">Jekyll</generator><link href="https://seanelvidge.com/feed.xml" rel="self" type="application/atom+xml"/><link href="https://seanelvidge.com/" rel="alternate" type="text/html" hreflang="en"/><updated>2026-11-17T11:46:11+00:00</updated><id>https://seanelvidge.com/feed.xml</id><title type="html">Sean Elvidge</title><subtitle>Articles, datasets and interactive tools by Sean Elvidge, covering football statistics, mathematics, probability and space weather.</subtitle><author><name>Sean Elvidge</name></author><entry><title type="html">Does a 4–0 Win Tell Us More Than a 1–0 Win?</title><link href="https://seanelvidge.com/articles/2026/Football_rankings_and_winning_margins/" rel="alternate" type="text/html" title="Does a 4–0 Win Tell Us More Than a 1–0 Win?"/><published>2026-10-24T08:20:00+00:00</published><updated>2026-10-24T00:00:00+00:00</updated><id>https://seanelvidge.com/articles/2026/Football_rankings_and_winning_margins</id><content type="html" xml:base="https://seanelvidge.com/articles/2026/Football_rankings_and_winning_margins/"><![CDATA[<p>A 1–0 win and a 4–0 win both earn three points. For a model trying to estimate how strong a team is, though, the score might tell us something extra.</p> <p>In <a href="/articles/2025/Football_team_rankings/">my original article on football rankings</a>, I described a model that tracks each team’s hidden strength, along with uncertainty about that strength. The ratings respond to how surprising a result was. An unexpected win moves a team more than a routine one, and a team we know little about can move more quickly than an established side.</p> <p>That article explains the main framework. This is an update to what the model learns after the final whistle: it now uses <strong>the margin of victory</strong>, as well as who won.</p> <p>My suspicion was that this might make very little difference. Football gives us deflections, red cards, missed penalties and the occasional afternoon when every shot goes in. A bigger score does not necessarily mean a much better team.</p> <p>But we have a long <a href="/articles/2024/All_England_football_league_results/">database of English league results</a>, so this is something we can test.</p> <h2 id="does-it-actually-help">Does it actually help?</h2> <p>The comparison uses <strong>20,096 matches across the top four divisions</strong>, covering 2016/17 to 2025/26. For every match, each model makes its prediction using only information available before that match. It then sees the result, updates the ratings, and moves on.</p> <p>The margin model’s settings were chosen using earlier seasons, 2000/01 to 2015/16. Both models are scored on exactly the same later matches.</p> <table> <thead> <tr> <th>Measure</th> <th style="text-align: right">Using results alone</th> <th style="text-align: right">Using results and winning margins</th> </tr> </thead> <tbody> <tr> <td>Log loss</td> <td style="text-align: right">1.035789</td> <td style="text-align: right"><strong>1.033491</strong></td> </tr> <tr> <td>Brier score</td> <td style="text-align: right">0.207478</td> <td style="text-align: right"><strong>0.206965</strong></td> </tr> </tbody> </table> <p>Lower is better for both. Log loss particularly penalises confident predictions that turn out to be wrong. The Brier score measures how far the predicted probabilities were from the outcome that happened; here it is averaged over the three possible outcomes.</p> <p>The improvement is small: about <strong>0.22% in log loss</strong> and <strong>0.25% in Brier score</strong>. Those are reductions in prediction error, not percentage-point increases in the number of winners correctly picked. Football remains very capable of ruining a good forecast.</p> <p>So I was right to suspect that margin of victory is not particular impactful in our predictions. However the improvement is consistent, it is present in all ten evaluation seasons:</p> <div class="row mt-3"> <div class="col-sm mt-3 mt-md-0"> <figure> <picture> <source class="responsive-img-srcset" srcset="/assets/img/football-margin-season-validation-480.webp 480w,/assets/img/football-margin-season-validation-800.webp 800w,/assets/img/football-margin-season-validation-1400.webp 1400w," sizes="95vw" type="image/webp"/> <img src="/assets/img/football-margin-season-validation.png" class="img-fluid rounded z-depth-1" width="100%" height="auto" alt="Positive reductions in log loss in all ten seasons from 2016/17 to 2025/26, ranging from about 0.0005 to 0.0046. The overall reduction is 0.0023." data-zoomable="" loading="eager" onerror="this.onerror=null; $('.responsive-img-srcset').remove();"/> </picture> <figcaption class="caption">Each bar compares the same matches with and without winning-margin information. Taller bars mean a larger improvement. The dashed line is the overall average, weighted by the number of matches.</figcaption> </figure> </div> </div> <p>The statistical checks also support an improvement across the period as a whole; the details are below. The results are encouraging evidence, with the usual need to keep checking what happens next.</p> <h2 id="a-big-win-needs-some-context">A big win needs some context</h2> <p>Suppose a strong favourite wins 4–0 at home. That might be a good performance, but a comfortable win was already plausible. An underdog winning by the same score is much more surprising.</p> <p>This update to the ranking model accounts for that distinction. It asks how large a winning margin was expected <strong>given the teams’ strengths and which team won</strong>. The score can then add evidence beyond the win itself.</p> <p>This also allows something that initially sounds odd: a strong favourite can win narrowly and have its rating fall. The team won, but the overall performance supplied less evidence of its strength than the model expected. Conversely, a team can lose narrowly and come out looking a little better.</p> <p>There is a deliberate limit here. The model uses goal difference, so 1–0 and 2–1 supply the same information. Likewise, 0–0 and 2–2 are treated alike. Modelling attacking and defensive strength separately would (and may well) be a future extension; it is not what this update does.</p> <h2 id="what-about-the-final-league-table">What about the final league table?</h2> <p>There is a related change to the <a href="/tableProbs">league table predictor</a>. Early in the season, its forecasts were too confident about the range of possible final points totals.</p> <p>The tool still holds team strengths fixed within a simulated season. But it now allows for uncertainty about what those fixed strengths are: for each simulated season it draws one strength for each team, uses it for all that team’s remaining matches, then draws again for the next simulated season.</p> <p>If we have underestimated a team, that can affect thirty remaining matches together. With only three matches left, there is much less time for the same uncertainty to change the table. This gives the early-season forecast a wider, more realistic spread without needing to ‘invent’ a sequence of future changes in form.</p> <p>In historical checks, an 80% range for a team’s final points total contained the actual total only <strong>59.75%</strong> of the time at the start of the season. With the updated ratings and shared strength uncertainty, that becomes <strong>81.08%</strong>. The chart below separates the contribution of the new ratings from the contribution of carrying their uncertainty into the simulations:</p> <div class="row mt-3"> <div class="col-sm mt-3 mt-md-0"> <figure> <picture> <source class="responsive-img-srcset" srcset="/assets/img/football-league-uncertainty-validation-480.webp 480w,/assets/img/football-league-uncertainty-validation-800.webp 800w,/assets/img/football-league-uncertainty-validation-1400.webp 1400w," sizes="95vw" type="image/webp"/> <img src="/assets/img/football-league-uncertainty-validation.png" class="img-fluid rounded z-depth-1" width="100%" height="auto" alt="At the start of the season, actual coverage of 80 percent final-points intervals rises from 59.75 percent with results-only fixed estimates, to 63.42 percent with margin-based fixed estimates, to 81.08 percent when rating uncertainty is sampled." data-zoomable="" loading="lazy" onerror="this.onerror=null; $('.responsive-img-srcset').remove();"/> </picture> <figcaption class="caption">An 80% forecast interval should contain the eventual points total about 80% of the time. Sampling one uncertain strength per team brings coverage much closer to that target, especially when most of the season remains.</figcaption> </figure> </div> </div> <p>Across all the tested stages of the season, the combined changes improve the final-position probability score by about <strong>5%</strong>. That is a separate result from the much smaller improvement in individual match predictions. The <a href="/articles/2025/League_table_prediction_probabilities/">league-position article</a> explains the simulation and its assumptions in more detail.</p> <h2 id="part-ii--if-you-dare-read-on-the-mathematics">Part II — If You Dare Read On: The Mathematics</h2> <h2 id="1-keep-the-result-model-add-a-conditional-margin">1. Keep the result model, add a conditional margin</h2> <p>The Davidson outcome model from the <a href="/articles/2025/Football_team_rankings/">original article</a> remains:</p> \[\Delta=\beta(s_H-s_A)+h,\] \[(p_H,p_D,p_A)=\frac{(e^\Delta,\kappa,e^{-\Delta})}{e^\Delta+\kappa+e^{-\Delta}},\] <p>with \(\beta=0.9\). Strengths are forecast to the match date using the same calendar-time evolution as before.</p> <p>For a decisive result, let \(z=1\) for a home win and \(z=-1\) for an away win. Define the <em>excess winning margin</em></p> \[k=|g_H-g_A|-1.\] <p>A one-goal win has \(k=0\); a four-goal win has \(k=3\). Conditional on the winner, model this with a geometric distribution:</p> \[P(k\mid z,\Delta)=\frac{\lambda^k}{(1+\lambda)^{k+1}},\] <p>with \(k=0,1,\ldots\) and</p> \[\lambda=\exp(\alpha+c z\Delta).\] <p>Its mean is \(\lambda\) and variance is \(\lambda(1+\lambda)\), allowing occasional large margins. The selected slope is \(c=0.75\), while \(\alpha\) is learned separately for each division tier.</p> <p>The joint likelihood is</p> \[P(z,k\mid\Delta)=P(z\mid\Delta)P(k\mid z,\Delta).\] <p>Conditioning on the winner matters: the extra term describes how large the win was, so the win itself is not counted twice. Summing over all possible margins recovers the original Davidson result probabilities. A draw has no additional margin term.</p> <p>The current score is used only after the pre-match prediction has been recorded. Its information changes later predictions.</p> <h2 id="2-updating-the-strengths">2. Updating the strengths</h2> <p>Write</p> \[\begin{aligned} m&amp;=p_H-p_A,\\ V_z&amp;=p_H+p_A-m^2,\\ t&amp;=\frac{\lambda}{1+\lambda}. \end{aligned}\] <p>For a decisive result, the score and observed information in \(\Delta\) are</p> \[g_\Delta=z-m+c z\{k-(k+1)t\},\] \[J_\Delta=V_z+c^2(k+1)t(1-t).\] <p>Here, “score” means the derivative of the log likelihood, rather than the numbers on the scoreboard. The first term in each expression comes from the result; the second comes from the margin. For a draw, use \(z=0\) and omit the margin terms.</p> <p>With the prior marginal variances \(v_H,v_A\), define</p> \[\begin{aligned} g&amp;=\beta g_\Delta,\\ J&amp;=\beta^2J_\Delta,\\ D&amp;=1+(v_H+v_A)J. \end{aligned}\] <p>The update is then</p> \[\begin{aligned} s_H^+&amp;=s_H+\frac{v_Hg}{D},\\ s_A^+&amp;=s_A-\frac{v_Ag}{D}, \end{aligned}\] \[\begin{aligned} v_H^+&amp;=v_H-\frac{v_H^2J}{D},\\ v_A^+&amp;=v_A-\frac{v_A^2J}{D}. \end{aligned}\] <p>This is one Gaussian update using the joint likelihood’s derivatives. For draws it reduces to the results-only categorical update. As before, the filter retains marginal variances and discards the covariance induced between the two teams, so it remains an approximation.</p> <p>After each complete date, the tier’s home advantage, log draw weight and margin intercept are updated using their likelihood scores, with learning rate 0.001. The margin intercept starts at \(\log 0.8\). Every match on a given date uses the shared parameters available before that date’s results, preventing the ordering of same-day matches from changing their forecasts.</p> <h2 id="3-how-convincing-is-the-improvement">3. How convincing is the improvement?</h2> <p>The development comparison tested conditional geometric and Poisson excess-margin distributions, five margin slopes, and four process variances. Results-only controls were allowed the same process-variance choices. The winning margin model selected the existing weekly process variance of 0.0017, so the gain is not simply the result of making it more responsive than the comparator.</p> <p>For log loss, the mean difference between the margin model and the results-only model is <strong>−0.002298</strong>. Resampling whole seasons gives a 95% interval of <strong>[−0.003089, −0.001565]</strong>. For the Brier score, the difference is <strong>−0.000513</strong>, with interval <strong>[−0.000700, −0.000346]</strong>. Both favour the margin model throughout their intervals.</p> <div class="row mt-3"> <div class="col-sm mt-3 mt-md-0"> <figure> <picture> <source class="responsive-img-srcset" srcset="/assets/img/football-margin-confidence-intervals-480.webp 480w,/assets/img/football-margin-confidence-intervals-800.webp 800w,/assets/img/football-margin-confidence-intervals-1400.webp 1400w," sizes="95vw" type="image/webp"/> <img src="/assets/img/football-margin-confidence-intervals.png" class="img-fluid rounded z-depth-1" width="100%" height="auto" alt="Relative reductions in log loss and Brier score are about 0.22 and 0.25 percent, with both 95 percent intervals wholly above zero." data-zoomable="" loading="lazy" onerror="this.onerror=null; $('.responsive-img-srcset').remove();"/> </picture> <figcaption class="caption">The same score differences expressed as percentages of the results-only baseline. Whiskers are 95% intervals from resampling whole seasons; zero would mean no improvement.</figcaption> </figure> </div> </div> <p>A two-sided paired sign-flip test at season level gives <strong>p = 0.001953</strong> for both measures. Using seasons as blocks preserves dependence between matches within a season, rather than pretending that every match is an independent experiment. There are only ten evaluation seasons, though, and adjacent seasons need not be completely independent either.</p> <p>The mean improvement is present in all four tiers, but the evidence is weaker when each tier is tested separately. In particular, the Premier League’s log-loss sign-flip test does not reach the 5% threshold. The strong pooled result should not be read as proof of significance in every division.</p> <h2 id="4-carrying-uncertainty-into-a-season">4. Carrying uncertainty into a season</h2> <p>For simulation \(b\), draw</p> \[S_i^{(b)}\sim\mathcal{N}(\widehat s_i,v_i)\] <p>once for each team, using its rating forecast at the simulation cutoff. Use that same \(S_i^{(b)}\) in every remaining fixture involving the team. Each fixture is drawn once, and its result awards points to both opponents.</p> <p>For a final points total \(T\), the relevant identity is</p> \[\begin{aligned} \operatorname{Var}(T)&amp;=\mathbb{E}[\operatorname{Var}(T\mid S)]\\ &amp;\quad+\operatorname{Var}(\mathbb{E}[T\mid S]). \end{aligned}\] <p>Fixing strength at its estimated mean misses the variation in expected points across plausible strengths. With many remaining games, that shared effect can be substantial. Independently redrawing strength for each match would average much of it away.</p> <p>The league comparison uses 152 historical forecast cutoffs and 20,000 simulations per cutoff, excluding the incomplete 2019/20 League One and League Two seasons. The rank probability score compares the cumulative probabilities of finishing at or above each position with what actually happened, averaging the squared differences across the possible cutoffs. Lower is better. Sampling rating uncertainty improves this score even when comparing against the same margin-based ratings held at fixed estimates: the difference is <strong>−0.002265</strong>, with 95% interval <strong>[−0.002873, −0.001653]</strong>.</p> <p>I also tested extra uncertainty about average future form. Its development-selected setting made the points distributions too wide in the later evaluation seasons, without improving the position score over sampling the existing rating uncertainty.</p> <p>The complete rating history has been replayed with the margin update. You can explore it through the <a href="/teamRankings">rankings history</a>, <a href="/matchProbs">match probability calculator</a> and <a href="/tableProbs">predicted league tables</a>.</p>]]></content><author><name>Sean Elvidge</name></author><category term="football"/><category term="mathematics"/><summary type="html"><![CDATA[How winning margins improve my football ratings, with ten seasons of validation and a closer look at uncertainty in predicted league tables.]]></summary><media:thumbnail xmlns:media="http://search.yahoo.com/mrss/" url="https://seanelvidge.com/assets/img/football-margin-season-validation.png"/><media:content medium="image" url="https://seanelvidge.com/assets/img/football-margin-season-validation.png" xmlns:media="http://search.yahoo.com/mrss/"/></entry><entry><title type="html">How long until a double hat-trick in the Premier League?</title><link href="https://seanelvidge.com/articles/2026/How_long_until_a_double_hattrick/" rel="alternate" type="text/html" title="How long until a double hat-trick in the Premier League?"/><published>2026-09-04T22:00:00+00:00</published><updated>2026-09-04T22:00:00+00:00</updated><id>https://seanelvidge.com/articles/2026/How_long_until_a_double_hattrick</id><content type="html" xml:base="https://seanelvidge.com/articles/2026/How_long_until_a_double_hattrick/"><![CDATA[<p>Five players have scored five goals in a single Premier League match.</p> <p>Andrew Cole did it for Manchester United against Ipswich in 1995. Alan Shearer followed for Newcastle against Sheffield Wednesday in 1999. Jermain Defoe, Dimitar Berbatov and Sergio Agüero later joined them.</p> <p>But nobody has ever scored six.</p> <p>A hat-trick is sufficiently common that we normally see several every season. Four goals in a match is unusual, but hardly unprecedented. Five is exceptional. Six — the <em>double hat-trick</em> — has still never happened in the Premier League.</p> <p>That raises an obvious question:</p> <p><strong>Is the Premier League overdue one?</strong></p> <p>Or, put another way, after more than three decades of Premier League football, should we actually be surprised that nobody has managed it?</p> <h2 id="how-rare-are-big-goal-hauls">How rare are big goal hauls?</h2> <p>From the start of the Premier League in 1992–93 through to 2025–26, there have been 30,561 occasions on which a player has scored at least one goal in a match, excluding own goals.</p> <p>They break down like this:</p> <table> <thead> <tr> <th style="text-align: right">Goals by one player</th> <th style="text-align: right">Number of times</th> </tr> </thead> <tbody> <tr> <td style="text-align: right">1</td> <td style="text-align: right">27,285</td> </tr> <tr> <td style="text-align: right">2</td> <td style="text-align: right">2,871</td> </tr> <tr> <td style="text-align: right">3</td> <td style="text-align: right">363</td> </tr> <tr> <td style="text-align: right">4</td> <td style="text-align: right">37</td> </tr> <tr> <td style="text-align: right">5</td> <td style="text-align: right">5</td> </tr> <tr> <td style="text-align: right">6 or more</td> <td style="text-align: right"><strong>0</strong></td> </tr> </tbody> </table> <p>There is already quite a striking pattern here.</p> <p>Roughly speaking, moving from one scoring level to the next happens about one time in ten:</p> <ul> <li>about 10.5% as many braces as single-goal performances;</li> <li>about 12.6% as many hat-tricks as braces;</li> <li>about 10.2% as many four-goal performances as hat-tricks;</li> <li>and about 13.5% as many five-goal performances as four-goal performances.</li> </ul> <p>That does not mean that somebody who has already scored five has a 10–13% chance of immediately scoring a sixth. These are frequencies across complete player performances, not probabilities during an individual match.</p> <p>But it does suggest something useful: the number of extreme scoring performances falls away rapidly, while the fall-off is reasonably regular.</p> <p>If that pattern continues, six goals should be very rare — but perhaps not impossibly rare.</p> <h2 id="a-statistical-model-for-goal-hauls">A statistical model for goal hauls</h2> <p>The obvious starting point for modelling goals is the <a href="https://en.wikipedia.org/wiki/Poisson_distribution">Poisson distribution</a>. It is widely used for counts of relatively rare events and often provides a reasonable first approximation to football scores.</p> <p>There is a problem here though.</p> <p>A Poisson model in its simplest form assumes a single underlying scoring rate. But clearly not every player in every match has the same chance of scoring.</p> <p>A centre-back playing for a relegation-threatened team does not have the same scoring prospects as Erling Haaland playing up front for Manchester City. The same player can have very different prospects from one match to another depending on the opposition, minutes played, penalties, red cards, tactics and simply how the match develops.</p> <p>So rather than assuming a single scoring rate, we can imagine lots of different scoring rates mixed together.</p> <p>Mathematically, doing this in a particular natural way leads to something called a <a href="https://en.wikipedia.org/wiki/Negative_binomial_distribution">negative binomial distribution</a>.</p> <p>The name is much less important than the idea. We are saying:</p> <blockquote> <p>goals might behave roughly like rare random events within a particular player-match, but the underlying chance of those goals occurring varies enormously between player-matches.</p> </blockquote> <p>That variation creates a slightly “fatter tail” than a simple Poisson model: very large scoring performances occur more often than we would expect if everyone had the same underlying scoring rate.</p> <p>And that turns out to describe the Premier League data remarkably well.</p> <h2 id="what-does-the-model-predict">What does the model predict?</h2> <p>Fitting the model to the observed one-, two-, three-, four- and five-goal performances gives the following expected numbers over Premier League history:</p> <table> <thead> <tr> <th style="text-align: right">Goals</th> <th style="text-align: right">Actually observed</th> <th style="text-align: right">Model expectation</th> </tr> </thead> <tbody> <tr> <td style="text-align: right">1</td> <td style="text-align: right">27,285</td> <td style="text-align: right">27,280</td> </tr> <tr> <td style="text-align: right">2</td> <td style="text-align: right">2,871</td> <td style="text-align: right">2,889</td> </tr> <tr> <td style="text-align: right">3</td> <td style="text-align: right">363</td> <td style="text-align: right">342</td> </tr> <tr> <td style="text-align: right">4</td> <td style="text-align: right">37</td> <td style="text-align: right">43</td> </tr> <tr> <td style="text-align: right">5</td> <td style="text-align: right">5</td> <td style="text-align: right">5.5</td> </tr> <tr> <td style="text-align: right">6+</td> <td style="text-align: right"><strong>0</strong></td> <td style="text-align: right"><strong>0.83</strong></td> </tr> </tbody> </table> <p>The agreement is perhaps better than I would have expected.</p> <p>Most importantly, the model predicts that by now we should have seen about 0.83 performances with six or more goals.</p> <p>At first sight that might sound as though we are overdue one. After all, 0.83 is getting fairly close to 1.</p> <p>But expected values do not work quite like that.</p> <p>If an event occurs randomly with an expected count of 0.83 over some period, the probability of seeing <em>none at all</em> is approximately</p> \[P(0)=e^{-0.83}\approx0.44.\] <p>So there is about a <strong>44% probability that the Premier League would still have had no double hat-trick by now</strong>.</p> <p>That is the result I find most interesting.</p> <p>The fact that nobody has scored six Premier League goals in a match is not particularly surprising.</p> <p>There was almost a fifty-fifty chance that we would still be waiting.</p> <p>At the same time, six goals is no longer so extreme that seeing it would be statistically extraordinary. The model says it is rare, but entirely plausible.</p> <h2 id="so-when-might-it-happen">So when might it happen?</h2> <p>The fitted model gives a probability of approximately</p> \[2.7\times10^{-5}\] <p>that a Premier League scoring performance contains six or more goals.</p> <p>That is about one in every 36,900 occasions on which a player scores in a match.</p> <p>A modern 380-match Premier League season produces roughly 880 scoring player-performances. That translates into about a</p> <p><strong>2.4% chance of at least one double hat-trick in any particular season.</strong></p> <p>Another way of expressing that is an average waiting time of roughly <strong>42 seasons</strong>.</p> <p>Again, that does not mean we should expect one exactly every 42 years. Rare events do not arrive according to a timetable. We might get one next weekend, or wait another century.</p> <p>It is more useful to look at the cumulative probability:</p> <table> <thead> <tr> <th style="text-align: right">From now</th> <th style="text-align: right">Chance of at least one six-goal performance</th> </tr> </thead> <tbody> <tr> <td style="text-align: right">1 season</td> <td style="text-align: right">2.4%</td> </tr> <tr> <td style="text-align: right">5 seasons</td> <td style="text-align: right">11%</td> </tr> <tr> <td style="text-align: right">10 seasons</td> <td style="text-align: right">21%</td> </tr> <tr> <td style="text-align: right">20 seasons</td> <td style="text-align: right">38%</td> </tr> <tr> <td style="text-align: right">30 seasons</td> <td style="text-align: right"><strong>51%</strong></td> </tr> <tr> <td style="text-align: right">40 seasons</td> <td style="text-align: right">62%</td> </tr> </tbody> </table> <p>On this model, therefore, there is roughly a one-in-two chance that somebody scores six or more goals in a Premier League match within the next 30 seasons.</p> <p>There is, of course, considerable uncertainty in that number. We are trying to estimate the frequency of something that has never actually happened, using only five five-goal performances and 37 four-goal performances to tell us about the very far end of the distribution.</p> <p>A reasonable uncertainty range puts the average rate at somewhere between approximately one every 27 seasons and one every 75 seasons.</p> <p>But the qualitative conclusion does not really change.</p> <h2 id="not-overdue--but-not-unimaginable">Not overdue — but not unimaginable</h2> <p>This is a useful example of the distinction between an event being unobserved and an event being surprisingly absent.</p> <p>Nobody has ever scored six in a Premier League match. That makes it historically exceptional, but it does not mean something statistically peculiar has happened.</p> <p>Based on the pattern of every scoring performance since 1992, we would have expected about 0.83 double hat-tricks by now. There is still around a 44% chance that the correct number would be zero.</p> <p>So the Premier League is not overdue a double hat-trick.</p> <p>But nor would the first one be some wildly improbable statistical freak.</p> <p>When somebody eventually scores six, the headlines will quite reasonably describe it as unprecedented.</p> <p>The mathematics will say something slightly different:</p> <p><strong>unprecedented, yes; unexpected, no.</strong></p> <h2 id="acknowledgements">Acknowledgements</h2> <p>The data in this post is derived from data provided in the <a href="https://github.com/schochastics/football-data">https://github.com/schochastics/football-data</a> repository by David Schoch, used under the Open Data Commons Attribution License (ODC-By) v1.0.</p>]]></content><author><name>Sean Elvidge</name></author><category term="football"/><category term="mathematics"/><summary type="html"><![CDATA[No player has scored six goals in a single Premier League match - how surprising is that?]]></summary><media:thumbnail xmlns:media="http://search.yahoo.com/mrss/" url="https://seanelvidge.com/assets/img/topScorers.png"/><media:content medium="image" url="https://seanelvidge.com/assets/img/topScorers.png" xmlns:media="http://search.yahoo.com/mrss/"/></entry><entry><title type="html">Space Weather - The Musical</title><link href="https://seanelvidge.com/articles/2026/Space_Weather_Sonification/" rel="alternate" type="text/html" title="Space Weather - The Musical"/><published>2026-06-17T22:00:00+00:00</published><updated>2026-09-06T00:00:00+00:00</updated><id>https://seanelvidge.com/articles/2026/Space_Weather_Sonification</id><content type="html" xml:base="https://seanelvidge.com/articles/2026/Space_Weather_Sonification/"><![CDATA[<p>What does a geomagnetic storm sound like?</p> <p>Not metaphorically. Literally.</p> <p>Imagine taking the indices we use to drive our models of near-Earth space (solar radio flux, sunspot number, geomagnetic activity, ring-current disturbance) and translating it into a piece of music. This isn’t a “space-inspired” soundtrack, but a deterministic piano score where every pitch, rhythm, chord, tempo and accent is driven by real space weather indices.</p> <p>That is what this blog post is all about. About how we turn space weather events into piano music. The result is a structured musical translation of the space environment, built so that the data can be heard.</p> <h2 id="the-may-2024-superstorm">The May 2024 Superstorm</h2> <p>For an introduction to the event’s physical setting, see <a href="/articles/2024/Solar_storms_are_like_buses/">solar storms and the solar cycle</a>. To explore how unusual storms are quantified, read the <a href="/articles/2019/Maths_on_a_Mug_8/">extreme value theory explainer</a>.</p> <figure> <iframe src="/assets/sonification/May_Storm_20240505-20240515.pdf" class="rounded z-depth-1" frameborder="0" allow="accelerometer; autoplay; clipboard-write; encrypted-media; gyroscope; picture-in-picture; web-share" allowfullscreen="" width="100%" height="700" title="May 2024 storm score"/> </figure> <figure> <audio src="/assets/sonification/May_Storm_20240505-20240515.mp3" controls=""/> </figure> <p>(Click play and listen along whilst you read the rest of the post explaining where the music comes from)</p> <h2 id="a-storm-compressed-into-the-hands-of-a-pianist">A storm, compressed into the hands of a pianist</h2> <p>The example above uses data from May 5th to 15th 2024. This 10-day window captures the period around the major May 2024 geomagnetic storm — an event of intense aurora, severe geomagnetic activity and unusually (at least in recent times) dynamic near-Earth conditions.</p> <p>Musically, the piece is placed onto a fixed daily grid:</p> <ul> <li>one day becomes four bars;</li> <li>each bar is in 6/4;</li> <li>one day is therefore 24 quarter-note beats;</li> <li>one hour corresponds to one quarter-note beat;</li> <li>thirty minutes corresponds to half a beat.</li> </ul> <p>So however expressive the notes become (more details below), the structure remains anchored in time. Every layer of space weather indices realigns at midnight. Each day takes up the same “musical space”, allowing the changing behaviour of the Sun-Earth system to become clear through changes in texture, pitch, rhythm and intensity.</p> <h2 id="five-indices-five-jobs">Five indices, five jobs</h2> <p>Here we use five space weather indices, and each are given a different job to do in defining the music:</p> <ul> <li>F10.7: the adjusted solar radio flux, controls the harmonic root in the left hand.</li> <li>Sunspot number: adds ‘density’ to the left-hand. When the sunspot number is sufficiently high, we add an octave root.</li> <li>Hp30: drives the right-hand melody. Because Hp30 is available every 30 minutes, it provides a natural melodic line.</li> <li>Kp: controls the broad rhythmic regime. Quiet geomagnetic periods lead to slower figures; storm periods create denser patterns.</li> <li>Dst: as it becomes more negative, the music becomes more forceful through dynamics, accents and tempo.</li> </ul> <p>I was keen to use as many indices as possible to give the music as much ‘texture’ as possible - entirely deterministically. I have tried to make solar activity shape harmony, geomagnetic activity shape motion and storm intensity shape tension.</p> <h2 id="the-right-hand-melody-hp30">The right hand melody: Hp30</h2> <p>The right hand is the most active. It uses a pitch drawn from F# minor (my favourite), ranging from F#4 to E6. Each Hp30 value is normalised and mapped to one of fourteen pitches.</p> <p>This means that higher geomagnetic activity tends to push the melody upward. But the mapping is not absolute. To create the score we do some local normalisation, i.e. we care both about how large Hp30 is on a physical scale and how large it is relative to the other values in the selected date range. This was needed so that even quiet periods (geomagnetically) stay interesting.</p> <p>The Kp (a logarithmic scale from 0 to 9) is used to define the rhythm:</p> <ul> <li>below Kp 5, the rhythm is relatively slow;</li> <li>from Kp 5 to 7, the rhythm becomes more active;</li> <li>above Kp 7, each hour becomes a four-note semiquaver figure.</li> </ul> <p>But the exact rhythm inside each hour also responds to local movement in Hp30 and Dst. If Hp30 jumps within the hour, or if Dst changes sharply from one hour to the next, the right hand becomes more animated. A quiet Kp period is therefore not forced to be dull if the other data are still moving.</p> <h2 id="the-left-hand-harmony-and-tension">The left hand: harmony and tension</h2> <p>The left hand provides the harmonic ‘frame’ of the piece. It chooses roots from an ordered set:</p> <p>F#, A, B, C#, D, E, G#.</p> <p>These are not arranged chromatically but arranged to give useful harmonies inside the F# minor world we’re working in.</p> <p>Each day has four bars, and each bar receives one harmonic root. F10.7 selects the base root. Then the day’s pattern of F10.7 and sunspot-number change determines how the four roots move. If solar activity and sunspots are both rising, the harmony tends to climb. If both are falling, it descends. If they disagree, the progression takes a mixed path.</p> <p>The chords themselves are simple diatonic triads: F# minor, A major, B minor, C# minor, D major, E major and G# diminished. This keeps the harmonic language coherent while still allowing the data to move the music through different regions of the scale.</p> <p>Dst and Kp then decide how much weight the left hand carries. In low-tension periods, a bar may simply be one long chord. In moderate tension, a bass note is added before the chord. In high tension, the left hand pulses with repeated bass-plus-chord figures. A storm is therefore not only heard in the treble melody but also changes the ‘weight’ of the piano texture.</p> <h2 id="tempo-dynamics-and-accents">Tempo, dynamics and accents</h2> <p>The score also has some performance notes in it. Whilst the overall tempo of the piece remains constant throughout, average note length is (slightly) determined by solar activity, Hp30, Dst, and Kp can all push the tempo upward (having an impact of making the piece feel like the speed is changing, but remaining playable (at least to my limits!)).</p> <p>We also use four dynamic levels: p, mp, mf and f. This allows for clear changes in intensity without going over the top. Accents appear when the normalised activity is high enough. Staccato is added to the shortest right-hand notes. During intense storm intervals, the music becomes not just higher or faster, but sharper and more articulated.</p> <h2 id="why-this-is-more-than-sonification">Why this is more than sonification</h2> <p>Many data-to-music projects work by assigning one variable to pitch and another to volume. That can be effective, but it often produces a thin musical result. This piano mapping is a little more ambitious. It treats the space weather system as a set of interacting musical lines:</p> <ul> <li>long-timescale solar conditions become harmony;</li> <li>short-timescale geomagnetic changes become melody;</li> <li>storm thresholds become rhythmic regimes;</li> <li>Dst depression becomes tension;</li> <li>sunspot number becomes chord density;</li> <li>daily evolution becomes harmonic progression.</li> </ul> <p>This is also all deterministic. Given the same date range then the same notes, rhythms, dynamics and accents are produced. There are no random choices. This is important because it means the musical output is reproducible. The piece can be discussed, analysed and regenerated.</p> <h2 id="hearing-the-may-2024-storm">Hearing the May 2024 storm</h2> <figure> <audio src="/assets/sonification/May_Storm_20240505-20240515.mp3" controls=""/> </figure> <p>In calm moments of the space environment, the music has ‘space’. The left hand can hold long chords while the right hand moves slowly through the scale. As activity increases, the melody becomes more agile. Rhythms subdivide. The tempo rises. Accents appear. The left hand gains pulse. The storm becomes audible not as noise, but as structure.</p> <p>That is the most compelling aspect of the approach. Space weather is often communicated through plots, maps, alerts and indices. Those are essential. But music offers a different route into the same system. It lets us hear change, pressure, release, escalation and recovery.</p> <p>The May 2024 storm was a scientific event, an operational challenge and, for many people, a spectacular visual experience. In this piano version, it becomes something else as well: a short, reproducible musical portrait of a disturbed geospace environment.</p>]]></content><author><name>Sean Elvidge</name></author><category term="spaceWeather"/><category term="mathematics"/><summary type="html"><![CDATA[Listen to the May 2024 geomagnetic superstorm as a piano score, with pitch, rhythm and dynamics derived from real solar and geomagnetic indices.]]></summary><media:thumbnail xmlns:media="http://search.yahoo.com/mrss/" url="https://seanelvidge.com/assets/img/May_Storm_20240505-20240515.jpg"/><media:content medium="image" url="https://seanelvidge.com/assets/img/May_Storm_20240505-20240515.jpg" xmlns:media="http://search.yahoo.com/mrss/"/></entry><entry><title type="html">Who wins in a 48 team World Cup - Panini</title><link href="https://seanelvidge.com/articles/2026/A_48_Team_World_Cup/" rel="alternate" type="text/html" title="Who wins in a 48 team World Cup - Panini"/><published>2026-06-11T19:30:00+00:00</published><updated>2026-06-11T19:30:00+00:00</updated><id>https://seanelvidge.com/articles/2026/A_48_Team_World_Cup</id><content type="html" xml:base="https://seanelvidge.com/articles/2026/A_48_Team_World_Cup/"><![CDATA[<p>Since 1970 in Mexico <a href="https://paninistore.com/">Panini</a> have published the official World Cup sticker book. It has become synonymous with the tournament, with a huge following and has become a multi-billion dollar business. It is estimated that for this World Cup between 500 and 800 million sticker packs will be sold <a href="https://www.marca.com/en/world-cup/2026/06/05/world-cup-stickers-1-4-billion-euro-business-with-an-underlying-controversy.html">[1]</a>.</p> <p>So what does it take to complete the sticker book? And how has this changed from the last 32-team World Cup (Qatar 2022) to this, the first 48-team World Cup? Well, the 50% increase in teams, “only” leads to a 10% increase in required stickers (but a 42% increase in costs…).</p> <p>For this analysis we assume that stickers are randomly distributed in packs and are each equally likely (which has been claimed to be the case by the Panini CEO, however <a href="https://academic.oup.com/jrssig/article/18/3/5/7038156?searchresult=1&amp;login=false">research has shown</a> that ‘shiny’ stickers are systematically rarer than others by about 2x).</p> <p>To see where these numbers come from, let \(N\) by the total number of stickers in the album and \(k\) be the number of stickers in one pack. So each pack contains \(k\) <strong>distinct</strong> stickers which are sampled uniformly at random from the \(N\) possible stickers. So each pack is equally likely to be any one of</p> \[{N\choose k}\] <p>which means “\(N\) choose \(k\)” and, simply, is used to represent the number of ways to choose \(k\) items from a set of \(N\) items where order does not matter. You can calculate its numerical value using:</p> \[{N\choose k} = \frac{N!}{(N-k)!k!}\] <p>where “!” means <a href="https://en.wikipedia.org/wiki/Factorial">factorial</a>.</p> <p>So the number we are trying to find is:</p> \[E_m = \text{ expected number of additional packs needed when you already have } m \text{ distinct stickers}\] <p>and in our specific case the goal is to compute \(E_0\) because we start with zero (0) stickers. The boundary condition is \(E_N = 0\) because once you already have all \(N\) stickers, you need to more packs. Now suppose you currently have \(m\) distinct stickers, then</p> \[N-m\] <p>stickers are still missing. In the next pack of \(k\) stickers, suppose exactly \(r\) of them are new stickers. To get exactly \(r\) new stickers:</p> <ul> <li>choose \(r\) stickers from the \(N-m\) missing stickers,</li> <li>choose the remaining \(k-r\) stickers from the \(m\) stickers you already have.</li> </ul> <p>So the number of packs that contain exactly \(r\) new stickers is:</p> \[\binom{N-m}{r}\binom{m}{k-r}.\] <p>Since all possible packs are equally likely, the probability of getting exactly \(r\) new stickers is:</p> \[p_{m,r} = \frac{\binom{N-m}{r}\binom{m}{k-r}}{\binom{N}{k}},\] <p>where invalid binomial terms are treated as zero. Then, the possible values of \(r\) are:</p> \[\max(0,k-m) \le r \le \min(k,N-m)\] <p>because you cannot get more new stickers than are missing, and you cannot get more than \(k\) new stickers from one pack. So now we need fewer stickers to complete our collection, specifically we move from ‘state’ \(m\) to state \(m+r\) with probability \(p_{m,r}\). Therefore:</p> \[E_m = 1 + \sum p_{m,r}E_{m+r},\] <p>where the “1” represents the pack you just bought. However, one possible outcome is \(r=0\), meaning the pack we bought contains no new stickers. In that case we remain at state \(m\), so \(E_m\) appears on both sides:</p> \[E_m = 1 + p_{m,0}E_m+\sum_{r\ge 1}p_{m,r}E_{m+r},\] <p>which we can rearrange to get:</p> \[\begin{eqnarray*} E_m - p_{m,0}E_m &amp;=&amp; 1+\sum_{r\ge 1}p_{m,r}E_{m+r},\\ E_m(1-p_{m,0}) &amp;=&amp; 1 + \sum_{r\ge 1}p_{m,r}E_{m+r},\\ E_m &amp;=&amp; \frac{1+\sum_{r\ge 1}p_{m,r}E_{m+r}}{1-p_{m,0}}. \end{eqnarray*}\] <p>This gives an exact recursive calculation. Since \(E_m\) depends only on \(E_{m+1}, E_{m+2},\ldots,E_N\) we compute backwards from:</p> \[E_N=0\] <p>down to</p> \[E_0.\] <p>In 2022 each Panini pack contained 5 stickers and the whole album required 670 stickers. So that gives \(N=670\) and \(k=5\), using the recurrence relation described above gives:</p> \[E\approx 946.9837488\] <p>so the expected number of packs required to complete the album is 947 packs.</p> <p>In 2026, each pack contains 7 stickers and an album needs 980 stickers. So with \(N=980\) and \(k=7\), and using the defined relation the expected number of packs is:</p> \[E_0 \approx 1042.36\] <p>or about 1042 packs. The increase from 947 packs to 1042 packs is an increase of about 10% (10.031…% to be more precise).</p> <p>A multi-pack of stickers at my local supermarket costs £7.50 for 6 packs. To get the expected number of 1042 packs requires 174 multi-packs at a cost of £1,305.</p> <p>For comparison, in 2022, a 6 pack multi-pack (remembering each contained 5 stickers per pack), cost £4.99, so to get the expected number of 947 packs, 158 packs were needed, at a total cost of £788.42. Adjusting for inflation that comes to £918.89.</p> <p>So whilst we only need 10% more stickers, we should expect to spend 42% more to complete the album.</p>]]></content><author><name>Sean Elvidge</name></author><category term="football"/><category term="mathematics"/><summary type="html"><![CDATA[Who is the biggest winner of the new, larger, 48 team World Cup? Panini, with the increase in the numbers of stickers to complete their album.]]></summary><media:thumbnail xmlns:media="http://search.yahoo.com/mrss/" url="https://seanelvidge.com/assets/img/panini_sticker_book.jpeg"/><media:content medium="image" url="https://seanelvidge.com/assets/img/panini_sticker_book.jpeg" xmlns:media="http://search.yahoo.com/mrss/"/></entry><entry><title type="html">When Good Ideas Don’t Survive Contact With Data - A December Football Myth Tested</title><link href="https://seanelvidge.com/articles/2025/A_December_Football_Myth/" rel="alternate" type="text/html" title="When Good Ideas Don’t Survive Contact With Data - A December Football Myth Tested"/><published>2025-12-31T19:30:00+00:00</published><updated>2025-12-31T19:30:00+00:00</updated><id>https://seanelvidge.com/articles/2025/A_December_Football_Myth</id><content type="html" xml:base="https://seanelvidge.com/articles/2025/A_December_Football_Myth/"><![CDATA[<p>Every so often I come up with what feels like a great idea for a blog post. A clever angle, an interesting hypothesis, something that surely must be hiding in the data just waiting to be revealed. And then, after hours of analysis, the result is… nothing. Completely flat. No effect.</p> <p>This is one of those stories.</p> <p>I wanted to look at whether the famously congested English football December schedule gives an advantage to teams (either weaker or stronger teams). With matches every few days, tired legs, rotation, winter weather, and general chaos, it seemed plausible that perhaps squad rotation from the stronger sides might be key, or perhaps stronger sides might stumble more than usual and that underdogs might pick up a few unexpected points.</p> <p>It’s a nice idea. Unfortunately, as we’ll see, a nice idea does not guarantee a nice result.</p> <h2 id="why-december-might-matter">Why December Might Matter</h2> <p>Consider what happens in December in English football:</p> <ul> <li>Fixture congestion increases dramatically.</li> <li>Elite teams face multiple competitions and may rotate heavily.</li> <li>Injuries accumulate and squad depth becomes critical.</li> <li>Weather worsens, pitches soften, and match conditions become less predictable.</li> </ul> <p>Perhaps if stronger teams depend more on structure and precision, and weaker teams rely more on disruption and variance, you can build a reasonable argument that December’s chaos environment might narrow the gap.</p> <p>In other words, if \(S\) is team strength and \(\epsilon\) is “football randomness”, maybe the effective strength looks more like:</p> \[S_{\text{effective}} = S + \epsilon,\] <p>and perhaps December has a larger \(\epsilon\).</p> <p>If so, underdogs might perform better than expected.</p> <h2 id="how-to-measure-underdog-performance">How to Measure “Underdog Performance”?</h2> <p>To test this we need a <a href="https://seanelvidge.com/articles/2024/All_England_football_league_results/">very large dataset of football results</a> and a way to <a href="https://seanelvidge.com/articles/2025/Football_team_rankings/">quantify which team was stronger <em>before</em> the match</a> was played.</p> <p>In the team strength database a higher values means a stronger team.</p> <p>Thus:</p> <ul> <li>The stronger team is whichever side has the higher rank.</li> <li>The weaker team (the “underdog”) is the side with the lower rank.</li> </ul> <p>And for each match, we compute:</p> <ul> <li>Underdog points</li> <li>Underdog goal difference</li> <li>Whether the match took place in December</li> </ul> <p>Everything you’d need to test whether December helps the weaker (or stronger) team.</p> <h2 id="december-vs-non-december-the-raw-numbers">December vs Non-December: The Raw Numbers</h2> <p>Aggregating across every division, every season, and every match where a clear favourite exists, we get:</p> <ul> <li>Outside December, underdogs earn: <strong>1.178 points per match</strong></li> <li>In December, underdogs earn: <strong>1.181 points per match</strong></li> </ul> <p>The difference, <strong>0.0026 points</strong>, is essentially zero.</p> <div class="row mt-3"> <div class="col-sm mt-3 mt-md-0"> <figure> <picture> <source class="responsive-img-srcset" srcset="/assets/img/ppm_dec_nonDec-480.webp 480w,/assets/img/ppm_dec_nonDec-800.webp 800w,/assets/img/ppm_dec_nonDec-1400.webp 1400w," sizes="95vw" type="image/webp"/> <img src="/assets/img/ppm_dec_nonDec.png" class="img-fluid rounded z-depth-1" width="100%" height="auto" alt="" data-zoomable="" loading="eager" onerror="this.onerror=null; $('.responsive-img-srcset').remove();"/> </picture> </figure> </div> </div> <p>The underlying distribution of goal differences tells the same story.</p> <div class="row mt-3"> <div class="col-sm mt-3 mt-md-0"> <figure> <picture> <source class="responsive-img-srcset" srcset="/assets/img/Underdog_Goal_Differences-480.webp 480w,/assets/img/Underdog_Goal_Differences-800.webp 800w,/assets/img/Underdog_Goal_Differences-1400.webp 1400w," sizes="95vw" type="image/webp"/> <img src="/assets/img/Underdog_Goal_Differences.png" class="img-fluid rounded z-depth-1" width="100%" height="auto" alt="" data-zoomable="" loading="eager" onerror="this.onerror=null; $('.responsive-img-srcset').remove();"/> </picture> </figure> </div> </div> <p>I will admit, this was mildly disappointing. But maybe aggregate data hides the truth? Perhaps the Premier League behaves differently? Or lower leagues, with smaller squads, show something?</p> <p>Nope.</p> <p>However you slice the dataset, Tier 1, Tier 2, Tier 3, Tier 4, the same conclusion emerges:</p> <ul> <li>The difference between underdog performance in December vs other months is tiny.</li> <li>In every tier, the effect is statistically insignificant.</li> <li>Some tiers lean slightly positive, others slightly negative, all within noise.</li> </ul> <p>So weaker teams, as a group, do <strong>not</strong> seem to benefit from the December schedule.</p> <p>You might reasonably ask whether the <strong>favourites</strong> show any seasonal change. If weaker teams don’t improve, perhaps stronger teams deteriorate?</p> <p>I ran the same comparison for the stronger side in each match.</p> <p>The result? Exactly the same story.</p> <p>Stronger teams’ points and goal differences in December are statistically indistinguishable from the rest of the season. No noticeable dip, no seasonal weakness, no hidden December curse.</p> <p>In short:</p> <blockquote> <p>Neither underdogs nor favourites show any meaningful change in performance during December.</p> </blockquote> <h2 id="a-more-formal-statistical-model">A More Formal Statistical Model</h2> <p>To be thorough, I fitted a regression model of underdog points:</p> \[\text{pts} = \beta_0 + \beta_1\,\text{December} + \beta_2\,\text{Home} + C(\text{Tier}) + C(\text{Season}) + \varepsilon .\] <p>This controls for:</p> <ul> <li>Home advantage</li> <li>League tier</li> <li>Season-by-season variation</li> </ul> <p>The coefficient for the December effect came out as:</p> \[\beta_1 = -0.0018 \pm 0.013,\] <p>which is, again, indistinguishable from zero.</p> <h2 id="when-a-good-hypothesis-fails">When a Good Hypothesis Fails</h2> <p>This leads to what I think is the most important lesson from the whole exercise.</p> <p>In science many ideas do not survive contact with real data. They’re plausible, they’re elegant, and they would make for a great story, but the universe simply refuses to cooperate.</p> <p>That does not make the work wasted.</p> <p>It makes it honest.</p> <h2 id="the-academic-problem-with-negative-results">The Academic Problem With Negative Results</h2> <p>In academia, “negative” or null results are notoriously hard to publish. Journals often prefer dramatic or surprising findings, which unintentionally encourages selective reporting:</p> <ul> <li>Effects that <strong>don’t</strong> exist are quietly forgotten.</li> <li>Effects that <strong>appear by chance</strong> get published.</li> <li>The literature accumulates exciting stories but not necessarily accurate ones.</li> </ul> <p>This December analysis is a textbook example: an interesting idea, diligently tested, clearly unsupported.</p> <p>Yet these non-effects matter. They give us a clearer picture of reality, and they remind us that the absence of a pattern is itself information, and sometimes, important information.</p>]]></content><author><name>Sean Elvidge</name></author><category term="football"/><category term="mathematics"/><summary type="html"><![CDATA[Every so often I come up with what feels like a good idea for a blog post. But they don't always work out, this is the story of my investigations into the congested English football December schedule and whether it gives certain teams an advantage.]]></summary><media:thumbnail xmlns:media="http://search.yahoo.com/mrss/" url="https://seanelvidge.com/assets/img/ppm_dec_nonDec.png"/><media:content medium="image" url="https://seanelvidge.com/assets/img/ppm_dec_nonDec.png" xmlns:media="http://search.yahoo.com/mrss/"/></entry><entry><title type="html">Christmas Day Football - The Lost Tradition</title><link href="https://seanelvidge.com/articles/2025/Christmas_Day_Football/" rel="alternate" type="text/html" title="Christmas Day Football - The Lost Tradition"/><published>2025-12-25T09:00:00+00:00</published><updated>2025-12-25T09:00:00+00:00</updated><id>https://seanelvidge.com/articles/2025/Christmas_Day_Football</id><content type="html" xml:base="https://seanelvidge.com/articles/2025/Christmas_Day_Football/"><![CDATA[<p>It’s been exactly 60 years since the last English league match was played on Christmas Day. On 25 December 1965, Blackpool hosted Blackburn Rovers in a festive Lancashire derby that would become the final flicker of a long-standing tradition. While Boxing Day remains a staple of the football calendar, the idea of heading to the ground after unwrapping presents has quietly faded into the past. Here’s a look back at how Christmas Day football began, what made it special, and why it came to an end.</p> <h2 id="victorian-beginnings">Victorian Beginnings</h2> <p>The Football League didn’t include Christmas Day matches in its inaugural season (1888), but by the very next year, Preston North End hosted Aston Villa on 25 December 1889. That 3-2 win for Preston drew around 9,000 fans and marked the beginning of Christmas football as a regular fixture.</p> <p>By the early 20th century, the tradition was well-established. Local derbies were often scheduled to reduce travel, and many clubs played the same opponent home and away over Christmas and Boxing Day. It wasn’t unusual for fans to watch a game in the morning and be home in time for turkey in the afternoon.</p> <h2 id="when-christmas-meant-football">When Christmas Meant Football</h2> <p>For much of the 20th century, football on Christmas Day was as normal as carols and crackers. The interwar years and post-WWII period were the high points, with full fixture lists and packed crowds. In 1949, over 3 million fans attended league matches during the Christmas week. Players would sometimes play three matches in four days. Fans shared flasks and sang carols on the terraces.</p> <p>Holiday scheduling quirks added to the charm. Tranmere once lost 4–1 on Christmas Day, only to win the return fixture 13–4 the next day. Matches were sometimes played in snow, fog, or biting cold, but still the crowds came.</p> <p>Christmas Day 1914 was particularly notable. Despite the shadow of World War I, nine First Division matches went ahead, drawing a combined crowd of around 173,000. Football was still seen as a morale-booster, and a newspaper at the time declared that to millions:</p> <blockquote> <p>Christmas without football would not be Christmas at all. (<a href="https://www.britishnewspaperarchive.co.uk/viewer/bl/0000530/19451221/019/0003">Essex Newsman, 21 December, 1945</a>)</p> </blockquote> <p>And across the Channel that same day, one of football’s most poignant and enduring moments occurred. British and German troops along the Western Front took part in the famous 1914 Christmas Truce. In the frozen fields of Flanders, soldiers from both sides briefly set down their arms and came together to sing carols, exchange gifts, and even play informal matches in No Man’s Land.</p> <h2 id="christmas-day-classics">Christmas Day Classics</h2> <p>There were high-scoring thrillers (Chelsea 7–4 Portsmouth in 1957), and even a match in 1940 where Norwich beat a scratch Brighton team 18–0. The largest Christmas Day crowds topped 60,000 at places like Newcastle and Arsenal.</p> <p>Some stories veer into folklore: players turning up tipsy after too much Christmas cheer, supporters pelting the pitch with orange peel, and squads sharing a turkey dinner on the train between back-to-back games.</p> <p>And for Coventry fans: our own Ken Satchwell scored the last ever Football League hat-trick on Christmas Day, in a 5–3 win over Wrexham in 1959. One of the last festive hurrahs before the tradition faded.</p> <h2 id="the-final-whistle-25-december-1965">The Final Whistle: 25 December 1965</h2> <p>60 years ago today, in 1965, Blackpool beat Blackburn Rovers 4–2 in front of just over 20,000 fans. Young Alan Ball, not yet a World Cup winner, scored one of the goals. The weather was chilly but the game was lively, and the matchday programme was wrapped in festive green and tangerine.</p> <figure> <picture> <source class="responsive-img-srcset" srcset="/assets/img/Blackpool-Xmas-Day-programme-480.webp 480w,/assets/img/Blackpool-Xmas-Day-programme-800.webp 800w,/assets/img/Blackpool-Xmas-Day-programme-1400.webp 1400w," sizes="95vw" type="image/webp"/> <img src="/assets/img/Blackpool-Xmas-Day-programme.jpg" class="img-fluid rounded z-depth-1" width="100%" height="auto" alt="" data-zoomable="" loading="eager" onerror="this.onerror=null; $('.responsive-img-srcset').remove();"/> </picture> </figure> <p>That game became, quietly, the last of its kind.</p> <h2 id="why-it-ended">Why It Ended</h2> <p>The reasons are a mix of culture, logistics, and changing times. More families preferred to stay home on Christmas morning. Public transport no longer ran on Christmas Day. Policing and stewarding became harder to organise. Players were increasingly reluctant to spend Christmas away from their families.</p> <p>The spread of floodlights and TV also meant fixtures could be moved to evenings or rescheduled around the holidays. Boxing Day, a public holiday since the 19th century, proved more popular for fans and clubs alike. By the early 60s, most clubs had already opted out. No ban was needed. The tradition just slipped away.</p> <h2 id="remembering-christmas-football">Remembering Christmas Football</h2> <p>Christmas Day football was sometimes chaotic, often charming, and always memorable. It belongs to another era, one of packed terraces, handwritten programmes, and roaring fires waiting at home. Brentford tried to make it come back in 1983 when the tried to schedule a game for Christmas morning. But fans rebelled and the club backed down.</p> <p>These days, players can enjoy their mince pies in peace, and fans save their voices for Boxing Day.</p> <p>But 60 years ago today, Blackpool beat Blackburn in the last of 1,243 Christmas Day fixtures, as English football took its final bow on December 25th.</p>]]></content><author><name>Sean Elvidge</name></author><category term="football"/><summary type="html"><![CDATA[It's been exactly 60 years since the last English league match was played on Christmas Day. Here’s a look back at how Christmas Day football began, what made it special, and why it came to an end.]]></summary><media:thumbnail xmlns:media="http://search.yahoo.com/mrss/" url="https://seanelvidge.com/assets/img/Blackpool-Xmas-Day-programme.jpg"/><media:content medium="image" url="https://seanelvidge.com/assets/img/Blackpool-Xmas-Day-programme.jpg" xmlns:media="http://search.yahoo.com/mrss/"/></entry><entry><title type="html">How to Calculate League Position Probabilities</title><link href="https://seanelvidge.com/articles/2025/League_table_prediction_probabilities/" rel="alternate" type="text/html" title="How to Calculate League Position Probabilities"/><published>2025-12-22T19:30:00+00:00</published><updated>2026-10-24T00:00:00+00:00</updated><id>https://seanelvidge.com/articles/2025/League_table_prediction_probabilities</id><content type="html" xml:base="https://seanelvidge.com/articles/2025/League_table_prediction_probabilities/"><![CDATA[<p>By mid-season, football fans instinctively start doing probability in their heads.</p> <blockquote> <p>If we win our next two, and they drop points away at… actually, hang on.</p> </blockquote> <p>This usually ends with a napkin full of scribbles and a strong emotional commitment to a particular set of results. The <a href="/tableProbs">predicted league tables</a> do the same exercise many times over, playing out the remaining fixtures to see where each team could finish.</p> <p>The result is a table of probabilities: a chance for every team to finish in every position. This post explains where those numbers come from and how to read them.</p> <h2 id="estimating-the-remaining-matches">Estimating the remaining matches</h2> <p>The starting point is my <a href="/articles/2025/Football_team_rankings/">team-strength ratings</a>. These learn from past results, including <a href="/articles/2026/Football_rankings_and_winning_margins/">the margin of victory</a>, to estimate how strong each team is and how confident we should be in that estimate.</p> <p>For each remaining fixture, the teams’ strengths, home advantage and the division’s tendency to produce draws give us three probabilities: a home win, a draw, or an away win. You can explore this part with the <a href="/matchProbs">match probability calculator</a>.</p> <p>A favourite is more likely to win, but an upset remains possible. That is rather the point of football.</p> <h2 id="allowing-for-uncertainty">Allowing for uncertainty</h2> <p>Imagine we have slightly underestimated a team. With thirty matches still to play, that small error has plenty of opportunities to matter. With three matches left, there is much less time for it to change the final table.</p> <p>To allow for this, each simulated season begins by picking one plausible strength for every team, using both its rating and the uncertainty around it. We keep those strengths fixed for all the remaining matches in that run. Then we pick again for the next simulated season.</p> <p>So a team drawn stronger than our central estimate is more competitive throughout that run. An error in our estimate would affect all its games together.</p> <p>This gives early-season forecasts a wider spread, when there is still plenty of time for our estimates to prove slightly wrong. The real ratings continue to update as actual results arrive; holding them fixed is an assumption about each simulated future.</p> <h2 id="playing-out-the-rest-of-the-season">Playing out the rest of the season</h2> <p>With those strengths chosen, we play each remaining fixture once, choosing a result according to its probabilities. A win gives three points to the winner and none to the loser; a draw gives one point to each. Both opponents share the same result.</p> <p>Add those points to the current standings, including recorded points deductions, and we have one possible final league table. Repeat the exercise many times and count how often each team finishes first, second, third, and so on.</p> <p>If a team finishes sixth in about 12 out of every 100 simulated seasons, its estimated chance of finishing sixth is about 12%.</p> <p>Teams tied on points are put in a random order. Real leagues use goal difference and other rules, so this is an approximation, particularly when teams are closely matched. The displayed percentages are also rounded estimates from a finite number of simulations.</p> <h2 id="reading-the-table">Reading the table</h2> <p>Find your division, then read across your team’s row. Each column is a possible finishing position.</p> <ul> <li><strong>12%</strong> means an estimated 12% chance of finishing in that position.</li> <li><strong>&lt;1%</strong> marks a very small estimated chance, including rare finishes for which an example has been found.</li> <li><strong>&lt;&lt;1%</strong> means the finish was not observed in the simulations and no example has yet been found. The calculation has not resolved how small its chance is.</li> <li><strong>–</strong> means the position has been ruled out by the tool’s checks on possible points totals.</li> </ul> <p>The strongest probability in each row is highlighted, and other positions are shaded relative to it. Look at the spread across the row: the most likely single position can still have a fairly small chance if several neighbouring places are plausible.</p> <p>Here is an example of the layout:</p> <div class="row mt-3"> <div class="col-sm mt-3 mt-md-0"> <figure> <picture> <source class="responsive-img-srcset" srcset="/assets/img/position-odds-2025-2026-EFL_Championship-480.webp 480w,/assets/img/position-odds-2025-2026-EFL_Championship-800.webp 800w,/assets/img/position-odds-2025-2026-EFL_Championship-1400.webp 1400w," sizes="95vw" type="image/webp"/> <img src="/assets/img/position-odds-2025-2026-EFL_Championship.png" class="img-fluid rounded z-depth-1" width="100%" height="auto" alt="Example table with Championship teams in rows and possible finishing positions in columns, shaded by probability." data-zoomable="" loading="eager" onerror="this.onerror=null; $('.responsive-img-srcset').remove();"/> </picture> <figcaption class="caption">An example of the table layout, from the 2025/26 Championship forecasts.</figcaption> </figure> </div> </div> <p>On a mobile screen, a compact summary shows the chance of finishing first, in the top six, or in the bottom three. You can switch to the full table, and download it as an image for closer inspection. Selecting a position in the full table can also show an example set of results that would produce that finish, where one has been found.</p> <p>The top-six and bottom-three figures describe finishing places. Promotion, play-offs and relegation depend on the rules of the particular division, so those labels do not always correspond directly to going up or down.</p> <h2 id="what-the-forecast-assumes">What the forecast assumes</h2> <p>Recent results inform the ratings, but the forecast does not try to guess the next injury crisis, transfer window, managerial change or future points deduction. It asks:</p> <blockquote> <p>Given what we know right now, how wide is the space of possible final tables?</p> </blockquote> <p>Sometimes that space is narrow. Sometimes it can be surprisingly large. A team can have the best chance of winning the league while still being more likely to finish somewhere else.</p> <p>The <a href="/tableProbs">tables</a> are refreshed as the <a href="/articles/2024/All_England_football_league_results/">football database</a> is updated. Check the season and update information shown on the page when reading a forecast.</p> <h2 id="does-it-give-sensible-probabilities">Does it give sensible probabilities?</h2> <p>We can test the approach by going back to past seasons and making forecasts using only what was known at the time. The checks cover 152 forecasts across the top four divisions from 2016/17 to 2025/26, made at the start and roughly one-quarter, one-half and three-quarters of each season. The incomplete 2019/20 League One and League Two seasons are excluded.</p> <p>One useful check is the range of final points totals. An <strong>80% range</strong> should contain the eventual total about four times out of five. At the start of the historical seasons tested, the earlier forecast managed only <strong>59.75%</strong>: it was too confident. With ratings that use winning margins and uncertainty carried through each simulated season, that becomes <strong>81.08%</strong>.</p> <div class="row mt-3"> <div class="col-sm mt-3 mt-md-0"> <figure> <picture> <source class="responsive-img-srcset" srcset="/assets/img/football-league-uncertainty-validation-480.webp 480w,/assets/img/football-league-uncertainty-validation-800.webp 800w,/assets/img/football-league-uncertainty-validation-1400.webp 1400w," sizes="95vw" type="image/webp"/> <img src="/assets/img/football-league-uncertainty-validation.png" class="img-fluid rounded z-depth-1" width="100%" height="auto" alt="Actual coverage of 80 percent final-points ranges is close to the intended 80 percent at all four forecast stages after sampling uncertain strengths. At the start of the season, coverage rises from 59.75 percent to 81.08 percent." data-zoomable="" loading="lazy" onerror="this.onerror=null; $('.responsive-img-srcset').remove();"/> </picture> <figcaption class="caption">An 80% range should contain the eventual points total about 80% of the time. Grey uses results-only ratings; orange adds winning margins; green also allows for uncertainty in team strength. The largest improvement is at the start of the season.</figcaption> </figure> </div> </div> <p>Across all four stages, the 80% ranges contain <strong>81.16%</strong> of the actual totals, and the 95% ranges contain <strong>94.18%</strong>. The combined changes also improve the score used to assess the full spread of finishing-position probabilities by about <strong>5%</strong>. Simply making every range wider would not guarantee a better score.</p> <p>There is still room to improve: the early-season 95% ranges remain a little narrow. These seasons had already been examined in earlier reviews of the model, so future results will provide a cleaner test. The <a href="/articles/2026/Football_rankings_and_winning_margins/">ratings and validation article</a> has the statistical details for anyone who would like to go further.</p>]]></content><author><name>Sean Elvidge</name></author><category term="football"/><category term="mathematics"/><summary type="html"><![CDATA[How the football league table predictor plays out the remaining fixtures, allows for uncertain team strengths, and estimates each club's possible finishing positions.]]></summary><media:thumbnail xmlns:media="http://search.yahoo.com/mrss/" url="https://seanelvidge.com/assets/img/position-odds-2025-2026-EFL_Championship.png"/><media:content medium="image" url="https://seanelvidge.com/assets/img/position-odds-2025-2026-EFL_Championship.png" xmlns:media="http://search.yahoo.com/mrss/"/></entry><entry><title type="html">Tracking Football Team Strengths with a Bayesian Kalman Model</title><link href="https://seanelvidge.com/articles/2025/Football_team_rankings/" rel="alternate" type="text/html" title="Tracking Football Team Strengths with a Bayesian Kalman Model"/><published>2025-12-15T09:20:00+00:00</published><updated>2026-10-24T00:00:00+00:00</updated><id>https://seanelvidge.com/articles/2025/Football_team_rankings</id><content type="html" xml:base="https://seanelvidge.com/articles/2025/Football_team_rankings/"><![CDATA[<p>Not all football-rating systems are the same. Many of the public ones, like the excellent <a href="http://clubelo.com/System">ClubElo</a>, do a fine job of ranking teams using elegant updates. Win and your rating rises, lose and it falls; the amount you move depends on how “surprised” the model was.</p> <p>This model begins in the same spirit but replaces those heuristic updates with a probabilistic engine: a <em>Bayesian Extended Kalman Filter</em> coupled to a modern version of the Bradley–Terry model (with added draws). The core idea is simple: treat each team’s strength as something hidden that we estimate and track through time, with uncertainty that expands between matches and contracts when new results arrive.</p> <blockquote> <p><strong>Update, 24 October 2026:</strong> The model now also learns from the margin of victory. The new article, <a href="/articles/2026/Football_rankings_and_winning_margins/">Does a 4–0 Win Tell Us More Than a 1–0 Win?</a>, explains that extension, the validation across ten seasons and the change to uncertainty in the league-table predictor. This article remains the introduction to the underlying rating framework and its results-only update.</p> </blockquote> <p>The results-only calendar model described below was reviewed in October 2026. On 20,096 evaluation matches from 2016/17–2025/26, it achieves a mean-per-outcome Brier score of <strong>0.207478</strong> and log loss of <strong>1.035789</strong>. Settings were selected using 2000/01–2015/16. The <a href="https://github.com/seanelvidge/England-football-results/blob/main/validation/REPORT.md">first mathematical review</a> explains the move to elapsed calendar time, nonzero initial uncertainty and division-specific parameters, together with the removal of the former innovation-dispersion controller. The filter remains a Gaussian approximation; the latest margin-based model and its scores are covered in the follow-up article.</p> <p>The rest of this post goes into the mathematical details of the ranking algorithm, but if you want to access the underlying data it is <a href="https://github.com/seanelvidge/England-football-results">available here</a> (specifically the file <a href="https://raw.githubusercontent.com/seanelvidge/England-football-results/refs/heads/main/EnglandLeagueResults_wRanks.csv"><code class="language-plaintext highlighter-rouge">EnglandLeagueResults_wRanks.csv</code></a>).</p> <h2 id="the-big-picture">The Big Picture</h2> <p>Try the model through the <a href="/teamRankings">interactive team rankings history</a> or the <a href="/matchProbs">match win, draw and loss probability calculator</a>. For background on updating uncertain estimates, my <a href="/articles/2023/Maths_on_a_Mug_15/">Bayes’ theorem explainer</a> and <a href="/kalman">Kalman filter notes</a> introduce the underlying ideas.</p> <p>Imagine every team has a hidden “true strength” \(s_i\). Before a match, the home and away teams carry beliefs about their current strengths, each with an associated uncertainty. When they play, the result provides new information that updates those beliefs.</p> <p>In the ClubElo framework this update is written directly as</p> \[R_{\text{new}} = R_{\text{old}} + K(S - E),\] <p>where \(S\) is the score (1 for a win, 0.5 for a draw, 0 for a loss), \(E\) is the expected probability, and \(K\) is a fixed responsiveness parameter.</p> <p>Here, the same logic is embedded in a <a href="https://en.wikipedia.org/wiki/Kalman_filter">Kalman filter</a>, which means the effective \(K\) is learned automatically. The update size depends on two things: how uncertain we are about the teams, and how surprising the result was. Upsets between uncertain teams lead to large updates; shocks between well-understood teams barely move the needle.</p> <p>Between matches, team strengths are not frozen. Instead, they evolve according to a <em>mean-reverting stochastic process</em>, allowing form to drift while preventing runaway behaviour. Newly promoted teams begin with large uncertainty and adapt quickly; established teams gravitate toward long-term baselines rather than rising indefinitely.</p> <h2 id="modelling-match-outcomes-wins-draws-losses">Modelling Match Outcomes (Wins, Draws, Losses)</h2> <p>Match outcomes are modelled using the <a href="https://www.jstor.org/stable/2283595">Davidson extension</a> of the <a href="https://en.wikipedia.org/wiki/Bradley%E2%80%93Terry_model">Bradley–Terry model</a>, which naturally incorporates draws. The probability of each outcome is</p> \[\Pr(\text{Home}) = \frac{e^{\Delta}}{e^{\Delta} + e^{-\Delta} + \kappa},\] \[\Pr(\text{Draw}) = \frac{\kappa}{e^{\Delta} + e^{-\Delta} + \kappa},\] \[\Pr(\text{Away}) = \frac{e^{-\Delta}}{e^{\Delta} + e^{-\Delta} + \kappa},\] <p>where</p> \[\Delta = \beta (s_H - s_A) + h.\] <p>Here, \(s_H\) and \(s_A\) are the latent strengths of the home and away teams, \(\beta\) is a scaling parameter, \(\kappa\) controls the draw rate, and \(h\) is the home-advantage term.</p> <h2 id="home-advantage---explicit-and-time-varying">Home Advantage - Explicit and Time-Varying</h2> <p>Home advantage is not treated as a fixed constant. Instead, it is explicitly modelled and allowed to vary over time. Historical analysis shows that home-win rates in English football have declined dramatically since the late 19th century, falling from well above 60% to closer to 40–45% in the modern era (see <a href="https://seanelvidge.com/articles/2025/Home_advantage_in_English_football/">this analysis</a>).</p> <p>The results-only model learns \(h\) and \(\log\kappa\) separately for each division tier. After each date, it updates them using that date’s outcome residuals; every prediction on the date uses the parameters available before those results. This replaces the earlier fixed calendar-year trend. The margin extension adds another tier-specific parameter, explained in the follow-up. Comparisons across remote eras and different divisions still depend partly on the tier priors.</p> <h2 id="validation-and-rating-scale">Validation and Rating Scale</h2> <p>Predictive accuracy is measured using the <a href="https://en.wikipedia.org/wiki/Brier_score">Brier score</a>, here averaged over both matches and the three outcomes. The summed multiclass convention would give a value three times larger. The later-season comparison above is the evidence for improvement; fitting or scoring the full history alone would not establish predictive value. Calibration is also checked by outcome and forecast probability, and is not perfect.</p> <p>Internally, the filter operates on a latent “skill” scale roughly spanning \(-3\) to \(+3\). However for presentation, these values are mapped linearly onto an Elo-style scale, centred on 1000 points, with elite teams reaching 1800+ and lower-league teams clustering about a thousand points below. This transformation is purely cosmetic; all inference happens on the latent scale.</p> <h2 id="part-ii--if-you-dare-read-on-the-mathematics">Part II — If You Dare Read On: The Mathematics</h2> <h2 id="1-state-evolution-ornsteinuhlenbeck-dynamics">1. State Evolution (<a href="https://en.wikipedia.org/wiki/Ornstein%E2%80%93Uhlenbeck_process">Ornstein–Uhlenbeck</a> Dynamics)</h2> <p>Each team’s latent strength evolves as</p> \[s_{i,t+1} = r_i\, s_{i,t} + (1 - r_i)\, \mu_{\text{tier}(i,t)} + \varepsilon_{i,t},\] <p>with</p> \[\varepsilon_{i,t} \sim \mathcal{N}(0, q_i).\] <p>For \(d_i\) days since the previous match, \(r_i=0.995^{d_i/7}\) and \(q_i=0.0017(1-r_i^2)/(1-0.995^2)\). The tier means are \(1.5, 0.3, -0.3, -0.6\) and a new team starts at its tier mean with variance \(0.25\). A division change retains its previous skill and multiplies its previous variance by \(1.5\) before the time transition. The latter is a modelling heuristic. These transitions allow uncertainty to tend towards the process’s stationary variance during inactivity.</p> <h2 id="2-observation-model">2. Observation Model</h2> <p>Given predicted strengths \(s_H\) and \(s_A\), the model produces a probability vector</p> \[\mathbf{p} = \begin{bmatrix} p_H \\ p_D \\ p_A \end{bmatrix},\] <p>using the Davidson–Bradley–Terry equations above. The Jacobian matrix \(H\) is computed by differentiating \(\mathbf{p}\) with respect to the state vector \([s_H, s_A]^T\), enabling linearisation of the nonlinear observation model.</p> <h2 id="3-the-results-only-extended-kalman-filter">3. The Results-Only Extended Kalman Filter</h2> <p>The categorical update provides the baseline for the model. The <a href="/articles/2026/Football_rankings_and_winning_margins/">new article</a> shows how the scalar update incorporates winning-margin information.</p> <p>Prediction step:</p> \[x^- = F x_t + (I-F)\mu,\] \[P^- = F P_t F^\top + Q,\] <p>where \(F = \operatorname{diag}(r_H,r_A)\) and \(Q=\operatorname{diag}(q_H,q_A)\). Home and away teams may have different elapsed times.</p> <p>Observation noise:</p> \[R = \operatorname{diag}(\mathbf{p}) - \mathbf{p}\mathbf{p}^\top.\] <p>Update step:</p> \[K = P^- H^\top \bigl(H P^- H^\top + R\bigr)^{+},\] \[x_{t+1} = x^- + K (y - \mathbf{p}),\] \[P_{t+1} = (I - K H) P^-.\] <p>The superscript \(+\) denotes a pseudoinverse: the three categorical probabilities sum to one, leaving only two independent residuals. The results-only update has an equivalent scalar score/information form, avoiding a singular matrix inversion. The filter retains each team’s marginal variance while discarding induced cross-team covariance. The old normalised-innovation controller has been removed; its target of two corrected a dimension error but did not establish exact categorical or Bayesian uncertainty calibration.</p> <h2 id="why-this-matters">Why This Matters</h2> <p>The model makes its evolution, observation and prior assumptions explicit, so they can be tested chronologically. Its strength estimates and uncertainty provide the starting point for both match forecasts and the <a href="/tableProbs">league table predictor</a>. The <a href="/articles/2026/Football_rankings_and_winning_margins/">winning-margin update</a> builds on this framework and tests how much more the score can tell us.</p>]]></content><author><name>Sean Elvidge</name></author><category term="football"/><category term="mathematics"/><summary type="html"><![CDATA[How a Bayesian extended Kalman filter estimates English football team strengths, uncertainty and match probabilities from historical league results.]]></summary><media:thumbnail xmlns:media="http://search.yahoo.com/mrss/" url="https://seanelvidge.com/assets/img/football_rankings.png"/><media:content medium="image" url="https://seanelvidge.com/assets/img/football_rankings.png" xmlns:media="http://search.yahoo.com/mrss/"/></entry><entry><title type="html">Football Bogey Grounds and How Statistics Can Prove Them</title><link href="https://seanelvidge.com/articles/2025/Football_bogey_grounds/" rel="alternate" type="text/html" title="Football Bogey Grounds and How Statistics Can Prove Them"/><published>2025-12-12T19:04:00+00:00</published><updated>2026-09-06T00:00:00+00:00</updated><id>https://seanelvidge.com/articles/2025/Football_bogey_grounds</id><content type="html" xml:base="https://seanelvidge.com/articles/2025/Football_bogey_grounds/"><![CDATA[<p>Football supporters are never short of folklore. Some of it heroic, some of it tragic, and some of it mathematically suspicious. Among the more enduring tales is the <em>bogey ground</em>: that one venue where your club never quite manages to win, no matter how many times the fixture computer sends you there with fresh optimism, a new manager and a good run of form.</p> <p>But one question always pops up in the back of my mind when I hear this: When is a bogey ground actually a bogey ground, and when is it just a trick of small numbers?</p> <p>The records below come from my <a href="/articles/2024/All_England_football_league_results/">historical English football league results database</a>. To investigate your own club’s record, use the <a href="/h2h?team1=Arsenal&amp;team2=Fulham">football head-to-head calculator</a>, including its home/away breakdown. These are league meetings, not all-competition records.</p> <p>Most fans instinctively understand the point. Losing your only ever visit to Carlisle does not make Brunton Park a cursed ground. A couple of failed trips to Luton are not evidence of eldritch forces at work. Yet, scattered across the long history of English league football, a few pairs of teams have met often enough, and still produced zero away wins, that bogey grounds become statistically real.</p> <p>In the Premier League an example of this is Fulham, and their trips to Arsenal.</p> <p>Fulham have played 32 league matches away at Arsenal. They have never won.</p> <p>Not once. Not in 1914 (lost 2-0). Not in 1964 (2-2). Not in 2014 (lost 2-0).</p> <p>Zero wins from thirty-two attempts. Seven draws. Twenty-five defeats.</p> <p>Even more striking, this isn’t just a bad example, this is, among every club in the entire Football League dataset, over 135 years of football, no team has a larger, more emphatic, more mathematically convincing record of away futility than Fulham at Arsenal.</p> <p>So let’s take a look, not just at the record itself, but at how we can quantify bogeyness in a way that recognises your intuition (“two games aren’t enough!”) but takes advantage of the huge dataset football provides.</p> <h2 id="why-never-won-isnt-enough-the-curse-of-small-n">Why “never won” isn’t enough (the curse of small \(n\))</h2> <p>Imagine you flip a coin twice and get two tails. Does that mean the coin is biased?</p> <p>Of course not. You need <em>more</em> flips before you start thinking that someone is up.</p> <p>The same goes for football. A team losing its only away visit to Manchester City tells you nothing. Losing twice? Still nothing except a faint sense of déjà vu. Losing five times? Now you’re paying attention. Losing ten times? Stop buying tickets to the away game.</p> <p>So what we need is a way of answering this question:</p> <p>Given a team has played \(n\) away matches at a ground and won none, what is a reasonable upper limit on how good their <em>true</em> chance of winning there might be?</p> <p>Fortunately statistics, as it always does, gives us a handy tool for working this out, the <a href="https://en.wikipedia.org/wiki/Binomial_proportion_confidence_interval#Wilson_score_interval">Wilson confidence interval.</a></p> <h2 id="a-gentle-introduction-to-the-wilson-interval">A (gentle) introduction to the Wilson interval</h2> <p>Don’t panic; no equations are necessary. Just an idea. A Wilson interval takes two bits of information:</p> <ul> <li>the number of games played (\(n\)), and</li> <li>the number of games won (0, in our bogey-ground case)</li> </ul> <p>and asks:</p> <p>“If their true underlying chance of winning were \(p\), how big could \(p\) realistically be before the observed data (zero wins) would start to look implausible?”</p> <p>For small \(n\) (not many games played) the answer is: “\(p\) could still be quite large.”</p> <p>For large \(n\) (a large number of games played) the answer is: “\(p\) must be quite small.”</p> <p>This lines up with our intuition.</p> <p>A team with 0 wins from 2 tries? Their true win rate might easily be 30%, 40%, even 50%.</p> <p>A team with 0 wins from 32 tries? Now the interval collapses. It tells you the true away win probability is almost certainly very small, low enough that even by bad luck alone, you’d be surprised <em>not</em> to have picked up a single win after that many attempts.</p> <p>The Wilson interval is good for this particular problem (compared to other approaches like the Wald Interval) because it behaves properly at the boundaries (like at 0% or 100%), where ordinary methods can breakdown. Football fans should love it because it gives scientific legitimacy to something they’ve always known: <strong>some bogey grounds are imaginary, but a few are real</strong>.</p> <h2 id="the-most-convincing-bogey-of-them-all-fulham-at-arsenal">The most convincing bogey of them all: Fulham at Arsenal</h2> <p>So, what does Wilson say about Fulham’s 0 wins from 32 away league games at Arsenal?</p> <p>It says this:</p> <blockquote> <p>In each of Fulham’s away matches against Arsenal their probability of winning must have been (at the very most) 10% for 0 wins out of 32 to not be statistically suspicious.</p> </blockquote> <p>Mathematically this comes from plugging \(n\) (number of games) and \(z\) (which is equal to 1.96 for 95% ‘confidence’) into the equation for calculating the upper bound of the Wilson interval:</p> \[\mbox{Wilson Upper Bound} = \frac{z^2}{n+z^2} = \frac{1.96^2}{32+1.96^2} = 0.107 = 10.7\%\] <p>So the only way we could avoid saying Arsenal is Fulham’s away team nemesis is if we believe Fulham, across the whole history of the fixture (starting in 1913; <a href="https://seanelvidge.com/h2h?team1=Arsenal&amp;team2=Fulham">see my head-to-head tool here</a>), have not had better than a 10% chance of winning those games. Whilst we can’t be (mathematically) certain that it is true, it is incredibly unlikely. Typical away win rates in the top division over history are roughly 25-30% range. Even clear cut underdogs often have pre-match win probabilities around 15-20%.</p> <p>So, therefore, I think it is safe to say to conclude:</p> <blockquote> <p>Fulham away at Arsenal is not just a bogey ground, it is the bogey ground.</p> </blockquote> <p>The Premier League’s most mathematically defensible curse.</p> <h2 id="other-bogey-teams">Other bogey teams</h2> <p>Fulham–Arsenal is the only pair in the <a href="https://seanelvidge.com/articles/2024/All_England_football_league_results/">entire database</a> whose Wilson upper bound hovers at just above the 10% threshold. But several others get close, here are the best (or worst) of the rest:</p> <table style="border-collapse: collapse; width: 75%;"> <thead> <tr> <th style="border: 1px solid black; padding: 8px;">Away Team</th> <th style="border: 1px solid black; padding: 8px;">Home Team</th> <th style="border: 1px solid black; padding: 8px;">Played</th> <th style="border: 1px solid black; padding: 8px;">Record</th> <th style="border: 1px solid black; padding: 8px;">Wilson Upper</th> </tr> </thead> <tbody> <tr> <td style="border: 1px solid black; padding: 8px;">Fulham</td> <td style="border: 1px solid black; padding: 8px;">Arsenal</td> <td style="border: 1px solid black; padding: 8px;">32</td> <td style="border: 1px solid black; padding: 8px;">0W–7D–25L</td> <td style="border: 1px solid black; padding: 8px;">10.7%</td> </tr> <tr> <td style="border: 1px solid black; padding: 8px;">Grimsby Town</td> <td style="border: 1px solid black; padding: 8px;">Blackburn Rovers</td> <td style="border: 1px solid black; padding: 8px;">28</td> <td style="border: 1px solid black; padding: 8px;">0W–9D–19L</td> <td style="border: 1px solid black; padding: 8px;">12.1%</td> </tr> <tr> <td style="border: 1px solid black; padding: 8px;">Mansfield Town</td> <td style="border: 1px solid black; padding: 8px;">Reading</td> <td style="border: 1px solid black; padding: 8px;">26</td> <td style="border: 1px solid black; padding: 8px;">0W–5D–21L</td> <td style="border: 1px solid black; padding: 8px;">12.9%</td> </tr> <tr> <td style="border: 1px solid black; padding: 8px;">Tranmere Rovers</td> <td style="border: 1px solid black; padding: 8px;">Barnsley</td> <td style="border: 1px solid black; padding: 8px;">26</td> <td style="border: 1px solid black; padding: 8px;">0W–11D–15L</td> <td style="border: 1px solid black; padding: 8px;">12.9%</td> </tr> <tr> <td style="border: 1px solid black; padding: 8px;">Oldham Athletic</td> <td style="border: 1px solid black; padding: 8px;">Charlton Athletic</td> <td style="border: 1px solid black; padding: 8px;">25</td> <td style="border: 1px solid black; padding: 8px;">0W–11D–14L</td> <td style="border: 1px solid black; padding: 8px;">13.3%</td> </tr> <tr> <td style="border: 1px solid black; padding: 8px;">Newport County</td> <td style="border: 1px solid black; padding: 8px;">Luton Town</td> <td style="border: 1px solid black; padding: 8px;">24</td> <td style="border: 1px solid black; padding: 8px;">0W–7D–17L</td> <td style="border: 1px solid black; padding: 8px;">13.8%</td> </tr> <tr> <td style="border: 1px solid black; padding: 8px;">Coventry City</td> <td style="border: 1px solid black; padding: 8px;">Preston North End</td> <td style="border: 1px solid black; padding: 8px;">24</td> <td style="border: 1px solid black; padding: 8px;">0W–9D–15L</td> <td style="border: 1px solid black; padding: 8px;">13.8%</td> </tr> </tbody> </table> <p><br/></p> <p>These are not small samples. These are not casual coincidences. When you are approaching 20 or 30 visits with no victory, it is time to stop thinking you are unlucky and start accepting you have a bogey ground.</p>]]></content><author><name>Sean Elvidge</name></author><category term="football"/><category term="mathematics"/><summary type="html"><![CDATA[Can a football club actually have a statistically verifiable football bogey ground or is it just bad luck?]]></summary><media:thumbnail xmlns:media="http://search.yahoo.com/mrss/" url="https://seanelvidge.com/assets/img/bogey_team.png"/><media:content medium="image" url="https://seanelvidge.com/assets/img/bogey_team.png" xmlns:media="http://search.yahoo.com/mrss/"/></entry><entry><title type="html">Why Our Next Big Solar Storm Warning Could Still Be a Guess</title><link href="https://seanelvidge.com/articles/2025/Guessing_at_Space_Weather/" rel="alternate" type="text/html" title="Why Our Next Big Solar Storm Warning Could Still Be a Guess"/><published>2025-11-30T19:30:00+00:00</published><updated>2025-11-30T19:30:00+00:00</updated><id>https://seanelvidge.com/articles/2025/Guessing_at_Space_Weather</id><content type="html" xml:base="https://seanelvidge.com/articles/2025/Guessing_at_Space_Weather/"><![CDATA[<p>Eighteen months ago, a beautiful auroral display swept across the world, accompanying the first extreme geomagnetic storm in more than two decades. It was a landmark event not just for the spectacle but for what it revealed about our growing global dependence on reliable space-weather forecasting. When another storm drew headlines this month (November 2025), with an early “G5 watch” issued by forecasters, governments and operators activated contingency plans in anticipation of another major impact (the geomagnetic storm scale, G-scale, ranges from one to five, with five denoting the most extreme events). Yet the storm ultimately peaked at only G3, a level we expect hundreds of times in a solar cycle. That mismatch exposed a persistent weakness in how we observe and interpret the Sun.</p> <p>The November events began with three coronal mass ejections (CMEs) erupting from the same active region over consecutive days. Using direct, head-on observations from the Sun–Earth line, scientists estimated their speeds and trajectories and issued a G4 storm watch for 12 November. When the first CME struck earlier than expected and with stronger parameters than the initial estimate, the risk escalated: successive CMEs often travel faster along the newly cleared path, and the potential for a combined impact grows. A G5 watch was released, triggering international preparedness actions. Power-grid operators revisited protection settings, communications providers reviewed continuity plans, and governments braced for potential disruption.</p> <p>Then nothing happened, at least not at the predicted intensity or time. The CME did arrive, but just before midnight, and with a magnetic field that was only briefly and weakly aligned in the southward direction needed to strongly couple with Earth’s magnetic system. Without that sustained southward field, even an otherwise fast and dense CME cannot deliver a major geomagnetic storm. The result was a modest G3. This forecasting gap did not arise from poor judgement but from a fundamental observational void: after we watch a CME leave the Sun, we lose sight of its evolution for roughly 99% of the journey to Earth.</p> <p>The limitations of this “single viewpoint” forecasting have been known for years. We can estimate initial CME speed, but we cannot track how it deforms, interacts with the solar wind or merges with other CMEs en route. Crucially, we cannot determine its magnetic structure, the key factor in storm severity, until the CME reaches the L1 point (Lagrange point 1), only about 30 minutes before Earth impact. Even this narrow window is presently fragile: the DSCOVR spacecraft, one of only two satellites providing L1 measurements, is offline, and ACE is approaching 30 years in space with ageing instruments.</p> <p>Given the rising global reliance on vulnerable infrastructure: power networks, navigation systems, satellite constellations, living with this degree of uncertainty is no longer tenable. Forecasts that overshoot create costly false alarms; forecasts that undershoot risk real damage. The November mis-forecast was harmless, but it underscored how close we are to the limits of what current assets can deliver.</p> <p>New capabilities are on the horizon, though they require sustained commitment. The NOAA–NASA SWFO-L1 mission is expected to restore robust solar-wind monitoring from mid-2026, stabilising the 30-minute warning baseline. Far more transformative would be the European Space Agency’s Vigil mission, planned for launch to the L5 Lagrange point in 2031. From that vantage point, Vigil would view CMEs from the side, enabling reliable speed and trajectory estimates and dramatically reducing arrival-time uncertainty. But its path to launch depends on political support through multiple rounds of ministerial scrutiny.</p> <p>Beyond L5, deeper innovation is needed. A constellation of sensors placed in distant retrograde orbits (DRO) could offer early in-situ magnetic-field measurements, potentially hours, not minutes, before a CME reaches Earth. ESA’s planned demonstration of a DRO spacecraft in 2026, HENON, is a first step, but only that. These observational advances must be matched by improved modelling of CME structure, solar precursors and the complex interactions that determine whether an incoming CME will actually be geoeffective.</p> <p>Accurate space-weather forecasting will always involve uncertainty, but it need not remain dominated by blind spots. The November storm was a reminder that our current system can predict the possibility of major events yet cannot confirm their true character until it is almost too late. If we want forecasts that are genuinely actionable, capable of guiding grid operators, satellite controllers and governments with confidence, then investment in new vantage points, new missions and new science is essential. Without it, our next “G5 watch” may leave us waiting once again, unsure whether a major storm is coming or whether the sky will stay quiet.</p>]]></content><author><name>Sean Elvidge</name></author><category term="spaceWeather"/><summary type="html"><![CDATA[Accurate space-weather forecasting will always involve uncertainty, but it need not remain dominated by blind spots. The November storm was a reminder that our current system can predict the possibility of major events yet cannot confirm their true character until it is almost too late.]]></summary><media:thumbnail xmlns:media="http://search.yahoo.com/mrss/" url="https://seanelvidge.com/assets/img/aurora.jpg"/><media:content medium="image" url="https://seanelvidge.com/assets/img/aurora.jpg" xmlns:media="http://search.yahoo.com/mrss/"/></entry></feed>