Algo FPL: 44. Man vs Machine Learning: Season Analysis & Conclusion

Gameweek 38 was an eventful gameweek in the Premier League and personally, I missed the times when we would have multiple games going on at the same time. It was good to have the fans back too instead of the fake noise. This season as a whole was a unique one, with several fixtures being postponed due to Covid, cup games, fan protests to result in a lot more fixture rescheduling than usual.

As far as FPL is concerned, The Machine finished on a rank of 473,356 to finish in the top 5% of managers in the world, the caveat of course being that 1 million teams were created late and a decent number of teams would be ditched by their managers before the end of the season.

Meanwhile, my team finished on a personal best rank of 2286 and I went on to win all my mini-leagues. This exceeded my expectations beyond all imaginations as among all my friends at least in the 12 years that I have been playing this game, this is the highest finish I have witnessed.

Thus, what started as an experiment to map out a Machine's team inadvertently ended up documenting my special season with screenshots of my team each week, my thought process, the decisions I made and the outcome all stored on this blog.

In this post, I will go into some depth about how the teams performed this season, evaluate decisions, point out where they gained & lost points, try to measure their luck and conclude with takeaways for both the ML model as well as my human decision making for next season.

Note: Advanced apologies as there are some screenshots which might be blurred - if you click on the images, you should see them properly.

Gameweek Ranks of Man & Machine's FPL Teams (green colour is chip usage)

The Machine's Team

Gameweek 38

The transfers made were: Mane and James in for Fernandes and Shaw
Mane's brace meant the last week punt was highly successful
Coufal and Kane scored big too, despite recent erratic performance
Harrison and Alexander-Arnold were the others to return to finish the season on a strong 68 points
Captain Salah got an assist and could have easily had more
The Machine got a green arrow to go from 543k to 473k

Gameweek History

The Machine had a solid start, reaching a rank of 400k by the first 10 gameweeks
However, by GW 17, it was out of the top million and spent the rest of the season going up and down between 500k and 1 million
The first time it broke into the top 500k again was the very last week of the season. It got 8 green arrows in the last 11 weeks to finish strongly.
Source: https://fantasy.premierleague.com/

Transfers

The Machine took 24 points worth of hits. Hits were rare because the projections had to indicate that the player coming in would have at least one return more than the player going out.
It gained a net of 92 points from transfers, which is on par with a decent manager but most transfers were spent on less impactful positions and thus contributed to its lack of major rise or fall throughout the season
As seen in the graph below, apart from two weeks of losses, most of the transfers were a positive return
Source: http://www.fplretro.com/

Captaincy

All the Machine's captaincy choices have a decent average return across the season - at the same time there isn't a captain average that is exceptional
It tends to pick the same captain for a stretch of fixtures, irrespective of difficulty. This is unlike a human strategy to look at both form and fixtures
This resulted in the Machine picking up unexpected hauls but missing out at times when a majority of the managers captained an 'obvious' candidate
Owning Vardy also meant it missed out on other premium players in that phase
Source: https://www.anewpla.net/fpl/report/index.php

Bench

The Machine did not lose too many points on the bench as it often made unconventional picks in the cheaper positions
This was yet another potential reason for its okay performance as it did not invest in the several highly owned cheaper options that were doing exceptionally well
Source: https://www.anewpla.net/fpl/report/index.php

Chips

This was one place where the Machine severely lacked strategy and also got extremely unlucky
When the Machine used its bench boost, the Manchester United-Liverpool game was unexpectedly postponed and it had 5 players from that game alone
The free hit was most successful with GW 18 being a blank gameweek - but the Machine never 'learnt' the best time to use the wildcards
Source: http://www.fplretro.com/

Point Distribution

Almost half the points of the team were obtained from minutes played, which is not out of the ordinary
It is interesting to see that almost a quarter of the points were from goals, which is only slightly lesser than the points obtained from assists and clean sheets - the model treated all returns equally in its formulation
Around 5% of negative points set the team back
Source: http://fploptimized.com/highlights.html

Formation

The Machine started the season preferring 4 at the back but as the season progressed it moved to a more 'human-like' 3 at the back formation
What's interesting is that the average formation shows almost 3 strikers picked in every game. This almost rules out 5 midfielders
The Machine did really well in its choices of midfielders and forwards, getting around 6 points per game from them. The same cannot be said of the defenders
Source: http://fploptimized.com/highlights.html

Decision Making

The goal success ratio is solid with almost one in three picks scoring a goal. Probably the near 50-50 captaincy selection hurt its chances of making those goals count big
Source: http://fploptimized.com/highlights.html

Key Moments

This analysis was done on the top 1 million players, as that is where the Machine belonged for most of the season
With the Machine picking a lot of differentials, the player list on the gains page has players like Vardy, Harrison, Ederson, Willian, Barnes who aren't commonly owned
Similarly, players like Antonio and Bowen were picked early despite West Ham having hard fixtures and the Machine benefitted from this
At the same time, the loss analysis shows that the main problem of the Machine is not making the 'obvious' picks that most human managers make (as shown by the high Effective Ownership)
Source: http://fploptimized.com/highlights.html

Risk & Luck Factor

This tool shows us that according to xG data, the machine should have finished at a rank of around 311k
Similarly, as per implied odds, the team that the Machine had should have obtained a rank of 207k
The luck% of the Machine is fairly low in the chart as a result
The relative risk of the machine is categorised as 93.6%, which shows that the Machine plays extremely differently from a human
The template correlation is extremely low as we see in the figure below and the Machine is categorized as a 'Risk Taker'
Source: https://fplreview.com/season-review/

Team of the Season

The Machine's season was built on a backbone of Fernandes, Salah, Kane and Bamford - much like that of a human player
It didn't own Son as often - the other high scoring premium player
It also differed significantly by picking Vardy and captaining him so many times. This period also coincided with the fall from 400k to 1 million in the ranks
It's quite telling that Ederson and Bowen are in the team of the season (very low ownership %) - owning triple City defence for a long time prevented it from owning Gundogan
Source: http://fploptimized.com/highlights.html

Conclusion

I feel that finishing 473k in a complicated season like this is a great achievement for a Machine based team, given the bad luck it had
The Machine managed to pick some differential gems at various stages of the season, even getting some ballsy captaincy calls right
However, there are multiple things that can be improved on
It made a lot of risky captaincy picks - often going for form over fixture and betting against the crowd
The Machine found it difficult to navigate blank gameweeks and double gameweeks, often ignoring players who had two fixtures to its own peril
Fixture rescheduling due to various reasons affected the Machine disproportionately as it was impossible to model that uncertainty
There was also no chip strategy that the machine had, which contributed to its downfall
It picked players who weren't certain to start. An injury flag prevented it from picking injured players but the same cannot be said about rotation risks
Building an optimizer to convert the projections into teams so that the team that is picked can be optimized can be done by converting return predictions to points
I am hoping that next season can see both the model and the rank outcome improve in some of these aspects
Useful links: https://www.fantasyfootballscout.co.uk/