The world of Machine Studying is huge and intriguing for builders, learners, and researchers. Once I needed to put together an interesting workshop for my colleagues at Trustfull, I made a decision to delve into this fascinating discipline. Fairly than choosing a contest from Kaggle, which usually options measurable efficiency metrics and ready datasets (and make it straightforward for myself), I made a decision to problem the workforce with a real-world instance rooted in considered one of my passions. This led to the inception of the “Pokémon VGC Predicts” mission. Be part of us on our one-day journey, not just for a narrative, but additionally to search out hyperlinks and sources on the subject.
Pokémon, one of many greatest franchises globally, has a aggressive online game format generally known as the VGC (Video Sport Championships). Our mission’s objective was to find out if it’s potential to foretell the winner of a match primarily based on the workforce preview, utilizing machine studying.
Whereas many are acquainted with Pokémon, not everybody is aware of the intricacies of its aggressive format. In a nutshell: In every match matchup, every participant brings a registered workforce of six Pokémon. They get a preview of the opponent’s workforce after which secretly choose 4 Pokémon for the battle, which is carried out in a doubles format (two Pokémon on all sides). The participant who wins the perfect of three battles wins the matchup.
With a fundamental understanding of workforce matchups, we wanted information. Pokémon being related within the e-sport world and being one of many prime media franchises will certainly share information concerning the aggressive scene and may have superb APIs and so forth, proper? nicely probably not. Nevertheless, we discovered an answer in Smogon (often known as Pokémon Showdown), an open-source web-based Pokémon battle simulator created by the group. Smogon gives logs of Pokémon battles and numerous statistics, making it a useful useful resource. Regardless that it’s unofficial, its utilization statistics are extremely consultant of the aggressive scene since most gamers use it for testing groups earlier than constructing them within the official video games (thoughts that constructing groups within the precise video games requires loads of effort and time whereas on Smogon is a “couple of clicks”). So after writing a number of hundred traces of code, we had a dataframe with practically 10,000 video games performed within the Gen9 Reg G 2024 VGC format.
Subsequent, we wanted to verify if our dataset was suitable for machine learning. Amongst different checks for instance our goal column was the “winner,” which might be “p1” or “p2.” For our mannequin to be efficient, this column wanted to be balanced. If we had considerably extra “p1” winners than “p2,” the mannequin may simply all the time predict “p1” because the winner. Fortuitously, our information was already evenly distributed.
We carried out different a number of key actions to organize the information, notably:
- Encoding Categorical Options: We used a easy label encoder for the “winner” column and one-hot encoding for the Pokémon groups.
- Feature Engineering: noticibly deletion of lacking information, some gamers didn’t carry a full workforce of six, which doesn’t occur in official tournaments so we excluded this video games from the information.
- Splitting the Data: We divided the information into coaching and testing units to coach the mannequin and consider its efficiency.
We initially tried a random forest mannequin and achieved a 48% accuracy fee, which was disappointing. After some parameter tuning, we managed to succeed in 51% accuracy. We additionally experimented with totally different strategies of encoding the groups, reminiscent of sorting and hashing the workforce composition to emphasise the workforce as a complete quite than particular person Pokémon, however these efforts had been unsuccessful.
Regardless of operating out of time, there have been extra steps we may have taken, reminiscent of function development. For instance:
- Group Sort Coverages and Weaknesses: which may affect sport outcomes.
- Group Archetypes: Figuring out if a workforce is a trick room workforce, climate workforce, and so forth.
- Detailed Group Builds: Together with skills, held gadgets, and transfer units disclosed in battle logs, with information imputation for lacking data. Word that this degree of particulars is disclosed in official tournaments workforce previews however not in Smogon logs.
We will’t say if these extra steps would have made any distinction, the meta could also be wholesome sufficient that the end result relies upon totally on participant talent and luck.
Whereas our preliminary objective was to refine the workshop end result and share it publicly, we ended up simply creating a fancy coin flip machine. It won’t be very helpful, however for those who -readers- present curiosity, we may change thought and publish it on GitHub anyway as is.
