Task description

Hearthstone: Heroes of Warcraft is an example of a turn-based collectible card game. During the game, two players choose their hero with a unique power and compose a deck of thirty cards. The cards represent various spells, weapons, and minions, which can be summoned in order to attack the opponent. There are nearly 1,500 cards available in Hearthstone, which can be included in a deck. A vast majority of them have special characteristics and abilities that make them unique. Some of them can only be used with a certain type of heroes, while other cards can be included in any deck. Using a well-composed deck is a pivotal success factor for Hearthstone players and can be seen as one of the most important aspects of the game. The main goal of our challenge is to develop algorithms for predicting win changes of players using various decks.

Data description and format: The data for this competition is provided in two different formats. The main one is a collection of JSON files which describe in details games played between four different bots using 400 Hearthstone decks. These games can be used to estimate win-rate of the decks and learn how particular cards are used by the bots. Based on this knowledge, the task for participants is to estimate win-rates of a different set of 200 decks, played by the same bots.

The decks used in the provided collection of training games are given in a separate file named trainingDecks.json. Each row of this file corresponds to a different deck and stores a JSON with a deck identifier, hero identifier, and a list of card names along with their cardinality in the deck (there can be one or two cards of the same type in a deck). Analogically, the descriptions of test decks are provided in the same format in the file testDecks.json. It is allowed to use external knowledge bases about Hearthstone cards as long as they are publically available and their source is clearly stated in the submitted competition report. One example of such a source is the HearthPwn portal.

For the convenience of participants, we provide an additional table training_games.csv, whose rows correspond to JSON files describing the games between bots. Each row of this table gives a simple summary of the corresponding game. In particular, it provides ids of the playing bots, decks which they were using and the result of the game.

The format of submissions: The participants of the competition are asked to predict win-rates of 200 Hearthstone decks described in the file testDecks.json for the four bots which were used to generate the training data. The predictions should be sent using the submission system which will become available on April 13, 23:59 GMT. A file with a solution should consist of 800 lines, formatted in the same way as the exemplary solution provided as the file testSubmissionTemplate.csv. In each line, there should be an identifier of a bot, id of a deck and the predicted win-rate separated by semicolons. The win-rates should be expressed as percentages (real numbers between 0.0 and 100.0).

Evaluation of results: The submitted solutions will be evaluated online and the preliminary results will be published on the competition leaderboard. The preliminary score will be computed on a small subset of the test records, fixed for all participants. The final evaluation will be performed after completion of the competition using the remaining part of the test records. Those results will also be published online. It is important to note that only teams which submit a report describing their approach before the end of the contest will qualify for the final evaluation. The winning teams will be officially announced during a special session devoted to this competition, which will be organized at the FedCSIS'18 conference.

The assessment of solutions will be done using the RMSE measure. In order to keep the rule that the higher the score the better result, scores on the Leaderboard will correspond to minus RMSE values. For every pair bot-test deck, the reference win-rates were computed only based on games with the bots and decks from the training data.

Last modified: Friday, 13 April 2018, 7:41 PM