Aug 18, 2023·edited Aug 18, 2023Liked by Emil O. W. Kirkegaard
There is ignorance luck in chess. You are not equally well prepared for every opening your opponent could play and can't know for sure which it will be. Sometimes your preparation lasts till the game is over, sometimes you are on your own after a couple of moves.
If you don't know how to read a chess board so that you can play against any opening that sounds like a skill issue. And indeed players who are high in that skill will deliberately make unorthodox moves to disconcert low-skill players who are merely regurgitating memorised rote “book” play.
It definitely depends on the skill level of the players playing the game in question. For example, Super Smash Bros is probably way more skill based than Chess at lower levels. Low level players in Chess tend not to plan ahead a great deal of moves and so opportunities sort of just open up out of the aether. But then when you get to upper level chess you have a much deeper awareness of everything on the board and so things happen less and less by serendipity.
Meanwhile with Smash (no items, tournament legal stage) assuming the players aren’t using some sort of gimmick, the experience is already pretty intuitive and lucky chances already require some deal of skill to take advantage of. But as you move up players gain more mastery of their characters, which means that character matchups matter more and more. One thing I experienced playing Smash bros. is that the tier list makes no sense when you’re bad at the game, but you understand it more and more as you get better. Obviously this means that between two random players, their raw skill matters less.
This just looks like a conceptual confusion. The question is how much room player skill has to influence the outcome of the game. Whether, in a particular match, either of the players actually possesses any skill is a different question.
> Meanwhile with Smash (no items, tournament legal stage) assuming the players aren’t using some sort of gimmick, the experience is already pretty intuitive and lucky chances already require some deal of skill to take advantage of.
Taking advantage of lucky chances in chess also requires skill. That's why you'll see new players completely fail to take advantage of obvious openings.
Note that this comment on character matchups also applies to white’s advantage in chess: basically nonexistent among newbies whose individual moves are far less impactful (and who therefore usually don’t even see the advantage) but at the highest levels so large that black is considered to be playing from behind the entire game and mostly hoping for a draw.
Due to things like this, I don't think it makes sense to talk about what percentage a game is skill or luck independently of the distribution of players.
This is something I've thought about a fair bit, in connection with trying to develop objective metrics for professional quality. (My conclusion on that score is that professionals are professionals precisely because it is hard to measure their output; once you can measure output, automation whittles away gains from expertise.) If you've not, you should look into Microsoft's trueskill system — they developed it to match gamers. Iirc it is like elo but Bayesian and with an allowance for chance
If two players are exactly the same skill (for example if you are playing yourself) there will be 100% luck involved.
Also, you should take the time involved into account. A chess game for four hours vs a rock paper scissors match for one second is not a good comparison. You should compare the outcome of a RPS match lasting for four hours, which would have very much skill involved.
I wonder how the (more) subjective and objective ratings of the relative importance of skill vs luck in games, would relate to the individual level correlations with intelligence and other psychological traits(i'm assuming positive, and there's been a few meta-analysis for like chess and other mental games). And given that you published a study finding a high correlation between(the general factor of), performance in mental sports and IQ on the national level, looking at the relationship betwen the factor loading on a general factor of mental sports performance(to the extent that the loading does not reflect just some countries preference towards go or scrabble etc.), or the correlation between the correlation of MS~NIQ and these measures of the role of skill could be an external validation of that method.
“Thus, the more heterogeneous the ratings are, the better we can predict the winner of a match.”
On game sites that I play on such as yucata.de we use a simplified eyeball-able version of this as an estimate of the game's skill level. Games with a high top rating are higher-skill than games with a low top rating.
You could collect the ranking data from sites like Yucata or BGA to create an estimate of skill for all those games fairly easily, I think.
Great topic! Chance actually has two dimensions. One, which you focus on, is the degree of asymmetric chance. If we roll dice, one of us gets a bigger number than the other, usually. It's asymmetric. If we are running a race, but the number of laps is only revealed after the 4th lap, the uncertainty is symmetric. This second chance of chance can actually increase the importance of skill, since the players must adapt to it (or, it reduces the importance of skill because it gets so hard that player can't adapt to it).
The 13 prize cards that everyone will bid on can be stacked either randomly (normal version) or from lowest to higher (deterministic version). But it is symmetric: either nobody knows the order, or everybody does.
Goofspiel (and poker) remind us of a third type of chance: mixed strategies, randomized strategies. Let's use poker. If I can't read the other players' faces, I randomly decided whether to bluff or not. Depending on what they do, one of those two strategies is best. But ex ante there is no telling which is best.
If you read the economist paper, their entrance the topic is that apparently US law regulates games of skill vs. games of chance (gambling) differently. So the law must make a distinction between skill games and luck games, whereas there is only a continuum. A typical legal decision vs. continuous reality problem (e.g. mental illness is categorical in diagnoses and legal decisions but continuous in real life).
No - less than 50% skill does not mean 0% skill. Gambling (eg. slots) is different as it is programmatically designed for players to lose to the casino over the long run.
Scrabble probably has more skill + luck than any other game. There's a very high skill ceiling, but among players of similar skill it's very luck based, even at the top level. The ELO methodology might result in high std dev (almost on par with chess), but I think this wouldn't do justice to how much luck is involved.
If there's a very high ELO difference (which there would be in Scrabble—source: I played in some competitions), it's because the apparent luck is actually very evened out by players' ability to control for that luck. It's what @Eric Rasmusen referred to elsewhere in this thread as “symmetric luck”. That often appears random to weak players, but experienced players understand that much of the skill consists in managing the luck and playing probabilistically.
There is ignorance luck in chess. You are not equally well prepared for every opening your opponent could play and can't know for sure which it will be. Sometimes your preparation lasts till the game is over, sometimes you are on your own after a couple of moves.
If you don't know how to read a chess board so that you can play against any opening that sounds like a skill issue. And indeed players who are high in that skill will deliberately make unorthodox moves to disconcert low-skill players who are merely regurgitating memorised rote “book” play.
It definitely depends on the skill level of the players playing the game in question. For example, Super Smash Bros is probably way more skill based than Chess at lower levels. Low level players in Chess tend not to plan ahead a great deal of moves and so opportunities sort of just open up out of the aether. But then when you get to upper level chess you have a much deeper awareness of everything on the board and so things happen less and less by serendipity.
Meanwhile with Smash (no items, tournament legal stage) assuming the players aren’t using some sort of gimmick, the experience is already pretty intuitive and lucky chances already require some deal of skill to take advantage of. But as you move up players gain more mastery of their characters, which means that character matchups matter more and more. One thing I experienced playing Smash bros. is that the tier list makes no sense when you’re bad at the game, but you understand it more and more as you get better. Obviously this means that between two random players, their raw skill matters less.
This just looks like a conceptual confusion. The question is how much room player skill has to influence the outcome of the game. Whether, in a particular match, either of the players actually possesses any skill is a different question.
> Meanwhile with Smash (no items, tournament legal stage) assuming the players aren’t using some sort of gimmick, the experience is already pretty intuitive and lucky chances already require some deal of skill to take advantage of.
Taking advantage of lucky chances in chess also requires skill. That's why you'll see new players completely fail to take advantage of obvious openings.
Note that this comment on character matchups also applies to white’s advantage in chess: basically nonexistent among newbies whose individual moves are far less impactful (and who therefore usually don’t even see the advantage) but at the highest levels so large that black is considered to be playing from behind the entire game and mostly hoping for a draw.
Due to things like this, I don't think it makes sense to talk about what percentage a game is skill or luck independently of the distribution of players.
This is something I've thought about a fair bit, in connection with trying to develop objective metrics for professional quality. (My conclusion on that score is that professionals are professionals precisely because it is hard to measure their output; once you can measure output, automation whittles away gains from expertise.) If you've not, you should look into Microsoft's trueskill system — they developed it to match gamers. Iirc it is like elo but Bayesian and with an allowance for chance
Yes, TrueSkill is easily the best rating system. There are online implementations of it too, as well as easily accessible code for Python etc.
If two players are exactly the same skill (for example if you are playing yourself) there will be 100% luck involved.
Also, you should take the time involved into account. A chess game for four hours vs a rock paper scissors match for one second is not a good comparison. You should compare the outcome of a RPS match lasting for four hours, which would have very much skill involved.
I wonder how the (more) subjective and objective ratings of the relative importance of skill vs luck in games, would relate to the individual level correlations with intelligence and other psychological traits(i'm assuming positive, and there's been a few meta-analysis for like chess and other mental games). And given that you published a study finding a high correlation between(the general factor of), performance in mental sports and IQ on the national level, looking at the relationship betwen the factor loading on a general factor of mental sports performance(to the extent that the loading does not reflect just some countries preference towards go or scrabble etc.), or the correlation between the correlation of MS~NIQ and these measures of the role of skill could be an external validation of that method.
“Thus, the more heterogeneous the ratings are, the better we can predict the winner of a match.”
On game sites that I play on such as yucata.de we use a simplified eyeball-able version of this as an estimate of the game's skill level. Games with a high top rating are higher-skill than games with a low top rating.
You could collect the ranking data from sites like Yucata or BGA to create an estimate of skill for all those games fairly easily, I think.
Great topic! Chance actually has two dimensions. One, which you focus on, is the degree of asymmetric chance. If we roll dice, one of us gets a bigger number than the other, usually. It's asymmetric. If we are running a race, but the number of laps is only revealed after the 4th lap, the uncertainty is symmetric. This second chance of chance can actually increase the importance of skill, since the players must adapt to it (or, it reduces the importance of skill because it gets so hard that player can't adapt to it).
The difference shows up in the card game Goofspiel. https://www.rasmusen.org/rasmapedia/index.php?title=The_Goofspiel_Card_Game
The 13 prize cards that everyone will bid on can be stacked either randomly (normal version) or from lowest to higher (deterministic version). But it is symmetric: either nobody knows the order, or everybody does.
Goofspiel (and poker) remind us of a third type of chance: mixed strategies, randomized strategies. Let's use poker. If I can't read the other players' faces, I randomly decided whether to bluff or not. Depending on what they do, one of those two strategies is best. But ex ante there is no telling which is best.
From a public health point of view, should we restrict people to playing Go, tennis, chess and Tetris, and give them warnings about everything else?
If you read the economist paper, their entrance the topic is that apparently US law regulates games of skill vs. games of chance (gambling) differently. So the law must make a distinction between skill games and luck games, whereas there is only a continuum. A typical legal decision vs. continuous reality problem (e.g. mental illness is categorical in diagnoses and legal decisions but continuous in real life).
No - less than 50% skill does not mean 0% skill. Gambling (eg. slots) is different as it is programmatically designed for players to lose to the casino over the long run.
Scrabble probably has more skill + luck than any other game. There's a very high skill ceiling, but among players of similar skill it's very luck based, even at the top level. The ELO methodology might result in high std dev (almost on par with chess), but I think this wouldn't do justice to how much luck is involved.
If there's a very high ELO difference (which there would be in Scrabble—source: I played in some competitions), it's because the apparent luck is actually very evened out by players' ability to control for that luck. It's what @Eric Rasmusen referred to elsewhere in this thread as “symmetric luck”. That often appears random to weak players, but experienced players understand that much of the skill consists in managing the luck and playing probabilistically.