Kasparov, Donner and the infinite regress of knowing

Last week, in an important team match, my opponent played an innocent sideline against my pet-opening. I had prepared quite well for this encounter, but had not looked at this particular variation since it is so rarely played. Over the board, I knew I had once looked at this line in detail, and had also found a nice way of dealing with it - but to my chagrin I couldn't remember it now.

Slighly altered version of the Homunculus Objection adapted by Dave Cantrell, originally published in Smithsonian 16 (1) (April 1985):97.

For some minutes I tried to retrieve the lost knowledge from the depths of my memory, but to no avail, so I decided to be practical and play a natural move instead. Further on in the game, in a normal middlegame position, instead of just playing some natural move, I suddenly saw a funny little intermediate move, weakening my kingside but weirdly complicating the position. I briefly started calculating the consequences of the move and I started to like it more and more. After some time, since I couldn't find a clear refutation, I played it.

The funny thing is that all the time while I was contemplating this crazy little move, I was totally aware of its 'ugliness', of how utterly 'unnatural' it looked and how unpositional its foundations really were. Despite this, I decided it was worth the risk and offered interesting fighting changes. Unfortunately, it turned out the move was rather easily refuted and I was left with a wrecked kingside, resulting in a zero for the team. (Though we did win the match.)

These situations are examples of a phenomenon called metacognition, or 'knowing about knowing': I knew I knew a particular opening line, but I just couldn't remember it. Likewise, in the middlegame, I knew my weird idea was against all positional rules, but I played it anyway.

A famous chess-related anecdote involving metacognition was once described by Tim Krabbé is his story A Walk with Kasparov. In this story, Krabbé describes how Garry Kasparov, having just lost to Jeroen Piket in the last round of the VSB tournament in Amsterdam, starts talking to him after the game. Kasparov tells Krabbé he had prepared the novelty he played in the game, but then to his horror couldn't remember the lines:

I ask him: 'Do you mean to say that Re4 might be in your computer and you forgot about it?' 'Maybe, maybe,' he says. 'I'm just curious to know. Are you curious too?' 'Yes I am.' 'Then come to my hotel and we'll check.'

Krabbé joins him and Yuri Dokhoian to Kasparov's hotel room, and watches as Kasparov opens his laptop, expecting to find the idea behind the novelty 19...Re4 hidden in his database. But even now, Kasparov can't find his forgotten home preparation, which Piket refuted with the reply 20.Bg3!

He is desolated, but he cannot find 19.Na4, let alone 19...Re4. He's absolutely sure Na4 is in one of his computers somewhere, but it doesn't seem to be in this one.

Interestingly, Krabbé kind of suggests that Kasparov may have been confusing lines and that he never analysed the move 19...Re4 in this position in the first place. Put differently, Kasparov was wrong about being sure he'd forgotten what he once knew! Of course, the interesting question is why, if he didn't remember the exact line, Kasparov had played the move anyway? When he played 19...Re4 he obviously felt the idea behind it must have been 'at the tip of his fingers', but how could he be sure he would retrieve this lost knowledge during the game?

There are several possibilities. Perhaps Kasparov had some experience in retrieving lost knowledge (after all, he does had a photographic memory) within reasonable thinking time. Or he just had to remember the precise move order and everything would be allright again. Or maybe a little distraction by walking around would be sufficient to get his mind back on track again. In my own experience, such 'aha!' moments usually come only days or even weeks after the actual event - not minutes.

In a recent blogpost on metacognition, Johan Lehrer of The Frontal Cortex describes the familiar 'tip-of-the-tongue' moment when you just can't think of the name of an acquaintance:

What's interesting about this mental hiccup is that, even though the mind can't remember the information, it's convinced that it knows it, which is why we devote so many mental resources to trying to recover the missing word. (...) But here's the mystery: If we've forgotten a person's name, then why are we so convinced that we remember it? What does it mean to know something without being able to access it?

The larger question is how the mind decides what to think about. After all, if we really don't know the name - it's nowhere inside our head - then it's a waste of time trying to find it. This is where metacognition, or thinking about thinking, comes in handy. At any given moment, we automatically monitor the flux of thoughts, emotions and errata flowing in the stream of consciousness. As a result, when a name goes missing we immediately analyze the likelihood of being able to remember it. Do we know the first letter of the name? Can we remember other facts about the person? Are we able to remember the first names of other acquaintances from high school? Based on the answer to these questions, we can then make an informed guess about whether or not it's worth trying to retrieve the misplaced memory.

The interesting thing in the Kasparov example is, in my view, that Kasparov did not remember correctly. As said, he was sure of something he shouldn't have been sure about. He was too confident of himself, perhaps: after all, someone like Kasparov has to remember such an awful lot of variations that inevitably, something goes wrong in his head from time to time. Maybe he would have been helped by something Jonathan Rowson describes in his book Chess for Zebras (2005): unlearning. Say what?

Jonathan Rowson in November 2008 at the Dresden Olympiad

Unlearning is really a way of of constantly looking at the baggage you bring to chess positions ans trying to work on the baggage that is most obviously problematic. It is also a way of trying to look at chess positions with fresh eyes, as free as possible from prejudices. When you succceed in doing this, you start to see the prejudices as prejudices, and not as absolute truths, and that's when real improvement becomes easier.

Basically, according to Rowson, Kasparov should have looked at the position after 19.Na4 with 'fresh eyes' and realized that even if he had once analysed 19...Re4, he didn't like it after 20.Bg3. He should have unlearned what he thought he had learned, so to say. He should have ignored the voice in his head telling him to play the 'analyzed' move Re4.

But wait a minute ... does ignoring the voice in your head also imply that in my own above mentioned game, I was actually right in playing that ugly-looking move', ignoring my intuition and going for concrete, 'fresh' complications? Stubbornly, consciously going against your intuition and your natural sense of reality was often subject of the chess writings of J.H. Donner. Here's how he described his thoughts during a game against Milic in 1950:

Until this moment I hadn't used much time to think, just fifteen minutes on my 19th move. Everything went fine, I saw everything and felt contempt for my opponent. But now a paralysing doubt overwhelmed me. Suddenly, I saw white pieces coming from all sides, while I was clearly aware that White didn't have a realistic chance. 'Stay calm,' I said to myself. 'You're winning.' But it didn't help. I couldn't calm down. I did see the best move in the position, h5, and I wanted to play it, but I touched my rook and played Rg8.

Whenever I read fragments like this (Donner describes many of them), or whenever I experience them myself, I become very pessimstic about books teaching you how to play chess. If even players like Donner and Kasparov (occasionally) couldn't handle the metacognitive voices in their heads telling them to play moves they only thought they remembered, or something they knew was bad, how does improving your chess knowledge by reading chess books actually help during a game?

During a game, you may be perfectly aware that a certain move violates some deep positional principle, but if another voice in your head tells you to 'unlearn' that learned principle and look at the position without preconceived knowledge (a phenomenon John Watson calls 'rule independence'), the result is the exact opposite. How to decide which voice is right? How to decide when to follow a rule and when to violate it? The problem of improving chess players is not that they don't know the definition of a weak square, but when the weak square is relevant and when not.

And even if, like me, you know that you're vulnerable to this kind of metacognitive confusion, it isn't easy to 'switch off' this mode of thinking just like that: after all, this only hands the question over to the next meta-level. Now you're ignoring the fact that you're ignoring that voice in your head telling you that you know a certain move must be played... And once you become aware of this, it starts all over again. How to avoid an infinite regress of metacognitive levels, leading straight to insanity?

I'm sure you'll say that the answer is to simply stop worrying about these different levels of cognition. Just focus on the chess! Does a super talent like Anish Giri think about metacognition? Of course he doesn't! He just plays the right move and that's the end of it! But most chess players are not super talents, and even a self-conscious grandmaster like Donner found it easier said than done. Heck, even Kasparov sometimes couldn't avoid this confusion in his head. At least for some of us, the problem is real.

Jonathan Rowson wrote that "if you want to become a better player, you need better habits, and you cultivate better habits through training", echoing Harvard psychologist Jerome Bruner's observation that "knowledge helps only when it descends into habits." Maybe they are right. Maybe training is the key to everything. Big like most chess players, I have never received any formal form of training, nor do I have time for it now. How to compensate this? Will we amateurs ever be able to make up for this lack of excercise? Or will any chess improvement book we read only increase our confusion?

Perhaps it is our fate that we'll always feel a bit like Faust, who during his quest for worldly knowledge, said:

Two souls, alas, are housed within my breast, and each will wrestle for the mastery there.

If you don't recognize this feeling, make sure you'll forget this article right away - before it's too late.