Black Monday for the Ruy Lopez

One of the nice things about chess is that it's a game played between humans. Despite an understandable tendency to approach chess as a rational game, I think there are a lot of irrational - human - things going on behind the scenes. One of the things that fascinate me most in chess is people's choice of openings. Last year, I compared chess openings with baby names and suggested that apart from being subject to fashion, the popularity of chess openings, like the popularity of baby names, may have something to do with the speed with which they rise to prominence. But after reading John Cassidy's How Markets Fail, his book on the economic crisis, I think herd behaviour plays a role that's at least as important. Cassidy introduces herd behaviour by quoting the second most famous economist of all time, John Maynard Keynes, who said "it's better to fail conventionally than to succeed unconventionally." And indeed, following a study from 1990 by economists Scharfstein and Stein in The American Review, evidence was found that under many circumstances, the best strategy for investors is to copy behaviour of others instead of trusting their own judgement:

When a manager of an investment fund follows the crowd and things turn out badly, he shares the embarrassment with everyone else; if he follows a completely different strategy, he is exclusively responsible for his own mistakes.

When I read this quote (which I translated back into English from the Dutch translation!), I was immediately reminded of Newton's famous metaphor, often echoed in chess-improvement books: the one about "standing on the shoulders of giants". Specifically, for chess openings this implies that it's better for amateurs to play the Sicilian because giants like Carlsen, Anand, Kasparov and Fischer play it - than Owen's Defence (1.e4 b6), because, well, nobody plays it. I have often witnessed such behaviour in chess players, myself included. Though a healthy advice in itself, it can often turn out disastrous. Who hasn't had the experience of faithfully copying a great player's game or analysis, only to realize that despite all good intentions, real understanding of the position is lacking? Or worse, that even though you understand the position, your opponent understands it better or has found an improvement over your hero's evaluation? (Yes, even the greatest chess players are sometimes proven wrong in mere amateur games!) But that's not what struck me in Cassidy's quote. What I found more interesting was the psychological "embarrassment" argument: the idea that it's somehow less shameful to fail in a way that everyone else fails. Suddenly, I understood this is also true for chess openings, though I think it is a much less-often discussed subject. As on Wall Street, in chess, too, there is such a thing as 'group pressure'. Funnily enough, I had even experienced it the very night before, during a blitz session with some friends. Time and again, my friend was having a rough time as White trying to refute a particular offbeat line I had been playing for some months now. He explained over the board that this time, he thought up some move at home but during play itself, he'd forgotten its exact point and lost anyway. After the game, the kibitzers jokingly asked him why he simply didn't play the main line against this obscure variation. And sure enough, in the next game, instead of trying to find the idea behind his home preparation - which, it turned out later, was 100% correct! - he switched to the book 'refutation' and, not knowing anything about it, I'm sad to report he lost again. This is a typical case of herd behaviour - going with the voice of the majority, even if you should really know better - but at least my friend had the excuse that he didn't actually know at the time what he should have done. This is, of course, contrary to what happened at Wall Street in 1987 and even more spectacularly during the Internet bubble of the late 1990s, where, according to Cassidy, "skilled and experienced investors consciously helped blowing up the bubble".

William Beard - The Bulls and the Bears (1879)

I remember the occasion very well. I was still a promising junior player with a slightly unconventional opening repertoire. It was an important team match and I felt my game could play a key role in it, if only because I knew my opponent well and so could prepare very concretely. A few minutes before the game, I overheard two of my stronger teammates saying to each other that they hoped I wouldn't play that 'reckless pet-line' of mine in such an important game. When my clock was started, I was utterly confused. Should I play it or should I not? Should I follow my own way - at the risk of being punished when things went wrong - or should I stick to the "safe" mainlines even though I knew much less about them? Well, you can probably guess how it ended. I chose to safely stand of the shoulders of giants - and lost without a fight. Of course, when I asked my opponent what he would have played against the line had had prepared, he told me he wanted to go for a line I had analyzed almost to mate! I never did dare to tell my team members. In fact, they praised my opening play after the game and sympathized with me for having failed, but at least tried in a decent way. I failed conventionally, and for them this was much better than the prospect of succeeding unconventionally. I vowed never to be such a coward again. (Although for me personally, it's still not easy being stubborn when interests are shared with 9 other team members!) In amateur chess, this type of thing - rational herd behaviour in economical terms - probably happens all the time. But I've often thought about the possibility that not only amateurs display rational herd behaviour, but that top grandmasters often do exactly the same thing. To be sure, some elite players are more unconventional than others, but some trends in chess openings are simply beyond comprehension - even according to the opening experts themselves. Magnus Carlsen recently showed that even the notorious King's Gambit can be employed successfully at lop level - yet so far, nobody in the world's top 50 has dared to follow the world's number one. (Though maybe Arkadij Naiditsch being crushed with Black by 2200-player Schwekendiek in Mainz might mobilize a new herd!) Could it simply be a matter of being afraid to become the laughing stock of the chess Olympus after a loss? In his book (in which he explains this and other phenomena in great detail), John Cassidy points out that there's considerable scientific evidence (for instance, research performed by neuroscientist S. Berns of Emory University) that the urge to follow the herd has a neurological basis. Of course, he assures us, not all market investors are lemmings which collectively plunge themselves to their doom, but generally speaking, the judgement of others really is incorporated into our own. At the very least, one influences the other to a considerable degree. And this is true for investors as well as chess players. So, is there a chess equivalent of the real estate bubble? What was the Dot-Com crash of chess openings? Will there some day be a Black Monday for the Ruy Lopez? When was the last time grandmasters collectively went for a variation that was subsequently refuted by a single simple, elegant little idea? Perhaps it's too early to write off Owen's Defense yet.