My six year old has been playing chess with me for about a year now. I let him win sometimes and make attempts at playing into positions where he can take advantage of a position he has seen in the past. He wants to play so we play. He likes it when we spend about 3-5 minutes going over the game. (Anymore and he gets bored.) He has his own chess book, Kasparov's My First Chess Book. (He saw dad had chess books so we got him one of his own. ) I wouldn't have thought a five year old would be ready for chess until my son wanted to play.
So I say as soon as they are ready to enjoy the game, let them play. He sometimes does funny things like mixing the pieces in with his Legos for a Lego Star Wars vs. Knights battle, but he has fun with it. Other kids are likely to do equally non-chess distractions or have fun at the boards expense, so what. At this point his enjoyment is what it's about. Your never too young to have fun.
You can begin introducing games such as chess and GO (3,000-year-old asian game of territories using black and white stones on a grid) to children as early as 4 years old. The path of their development would begin from their reactions from thereon.
You will read stories of elite chess players starting to show interest early in their lives, but the key word is INTEREST.
You can copy the Polgar Experiment to nurture his three daughters into strong chess players through home schooling and continuous chess curriculum, but be mindful NOT to become the infamous chess father who almost destroyed Gata Kamsky .
Finally, If you left them alone, they could grow up and strive to become chess champions on their own like Bobby Fischer.