Half Sigma discusses chess and the nerd factor in his post today. He cites the Peter Principle (or perhaps more accurate the Peter Rule) which is "non-athletic activities which only men participate in, yet are not viewed as traditionally masculaine, are the nerdiest activities."
From my experience, I agree that non-athletic, male-dominated, non-traditionally-masculine activities are nerdy. I also think an activity is nerdy if it has no real economic, pratical or intelligent value.
True, the argument could be made that many board games and computer games have some intelligent value, but I think chess tends to be associated with greater value than D&D and MMORPGs. And the reason I think that chess has the greater value is because in chess, you the player control all the pieces and are in charge of all operations, strategy and tactics on the board. In D&D and MMORPGs, you are one of many players and relinquish the "big picture". In chess, you're above the fray and not so caught up in the game while in D&D and MMORPGs you could get lost in time and reality.
And maybe that is another factor in the nerd factor: how detached are you from reality? The less detached from reality, the less nerdy.