The non-believer doesn't object to belief; he objects to certainty.
When a child is told that he knows what he doesn't know, the child bypasses an essential stage in the development of his humanity. Substituting the security of knowledge for the authenticity of experience, the child never learns confusion, wonder, curiosity, doubt. He never enjoys an awareness of his own stupidity. That awareness softens a person, humbles him, and gives him the sense of humor that will permit him the kinds of recklessness (intellectual, physical, emotional) from which he can acquire wisdom.
From that recklessness he might also acquire a religious sensibility grounded in humility, which is a prerequisite for kindness, the stated goal of all religions, everywhere. (The Dalai Lama: "My religion is very simple. My religion is kindness.")
The religious catastrophe—a catastrophe, too, for religion—is that men learn to think they know what they don't know. So they don't know how it feels to actually know something, how it feels to love knowledge.
As a result, they see knowledge as a source of power instead of a source of delight, and they turn on the world, intent upon making everyone else into mirrors of themselves, crusading slaves of certainty.