Babies, due to not believing in anything theologically at birth, are by definition atheists. After all, to be an atheist you don't have to say 'there is no god'. It is a statement of uncertainty which can include 'I don't know if I believe'. In the case of the baby, since the baby has not(and cannot have) considered any theological ideas makes them atheists by definition.

If they are all secretly Muslims or Christians, I have never seen any evidence to suggest that is the state of the world.