I get judged about going to Disney too - but the people that know me really well, kind of understand it now.
It does annoy me however, when I'm told to start saving so I can go on a beach holiday with friends - if I have money to save, its being saved for a Disney Holiday with my family, not for a beach holiday with 'the girls', that I won't enjoy, and besides, I don't want to go on holiday without my little girl!! (These are obviously child-less friends - who don't quite understand why I wouldn't possibly grab the chance of a child free holiday)
To me, a holiday wouldn't be the same without my family - and I'm lucky that my husband shares my love of Disney, and that he doesn't want to holiday anywhere else either. =)
Our daughter doesn't stand a chance - she HAS to love Disney as much as her Mummy and Daddy.. ha ha (But it does help that she's a big fan too)