Do I have to let the buyer try on the dress?

It's completely up to you. If you don't feel comfortable meeting a buyer in person, just explain this when contacted (or include it in your listing). Some buyers prefer the opportunity to try on the dress and some prefer to purchase online.