Not really (I don't go to church). All swear words are pronounced differently from the actual word (see 'joual' on google). The priest (or anybody else) can say the word in proper french and it won't be heard as a swear word.
Also, I think it has to do with how you use the word. When it's a noun, it's pretty clear that you're not swearing.