Danny Hillis discusses the thinking behind locating it in a remote mountain here https://soundcloud.com/longnow/progress-on-the-10000-year-cl...

But the entire talk is worth listening to.

It's not intended to be a secret - it's there to be visited (I hope one day to visit it myself), the mountain is part of a pilgrimage to better frame the experience.