Permanent magnets are locked in place. If they're floating freely, the "tiny magnets" just snap together and cancel each other out. That's why heating (increasing particle mobility) a permanent magnet makes it lose its magnetization.