So this is basically a *safe* optimization, since the index will always be valid and there's no need for the compiler to do a check and unwrap, panic.
This is how unsafe {} should be used. Sometimes some things are true but the compiler can't know that. And here the unsafe {} means that it dereferences a raw pointer (the index that we know is valid). If the unsafe {} safety condition is valid, unsafe {} is, well, safe.
Furthermore, it's an optional optimization (you could just copy the code and replace the unsafe access with a safe one, if you're paranoid) and it's not like if you write it in C++ it will be any safer than Rust's unsafe?!
This is how unsafe {} should be used. Sometimes some things are true but the compiler can't know that. And here the unsafe {} means that it dereferences a raw pointer (the index that we know is valid). If the unsafe {} safety condition is valid, unsafe {} is, well, safe.