Diamonds are actually tricky, because their practical brittleness is mostly due to breaks in the lattice caused by impurities. The lines of weakness are defined as cleavage, and diamonds will typically cleave along those lines when the break.
So there's a bit of ambiguity between the actual brittleness of the pure carbon structure and the practical brittleness where breaks in that structure naturally exist.