I'd talk to a lawyer. There are presumably some legal arguments that might apply. There definitely have been cases revolving around it, but I have no idea what the outcomes were.
From a moral point of view: robots.txt states an intent, and intentionally ignoring that is not a nice thing.
And often, "not a nice thing" translates into legal action. So, if this matters to you, you should find an expert :)
> From a moral point of view: robots.txt states an intent, and intentionally ignoring that is not a nice thing.
As a lawyer asked inane questions by friends, a common answer is: "I am no sure if its legal to do, but I'm absolutely sure you're a asshole if you do it."
From a moral point of view: robots.txt states an intent, and intentionally ignoring that is not a nice thing.
And often, "not a nice thing" translates into legal action. So, if this matters to you, you should find an expert :)