HN has a bit of the opposite policy, but it would be interesting to only allow downvotes if you have responded to the comment.

You would need a separate mechanism for flagging though to filter low effort contributions and it is questionable if that wouldn't just replace downvotes with flagging. Alternatively maybe only have that rule for comments above a certain length, although that isn't a good metric either.