I doubt this entirely. It might be quite difficult for said human to discern whether a simple passage were generated sans such accumulated experience in reading AI text, true. But LLMs do not converse like humans in ways that have always been essentially immediately obvious.
Sometimes I wonder if LLMs are just revealing a section of the population with untreated mental illness or if LLMs are actively exacerbating mental illness.
We might eventually regret exposing the general population to such a new technology without almost any safeguards.
Have you seen how many people form one-sided bonds with stars that don’t know they exist? Lonely people suspend disbelief to find some comfort. It’s not proof that the chatbot is indistinguishable from a human companion.
So? The people who can do it prove it can be done - the people who can't don't prove the opposite. Might as well claim all math is wrong because most people don't understand it.