In my experience as a published academic author all the LLM will make up all kinds of plausible papers I “wrote” that don’t exist, academic positions I’ve never held, and the like.
Even if you give it a paper directly I’d not believe it to be reliable. Maybe it could help search for papers, but that’s it.
right now, I download pdf and upload it to chatgpt to bounce ideas.