Vibe Artifacts, Not Vibe Papers
Asymmetric warfare on attention is not good for science
Recently, companies both big and small have released tools for “vibe writing” papers. While AI tools will and should change many of the ways that we do science, using AI to do most of the writing for traditional papers (a la arXiv or Nature) is not one of them. Here is a short argument why and, in the spirit of forward-looking positivity, some productive ways AI can help transmit scientific knowledge.
At the end of the day, a modern LLM’s job is to predict the maximally probable next token. One of the jobs of a scientific paper is to convey novel-to-humanity information. These two MOs are fundamentally incompatible. In a well-written paper, every single detail matters. This is a way in which good scientific communication differs from most other writing. (Although ideally more nonfiction would be like this.) Until all the experiments were done by the AI, it will never have all the context and nuance of the scientist who did the work.
Yes, many scientists are terrible at writing. And, for many scientists, English is not their first language. It’s frustrating to be stuck in a system that depends on well-written English to communicate scientific results. But several things are true at once:
Communicating results well is part of doing good science.
There are many ways to use LLMs to improve writing without having them generate large swaths of text.
The point of a paper is to be read. There’s an implicit compact between a writer and a reader — “you put attention and work into writing this and I will put attention and work into reading it.”
The world is already deluged with garbage papers.
Whether or not the paper is still the best way of conveying research results (it’s not), encouraging asymmetric warfare on attention is not good for science.
The real question we should be asking is “how can we use AI to disseminate scientific results well?” not “how do we exacerbate the signal-to-noise problem in science?”
Here are two ideas:
Use LLMs to create non-paper digital artifacts. Things like interactive websites, modifiable code and figures. These artifacts have the property where they demand far less attention to interact with, compress a lot of information, and it doesn’t matter who writes the code that generates them as long as the output is nuanced and accurate.
Use LLMs to create artifacts for other LLMs. Do analysis work in conjunction with Codex or Claude Code and then ask it to create a context file that would let another instance quickly ingest the calculations. You could use this trick to make it easy for someone else to replicate an analysis you did with spaghetti code and a bespoke environment. (We’ll leave the question of whether it counts as science if only LLMs understand it for another day.)
We write Reinvent Science to broaden the conversation around science and science funding, and we rely on you to help us reach as many readers as possible. Please support our work by subscribing and sharing.
