Study Overview and Design
Recently, MIT researchers, led by Nataliya Kosmyna, explored brain activation differences between LLM users and others and LLM effects over critical thinking Moreover, the study included 54 participants, aged 18-39, divided into three distinct groups. First, one group used LLMs; next, another used search engines; finally, a third group used no tools at all. During each session, participants wrote SAT essays on various topics, but they only had 20 minutes per essay. Furthermore, the study spanned four months and featured four sessions, with the first three following the same structure.
However, in the fourth session, 18 participants switched groups: the Brain-only group tried LLMs, and LLM users wrote without LLMs. Throughout the study, researchers measured brain activity using EEGs and analyzed n-gram patterns, among other metrics. Additionally, both human teachers and an AI judge graded all essays for quality and style. Consequently, researchers focused on brain connectivity, neural responses, behavioral patterns, and linguistic performance across all groups.
Key Findings: Brain and Behavior
Meanwhile, the LLM group showed ambition-based writing styles, closely matching the generative output of their LLM tools. As a result, LLM users exhibited the least neural activity and reduced engagement in attention control and semantic integration. Similarly, these participants relied less on high-order cognition, such as moral reasoning and emotionally-grounded decision-making. Therefore, the paper concluded that LLM users relied on tool-based composition rather than self-generated reflection.
On the other hand, the search engine group demonstrated more integration of external material into their essays. Subsequently, these users showed heightened cognitive effort, processing essays through more effortful semantic and attentional pathways.
In contrast, the Brain-only group displayed the most brain activity and greater engagement in moral and emotional reasoning. Moreover, these subjects produced more abstract thought and self-reflection, free from tool-based composition influences. Meanwhile, some Brain-only essays even reflected social media influences in their content and style.
Linguistic and Cognitive Differences
From NLP analysis, the Brain-only group exhibited the most variability in their approaches to each essay. Conversely, LLM users generated static paragraphs with minimal deviation between essays. Similarly, the search engine group’s ideas and keywords were shaped by their own queries, not structured LLM prompts. Ultimately, LLM users rarely used critical thinking to evaluate LLM output, and few relied on their own ideas.
Final Conclusions and Implications
Furthermore, Brain-only participants reported higher satisfaction with their essays, while LLM users valued their own work less. Additionally, teachers grading essays could easily identify LLM-generated content by its similarity and lack of expressiveness. Finally, Nataliya Kosmyna warned that LLM effects over critical thinking could hinder development in cognitive abilities among its younger audience.