# Giving University Exams in the Age of Chatbots Source: https://www.youtube.com/watch?v=zjvnKqf2uys Recap page: https://rapidrecap.app/video/zjvnKqf2uys Generated: 2026-01-23T03:03:35.864+00:00 --- ## Quick Overview Professor Plum's experiment revealed that students who relied heavily on AI tools like ChatGPT during an open-book exam performed significantly worse (scoring 8-11 out of 20) than those who did not, highlighting the danger of over-reliance on AI, which stifles learning and critical thinking skills necessary for true mastery. **Key Points:** - Professor Plum conducted an experiment on January 20, 2026, involving 60 university students to test the effect of AI on exam performance. - Students who used AI (like ChatGPT) for an exam scored significantly lower (8-11 out of 20) compared to those who relied on their own knowledge (scoring 18 or 19 out of 20). - The experiment involved two groups: Option A (traditional method, no AI) and Option B (allowed to use AI/internet). - Students in Group B, who used AI, were terrified of the accountability clause requiring them to log every prompt, leading them to avoid using the tools for the actual exam content. - The professor noted that students relying on AI were essentially outsourcing their thinking, leading to poorer outcomes, illustrating that AI can make users 'dumber' in the moment. - The log files confirmed that students in Group B knew the logic behind the answers they were submitting, even when they failed, suggesting they understood the underlying concepts but failed to execute them under pressure or due to fear. - The professor's ultimate goal was to show that fear of AI dependency stifles true learning, contrasting the results with the inherent collaborative nature of open-source tools like Git. ![Screenshot at 00:19: Professor Plum's experiment comparing the performance of students using AI \(described as an inflatable T-Rex costume\) versus those who relied on their own knowledge during an exam.](https://ss.rapidrecap.app/screens/zjvnKqf2uys/00-00-19.jpg) **Context:** The discussion revolves around an experiment conducted by Professor Plum in January 2026 concerning the impact of generative AI tools, specifically ChatGPT, on university students taking an open-book exam. The context is set against the backdrop of increasing anxiety about AI dependency in education, where many students are tempted to use these tools for academic work, despite fears of institutional repercussions like a three-year ban for cheating. The experiment sought to compare the performance of students who used AI assistance versus those who relied on traditional study methods. ## Detailed Analysis The video details an experiment conducted by Professor Plum on January 20, 2026, involving 60 university students to assess the impact of AI use on exam performance. Students were split into two groups: Option A (no AI/internet) and Option B (allowed AI/internet access). The results were shocking: students using AI scored significantly lower (8 to 11 out of 20) than the students who did not use AI (scoring 18 or 19 out of 20). The professor found that the heavy AI users, despite having access to tools like ChatGPT, performed the worst. One student who failed the exam even admitted to frantically copying AI-generated text without understanding it, illustrating that outsourcing thinking makes users 'dumber' in the moment. Furthermore, students in Group B were terrified of the accountability clause requiring them to log every prompt, leading them to avoid using AI for core exam content, opting instead to use it as a 'sounding board' or for simple arithmetic. The professor concludes that the fear of AI over-reliance is stifling genuine learning and collaboration, forcing students into isolation, whereas tools like Git exemplify healthy collaboration. The core takeaway is that when students rely on AI as a crutch rather than a tool, they fail to develop the foundational critical thinking skills needed for mastery. ### Experiment Setup - Professor Plum tested 60 students on January 20, 2026, using an open-book exam format with two options: AI allowed (Option B) or AI disallowed (Option A). - Students using AI scored 8-11/20; students avoiding AI scored 18-19/20 - The AI users were afraid of an accountability clause requiring prompt logging. ### The Failure of AI Reliance (Group B) - Students who used AI were terrified of the transparency requirement; they only used it for simple tasks like checking 2+2 or to check their own logic, not for core content creation. - The students who relied on AI failed spectacularly, scoring worse than the students in Group A. ### The Psychological Impact - The professor calls the fear of dependency 'radioactive' and notes that the fear of being caught (the accountability clause) caused students to self-isolate into 'silos' rather than collaborate. - The professor explicitly compares this outcome to teaching someone to drive a Ford but only showing them how to use the steering wheel, not the pedals. ### The Git Analogy - Open-source tools like Git exemplify healthy, collaborative creation, which contrasts sharply with the fear-driven isolation caused by current AI policies. ### Conclusion - The experiment proves that over-reliance on AI tools that provide ready-made answers, without understanding the underlying logic, ultimately harms learning and critical thinking, leading to failure. ![Screenshot at 00:00: Introductory graphic showing two podcasters and the call to action: 'BECOME A MEMBER TODAY!'](https://ss.rapidrecap.app/screens/zjvnKqf2uys/00-00-00.jpg) ![Screenshot at 00:17: Professor Plum dressed up in an inflatable T-Rex costume, symbolizing the absurdity or distraction of the AI reliance issue.](https://ss.rapidrecap.app/screens/zjvnKqf2uys/00-00-17.jpg) ![Screenshot at 02:28: The student who failed the exam revealed that he copied AI text without understanding it, highlighting the lack of internal processing.](https://ss.rapidrecap.app/screens/zjvnKqf2uys/00-02-28.jpg) ![Screenshot at 05:55: The professor contrasting the 'heavy users' who didn't use Git \(open-source collaboration\) with the students who failed.](https://ss.rapidrecap.app/screens/zjvnKqf2uys/00-05-55.jpg) ![Screenshot at 10:48: A slide or graphic illustrating the contrast between the student who used AI \(low scores\) and the student who didn't \(high scores\).](https://ss.rapidrecap.app/screens/zjvnKqf2uys/00-10-48.jpg)