How to Use AI to Study

There is a study that should change how you use AI to learn, and its most important number is the one that looks like good news.

In Generative AI Without Guardrails Can Harm Learning: Evidence from High School Mathematics — Bastani, Bastani, Sungu, Ge, Kabakcı and Mariman, published in PNAS — nearly a thousand high school maths students were given access to one of two GPT-4 tutors. GPT Base was a standard chat interface. GPT Tutor was the same model with prompts designed to safeguard learning: it gave teacher-designed hints instead of answers.

During the practice sessions, both helped enormously. Grades improved 48% with GPT Base and 127% with GPT Tutor.

Then the researchers took the tools away and ran an exam.

The GPT Base students scored 17% worse than students who never had AI access at all. Not worse than the tutor group — worse than the control. The students who had the most help while practising ended up knowing the least. GPT Tutor's guardrails largely eliminated that effect.

The authors' explanation is blunt: without guardrails, students used GPT-4 as a crutch during practice, and subsequently performed worse on their own.

The number that feels like progress is the warning sign

This is the trap, and it is almost impossible to notice from the inside. When you paste a problem and get a clean worked solution, the session feels productive. You understood every line. Nothing was confusing. You covered twenty problems instead of six.

That feeling is the symptom. Following a correct explanation is a different cognitive act from generating one, and only the second is the thing an exam — or a job — actually tests. Fluency while reading is not retrievability later.

So the practical rule is uncomfortable but simple: a study session that felt hard and slow is usually the one that worked. If your AI study session feels smooth, you are probably in the GPT Base condition of your own life.

Make the model withhold the answer

The single highest-value change is to instruct the model not to solve things for you. This is what separated the two conditions in the study, and it costs one paragraph:

Act as a Socratic tutor. Do not give me the answer or the next step. Ask me one question at a time to find where my reasoning breaks, and give me the smallest hint that would let me continue. If I ask you outright for the answer, ask me what I think it is first and why.

The last sentence matters more than it looks. You will ask for the answer — under time pressure, everyone does — and without that clause the model will simply comply. It is trained to be helpful, and handing over a solution is the locally helpful move. You have to tell it that being helpful here means refusing.

This is the design principle behind the learning pack: the tutor prompt guides with questions and hints rather than solutions, the practice quiz withholds the answer key until you have committed, and the worked-solution prompt always ends with fresh problems for you to attempt unaided.

Generate, then check — not read, then nod

Reverse the default direction of every study interaction. Produce first, get feedback second.

• Instead of explain photosynthesis → write your own explanation, then paste it and ask what is wrong, missing, or imprecise.
• Instead of summarise this chapter → summarise it yourself from memory, then ask the model to mark what you left out.
• Instead of make me flashcards → draft the cards yourself, then ask it to find the ones that are ambiguous or that test recognition instead of recall.

Each version is slower and each version is the one that works, because you cannot get feedback on retrieval you never attempted. The model is far better as a marker than as a lecturer.

The three places AI genuinely helps a learner

Not everything should be made harder. Three uses are unambiguously good:

Generating practice items. Writing thirty varied problems on a topic is tedious and mechanical, and having many attempts is one of the most robust findings in learning research. Ask for problems, not solutions — and ask for the answer key in a separate message you don't read until you're done.

Diagnosing the misconception. Here is my wrong answer and my working — don't correct it, tell me which specific misunderstanding would produce exactly this error. This is genuinely hard for a human tutor to do at 11pm and it is the highest-leverage question in the entire field.

Scheduling and structure. Turning a deadline and your available hours into a day-by-day plan with spaced review is planning, not learning. There is no crutch risk in outsourcing it, and spacing is the thing students reliably get wrong on their own.

Calibrate against something the model didn't write

The failure mode this creates: if you study with AI, self-assess with AI, and get graded by AI, you never learn whether the whole loop is drifting. Models are agreeable, and an assessment from something that wants you to feel good is not an assessment.

At least once per topic, test yourself on material the model has not seen and cannot mark — a past paper, a textbook problem set, a friend asking questions. That is your only reading on whether the practice gains are real or are the 48% that evaporates when the tool goes away.

The students in the study didn't know which condition they were in. You do, and you get to choose. Choose the one that felt worse.

---

Sources: Bastani, H., Bastani, O., Sungu, A., Ge, H., Kabakcı, Ö. and Mariman, R., Generative AI Without Guardrails Can Harm Learning: Evidence from High School Mathematics, PNAS (2025), doi:10.1073/pnas.2422633122.