Background
I have been working on this idea for our course AI Engineering
Causal ML has a lot of potential in itself - but I see it more as a combination of AI (llms) and causal. In this post, I explain a relatively simple approach to use LLMs and causal ML together - in an easy to understand manner. I also explain the significance of causal ML in comparison to traditional machine learning techniques.
By keeping it simple - I think we could create real solutions to complex problems that combine these two approaches.
The overall problem statement could be formulated as follows:
Helping secondary school students improve their maths performance. Goal: Find out what actually causes better results (not just what’s correlated).
Think of a causal machine learning system as a very careful detective:
It doesn’t just notice patterns (“these two things move together”), it tries to figure out what actually causes what.
We build this system in two layers:
Questionnaire → to collect structured information from people on the ground
LLM (like ChatGPT) → to help design questions, interpret answers, and support the causal reasoning
Process
1. Define the decision question
“Which actions actually improve students’ maths outcomes?”
E.g.
Extra after-school tutoring
Access to an online learning app
Smaller class size
Teacher training in new methods
Parental homework support
This is a decision question:
“If we invest in X, does it cause better maths results?”
You can ask an LLM: “I want to reduce maths failure rates in secondary school. Suggest 5–7 concrete decision questions a policymaker might ask.”
2. Map possible causes and influences (in plain language)
We brainstorm what might affect maths outcomes:
We can use the LLM to prompt ideas like: “List factors that might influence secondary school maths performance, and separate them into:
things we can change,
background/context,
possible confounders.
This becomes your design checklist
For example, in the tutoring example:
Possible causes (levers we can change)
Extra tutoring (yes/no, hours per week)
Use of a maths learning app at home or school
Teacher training in new teaching methods
Frequency of formative feedback (quizzes, practice tests)
Parental support with homework
Context factors (not easily changed but important)
Student’s prior maths ability
Socio-economic background
School resources (number of teachers, computers)
Class size
Language spoken at home
Confusing influences (confounders)
Highly motivated students might Seek extra tutoring, Use apps more and Have higher scores anyway
Schools with more resources might offer Smaller classes, Better-trained teachers and
Better results
We need to measure these to compare fairly.
3. Turn the causal story into a questionnaire
Now we convert these ideas into questions for students, parents, and teachers.
3.1 Outcome questions (what we’re trying to improve)
“What was your grade in the last maths exam?” (letter or numeric)
“On average, how confident do you feel in maths?” (1–5 scale)
(From school records) pass/fail, score out of 100
3.2 Questions about interventions (potential causes)
a) To students:
“In the last 3 months, did you attend extra maths tutoring?” No/ Yes, once a week / Yes, 2–3 times a week / Yes, more than 3 times a week
“Do you use a maths learning app (like Khan Academy, etc.)?” - Never / 1–2 times a week / 3–5 times a week /Nearly every day
b) To teachers:
“Have you received specific training in new maths teaching methods in the last 2 years?” [Yes/No]
“How often do you give short practice quizzes?” - Rarely / Once a month / Once a week /More than once a week
c) To parents:
“How often do you help your child with maths homework?” Never / 1 day/week / 2–3 days/week / 4+ days/week
3.3 Questions about background / confounders
a) To students/parents:
“Did you usually pass maths in previous years?” (Yes/No)
“How many books are there at home (roughly)?” (as a rough proxy for home learning environment)
“How many siblings do you live with?”
“What language do you mostly speak at home?”
b) From school data:
Class size
Teacher years of experience
School resource index (computers per student, etc.)
3.4 Open-ended questions
To students: “What makes maths easier or harder for you?”
To teachers: “What do you think most helps students improve in maths?”
To parents: “What stops you from helping more with homework (if anything)?”
Where LLM helps
Drafting clear, age-appropriate questions
Checking for leading or confusing wording
Translating into simpler English or other languages
4. Collect the data
Run the survey across many classes/schools
Merge data: Questionnaire data (students, parents, teachers)
and School records (exam scores, class size, etc.)
Each student becomes one row in a table, with columns like:
extra_tutoring_hours
uses_app_frequency
teacher_trained
parent_homework_help
prior_maths_result
class_size
exam_score
5. Write down the causal story in simple terms
We now say explicitly:
“We believe that extra tutoring, use of a maths app, and teacher training can cause better maths results. But we also know that prior ability, family background, and school resources might influence both the chances of getting these interventions and exam scores.”
The key idea:
“We want like-for-like comparisons: students with similar background and ability, but different exposure to, say, extra tutoring.”
Where LLM helps Ask: “Given this context, describe a causal diagram in words, listing arrows between key variables.”
This helps you and stakeholders reason clearly before any statistics.
6. Causal machine learning (explained without maths)
Now the causal ML system uses the data to answer questions like:
“If a student starts attending weekly tutoring, how much will their maths score likely improve, on average?”
What the system does in plain terms
Finds similar students
Same prior maths level
Similar socio-economic background
Similar class size / school resources
Looks at who got the intervention
Group A: students who began extra tutoring
Group B: very similar students who didn’t
Compares outcomes
Average exam score in Group A vs Group B
Difference = estimated effect of tutoring, after considering background factors
Repeats for other interventions
Using a maths app
Teacher training
More frequent quizzes
Parental homework help
So we can say things like:
“For students with similar background and prior ability, those who attended weekly tutoring scored on average 6 points higher out of 100.”
“Using the maths app at least 3 times a week is associated with 4 points higher, controlling for other factors.”
This is causal ML in friendly language:
“Carefully comparing similar students to estimate what difference an intervention makes.”
7. How LLMs complement the causal analysis
LLMs don’t do the statistics, but they are powerful assistants around it.
7.1 Explaining results to non-experts
Technical output might say:
“Average treatment effect of tutoring = +6.2 points (95% CI 3.5–8.9).”
You give this plus context to an LLM and ask:
“Explain these findings for head teachers and parents in simple language.”
It can produce:
“For students similar to those in this study, starting weekly maths tutoring raised exam scores by about 6 points out of 100, on average. Some improved more, some less, but the evidence suggests tutoring meaningfully helps.”
7.2 Analysing open-ended answers
Feed the free-text responses to an LLM and ask it to:
Group common themes:
“Maths anxiety”
“Lack of quiet study space”
“Internet/data problems”
“Parents don’t understand the maths”
Provide quotes to illustrate each theme
Link these to interventions
Example narrative:
“Many students mention they feel anxious during tests. This suggests that tutoring might work even better when combined with simple test-anxiety strategies.”
7.3 Generating “what-if” stories for policy
Once you know the estimated effects, you can ask:
“Based on our findings, write a 1-page policy brief for the school board about whether to invest more in tutoring, apps, or teacher training, using plain English.”
The LLM will:
Weave numbers + context into a story
Present trade-offs (e.g., cost, feasibility)
Suggest phased pilots (“start with Year 9, evaluate, then scale”)
7.4 Checking assumptions and limitations
Ask:
“Given our causal setup (tutoring, apps, teacher training, background factors), what assumptions might fail, and what biases could affect our conclusions?”
The LLM might highlight:
Some students self-select into tutoring because they are more motivated
Tutors might differ in quality
Some schools may have unmeasured advantages
This helps you design better follow-up studies or randomised pilots.
8. One-paragraph story you can tell stakeholders
“We surveyed students, parents, and teachers to understand who gets extra tutoring, uses maths apps, and receives trained teaching—and we also recorded their backgrounds and exam scores. Then, using causal machine learning, we compared students who are similar in background and prior ability but differ in these interventions. This allowed us to estimate how much tutoring, app use, or teacher training actually causes improvement in maths scores. Large language models (like ChatGPT) helped us design clear questionnaires, summarise people’s written comments, and explain statistical results in plain language. Together, this system turns everyday school experiences into reliable evidence about what really helps students learn maths.”
Causal vs non causal
We can also compare causal and non causal approaches.
This is a very important question—because it makes clear why you bothered with causal ML at all.
We can explain this idea using the same example - Helping secondary school students improve maths performance” problem.
Think of non-causal techniques as answering: “What usually goes together?”
and causal techniques as answering: “What will happen if we change something?”
Non-Causal Techniques for Improving Maths Performance
1 Descriptive Statistics
(Averages, percentages, charts)
What it looks like
You might calculate:
Average maths score of students who attend tutoring
Average maths score of students who don’t
Percentage of high-scoring students using a maths app
Example finding:
“Students who attend tutoring score 15 points higher on average.”
Why it’s attractive
Simple
Easy to explain
Fast to compute
Great for summarising data
Key limitation (non-causal)
This does not tell you why the difference exists.
Possible explanations:
Tutored students were already stronger
Their parents are more involved
Their schools have better resources
Core problem:
Descriptive stats confuse cause and coincidence.
They answer: “What does the world look like?”
But not:“What would change if we intervene?”
2️Correlation Analysis
(“X is correlated with Y”)
What it looks like
You compute correlations such as:
App usage ↔ exam score
Homework help ↔ confidence
Class size ↔ performance
Example finding:
“Use of maths apps is strongly correlated with higher scores.”
Why it’s attractive
Slightly more “scientific” than averages
Identifies interesting relationships
Good for hypothesis generation
Key limitation (non-causal)
Correlation does not tell you direction or mechanism.
For example:
Do apps improve maths?
Or do good maths students enjoy apps?
Or do better schools provide both apps and good teaching?
Core problem:
Correlation cannot distinguish:
Cause → effect
Effect → cause
Shared underlying cause
So it cannot support policy decisions.
3️Predictive Machine Learning
(Regression, random forests, neural networks)
What it looks like
You train a model to predict maths scores using inputs like:
Tutoring hours
App usage
Parental help
Class size
Past performance
The model predicts:
“This student will likely score 72/100.”
Why it’s attractive
High accuracy
Handles many variables
Feels “advanced” and modern
Useful for early warning systems
Key limitation (non-causal)
Predictive models learn patterns in existing data, not consequences of change.
If you ask:
“What if we give this student tutoring?”
The model cannot answer reliably because:
It has never seen the counterfactual (what would have happened otherwise)
It treats tutoring as just another feature, not an intervention
Core problem:
Prediction ≠ intervention.
These models answer: “What is likely to happen?”
Not: “What will happen because we act?”
4️Feature Importance & Explainable AI
(SHAP values, feature rankings)
What it looks like
You might see outputs like:
“Top predictors of maths success:
Prior maths score
App usage
Parental education
Tutoring”
Why it’s attractive
Feels interpretable
Shows “what matters most”
Useful for communication
Key limitation (non-causal)
Important predictors are not necessarily good levers.
Example:
Prior maths score is the strongest predictor → But you can’t change it directly.
Parental education predicts success → But schools can’t intervene on it.
Core problem: Feature importance ranks predictors, not causes you can act on.
5️Clustering & Segmentation
(Grouping students)
What it looks like
You cluster students into groups:
“High achievers”
“Struggling but motivated”
“Low engagement”
“High anxiety”
Why it’s attractive
Helps tailor support
Useful for personalised learning
Good for targeting resources
Key limitation (non-causal)
Clusters describe who is similar, not what will help them.
Example:
You identify a “low engagement” group
But you still don’t know:
Should you give them tutoring?
Apps?
Counselling?
Teacher changes?
Core problem: Segmentation tells you who, not what to do.
Why These Methods Fail for Decisions
All non-causal methods share one limitation:
They describe the world as it is — not the world after we intervene.
In decision-making, the real question is always:
“If we do X instead of Y, what will change?”
Non-causal techniques:
Cannot simulate “what if”
Cannot separate cause from selection
Cannot estimate true impact
Cannot support cost-benefit trade-offs
Why Causal ML Is Different
Causal ML is built specifically to answer:
“What will happen if we intervene?”
For example:
If we add weekly tutoring, what happens to scores?
If we train teachers, does that cause improvement?
If we increase app usage, does it really help?
This is why the approach:
Questionnaire + causal thinking + LLM support is a practical architecture for real-world decision-making.
“Traditional analytics can tell us who is doing well and who is struggling, but only causal analysis can tell us which actions will actually improve student outcomes.”
The intriguing question
For me, the intriguing question is as follows
If we create the dataset in step 4 (which we have to in any case)
ex
Run the survey across many classes/schools
Merge data: Questionnaire data (students, parents, teachers)
and School records (exam scores, class size, etc.)
Each student becomes one row in a table, with columns like:
extra_tutoring_hours
uses_app_frequency
teacher_trained
parent_homework_help
prior_maths_result
class_size
exam_score
Then, could we directly prompt against that dataset and get a causal analysis? without going through a causal system (ex Python coding)?
Its worth exploring!
Our course at Oxford is AI Engineering
Image source: unknown
Such a useful article! Thank you