Since the last issue I…
went to Austin for SXSW EDU;
came back home to a bank run;
announced Reach IV and Reach’s first Founders’ Fund;
went to NYC for an AI edtech demo showcase; and am now
making my way to ASU GSV (if you’re there, my colleagues and I will be onstage across five sessions. come say hi!)
I find airplanes to be one of the most conducive environments to writing. Maybe it’s the ambience. Maybe it’s the fact that I’m too cheap to pay for wifi so I force myself to write offline without the distraction of having a million tabs open. My dispatch on the flight back from NYC to SFO…
Benjamin Bloom is back. It’s remarkable how often I come across references today to his 2 Sigma study, which found that personal tutoring had the greatest effect on student learning. Published in 1984, that paper was widely cited as the research basis for dozens of “adaptive” and “personalized” learning software in the 2010s. Now it’s resurfaced as the Holy Grail that’s now within reach, thanks to LLMs and AI.
(It’s worth pointing out that Matt Barnum of Chalkbeat has called into question the small sample size in Bloom’s study and how relevant a nearly 40-year-old study is today.)
Today there are many fledgling efforts to create AI tutors (though I think ChatGPT and GPT-4 surpasses most of them). And it seems like every edtech company is working on adding an AI tutor as a feature — and why not? It’s become a low hanging fruit, a commodity of sorts.
Earlier this week, Reach Capital and Union Square Ventures co-hosted an AI edtech showcase featuring Quizlet, Elicit, Pressto, Amira Learning, Khan Academy and an interview with Duolingo’s CTO Severin Hacker. The highlight was seeing Sal Khan give a demo of Khanmigo, the AI assistant developed through their partnership with Open AI. They had several months’ early access to GPT-4 and the head start showed: Khanmigo is a AI tutor fine-tuned with complex prompts so that it doesn’t give answers but instead responds with questions and conversational dialogue to help students arrive at the answers themselves. It mimics Socratic pedagogy as well as one could reasonably expect from a machine.
Beyond this, the team has also been working on a suite of teacher tools for lesson planning, rubrics, question hooks for lessons and other instructional support features. There is a writing feedback tool in the works that will ask students for citations and show their work.
The breadth is pretty remarkable. It should also give pause for the many developers working on similar tools. I have seen startups pitch each feature that Khan is working on as companies.
Every investor is concerned about technical defensibility in AI for good reason, because every few months OpenAI (and soon other LLM developers) will release updates that will render a slew of tools obsolete. My favorite non-Reach edtech VC likened the dynamic between OpenAI and startup founders to Francisco Goya’s painting “Saturn Devouring His Son.”
Take Jasper, the content marketing assistant built on GPT, for instance. The Information wrote about the surprise and concern expressed by Jasper when ChatGPT was launched. Jasper was an OpenAI partner, paying handsome sums in API call fees, but is now wrestling with the challenge: “Who would pay $80 a month for Jasper when they could get ChatGPT for free?”
I’m coming around to the fact that technical defensibility isn’t a disqualifier. Most successful tech tools are not airtight defensible from a technology standpoint, and in education, better tech doesn’t always win. In fact, it can backfire. I am reminded of a comment from the former CEO of Knewton, which was building a supposedly advanced adaptive learning engine that struggled to get uptake from schools and publishers (and eventually sold for pennies on the dollar): “Knewton is a Ferrari, but we’re in a Kia market. Ferraris require more maintenance. It’s more complicated to use Knewton.”
What wins? Execution, implementation and go-to-market know-how. Unique (unfair) insights into the idiosyncrasies of our fragmented education systems and how to operate and sell within it. How do you get something in front of teachers and students at scale, and get schools and parents to buy it? How do you bake in rich, complex pedagogical frameworks into elegant products and user experiences? Back to basics, y’know.
Who is in a better position to address some of these questions than teachers themselves? Playlab.ai, which I learned about at the event, has been running AI hackathons for teachers to build their own AI tools. I saw efforts to prompt GPT with project-based learning rubrics so that it gave scaffolded lesson plans and assignments aligned to that pedagogy.
Let the people themselves build the things they need. Democratizing the power to create is perhaps the most exciting thing about our current AI age.
This Month’s Story Assignment
I miss journalism every now and then. I don’t run a newsroom any more, but if I did here is a feature I would assign.
For all the hand-wringing around how AI will change writing, there is surprisingly little info on the current state of writing proficiency. The most recent writing benchmark we have is a dozen years old. Results from NAEP writing tests in 2011 found that only 27% of eighth- and twelfth-graders scored “Proficient” or above. There was another test given in 2017 but it was invalidated because it was given on faulty devices.
The next scheduled test? 2030.
The world will look a lot different then! Is NWEA really going to wait until 2030 to do its writing assessment? What does it mean to write in the age of AI, and how will we assess proficiency?
One Last (Short) Thing to Read
Dan Meyer, a former math teacher, researcher and current Director of Research at Desmos, is one of my favorite edtech writers.
You should read and subscribe.