Building a General-Purpose Agentic Learning Pipeline

Twinbook Trust & Engineering series · ~9 min read · product engineering

A note on scope: this post is about what we set out to build and why it's hard — the general-purpose problem and the principles we used to solve it. It deliberately does not disclose our model choices, prompts, or detailed internal architecture. Those are part of how Twinbook works, and we keep them under the hood. What's worth sharing is the harder, more interesting question: how do you make a learning pipeline that works on anything a learner throws at it?

The trap of building for one kind of content

Almost every study tool quietly assumes a shape. It's built for clean PDFs, or for typed notes, or for a specific textbook format, or for one subject. The demo looks great — on the content it was built for. Then a real learner shows up with a scanned slide deck, a photo of a whiteboard, a problem set, a recorded lecture, and a dense medical textbook with figures, tables, and chapter structures that don't match any template, and the tool falls apart.

We decided early that the only version of Twinbook worth building was a general-purpose one: it should turn whatever a learner actually has into structured, studyable, personalized material. Not one format. Not one subject. Not one "happy path." That's a much harder problem, and most of our engineering went into not cheating on it.

What "general-purpose" actually means here

Concretely, the pipeline has to handle:

Any input modality — typed text, PDFs, slide decks, images and scans, and video.
Any subject and any structure — a tidy textbook with "Chapter 1" headings, a messy slide deck with none, a problem set, a lecture transcript.
Any scale — a two-page handout or a two-thousand-page textbook.
Different kinds of learning — reading-style material, problem-solving material, and lectures are genuinely different and shouldn't be processed identically.

And the output has to be consistent regardless of what went in: clean structure, usable study materials, and a learning memory that personalizes review.

The principles that got us there

We didn't get to general-purpose with one clever trick. We got there by committing to a few principles and refusing to take shortcuts that would have made the demo easier and the product narrower.

1. Structure first, content second

Before generating anything, the pipeline works out the shape of the material — its sections, its chapters, its boundaries. Getting structure right is what lets everything downstream be consistent, whether the source was immaculate or chaotic. A lot of the hardest engineering lives here, because real-world documents almost never announce their own structure cleanly.

2. Don't assume the format — detect it

Instead of hardcoding "this is what a chapter heading looks like," the pipeline is built to figure out the organization of unfamiliar material. That's the difference between a tool that works on the textbook you tested and a tool that works on the one your learner actually owns. When detection is uncertain, we'd rather degrade gracefully than confidently mangle the content.

3. Respect the kind of learning

A reading assignment, a problem set, and a lecture are not the same learning task, so we don't pretend they are. The pipeline routes material into the appropriate kind of processing, and — importantly — the learner's intent is authoritative. When the product knows what kind of studying you're doing, it uses that, rather than guessing from the file type. Guessing is exactly how general-purpose tools quietly become wrong.

4. Build for the worst case, not the demo

Big, messy, real content is the test that matters. Designing for thousand-page textbooks and noisy scans — not just a clean ten-page PDF — forced the pipeline to be durable, resumable, and honest about its limits. The features that make hard content work (handling scale, recovering from partial failures, verifying results) are the same features that make easy content boringly reliable.

5. Ground everything, then remember it

The end of the pipeline isn't a pile of generated text — it's a learner's memory: the concepts they've met and what they're likely to forget, scoped privately to them. Generation is grounded in the learner's own material and tied back to sources, and the result feeds a personalization layer that makes the next study session smarter. (How that memory is kept private and under your control is its own post: What Is Learning Memory — and How Students Control It.)

6. Treat AI output as a draft to be checked

General-purpose breadth raises the stakes on correctness — the more varied the input, the more ways generation can go wrong. So AI output is validated and evaluated before it's relied on, not shipped raw. We'd rather catch an off-task generation than show it. (More in Our Approach to AI Safety in Education.)

Why an agentic pipeline

"General-purpose" and "agentic" go together. A rigid, linear pipeline can only do exactly what it was scripted to do — which is fine until it meets content that doesn't fit the script. An agentic approach lets the system make decisions about unfamiliar material: assess what it's looking at, choose how to handle it, and adapt when the first approach doesn't fit — within firm boundaries and with its output checked. That adaptivity is what lets one pipeline serve the enormous variety of real study material instead of a curated subset.

It's the same philosophy as our harness engineering post, pointed at content instead of operations: the value isn't a single model doing magic, it's a well-designed system that can act, observe, correct, and stay within guardrails.

What we're honest about

General-purpose is a direction, not a finished destination. Some content is still hard — unusual layouts, ambiguous structure, edge cases we haven't seen. When the pipeline is unsure, we aim to degrade gracefully and visibly rather than fail silently or fabricate structure. We keep expanding the range of material it handles well, and we treat each new kind of "hard" content as the next problem to solve rather than something to paper over.

That's the whole bet: build for the messy reality of how learners actually study, not for the clean case that demos well. It's harder, it took longer, and it's the only version of Twinbook worth shipping.