In a garage somewhere, someone is building a prototype. The workbench is covered in parts. A drawer of screws hangs open, a half-assembled frame sits under the lamp, and a sketch with three crossed-out versions is taped to the wall. The first attempt didn't fit. The second one broke. The third one might work, and finding out cost an afternoon and a few spare parts.
Across town, a factory line is running for a product nobody has tested. The conveyors are installed, the stations are staffed, and every morning the line stamps out a few thousand units of something that may not be what anyone wanted.
Nobody builds the factory before the prototype works. Data teams do it all the time.
A new question comes in, and the first move is to stand up a pipeline, a warehouse table and a schedule, before anyone has answered the question once. For a long time that habit was sensible. It makes much less sense now that a laptop can do the work of a small warehouse and an agent can do most of the building.
Why data teams built the factory first
The factory habit came from real constraints. A laptop couldn't scan a serious dataset without grinding to a halt. Storage was expensive, so teams summarized data early and threw away what they couldn't afford to keep. And the application databases that held the data weren't safe places to run analysis, because heavy queries slowed the application for everyone using it, which I explained in Before You Point an Agent at Your Data.
So teams built assembly lines. Pipelines moved data off the operational systems. Transformations reshaped it into standardized parts, with tables pre-joined and laid out like pre-cut pieces on a jig, so every report was assembled from the same components. Orchestration kept the stations running in order, because one missed step could spoil the whole batch.
For its time, the assembly line was the only reliable way to deliver analytics. It brought order and trust to work that would otherwise have been fragile and slow. The trouble is that it got built for every question, including the ones nobody had tried to answer yet.
What landed on the workbench
Three things changed, and all of them landed on the bench.
The power tools arrived. DuckDB is an analytical database that runs inside your own program, with no server to set up. It reads CSV, JSON and Parquet files right where they sit, uses the same column-by-column layout that makes warehouses fast, and can work through datasets larger than your memory by spilling to disk. The DuckDB team ran the full TPC-H benchmark against roughly 3 TB of data on a Framework laptop, on battery, in under an hour. Most of the questions a business asks involve far less data than that.
Storage got cheap. It's now cheaper to keep the lumber than to keep the line running. Raw data can sit in open formats like Parquet until someone needs it, and the same files work on the bench and in the factory.
The builder changed. An agent can write the queries, read the errors and try again without anyone typing each attempt. That removed most of the engineering time an experiment used to take, which makes the cost of each attempt the thing to watch. On a shared cloud platform, fifty exploratory queries can mean fifty charges or a chunk of capacity that production reports were counting on. On your own laptop, they cost next to nothing, which is the case I made in Give the Agent a Working Environment.
Put those together and the workbench can handle most of what a question needs before anyone knows whether it deserves a factory.
The workbench recipe
Most of my new data work starts the same way now. A new API shows up. Or a folder of datasets downloaded from the web. Or the pile of exports and spreadsheets that comes with starting something new. The first question is always the same: what's actually in here?
I pull a representative copy onto my laptop and let an agent look at it in DuckDB. Which fields are filled in, what a single record represents, where the types don't agree with each other, which JSON is nested three levels deep. Most of what it finds at this stage is mess, and that's the point of the bench. A wrong guess costs a few seconds.
Then it tries joining the new data to the datasets I already have. Some keys match. Some turn out to mean something different from what their names suggest. Some joins quietly duplicate rows until a count doesn't add up. The agent revises the mapping and runs it again, as many times as it takes, without anything leaving my machine.
Once the data holds together, DuckDB becomes the backend for a small FastAPI app and a quick dashboard. That step matters more than it sounds. A table of query results tells you the data exists. A dashboard on screen tells you whether it answers a question anyone cares about, and it's what you can put in front of someone else for a reaction.
Giving the agent that recipe up front keeps it focused: DuckDB for the data, FastAPI for the backend, a simple dashboard on top. It doesn't have to invent infrastructure, choose a cloud service or wire up permissions for something that may not survive the week. The whole prototype lives on the bench, and none of it runs up a bill while I find out whether it's worth anything.
The end of the bench
Every workbench session ends with the same decision.
Sometimes the answer is right there. The question needed answering once, the dashboard showed it, and the work is done. The bench was the right tool, and there's nothing to industrialize.
Sometimes the question turns out to be one people will ask every week, that a team needs to share, or that has to refresh on a schedule. That's when it goes to the factory: a cloud warehouse or lakehouse, a real pipeline, governed access. What moves is everything the bench proved, including the definitions, the joins that worked, the quality checks that caught the duplicates and the dashboard layout people actually used. The factory gets built for a product that's been tested.
What the workbench can't do
The bench has one real limit: it serves one person. There's no central place where everyone sees the same numbers, no shared definitions the whole company works from, no access controls deciding who can see what, and no schedule keeping it fresh while you're away. That's the factory's real job, and it's worth paying for once the work needs it.
A few other things are worth keeping straight:
- Local isn't free. Your machine, your agent's tokens and getting the data in the first place all cost something, and sensitive data still has to follow your organization's rules about where it can live. What's nearly free is running the next query against data that's already on the bench.
- Cloud queries aren't always billed one by one. Many platforms offer capacity pricing, free allowances or cached results. The risk is less any single query than the unpredictable cost of uncontrolled exploration.
- DuckDB isn't a small version of a cloud warehouse. It's an engine that happens to be extremely convenient for local work. Moving bench SQL into a cloud platform can take some adjustment.
Let the work earn the factory
The factory is still the right answer for work that's repetitive, shared and stable. The mistake is building it before the work proves it needs one. A few habits keep the order right:
- Defer decisions until the question stabilizes. Early answers can be rough. Modeling, pre-aggregating and scheduling before you understand the shape of the problem just locks in assumptions.
- Keep data movement to a minimum. Every hop between systems adds cost, failure points and governance work. Pull what the question needs, not everything that exists.
- Use open formats. Work done on the bench in Parquet and plain SQL moves into the factory without starting over.
- Add orchestration last. Schedule the work that has become a habit, not the work you hope will.
The signal to build is pain that keeps coming back: the same query running every week, burning compute, slowing someone's dashboard. When that happens, the factory pays for itself, because it's reducing a real, recurring cost instead of anticipating one.
Build on the bench. Let the work tell you when it's time to pour concrete.