The one rule that shaped every file
Everything I build now runs into the same wall: the tool is 80% of what I need, the last 20% is where the money is, and the vendor will never build it because it's specific to me. So I pay $400 a month to be almost right.
Leadd started as a rule written at the top of the repo, before a single component existed:
That rule does a lot of work. It means no homemade scraper — Outscraper already talks to Google Maps better than I will. No contact database — Prospeo, Findymail and Dropcontact already have one. No email validator — ZeroBounce is $0.004 a check and better than anything I'd write.
What it leaves me is the part nobody sells: deciding what to look for, knowing when to stop, and learning from what came back. That turned out to be roughly 29,000 lines of it.
The stack, and why
| Layer | Pick | The reason it won |
|---|---|---|
| App | Next.js 16 · App Router | One repo for the console, the API and the cron surface. Turbopack keeps a 681-file project honest. |
| Orchestration | Inngest v4 | The whole thesis. Every stage is a step.run — crash mid-run and it resumes at the exact step, not the top. |
| Data | InstantDB | Live queries straight to React. The console updates while a run is executing, with no socket code. |
| Intelligence | OpenRouter | One adapter, every model. Swap the scoring model from a dropdown; no vendor SDK ever gets imported. |
| Validation | Zod v4 | Nothing from an external API reaches the database unparsed. Not once. |
| Media | Backblaze B2 | Learned the hard way. More on that below. |
sender reputation protected · bounce rate 0.4%
| Casa Verde Dental | hot | 91 | Alajuela |
| Studio Mar Pilates | hot | 88 | Escazú |
| Nómada Coworking | hot | 84 | Tamarindo |
| Clínica Rivas ORL | warm | 69 | Heredia |
| Boutique Solaz | warm | 61 | Santa Ana |
| Terraza Lunática | warm | 57 | Jacó |
The hardest problem was not scraping. It was repeating yourself.
Any tool can pull 200 dentists in Madrid. Pull them again next Tuesday and you get the same 200 dentists. Every lead tool I've used quietly hands you the same list forever and lets you feel productive about it.
The provider's own answer is a skip parameter — ask for results 200–400 instead of 0–200. I built on that first. Measured live, it returned 67–80% duplicates. The pagination doesn't hold across calls the way you'd hope.
So I threw it out and made freshness a property of the system instead of the provider. Every search is reduced to a fingerprint of (vertical, location, region, language). That fingerprint goes into a global 90-day lock — global meaning across every plan, not per-plan. If any plan searched "dentists / Alajuela" six weeks ago, no plan searches it again for another six.
The plan then walks a ladder of markets rather than paging deeper into one. Duplicate rate measured live after the change: zero.
| Approach | Duplicate rate | Why |
|---|---|---|
Provider skip pagination | 67–80% | Offsets don't hold across calls; the same rows come back. |
| Ladder-first rotation + 90-day global lock | 0% | Never asks the same question twice inside the freshness window. |
When a plan exhausts its entire fresh space, it doesn't fail and it doesn't spin. It sets a 30-day cooldown, closes the run at zero, and reschedules itself past the cooldown. A criterion is never left perpetually "due" — that was a bug I shipped once and it cost me a week of phantom runs.
Every step is idempotent, or the whole thing is a toy
This is the constraint that made the codebase what it is. It's written into the project rules in one line: every step idempotent; dedupe by placeId; re-running a flow never double-writes.
Sounds obvious. It isn't, and here's the shape of the bug that teaches you. Inngest replays a function from the top on every step boundary — code outside a step.run re-executes each time. So this:
| Broken | Fixed |
|---|---|
| const runId = id() | const runId = await step.run("init-run", () => id()) |
A bare id() generates a different id on every replay. Step 3 writes to a run that step 8 can no longer find. The run looks fine, finishes green, and silently orphans its own record. Memoising it inside a step pins it for the life of the execution. |
|
Every stage after that is written to survive being run twice: enrichment returns early if the lead already has an email; scoring returns early if it already has a score unless you explicitly force a re-score; the discovery upsert dedupes on placeId before it writes anything.
| # | Step | State | Duration | Detail |
|---|---|---|---|---|
| 01 | init-run | memoized | 2ms | runId pinned for all replays |
| 02 | resolve-model | memoized | 41ms | openrouter · settings default |
| 03 | start | done | 88ms | runs row → status running |
| 04 | discover:sync | done | 6.2s | outscraper · 118 businesses |
| — | ▲ 502 from provider | retry | — | backoff 4s · resumed at step 05 |
| 05 | persist | done | 1.1s | 94 new · 24 deduped on placeId |
| 06 | enrich:ChIJd8…d4Q | done | 820ms | prospeo → hit |
| 07 | verify:ChIJd8…d4Q | done | 96ms | zerobounce · valid |
| 08 | score:ChIJd8…d4Q | done | 1.9s | llm · 84 · tier hot |
| 09 | images:ChIJd8…d4Q | done | 2.4s | 3 photos → B2 (re-hosted) |
| … | 93 more lead chains | executing | — | concurrency limit 3 |
| 98 | route → smartlead | queued | — | dispatchable only |
| 99 | finish | queued | — | stats + telegram notify |
Three providers, one waterfall, first hit wins
A business listing gives you a name, a website and a phone. It rarely gives you an address a human reads. So enrichment is a waterfall: Prospeo → Findymail → Dropcontact. First one that returns an email wins, and the lead records which provider found it.
That last part is the whole point. Storing enrichmentSource means I can ask, months later, which provider actually earns its invoice on my verticals in my countries. That's not a question any of the three will answer for you.
Every adapter in the repo — all twenty-plus of them — follows the same contract, because they were painful in the same ways:
- Keyless-tolerant. No API key configured means a logged no-op, not a crash. I can run the whole pipeline with half the integrations dark. Flip
INTEGRATIONS_STRICT=trueand missing keys become hard failures instead. - Exponential backoff on 429 and 5xx, in one shared HTTP layer. Written once, not twenty times, not slightly differently each time.
- Normalised return shapes. Three email-finders with three different response schemas become one shape at the boundary. Nothing downstream knows which vendor it came from.
- Zod at the edge. External JSON is parsed before it's trusted. No
anyreaches a persisted path.
The gate that says no
This is the single most valuable line in the codebase and it's a Set with two strings in it.
valid and unknown are ever dispatchable. Never invalid. Never risky. And never, ever catch_all.Catch-all is the trap. A catch-all domain accepts every address, so it looks like a pass. It's actually unverifiable — you're guessing, and the mailbox may not exist. Send enough of them and your sender reputation degrades quietly over weeks. By the time you notice deliverability dropping, the damage is already priced into your domain.
In the mock week above that gate withheld 272 addresses. Those are 272 sends I'd otherwise have made, and the reason the bounce rate stays under half a percent. The most profitable thing this system does is refuse to send email.
Rank, don't park
Scoring is the one place an LLM belongs in this pipeline. The model gets the whole picture — name, bio, category, rating, review count, socials, website state, whether the email verified, how many times we've seen this business before — and returns four things: a score, a tier, a whyNow, and an angle.
The last two matter more than the number. whyNow is the reason this business is worth contacting this week. angle is the opening line's actual argument. A score sorts a list; those two write the email.
All model calls go through one file. Never a vendor SDK, always OpenRouter, and the model is picked from a dropdown in the UI. Changing the scoring model is a settings change, not a deployment.
Failure is a design decision
Here's the behaviour I'm proudest of. When the LLM is unavailable — and it has been, mid-run, out of credit, with a 402 — scoring returns null and the pipeline keeps going.
The temptation is to throw. It feels correct: the step failed, fail the run. It's wrong. Enrichment and verification run on different, already-paid-for providers, and they're downstream. Aborting on an LLM outage throws away the paid contact data to protect a number I can backfill for free tomorrow. So the gate ranks, it doesn't park. An unscored lead flows on, and a backfill scores it later.
| Name | Score | Tier | Channels | City | Next action | Rating | ||
|---|---|---|---|---|---|---|---|---|
| ▣ | Casa Verde Dental | 91 |
hot | valid | ✉ ☏ ⌘ ⓘ | Alajuela | dispatch · seq 2 | 4.8 |
| ▣ | Studio Mar Pilates | 88 |
hot | valid | ✉ ⌘ ⓘ | Escazú | dispatch · seq 1 | 4.9 |
| ▢ | Nómada Coworking | 84 |
hot | catch_all | ☏ ⌘ | Tamarindo | held · unverifiable | 4.7 |
| ▢ | Clínica Rivas ORL | 69 |
warm | valid | ✉ ☏ ⓘ | Heredia | draft reply | 4.4 |
| ▢ | Boutique Solaz | 61 |
warm | unknown | ✉ ⌘ | Santa Ana | dispatch · seq 1 | 4.2 |
| ▢ | Gimnasio Atlas | — |
unscored | valid | ✉ ☏ | Cartago | backfill queued | 4.1 |
Handing off to the sequencer
Qualified leads go to Smartlead, grouped into a campaign named after the query that found them. That's it — I'm not rebuilding an email sender, and warming domains is somebody else's full-time job.
What I did build is everything around the handoff: the offer matrix, the segment taxonomy, and a message studio that composes DMs and emails from templates against the lead's angle and whyNow. The sequencer sends. The engine decides what's worth sending and what it should say.
The part that makes it a machine instead of a script
Stages 01–06 are a pipeline. Plenty of people have built one. Stage 07 is the reason this is a different category of thing.
Every plan carries a rolling quality average — an EWMA over its runs, α = 0.5, so it's responsive to the most recent result rather than anchored to ancient history. Before each run the plan reads that number against a bar of 55 and makes one of two decisions:
| Rolling quality | Decision | What actually happens |
|---|---|---|
| ≥ 55 | KEEP | The vertical is working. Hold it, take its top-ranked city that isn't inside the 90-day global lock. |
| < 55 | PIVOT | The vertical is weak. Advance to the next vertical entirely, and take a fresh market there. |
| all mined out | COOLDOWN | Whole fresh space exhausted. 30-day sleep, run closes at zero, schedule advances. |
The plan corrects its own aim. Nobody logs in on Monday to notice that dentists in Costa Rica have stopped converting — the rolling average notices, and next Tuesday it's running physiotherapists instead.
And because the forecast is computed by the same function the runtime uses, the console can show you the next eight runs before they happen. Not an estimate. The actual picks.
advance city only
217 fingerprints locked · 0% dup
| Date | Vertical | Market | Decision | Why |
|---|---|---|---|---|
| Aug 11 | pilates studios | Escazú · CR | keep | quality above bar |
| Aug 18 | pilates studios | Santa Ana · CR | keep | next untried city |
| Aug 25 | pilates studios | Heredia · CR | keep | next untried city |
| Sep 01 | pilates studios | Valencia · ES | keep | CR cities mined out → region roll |
| Sep 08 | physiotherapy | Málaga · ES | pivot | projected quality < 55 |
| Sep 15 | physiotherapy | Sevilla · ES | pivot | continue new vertical |
| Sep 22 | physiotherapy | Zaragoza · ES | pivot | continue new vertical |
| Sep 29 | dental clinics | Alajuela · CR | pivot | 90d lock expires · re-eligible |
Then I gave it a second brain that reads the world
Stage 07 optimises inside a space I defined. It gets better at my verticals in my markets. It will never tell me the vertical is dying, or that a category I've never touched is about to spike.
So the Oracle is a separate spine: sensors → signals → opportunities. Sensors run on a schedule and normalise wildly different sources into one signal shape — Google Trends, GDELT news, Polymarket prediction markets, Reddit, Wikipedia pageviews, CoinGecko, X. Each source carries a trust weight. Signals get a cheap deterministic salience score first — momentum log-scaled, weighted by kind, with a geo-relevance bonus — and only the survivors are worth an LLM call.
Opportunities land in four tiers, at two altitudes: tactical, which writes back into the lead engine, and strategic, which is the "what business should exist" question.
| Tier | Question it answers | Wired action |
|---|---|---|
| business | Something is moving inside a vertical I already work. | boost_market |
| niche | An adjacent category is heating up. | add_seed · new_niche |
| disruption | Something structural is about to change this market. | outreach_angle |
| event | A dated thing is coming that creates a window. | prepare · watch |
Two things keep it from being an astrology generator. First, every opportunity carries its receipts — the exact signal and topic ids it was built from, plus one deterministic line of what the data actually shows. Nothing gets asserted without a citation you can click.
Second, it learns my taste. I mark calls gold, good, wrong or irrelevant, and those verdicts tune weights along four dimensions — signal kind, tier, altitude, and source. Mark enough Polymarket-derived disruption calls as irrelevant and the Oracle stops leading with them. It's a preference model with a human in the loop, not a feed.
| ◈ | cryotherapy | breakout | 94 |
| ◈ | contrast therapy | +840% | 88 |
| ◈ | card fee ruling | prediction | 81 |
| ◈ | padel courts ES | rising | 67 |
| ◈ | sauna clubs | geo_rise | 54 |
| ◈ | dental CR | −18% | 41 |
kind breakout 1.24× · prediction 0.71×
source trends 1.18× · x 0.64×
tier niche 1.31×
| ● | google-trends | 6h | 1.0 |
| ● | gdelt | 1h | 0.8 |
| ● | polymarket | 12h | 0.9 |
| ● | 6h | 0.6 | |
| ● | wikipedia | 24h | 0.7 |
| ● | coingecko | 1h | 0.5 |
| ● | x | rate-limited | 0.6 |
I refused to build another admin panel
A machine that runs itself still needs a cockpit, and I had a strong opinion about what it should feel like: the Nostromo, not a SaaS dashboard. Phosphor green on black, monospace numerals, scanlines, hairline grids, glowing readouts. Three themes ship — Phosphor CRT (default), Industrial, and Prometheus — all driven by CSS custom properties, so the entire console re-skins from one token block.
This is not decoration. Density is the feature. A dense readout means one glance answers "is anything wrong", and the aesthetic gave me permission to put six numbers where a normal dashboard puts one.
The console covers 26 desktop routes and a separate 10-route mobile build, plus a command palette, a chat terminal wired to the same model layer, push notifications, and a Telegram bot for when I'm not at a desk.
| Campaign | State | Spend | Ads |
|---|---|---|---|
| RT · site visitors 30d | paused | $612 | 14 |
| LAL 1% · purchasers | paused | $488 | 11 |
| Broad · CR wellness | paused | $377 | 9 |
| 59 more | paused | $1,480 | 168 |
How it fits together
Two loops. The inner loop is the pipeline: discover, persist, enrich, verify, score, route. The outer loop is what makes it a machine — outcomes feed the rolling quality average, which changes what the next run looks for. The Oracle sits outside both, watching the world and writing new seeds into the plan set.
Outcomes
These are engineering outcomes, measured against real providers and a real database — not projections, and not customer metrics. Where a number describes my own operation I've said so.
| Before | After | How it was verified |
|---|---|---|
| 67–80% duplicate leads per re-run using provider pagination | 0% duplicates | Live runs against Outscraper after switching to ladder-first rotation with a global 90-day fingerprint lock. |
| Spend visible on 1 of 3 paid endpoints; tracked $0.64 vs actual $9.37 | Itemised cost on every endpoint | Provider balance API reconciled against the internal ledger; the budget cap now reads a number that exists. |
| A failed run restarted from the top, re-paying for completed work | Resumes at the failed step | Inngest step boundaries + idempotent writes; verified by killing runs mid-flight and watching them resume. |
| Unverifiable addresses reaching the sequencer | catch_all / risky / invalid never dispatch | A hard gate at the routing boundary, with unit tests asserting each status. |
| Lead photos silently going blank after a few days | Re-hosted on first sight | Root cause was expiring Google Maps URLs, not a schema bug. Confirmed in production logs, then backfilled live with zero upload failures. |
| An LLM outage aborting a run and wasting paid contact data | Degrades, doesn't abort | Scoring returns null on provider failure; enrich and verify complete; a backfill scores later. |
Benefits
- Unit cost you can actually see. Every paid call reports its itemised cost into one ledger, so cost-per-contactable-lead is a number the console displays rather than something I reverse-engineer from three invoices at month end.
- Attention only where judgment is needed. The pipeline runs on a schedule and lands qualified leads in a queue. I open the console to decide, not to operate.
- Provider independence. Twenty-plus adapters behind one normalised contract. If Prospeo's hit rate drops on my verticals, I see it in the data — because I store which provider found each email — and swapping the order is a config change.
- The sender reputation stays intact. Refusing to send is a first-class outcome. That single gate is what keeps the whole channel alive for the next campaign.
- Failures are cheap. Durable steps mean an outage costs seconds, not a re-run — and re-runs are what actually burn provider credit.
- It compounds. Every run produces an outcome, every outcome moves a rolling average, every rolling average changes the next run's aim. The system is better in week eight than week one without me editing anything.
Why the lead-tool category is in trouble
Not because of this project. Because of what this project cost to build.
Look at what Leadd actually is: a thin orchestration layer over commodity APIs, plus my specific judgment about who's worth contacting and when to stop looking. The commodity half is rented and interchangeable. The judgment half is 29,000 lines that took three weeks — because the tedious parts, the adapter boilerplate, the schema wiring, the test scaffolding, no longer take the time they used to.
Three shifts stack here, and each one on its own is survivable:
| Shift | What it removes |
|---|---|
| Every capability is an API | You no longer need to own a contact database, a scraper, or a validator to have one. The vendor's data advantage is available to their competitor for four-tenths of a cent. |
| Durable execution is a primitive | Reliable long-running orchestration used to be the hard engineering that justified the seat price. It's now a library and a decorator. |
| Assembly cost collapsed | The integration work that made "just build it" unrealistic for a one-person operation is now days, not quarters. |
What survives that is not features. It's the two things a vendor structurally cannot sell you:
- Judgment specific to your business. My scoring prompt encodes what a good lead looks like for me. A generic tool has to average across every customer, and the average is worthless to all of them.
- A feedback loop that closes on your own outcomes. A vendor can't run my evolution loop, because they don't get to see which of my replies turned into money. That loop is where the compounding lives, and it only closes inside the operation that owns both ends.
So the disruption isn't "AI writes your outreach." Everyone shipped that. It's that the vertical SaaS layer between the APIs and the operator is getting squeezed to nothing — because the operator can now build the layer themselves in the time it takes to evaluate three vendors, and the version they build knows things the vendor never will.
The obvious counterargument is maintenance: I now own twenty API integrations that will break. That's real, and it's the honest cost. But it's also what the $400/month was buying, and the difference is that when a provider changes its response shape I fix it in an afternoon rather than filing a ticket and waiting for a quarter.
Where it stands
The engine is functional, hardened and feature-complete. The console covers 36 routes. The Oracle spine is live with seven sensors and a working operator-feedback loop. Paid ads reporting reads real account data end to end.
Open work, in the order it matters:
- Webhook authentication. Two inbound webhooks currently have no auth. They need a shared secret rather than a session check, because one of them is called by a third-party dashboard. This is the top of the list and it's the kind of thing an audit finds precisely because you built the audit.
- Write-back on ads. Reporting is read-only today. Budget and state changes are next, then attribution back to the lead engine.
- Composable discovery systems. Chained multi-step pipelines — location → posts → users → enrich → link-crawl — as typed, composable steps rather than one-shot flows.
- Production deployment with real auth. The engine runs locally against real providers; the public deployment is gated behind finishing the auth story properly.