Building a Cost-Effective Vector Embedding Sync Pipeline for Semantic Search
The expensive part of a vector sync pipeline is almost never the embedding call. It's how often you make it, and how often you didn't need to.
This is the sync pipeline behind a semantic candidate search: recruiters find people by meaning, so every candidate carries an embedding built from their profile, and that embedding has to stay current as the profile changes.
A candidate’s profile changes constantly: a new skill, a finished certification, an edited work description. Every one of those edits means the vector sitting in the database no longer matches who they actually are.
The real question isn’t whether to keep vectors fresh. It’s how you know which profiles changed, so you re-embed those and nothing else.
A two-queue architecture solves this, because marking what changed and acting on it can’t happen in the same step.
Knowing what changed
entity changes → decorator checks watched columns → sync-state queue marks candidate dirty
A decorator sits on every entity that makes up a profile, skills, education, work experience, certifications, and tells a single subscriber which columns actually matter. A skill’s proficiency level is watched. An unrelated internal field isn’t. Touch a column nobody’s watching, and nothing happens at all, no dirty mark, no queue, no call.
When a watched column does change, the subscriber doesn’t touch OpenAI or pgvector itself. It sends one message to a sync-state queue, whose only job is working out which candidate this actually affects and marking them dirty.
That queue earns its keep on the cases that aren’t obvious. Most changes are direct, a candidate edits their own work history, and the affected ID is right there. But one change to a batch can affect hundreds of candidates at once. Figuring out which hundred means a database query, and that query can’t safely run inside the subscriber’s own transaction. So the sync-state queue resolves that fan-out asynchronously, outside the transaction that triggered it, then writes the result to a small tracking table.
| Column | Purpose |
|---|---|
candidate_id |
Which candidate this row tracks |
last_updated_on |
Set whenever a watched column changes |
last_synced_on |
Set after a successful embed, null if never synced |
A candidate is dirty whenever last_updated_on is newer than last_synced_on. That single comparison is the entire signal the rest of the pipeline runs on.
So why not embed a candidate the moment it’s marked dirty?
Draining the dirty set
sync-state queue marks dirty → cron drains every 10 min → embedding queue rebuilds, embeds, upserts
Because embedding the moment a change lands means every save costs a call, and that restraint is the whole trick. So the marking and the embedding live in different queues. A cron drains the dirty set every 10 minutes, and several edits to the same candidate collapse into one rebuild, embed, and upsert into pgvector.
The trade-off is a 10-minute window where a candidate’s vector can be stale. In exchange, someone who can’t decide between “Software Engineer” and “Senior Software Engineer” and edits their title four times in five minutes still costs one embedding call, not four, because all four edits are still sitting in the same dirty row when the cron finally runs.
Worth calling out on its own: the embedding call itself goes out in sub-batches of 25 candidates per request, not one request per candidate. Fewer, larger requests keep the pipeline further from OpenAI’s per-minute rate limits, not just its per-token price, which matters most exactly when a busy sync window has the most candidates queued up at once.
That is the whole pipeline, end to end:
candidate_id, last_updated_on, last_synced_on
Making it cheap
relevant-only sync + coalescing + caching (a cheap model too) → small bill, most of it invisible
Three levers do most of the work, and none of them is the embedding model.
- Watch only what matters. The subscriber fires only on columns that actually feed the embedding text or the filter metadata. Touch a column nobody’s watching and nothing happens at all, no dirty mark, no queue, no call.
- Coalesce instead of reacting. Dirty-tracking plus the 10-minute drain means only candidates who actually changed get re-embedded, and several edits in one window collapse into a single call.
- Cache identical text. Every embedding call checks Redis first, keyed on a hash of the exact text, 1-day TTL, so identical text never reaches OpenAI twice. FNV-1a is the deliberate choice here: a non-cryptographic 64-bit hash is fast and stable, and cryptographic strength buys nothing for “same input, same key.”
Two things underneath help without being levers you tune: the vectors live on the same Postgres instance as everything else, so there’s no separate vector-database bill, and the model itself is cheap, text-embedding-3-small at 1536 dimensions, 6.5x under the large one. Neither is where the cost lived.
The coalescing lever is the one worth doing the math on. Picture the pipeline at a larger, steady-state scale, say 250,000 candidates. Each document is capped at the top 5 entries per section, so it lands around 550 tokens whether a candidate has two years of history or fifteen. A new candidate typically builds their profile in about 8 separate saves: an introduction, an education entry, one or two work entries, skills, a certification, a portfolio piece.
naive (embed on every save):
250,000 × 8 calls × 550 tokens ≈ 1.1B tokens → ~$22.00
coalesced (drained every 10 min, ~2.5 embeds/candidate):
250,000 × 2.5 × 550 tokens ≈ 343.75M tokens → ~$6.88
≈ 69% fewer embedding operations for the same onboarding burst
The dollar figure looks small either way, embedding tokens are just cheap. What actually matters is the call count when it scales: fewer calls means less exposure to rate limits and less queue throughput spent reprocessing text nobody asked to change.
Caching’s real win is the quiet one. Dirty-tracking is coarse: it knows a watched column changed, not whether the embedded text changed. Often it didn’t. Three routine edits mark a candidate dirty but rebuild to the exact text already embedded:
- a column that only feeds filter metadata, never the document
- an entry edited beyond the top 5, so it never enters the document
- a value edited, then reverted before the cron runs
Each becomes a Redis lookup, not an OpenAI call. Dirty-tracking flags what might have changed; the cache catches what didn’t.
Retries are the smaller win. A batch fails partway, the retry rebuilds identical text, and the cache spares the duplicate call. Rarer, but real money on the runs that would otherwise pay twice.
The one thing to take from this
The expensive part was never the embedding model, and never really the embedding call. It was making that call more often than the data actually changed.
Watch only what matters, coalesce it, cache it: three ways of not making a call you didn’t need. The cost problem disappears on its own, not because saving money was ever the goal, but because keeping every candidate’s vector correct and current was. Cheap is just what that looks like when you’re not guessing.
Once these vectors exist, the listing page has to search them by relevance and still come back fast. Here’s how that page stays under 300ms.
How are you handling embedding freshness at scale? I’m curious whether people coalesce or just embed on every write. Let’s talk on LinkedIn.