Back to Blog
Applied Vision Research · Cost Optimization · AI Architecture

Build Your Own Vision Model. Save Costs.

13 tasks are free. This model does the 6 that actually need judgment — the 6 strictly-vision-LLM tasks out of 20 a frontier model like Gemini covers, just 30.0% of the total. Own the rest outright. This is the complete build plan: architecture, data, training, API, deployment, cost, latency, reliability, security, and the decision framework.

ArchitectureCost AnalysisTrainingDeploymentSecurityDecision Framework
Published August 19, 202630 min readSpecification Dossier POC-VLM-001 · Rev 09

Sheet 01 · The Trap

Why Founders, CEOs, and Developers Keep Paying for Math

Nobody Told You Where the Line Is — Because It Isn't in Their Interest To

Frontier vendors' own documentation markets breadth, not fit. Google's Gemini docs say the model can "reduce the need to use specialized ML models" — true, and exactly the message that keeps every call, even the free ones, routed through a paid endpoint. No vendor publishes the opposite guidance: "you don't need us for this." That guidance would cost them revenue, so it doesn't exist in their docs. It exists in this one.

This system covers 30% of what a frontier vision LLM like Gemini can do — and that 30% is not a limitation, it's the exact slice this pipeline gets billed for that actually needs judgment. The other 70% — OCR, video, dialogue, multi-image reasoning, translation, and more — covers real use cases, just not this one. Knowing that difference is the entire wake-up call.

How the Trap Actually Works

Every quickstart doc, every demo, every "get started in 5 minutes" guide routes a single image through the same general endpoint, whether the task is deciding if a photo is worth keeping or checking if a file is corrupted. Each individual call feels trivial — a fraction of a cent, a few hundred milliseconds. Nobody stops to count that 13 of the 19 tasks in a typical photo pipeline never needed that call at all. The cost doesn't arrive as one bad decision; it accretes silently, call by call, until it's a real line item nobody audited.

Why This Specifically Catches Founders and CEOs, Not Just Engineers

Technical leaders read "AI-powered" as a single undifferentiated capability rather than a stack of very different tasks with very different costs. A CEO approving a vision-AI line item sees one vendor invoice, not the 13-task breakdown that would show most of it is math a spreadsheet could do. The fix isn't hiring more engineers — it's asking one question before any image touches a paid model: does the correct answer here have a fixed, known format? If yes, it never needed to be "AI" in the first place.

What Waking Up Looks Like

Every free task in the pipeline runs today, for free, on commodity hardware, with a one-line justification for why a vision LLM was never required. Read it once and the line stops being invisible.

Sheet 02 · Objective & Scope

What the System Needs to Do, and Why a General Endpoint Doesn't Fit It

The Bet This Dossier Makes

A small number of frontier labs do not get to decide what your product costs, how fast it moves, or where your users' photos end up. Every API call you send them for a task a small model can learn is a piece of that decision handed away for free. Own the judgment. Rent nothing you don't have to.

The task in question is narrow, repeatable, and judgment-based: given a photograph and the context it was taken in, decide how visually and narratively significant it is, and produce short, grounded language describing it. That is the entire job.

A narrow, well-defined task is exactly the case where a small, purpose-trained model outperforms a large general one on the metric that actually matters — consistent judgment against a fixed rubric — while costing a small fraction to run at volume.

InputSingle image + context tag
OutputStructured JSON
Model classSmall VLM (< 1B params)
OwnershipFully self-trained
DependencyNo third-party inference API
StageProof of concept

Explicitly Out of Scope for This Phase

General-purpose visual chat, open-vocabulary object localization, identity recognition, and video understanding. Those are extensions once the core judgment task is proven.

Sheet 03 · Glossary

Terms Used Throughout This Dossier

Not knowing these terms yet is exactly why frontier labs can charge what they charge. Ten minutes with this page and the gap closes for good.

TermMeaning
VLMVision-language model — takes an image (and usually text) as input, produces text as output.
Forward passOne complete run of input through the model to produce output. Cost and latency scale with the number of forward passes.
EmbeddingA fixed-length numeric vector representing an image's content, used for similarity comparison. Not readable by a text-only model.
QuantizationCompressing model weights (e.g. 16-bit → 4-bit) to cut memory and speed up inference, at a small accuracy cost.
TokenThe unit an LLM processes text in. API pricing is almost always per input/output token.
DistillationTraining a small model to reproduce a larger, more expensive model's outputs — used to bootstrap labeled data cheaply.
Fine-tuningContinuing training on an existing pretrained model's weights for a new task. Faster than training from scratch; less architectural ownership.
LLM-as-judgeUsing a language model to score or rank other outputs against a rubric.
RubricThe fixed criteria a model (or human) uses to score consistently across many items.
Cold startThe delay before a scaled-to-zero GPU serves its first request after sitting idle.
Throughput vs. latencyThroughput = requests handled per second in aggregate. Latency = time for one request to complete. Optimizing one can hurt the other.
Batch inferenceProcessing many images in one queued request for efficiency — trades per-item latency for total throughput and lower cost.

How to Read This Dossier Depending on Who You Are

First-Time Builder

Start here, then Sheets 02, 09, and 29. You almost certainly don't need to read the training or architecture sheets to make your first decision.

Freelance Developer

Jump to Sheets 29 and 26 first — your volume is probably too low for a full build to pay off. Then Sheet 28 for the honest comparison.

Expert / Pushing Limits

Sheets 24, 21, 22, 23, and 26 cover the operational depth. The rest of the dossier is the system those sheets assume.

Sheet 04 · Design Reasoning

Working Back from the Task to the Smallest System That Can Do It

Break the required capability into what it actually consists of, and each piece points to a different, buildable component — rather than one large model asked to do everything at once.

Decomposing the Capability

  • Technical quality — blur, exposure, framing. Deterministic signal processing. No learned model required.
  • Redundancy / similarity — embedding distance between images. A small pretrained encoder, not a language model.
  • Contextual significance — the one genuinely open-ended judgment call. The only piece that needs a trained reasoning model.

Why Train Rather Than Call an Existing Endpoint

An external multimodal endpoint bills per image, exposes every input to a third party, and offers no way to make the underlying judgment consistent with a specific rubric beyond prompt text. A small model trained directly on labeled examples of the target judgment converges on that rubric structurally — it is what the weights encode, not what a prompt hopes to steer at inference time.

Why Train Rather Than Only Fine-Tune an Open Checkpoint

Fine-tuning an existing open model is the faster, pragmatic default for production. This proof of concept is deliberately scoped to a full independent build instead, so every component and every architectural choice is understood and owned outright rather than inherited.

Sheet 05 · System Architecture

Three Components, Each Doing One Job

Image
Vision Encoder

~85M params

Projection

pixel-shuffle + linear

Language Decoder

~135–360M params

Output

structured JSON

1

Vision Encoder

A standard Vision Transformer, trained to represent image content as a sequence of patch embeddings. Consumes the raw photo; produces no language, only structured visual features.

2

Modality Projection

A compact bridge — a spatial compression step followed by a linear layer — that reshapes visual features into the same embedding space the language decoder reads natively. This is also why an external LLM-as-judge can't consume this model's embeddings directly.

3

Language Decoder

An autoregressive transformer decoder that reads the projected visual tokens alongside a short context instruction and generates the structured output as text.

Sheet 06 · Extracted Information

Exactly What the Model Reads Out of a Photo

Every field below is produced by a single forward pass through the model for one image and one context tag. Nothing here is identity recognition — people are counted, never identified.

FieldTypeWhat It Captures
scorefloat, 0–1Overall significance / curation-worthiness
context_alignmentfloat, 0–1How well the image matches the supplied context tag
captionstringShort, factual description of what's in the frame
rationalestringWhy the score landed where it did, grounded in context
tagsstring[]Notable subjects and keywords detected in the scene
people_countintegerNumber of people present — count only, no identity
narrative_fragmentstringA short narrative-style sentence for downstream editorial use

What Is Deliberately Not Extracted

Facial identity, demographic inference, exact geolocation, and any attribute not directly observable in the frame. Technical quality (blur / exposure) and near-duplicate detection are computed upstream by classical methods and are not part of this model's output.

Sheet 07 · Coverage vs. Frontier

30.0% of What a Frontier Vision LLM Can Do — On Purpose

This Was Never Trying to Be Gemini

A frontier model like Gemini 3.1 Pro does far more than curate photos — OCR, document understanding, video, dialogue, translation, object localization, and more. Against a 20-capability baseline representative of that surface area, this system implements exactly 6. That's not a shortfall. It's the whole point: pay for the 30.0% this pipeline actually uses, not the other 70.0% sitting idle in every API call.

The Calculation

Baseline capability categories20
Covered by this model6
Coverage6 ÷ 20 = 30.0%
What This System CoversAPI Field
Contextual significance / curation-worthiness scoringscore field
Context-to-image alignment scoringcontext_alignment field
General image captioningcaption field
Judgment rationale generationrationale field
Open-vocabulary scene / subject taggingtags field
Narrative-style text generation grounded in an imagenarrative_fragment field

Sheet 08 · The Other 70%

What Every Uncovered Capability Would Take to Build

The 14 rows below are what separate this system from a general-purpose frontier vision LLM. None of them are needed for this pipeline — but the honest cost of closing each gap is listed anyway, so the scope decision is an informed one, not an assumed one.

General Vision-LLM CapabilityWhat's Needed to Build It
Open-ended free-form visual question answeringFree-form Q&A training data + a decoder not constrained to fixed JSON fields
Object detection & bounding-box localizationA localization head + box-annotated training data (COCO-style detection, not just captions)
OCR / text extraction from imagesDedicated text-recognition training data and typically higher input resolution
Document, chart & table understandingA layout-aware encoder + a structured-document training corpus
Spatial & depth reasoningDepth- or geometry-annotated training data
Multi-image comparison / reasoning across a setCross-image attention — this architecture takes one image per forward pass — plus interleaved multi-image training data
Video understanding (temporal reasoning)A temporal/frame-sequence encoder and a video training corpus — explicitly out of scope
Multi-turn image-grounded dialogueConversation-history handling and dialogue training data — explicitly out of scope
Instruction-following image-edit descriptionPaired before/after edit-instruction training data
Aesthetic / quality critique as a standalone outputDedicated aesthetic-critique labels — currently folded informally into rationale
Content moderation / safety classificationA separately trained, dedicated safety classifier — tags/caption are a signal, not a substitute
Facial identity recognitionIdentity-linked training data — excluded on purpose for privacy, not a capability gap
In-image text translationOCR as a prerequisite, plus translation training pairs
Open-set object counting (beyond people)Counting-specific training data across arbitrary categories

What Closing the Gap Would Actually Cost

Every row above adds its own training data, and several — multi-image reasoning, video, dialogue — add architectural surface area this design deliberately avoids. Building all 20 would mean re-deriving a general-purpose frontier model from scratch: the exact cost and dependency profile this dossier argues against. The 30.0% covered is the 6 capabilities this pipeline actually bills for — nothing rented, nothing idle.

Sheet 09 · Commercial Use Cases

What the Model Fields Enable in a Real Product

Every use case below is built directly from score, context_alignment, caption, rationale, tags, people_count, and narrative_fragment — no additional model output required.

Automated culling / curation

score

Highlight reel generation

score

Accessibility alt-text

caption

Auto-assembled photo stories

narrative_fragment

Search & filtering

tags

Editorial audit trail

rationale

Personalized recaps

score + tags

Photo-book auto-layout

score

Client proofing tools

score + rationale

Listing quality scoring

context_alignment

Social share recommendation

score

Context-mismatch flagging

tags + context_alignment

Common Thread

Every one of these is a judgment or ranking task, not a retrieval or transformation task. That distinction is what determines whether a use case belongs here or in the free tier.

Sheet 10 · Task Split

13 Tasks Are Free. 6 Need the Vision LLM.

Don't Estimate What You Can Count

Every task this pipeline needs is enumerated here — nothing is rounded. 13 of the 19 total tasks are solved for free, today, on commodity hardware. The other 6 require language-grounded visual judgment, and those 6 are exactly the strictly-vision-LLM tasks counted against Gemini's own capability set.

The Count

Tasks requiring the vision LLM6
Free, deterministic tasks13
Total distinct tasks in the pipeline19
Requires the Vision LLM — TaskWhy Nothing Else Can Produce It
scoreJudgment against a rubric, not a measurement
context_alignmentRelates image content to an arbitrary text label
captionOpen-ended language generation grounded in the image
rationaleExplaining a judgment in language
tagsOpen-vocabulary content description
narrative_fragmentFree-form narrative language generation

Sheet 11 · The Free Tier

The 13 Tasks That Never Touch the Model

Each one has a fixed, known answer format, so it's solved once as an algorithm and stays solved regardless of scale.

Free, No Vision LLM — TaskHow
Blur / sharpness detectionLaplacian variance — pure math
Exposure / contrast scoringHistogram analysis — pure math
Duplicate / near-duplicate detectionPerceptual hashing — deterministic
Resize, crop, rotate, convertStandard image libraries
EXIF metadata extractionDirect file parsing
Thumbnail generationStandard resizing
Color palette extractionPixel histogram — pure math
Red-eye reduction, white balanceClassical CV filters
WatermarkingDirect pixel compositing
Corrupt-file validationFile header checks
Panorama stitchingClassical feature matching, no learned model
Basic face detection & countingLightweight open-source detectors, not an LLM
Fixed-category object taggingSmall pretrained classifiers, not generative

What This Buys You

Everything above runs entirely offline, at effectively zero marginal cost, on commodity hardware — no API call, no GPU inference server, no per-image billing of any kind. Every photo touches this tier; only the curation-worthy subset ever reaches the paid layer.

Sheet 13 · The Cost of Not Waking Up

$59,280 a Month — The Exact Price of Skipping the Free Tier

This Is What the Trap Actually Costs

At 3,000,000 images/month, routing all 13 free tasks through a frontier vision LLM instead of the free tier costs approximately $59,280/month — 6.2× more than the entire cost of the judgment task this pipeline actually needs, for work a Laplacian variance calculation and a file-header check already do for nothing.

The Calculation

Free tasks routed through the vision LLM13
Input tokens/image (image + instruction)700 × 13 = 9,100
Output tokens/image (short answer per task)10 × 13 = 130
Reference volume3,000,000 images/month
Input cost27,300,000,000 tokens × $2/M = $54,600
Output cost390,000,000 tokens × $12/M = $4,680
Total monthly waste, list price$59,280
Total monthly waste, batch discount$29,640

Annualized, list price

$711,360

Annualized, batch discount

$355,680

vs. the judgment task that's actually necessary

6.2×

Sheet 14 · Worldwide Scale

$0.8B–$6.5B a Year, Worldwide — A Fermi Estimate, Not a Fact

This Sheet Is Different From Every Other Number in This Dossier

Every other calculation here counts something enumerable — fields, tasks, tokens, dollars at a stated volume. A worldwide figure can't be counted; it has to be estimated from published market sizes that disagree with each other by 5× or more, plus at least one real judgment call with no data behind it at all. Shown here anyway, labeled for exactly what it is.

InputValueConfidence
Global annual spend on vision-capable LLM API calls$3B–$15B/yrSource data disagrees 5×+
Share following the naive one-call-per-task pattern30%–50%Not sourced — stated judgment call
Waste share of naively-routed spend86.1%Exact — computed from Sheet 13 figures

The Calculation

Low estimate$3B × 30% × 86.1% = $0.77B/yr
High estimate$15B × 50% × 86.1% = $6.45B/yr

Sheet 15 · Hybrid Scoring

A Better Score Is a Composite, Not a Single Signal

The VLM score alone can't see the batch, so it can't penalize redundancy. Embedding distance alone can't see content, so it can't tell you which duplicate to keep. The most accurate score fuses both, gated by objective technical quality, with no additional LLM calls.

final_score = quality_gate × (α·vlm_score + β·context_alignment) × distinctiveness_factor
1

Quality Gate — Hard Multiplier, 0–1

Blur / exposure / composition from the classical CV layer. Not a weight — a cutoff. A technically ruined photo shouldn't outrank a clean one regardless of content: 0 if blur_score < threshold, else a soft ramp toward 1.0.

2

Base Score — VLM Output, Unmodified

α·vlm_score + β·context_alignment, straight from the model's fields. Start at α=0.7, β=0.3 — content significance dominates, context-fit is a secondary correction — and tune against the held-out reference set.

3

Distinctiveness Factor — Embeddings, Batch-Level

Cluster the batch by embedding cosine distance (~0.9 similarity = same cluster). Within each cluster, rank by base score and decay the rest: decay_rate^(rank_within_cluster). Singletons keep a factor of 1.0.

Sheet 16 · LLM-as-Judge

Where an LLM-as-Judge Still Earns Its Keep — and Where It Doesn't

An external LLM-as-judge has no trained alignment to this model's embedding space. A text judge can only ever consume the structured JSON output, never embeddings and never raw images.

Call Ordering That Maximizes Use of the Vision Model

Image → VLM (score + caption + embedding) → hybrid fuse → optional LLM-judge batch rerank → final ranked output

What Stays on the Self-Hosted Model

Every image is scored once by the purpose-built VLM — the cheap, self-hosted, per-image pass. Redundancy is resolved by embedding distance, never by a judge call. Together these two free/cheap layers already cover both per-image judgment and batch-level duplication.

The One Job Left for an LLM-as-Judge, If Kept at All

Batch-level reasoning a single forward pass structurally cannot do: enforcing narrative diversity across a full curated set, cross-checking captions for redundancy the embedding threshold missed. Fed the already-scored, already-deduplicated structured JSON for one batch — not per image — this is a single, small, infrequent call instead of one LLM call per photo.

Net Effect

Token-billed LLM calls drop from once-per-image to at most once-per-batch, and only for the narrow slice of judgment — cross-item narrative balance — that the vision model was never positioned to do. Every call you don't make is a call nobody else can meter, throttle, or bill you for.

Sheet 17 · Training Data

One Public Foundation, One Bootstrapped Layer

General Grounding — Public, Freely Licensed

COCO Captionscocodataset.org
Conceptual Captions (3M / 12M)Google Research, web-mined alt-text
The CauldronHugging Face Hub — aggregated open VQA/instruction sets
Large-scale image-text pairsDataComp / open web-scale alternatives

Representative Photo Sample — For Bootstrapping

A diverse pool of real-world photos across the target contexts is needed before the judgment layer can be labeled. Free-tier stock-photo APIs (e.g. Unsplash, Pexels) provide a broad, permissively licensed starting pool; supplemented over time by real usage once the system is live.

Task-Specific Judgment Layer — Bootstrapped by Distillation

The representative sample is passed through a strong existing reasoning system configured with the target rubric, producing labeled examples of image, context, score, caption, and rationale. This converts an expensive, judgment-capable reference into a training set for a small model that reproduces the same judgment far more cheaply at inference time.

Continuous Refinement — After Launch

Every real acceptance or correction of the model's output by an end user is a labeled example in a real context. Folding this feedback into retraining is what lets the system keep improving on exactly the judgment it's used for.

Sheet 18 · Training Plan

Three Passes, Each Validated Before the Next Begins

1

Baseline Reproduction

Train the full architecture on general-grounding data only; confirm it reaches known reference accuracy on a standard visual benchmark before any task-specific work begins.

2

Blended Fine Pass

Continue training on roughly three-quarters general data, one-quarter bootstrapped task data — evaluated against agreement rate with reference judgments held out from training.

3

Iteration on Disagreement

Cases where the model diverges from the reference judgment are categorized and used to target additional bootstrapped examples for the next pass.

Approach

Full training, staged

Compute

Single high-end GPU, days not weeks

Tracking

Held-out eval each pass

Primary metric

Agreement with reference judge

Sheet 19 · API Endpoints & Payloads

POST /v1/assess

Request

{
  "image": "https://... or base64",
  "context_tag": "trip",
  "image_id": "img_0192",
  "metadata": {
    "captured_at": "2026-06-11T14:22:00Z"
  }
}

Response · 200

{
  "image_id": "img_0192",
  "score": 0.87,
  "context_alignment": 0.93,
  "people_count": 2,
  "tags": ["coastline","sunset"],
  "caption": "...",
  "rationale": "...",
  "narrative_fragment": "...",
  "model_version": "poc-0.3"
}

POST /v1/assess/batch · GET /v1/jobs/{id}

Batch Request

{
  "context_tag": "trip",
  "items": [
    {"image_id":"a","image":"url"},
    {"image_id":"b","image":"url"}
  ],
  "callback_url": "https://..."
}

Job Status · 200

{
  "job_id": "job_7f3a",
  "status": "processing",
  "progress": {
    "completed": 1400,
    "total": 3600
  }
}

GET /v1/health → { "status": "ok", "model_version": "poc-0.3" } · Batch accepts up to 3,600 items per job.

Sheet 20 · Deployment

Where Each Piece Runs

This is the moment the model stops living on someone else's infrastructure under someone else's terms of service. Every line below runs where you say it runs.

Serving Stack

Weights4-bit quantized (AWQ/GPTQ)
Inference serverOpenAI-compatible batched server (vLLM-class)
PackagingSingle container image, versioned
Weight storagePrivate model registry / object storage

GPU Hosting — Options Considered

Scale-to-zero serverless GPUPrimary choice for this stage
Reserved / committed GPU capacityOnce utilization is steady
Hyperscaler managed instancesOnly if SLA/compliance requires it

Sheet 21 · Reliability & Operations

What Breaks in Production, and How This Is Designed to Fail Safely

Rate Limiting & Backpressure

The batch endpoint's 72,000-image in-flight ceiling is enforced at the API gateway, not inside the model server — callers over the limit get a 429 immediately rather than queuing indefinitely. Autoscaling triggers on queue depth, so sustained overload adds GPUs before it adds latency.

Retries & Idempotency

image_id is the idempotency key for both endpoints — a retried request with the same image_id should return the same cached result rather than re-running inference. Batch jobs retry callback_url delivery with exponential backoff.

Model Versioning & Rollback

Every response carries model_version. New training passes roll out as canary traffic before becoming default; if agreement rate against the reference judge drops post-deploy, rollback is switching default traffic back to the previous model_version — no data migration required.
What to Actually MonitorHow
Agreement-rate driftvs. reference judge, per Sheet 18
Latency percentilesp50 / p95 / p99
GPU utilization & queue depthautoscale trigger health
Cost per image, trailing 7-daycatches storage/CDN creep

Sheet 22 · Security, Privacy & Data Ownership

Where the Photos and Their Derived Data Actually Live

Self-hosting is the privacy lever this whole design pulls — data never leaves the environment, every call, by design. That default doesn't remove the need for basic data governance; it just means the governance decisions are entirely this team's to make.

What the Model Itself Withholds

Facial identity, demographic inference, and exact geolocation are deliberately not extracted. Photos of minors, in particular, deserve retention and access policy attention regardless of what the model extracts from them.
Encryption & AccessImplementation
In transitTLS on every API call, internal and external
At restObject storage server-side encryption, zero-egress tier
API accessPer-key auth with rotation; scope keys to context_tag if multi-tenant

Compliance — Not Legal Advice

This dossier is planning-grade engineering guidance, not a compliance determination. Photos of people trigger data-protection regimes (GDPR, CCPA, and similar) that vary by jurisdiction and by whether minors are depicted; involve legal/compliance review before launch if the pipeline handles user-uploaded photos at any real scale.

Sheet 23 · Peak Load Capacity

What the Fleet Can Absorb at Once

Throughput Model

Per-GPU throughput~20 images / second
Reference autoscale ceiling4 GPUs, burst
Fleet peak throughput~80 images / second
FormulaGPUs × 20 img/sec = peak req/sec
ScaleCapacity
4 GPUs (reference)~80 req/sec, ~15 min clear for 72K backlog
8 GPUs~160 req/sec, ~7.5 min clear
20 GPUs~400 req/sec, ~3 min clear

Sheet 24 · Latency End to End

Where the Milliseconds Actually Go

StageTypical Range
Network round-trip (client → API)20–150ms, region-dependent
Queue wait (fleet saturated)0ms typical, up to several seconds at peak
Cold start (scale-to-zero, first request after idle)3–15s
Model inference (single image, warm GPU)50–150ms at this model size
Response serialization<5ms

Cold Start Is the Single Biggest Latency Risk

Scale-to-zero is the cheapest hosting option and the one most exposed to cold-start latency — a request landing on an idle fleet can take 20–100× longer than a warm one. If a p95 latency SLA matters, keep one warm instance standing and let autoscale handle burst above it.

Sheet 25 · Monthly Cost, Itemized

Reference Volume: 100,000 Images / Day

Line ItemBasisMonthly
Inference compute~1.4hr GPU-time/day × safety margin × $0.70/hr$90–150
Object storage — rolling raw~9TB rolling window × $0.015/GB$130–150
Object storage — retained~1.35TB/mo growth × $0.015/GB$20–30, compounding
CDN / deliveryUsage-based, cache-hit dependent$50–150
Queue + metadata storeManaged small tier$50–200
Monitoring / loggingStandard managed tier$20–50
Total~$360–730 / month

Marginal Cost Per Image

At reference volume (~3M images/month), total cost lands at roughly $0.0001–0.0002 all-in per image processed — dominated by rolling storage and delivery, not compute, at this model size.

Sheet 26 · Cost at Different Scales

ScaleVolumeApprox. MonthlyNotes
Freelance / side project500/day (~15K/mo)$15–40Fixed costs dominate; compute is nearly free
Small startup10,000/day (~300K/mo)$80–180Storage and CDN still dominate over compute
Reference100,000/day (~3M/mo)$360–730Compute becomes a real line item
Growth stage1,000,000/day (~30M/mo)$2,500–5,000Committed-capacity discounts (20–40%) start to matter

Below Roughly 5,000–10,000 Images/Day, Check the Crossover

A frontier API's per-call cost is strictly linear with no fixed overhead. This system's cost is mostly fixed (storage minimums, a standing queue and metadata tier) with a small linear component. At low enough volume the fixed overhead of self-hosting can exceed what a frontier API would simply bill per call — calculate the actual crossover for your volume before committing engineering time to a build.

Sheet 27 · Cost vs. Frontier API

Same Reference Volume, Priced Against Gemini 3.1 Pro

This Is the Number That Should Change Your Mind

Up to 7× cheaper, self-hosted, at the exact same volume. That's not a rounding error — that's the entire margin of a business, sitting in a vendor's invoice instead of your pocket.

ApproachBasisMonthly
This system, self-hostedSheet 25 total~$360–730
Gemini 3.1 Pro, list price~700 input + ~150 output tokens/img × $2/$12 per M × 3M images~$9,000–9,600
Gemini 3.1 Pro, batch discount50% batch-API rate applied to the above~$4,500–4,800
Ratio, worst case for this system~6–7× cheaper self-hosted

Why the Gap Widens With Scale, Not Narrows

The API's cost is strictly linear — every image costs the same whether it's the first or the ten-millionth. This system's marginal cost per image falls as volume grows, because the fixed cost (the trained model, the serving infrastructure) is already sunk. The two cost curves diverge further the more the system is used, not less.

Beyond Raw Cost

Data leaves the environmentEvery call, by design
LatencyExternal network roundtrip
Rate limitsSet by the provider, not this team
Model churnProvider can deprecate on any schedule

Sheet 28 · Three Paths, Compared

Full Build, Fine-Tuned Open Weights, or a Frontier API

DimensionFull Build (This Path)Fine-Tune Open WeightsFrontier API
Weight ownershipCompleteCompleteNone
Architecture ownershipCompleteInheritedNone
Upfront costLow–moderateLow~None
Marginal cost at volumeLowest, fallingLowest, fallingHighest, linear
Data leaves the orgNeverNeverEvery call
Time to first version6–10 weeks1–3 weeksImmediate
Dependency riskNoneNone once trainedHigh

Why Not the Frontier API, Beyond Cost

Cost alone is enough to disqualify a frontier API at any real volume for a narrow, repeated task — but it isn't the only reason. A general-purpose model iterates on a schedule this team doesn't control: a flagship vision model in this market was superseded and deprecated within roughly four months of a prior release, requiring every downstream integration to re-validate against a new model version on short notice. Someone else's roadmap should never be a load-bearing part of your product.

Sheet 29 · Decision Framework

Which Path Fits You

Sheet 28 compares the three paths on their merits. This sheet maps reader type to a starting recommendation — still worth checking against your actual volume before committing.

First-Time Builder

Start with a frontier API. Optimize for speed to a working first version, not cost — you don't have usage data yet to size anything against. Revisit once you do.

Freelance / Small Client Project

Check actual volume first. Below roughly 5,000–10,000 images/day, a frontier API is usually still cheaper all-in once your time is priced in. Fine-tuning only pays off if the same need recurs across multiple client projects.

Startup, Proven Need

Fine-tune an open checkpoint. Fast enough to ship, full data ownership, marginal cost already falling with volume.

Expert, Pushing Cost / Latency / Ownership to the Limit

Build it. Own the architecture outright, keep a warm instance once real traffic justifies it, and negotiate committed compute pricing once volume is proven. This is the path that puts every future model version, every pricing change, every rate limit — entirely under your control. That's not a technical nicety. That's the whole point.

Reader TypeRecommended PathPrimary Constraint
First-time builderFrontier APISpeed to first version
Freelancer, low volumeFrontier APITotal cost incl. your time
Startup, proven needFine-tune open checkpointCost / ownership balance
Expert, max controlFull buildCost + latency + ownership ceiling

Sheet 30 · Implementation Detail

What Actually Gets Built, File by File

File / PathWhat It Is
models/vision_transformerVision encoder
models/modality_projectionPixel-shuffle + linear bridge
models/language_modelDecoder-only transformer
models/vision_language_modelCombined forward pass
data/bootstrap.pyReference-model distillation script
data/blend.pyGeneral + task-data mixing
train.pyStaged training loop
serve/Quantization + inference server config, API layer

Build Sequence

  • Stand up the environment — Install the training stack; confirm a single GPU runs one forward and backward pass end to end.
  • Reproduce the baseline — Train on general data only; confirm known reference accuracy before changing anything.
  • Build the bootstrap dataset — Run the distillation script against a reference judgment source; convert output into the training schema.
  • Blend, train, evaluate, serve — Retrain on the blended mix, iterate against held-out agreement, quantize, and stand up the API.

Sheet 31 · Timeline & Resourcing

A Proof of Concept Measured in Weeks, Not Quarters

PhaseDurationOutput
Environment & baseline reproduction1–2 weeksValidated pipeline
Task dataset bootstrap2–3 weeksLabeled training set
Blended training & iteration3–4 weeksCandidate model
Evaluation, API & serving setup1–2 weeksWorking proof of concept
Total6–10 weeksIndependently trained system

Resourcing for This Phase

One engineer with applied machine-learning experience can carry this proof of concept end to end. Training compute cost is modest, measured in the low hundreds of dollars; ongoing hosting cost at the reference volume is likewise modest relative to a per-call external API at the same volume.

What This Proof of Concept Establishes

A working, independently trained, privately held vision-language system with a defined API contract, that reproduces a specific judgment task at a quality worth building on — and a validated pipeline for producing the next version, larger or more capable, from the same foundation. Six to ten weeks from today, this stops being a vendor's model running on a vendor's schedule at a vendor's price — and starts being yours.

36 views
0 likes

Start a Critical Discussion

These questions don't have consensus answers. Share one to LinkedIn or X and see what your network actually thinks.

"Is AI infrastructure more like railroads in 1870 or the internet in 1999 — and does the distinction matter for your career bets?"

"Jensen says Layer 5 (applications) hasn't exploded yet. What's actually blocking it — talent, trust, or tooling?"

"The trades (electricians, cooling engineers) are the #1 AI job category by volume. Is this fact criminally underreported?"

Share this analysis

If this changed how you think about something, share it. The AI workforce conversation needs more data and less hype.

We use cookies

Essential cookies keep the platform running (authentication, session). We also use analytics cookies to improve your experience. EU/UK users: non-essential cookies require your explicit consent under GDPR Art. 6(1)(a) and the ePrivacy Directive. See our Privacy Policy for details.