Single-camera produce intelligence · edge-deployed

Point a camera
at fruit.
Get its weight.

One RGB camera, no scale, no depth rig. A vision pipeline segments each fruit, recovers real-world scale, estimates volume, and multiplies by density — running live on an NVIDIA Jetson edge board. Built as a framework: teach it a new fruit by registering one number.

See the validation ↓ Run it yourself

+4%apples vs water displacement

80/80unit tests green

Jetson Orin Nanorunning live on the board

fruitvol · live estimate capturing…

Detected

— apples

Volume

— cm³

Est. weight

— g

vs ground truth

—

New here? Start with this

What it is, why it's hard, and how it works.

Grading stations and kitchens need to know how much produce they have — but weighing every item by hand doesn't scale, and a scale can't tell apples from mangoes. So we ask a single ordinary camera to do it.

What

A camera that reads weight

Point one RGB camera at produce. The system finds each fruit, works out how big it really is in centimetres, estimates its 3-D volume, and multiplies by the fruit's known density to print a weight in grams — no scale, no special sensor.

Why it's hard

A photo has no size or depth

A picture is flat pixels. The same apple looks big up close and small far away, and you can never see an object's back. Turning "some pixels" into "so many cubic centimetres" means recovering real-world scale and inferring the hidden 3rd dimension — the two problems this project is built around.

How

Segment → scale → solid → grams

An open-vocabulary AI model cuts out each fruit's exact shape. A reference object (or an AI depth model) fixes the millimetres-per-pixel. Simple geometry spins that outline into a 3-D solid for its volume. Density turns volume into weight.

The one-sentence version

fruitvol converts a single photo of produce into a weight estimate — and it's built as a reusable framework, so teaching it a brand-new fruit takes one number (that fruit's density), not a new dataset or a retrained model. On real apples it landed within 4% of the true weight measured by water displacement.

The demo you can watch from your laptop

The Jetson hosts the feed. The Mac watches it — wirelessly.

The Jetson Orin Nano serves its own camera as a live MJPEG stream over Wi-Fi. Any machine on the same network — my MacBook, across the room — just opens http://ImageProcessingJetson.local:8000 in a browser and watches the produce line like a security camera. No cable to the laptop, no app to install. Flip on the open-vocabulary overlay and it names and boxes the fruit in real time — no labels, no training, you just type the class names.

STREAMING CORE

Zero-dependency feed

A stdlib HTTP + MJPEG server — no OpenCV on the capture path (it's broken on this L4T image), so it runs on the bare board. The frame source is a swappable interface: CSI camera, webcam, or a camera-free synthetic pattern for testing off-device.

RECOGNITION

Open-vocabulary, on-GPU

Grounding DINO runs on the Jetson GPU in its own thread so the video stays smooth. Detection and volume estimation share the same swappable slots — the recognition model is a config flag, not a rewrite.

CONTROL DASHBOARD

Snap → measure → download

The browser page is a control panel: photo mode snaps a few frames with a rotate-the-fruit countdown, runs volume→weight, and hands back an annotated result. Live CPU/GPU/temp/fps metrics sit alongside — one command (jetson_up.sh) brings the whole rig up in Docker.

The Jetson Fruit Vision browser dashboard: live feed panel, photo-mode snap-and-analyze controls, system metrics, and a Jetson CLI reference — The full control panel, open in a browser — live feed (shown here with the camera-free synthetic test source), the **snap → measure → download** photo-mode workflow, live system metrics, and an on-page Jetson command reference.

What happens between shutter and scale

Five stages. Every one of them swappable.

The measurement path is a chain of typed interfaces. Any stage can be replaced without touching the others — the scale source, the segmenter, the depth model are all config, so a research bake-off stays a one-line change.

01 · INPUT

Image

One RGB frame from a single camera.

csi · webcam · file

02 · MASK

Segmenter

Grounding DINO → SAM isolates each fruit, pixel-clean.

grounded-sam · threshold

03 · SCALE + Z

Depth / ref

Metric depth, or a reference object → mm-per-pixel.

depth-anything · mm-per-px

04 · GEOMETRY

Volume

Pure-numpy solids, unit-tested against analytic shapes.

ellipsoid · revolution · depth-mirror

05 · OUTPUT

× density

Volume × the fruit's density → grams.

density registry

Real fruit · water-displacement ground truth · 13 Jul 2026

It measured apples to within 4%. It also shows you exactly where it breaks.

Same rig, same day, three fruits — scored against water displaced in a measuring jug. The honest part matters as much as the headline: a single top-down view has a real boundary, and the results map it.

Four apples with per-fruit segmentation mask overlays — apple_masks.png — per-box SAM masks, four apples measured individually

✓ Works — well-constrained

4 apples, round and separated

Grounding DINO boxes each apple → SAM mask → ellipsoid volume with an oblate 0.80 thickness prior and a measured reference scale.

Measured

626 cm³

Actual (displaced)

600 cm³

Error

+4%

Why it holds: round fruit + a measured 15 cm scale bar in-plane. The error budget below shows the scale bar — not any model tweak — is what pulled +80% down to +4%.

Apple error budget · scale is the dominant lever

A 10% scale error cubes into ~40% volume error.

Eyeballed ruler + sphere prior

1077 cm³ · +80%

+ oblate apple prior (0.80)

861 cm³ · +44%

+ measured reference scale

626 cm³ · +4%

— — — ground truth = 600 cm³ (water displacement) — — —

The actual ambition

Not a fruit model. A machine that learns any fruit.

Every design decision answers one question: does this generalize to a new fruit with minimal input, or does it bake in one fruit?

The segmenter is open-vocabulary — a new fruit is a text string, not a labeled dataset.
Onboarding = register one density (register_density()). Optionally a shape ratio for non-round fruit.
Scale source is an interface — reference object today, metric depth tomorrow, no pipeline rewrite.
Geometry is pure numpy, unit-tested against analytic spheres and ellipsoids — the math is auditable.

BUILT WITH CLAUDE CODE

The whole rig — code, docs, and design study — was built with Claude.

Claude Code drove the Jetson bring-up (headless SSH, the R39.2 camera-enable workaround), wrote the streaming server and the volume pipeline, generated the unit tests, ran the water-displacement validation analysis, and maintains a versioned Word design study under track-changes plus an append-only prompt log. This showcase page is a Claude artifact too.

A single builder + Claude, from bare board to +4%.

Chosen for a product that ships

Every model on the path is permissively licensed — on purpose.

This is meant to be commercialized, so AGPL was designed out from day one. The default detector is Grounding DINO (Apache-2.0), not the AGPL YOLO-World that would force open-sourcing the whole product. The active license is printed in the health check and the banner.

Stage	Model / component	License	Role
Detect	Grounding DINO	Apache-2.0	Open-vocab detection (default)
Segment	SAM 2	Apache-2.0	Pixel-clean masks
Depth	Depth Anything V2 · Metric Indoor Small	Apache-2.0	Monocular metric scale
Detect (fast)	YOLOX / RTMDet	Apache-2.0	Trainable edge detector
Geometry	fruitvol.volume	in-house	Pure-numpy solids

◆ NVIDIA Jetson Orin Nano Super Developer Kit

Everything you've seen runs on one board.

The whole pipeline — Grounding DINO detection, SAM 2 segmentation, Depth Anything V2, and the volume math — runs live on the NVIDIA Jetson Orin Nano Super Developer Kit, using its CUDA GPU and TensorRT. It's the "prove-it" rig: if it works here, the cost-down work is about shrinking to cheaper boards, not proving the idea. Camera is a Raspberry Pi Camera Module 2 (IMX219, 8 MP) on a 15↔22-pin CSI cable. Development is fully headless — SSH/VNC from the Mac, no monitor on the board.

The test bench: a MacBook running the code, wired to the Jetson Orin Nano Developer Kit with the camera mounted over a measuring jug, apples on a towel — The actual test rig: MacBook (left) running the code and viewing the feed, the **Jetson Orin Nano** driving the camera mounted over a water-displacement jug, apples staged on a towel. ⚠ Yes, it's a book-and-bucket rig on a bare desk — this validation was run while travelling, so the setup is deliberately scrappy. The measurement discipline (15 cm scale bar, water displacement) is what matters, not the furniture.

Board / part	Qty	Status	Role
Jetson Orin Nano Super Dev Kit	1	RUNNING ON IT	Primary dev + prove rig — full GPU pipeline
Raspberry Pi Camera Module 2 (IMX219)	2	IN USE	The project camera — works on every board
Raspberry Pi 5	1	DEPLOY TARGET	Cost-down tier — not yet run on it
Raspberry Pi Zero W	2	CAPTURE-ONLY	Future camera nodes — no on-device inference
Scale · caliper · measuring jug · fiducials	—	GROUND TRUTH	Water-displacement + dimension validation

An honest note on the ₹10,000 target & the smaller boards

So far it has only run on the Jetson — the Pi 5 and Pi Zero W are owned but not yet deployed to. The Jetson is a development rig, not the shipped device; the < ₹10,000 hardware target is for the eventual cost-optimized build. That's exactly the roadmap: bring the compute down — quantize the models, lean on TensorRT/NCNN — so the pipeline fits a Raspberry Pi 5 (deploy tier) with the Pi Zero Ws as cheap capture nodes. Sensing stays RGB-only for now; a depth/ToF camera is gated behind an accuracy bake-off and kept behind a swappable interface.

Where it goes next

What we're building toward.

The apple result proves the path. The roadmap is about removing the assumptions a single top-down photo forces, and pushing the same pipeline onto cheaper hardware.

See thickness, drop the shape prior

The mango/banana results all trace to one gap: a single view can't measure depth. Add the depth-mirror method (real thickness from metric depth) and a multi-angle sweep so volume stops depending on a guessed shape ratio.

Near

Handle groups & occlusion

Separate touching produce (better instance segmentation) and estimate hidden/occluded items so a pile — not just tidy, spaced fruit — can be measured. This is the banana-bunch failure, turned into a feature.

Cost-down

Shrink onto a Raspberry Pi 5

Quantize the models and lean on TensorRT/NCNN to fit the deploy-tier board, with Pi Zero Ws as capture nodes — driving the hardware bill toward the < ₹10,000 target.

Framework

Guided learning for any fruit

Fold density + an optional shape ratio into a short onboarding flow: show the system a new fruit, give it a couple of numbers, and it's ready — no dataset, no retrain. The generalization goal, made a product step.

Later

Accuracy target & Part 2

Pin a formal error target across a fruit panel, then revisit the deferred defect detection (Part 2) once volume/weight is solid.

On your own machine — no camera needed

Run the pipeline yourself.

The measurement path runs anywhere with Python ≥3.10; only the live camera feed wants the Jetson. Off-device, the synthetic source and threshold segmenter exercise the whole thing with no hardware.

# install (editable + test deps)
pip install -e ".[dev]"

# one-time model fetch (~1 GB, all Apache-2.0)
python3 scripts/fetch_volume_models.py

# measure a fruit from one or more images → volume + weight
python -m fruitvol.cli scene.jpg --fruit mango --mm-per-px 0.16

# teach it a new fruit — one number
python -m fruitvol.cli --register-density guava --weight-g 180 --volume-cm3 230

# watch the live edge feed from any browser (on the Jetson)
python3 scripts/live_stream.py --detect --classes "apple,banana,mango,carrot"
# → open http://ImageProcessingJetson.local:8000 · test anywhere with --source synthetic

Point a camera
at fruit.
Get its weight.