Two passes, not one

An image model given a product photo and a brief does two different jobs at once. It builds a scene, and it reproduces one specific object inside that scene. It is good at the first and unreliable at the second.

The output is not obviously broken, which is the problem. The logo is nearly the logo. The hardware is the right shape and the wrong finish. The colour has moved half a shade. Nothing in the frame looks wrong until it sits next to the real product on a listing page.

For a moodboard that is acceptable. For a catalogue it is not, and there is no partial credit: the object in the picture is the object a customer is being asked to buy.

A model asked to invent and to reproduce in one pass will invent. That is not a prompt problem.

We spent a while treating it as one. Longer briefs, stricter wording, explicit “do not alter the logo” clauses. All of it helped a little and none of it held across a set, because a single pass has no separate step where the product is the only thing being decided.

So the pass got split. Stage one renders the frame with nothing pinned in it. Stage two puts the real product into that frame.

Stage one is a base model built for real people and real light. It decides scene, pose and composition. It is never asked to imagine the product, and it could not edit one in afterwards if we wanted it to: it generates and cannot edit. That limitation is what keeps the two stages honest about which one owns what.

Stage two has the frame already. It is not deciding lighting or pose, so the one thing it is deciding is the one thing the first engine was worst at.

When nothing is pinned, there is nothing for stage two to do. It degrades to a light refine or is skipped, and the shot is not billed for a pass that had no work in it. Running it anyway for symmetry would have been the tidier code and the wrong bill.

The first version of stage two built its edit instruction from the product record: name, category, the fields we already had. That is the obvious implementation. It is also wrong, and it fails in the most expensive way available, which is confidently.

A white tank top came back as a shoulder bag. Well lit, well placed, correctly composited, and the wrong category of object entirely.

The editor was not broken. The editor did exactly what it was told. We had told it the wrong thing.

The instruction writer had never seen the garment. It had seen a string, inferred an item type from it, and written a fluent instruction about an object that did not exist. Every layer downstream then executed that instruction correctly.

The fix was to stop describing and start showing. Stage two is now a vision pass: it is handed the base render and the actual reference photographs, and writes the instruction from what it can see. It names the real object because it is looking at the real object.

This is a general lesson we keep relearning. Anywhere a model is asked to reason about a specific real thing, give it the thing. A name is a lossy encoding and the model will fill the loss in silently.

The instruction is narrow on purpose, and most of the rules are there because the broad version misbehaved.

It never re-describes the scene. Background, pose and light already exist in the base render. An instruction that mentions them invites the editor to redraw them, and a redrawn background means stage one's work was wasted.

It states what to preserve, not only what to change. An edit that only names the change is read as permission to touch everything else.

It is written in editing verbs. Replace, keep, match. Generation verbs pull the model back toward making a new picture.

When the vision pass fails

It falls back to a deterministic instruction and the render continues. An ordinary shot beats a failed shot, and a pipeline that stops because one reasoning step timed out is a pipeline that loses work it already paid for.

The instinct when pinning something is to hand over every angle you have. It makes the output worse. Multiple references pull the subject in different directions and it drifts from shot to shot across a set, which is the exact thing pinning was supposed to prevent.

So: one angle per product, one photograph per character, face on and well lit with nothing else competing in the frame. The restriction is the feature.

Almost none of the work here was discovering that. It was enforcing it in the places a well-meaning user or a well-meaning agent would route around it.

Two passes cost more than one. A composited shot bills both stages. The skip path in figure 1 is not an optimisation, it is the only reason the cheap case stays cheap.

Stage two cannot rescue stage one. If the base render puts the subject in a pose where the product would be half out of frame, the composite has nowhere to put it. It can only place the object into the picture that exists.

A described place is not a real place. Generated locations hold up. A specific real building, a client's actual store, cannot be produced from words and still needs a photograph.

The vision pass is another model call. It is one more thing that can be slow, and its failure mode is a worse instruction rather than a loud error, which makes it harder to notice than a crash would be.

The base models come from Higgsfield.^[1] They are why the base frame holds a real face and real light.

The compositing pass and the rest of the image and video models run through fal.^[2] Which editor a shot gets is configuration rather than a constant, which is what lets a flagged render route to a model that can take it instead of failing at the gate.

What is ours is the split, what stage two is shown, and the reference discipline. The other half of the system, deciding what a given brand's shot should look like in the first place, is a separate note.