imagen

Pictures from a model that only answers questions.

imagen never asks Jev to draw. It asks multiple-choice, rating and yes-or-no questions: what's the subject, what shape, what color, where does it sit. Code turns the answers into a picture.

Five prompts

imagen render for the prompt crescent moon
crescent moon4.9 s · 16 Jev calls
imagen render for the prompt full moon
full moon5.2 s · 21 Jev calls
imagen render for the prompt sun
sun4.9 s · 20 Jev calls
imagen render for the prompt rose
rose3.6 s · 12 Jev calls
imagen render for the prompt beach
beach3.6 s · 15 Jev calls

64 by 64 cells · unedited

How it draws

Dashed outline of a circle with a glow ring, the layout Jev chose for full moon 1 · Layout

Jev picks the subject, a circle, its size, a white fill, a grey rim and a glow.

The 64 by 64 grid of colors Jev chose, cell by cell 2 · Pixels

Jev answers 1,668 color questions. The rest of the 64 by 64 cells inherit from their parent. This is the finished picture.

Side experiment: sysone-vl

Before imagen, we trained our own typed decision model on camera images. It answers four questions about a tabletop scene in 110 ms, with no text generated.

Top-down view of a wooden table: a grey block sits on a small green triangle next to a two-finger gripper, with a yellow cube, a blue cube and a green puck nearby
Task: pick up the object closest to the gripper
  • Which object?green triangle0.99997
  • In reach?yes0.991
  • Will the grasp work?probably not0.308 · true 0.25
  • How cluttered?heavy1.00

The green triangle hides under the grey block, so the grasp works one time in four. The model says 0.31. Reinforcement learning on simulated grasp outcomes cut its calibration error from 0.21 to 0.06.

Reliability plots: before, predictions bunch near 0 and 1; after RLCD they follow the diagonal

500 held-out scenes · dashed line is perfect calibration

Built on

Jev is TypeSafe's hosted decision model. imagen calls its API. sysone-vl is our own build of the same idea on an open vision model.