Prompting video and sound

The video model produces picture and sound together from one prompt: motion, camera, ambience, music and spoken lines. It responds to how a prompt is written as much as to what it says. Prompts in the style below are followed far more closely than keyword lists or prompts written for other video models.

Prompts can be up to 4000 characters. Video has no negative prompt.

The short version #

Write one flowing paragraph in the present tense, four to eight sentences (roughly 60 to 200 words), covering these six things in this order:

1. The shot: scale and angle. A medium close-up, a wide establishing shot from a low angle.
2. The scene: place, lighting, colours, textures, atmosphere.
3. The action: what happens, as a sequence of verbs in time order.
4. The characters: what they look like and how their feelings show physically.
5. The camera: how and when it moves, and where it ends up. Or say it is static.
6. The sound: ambience, effects, music, and any speech in quotation marks.

A medium wide shot of a narrow street at dusk, overcast and grey, wet asphalt holding a little reflection. A courier in a yellow rain shell walks a bicycle in from the left, stops at the centre and leans it against a closed metal shutter, then looks up at the windows above. Her shoulders are loose and she breathes out slowly. The camera tracks her from the side at chest height and settles with her centred against the shutter. A tyre hisses on the wet ground, a car passes in the distance, and the shutter rattles as the bicycle touches it.

Your prompt is expanded into a fuller shot description before rendering, so a short prompt still works. The expansion can only guess at what you left out, though: anything that matters (an action, a line of dialogue, a camera move, silence) should be written by you.

Rules that make the difference #

Give every sentence a verb that does something. She turns, reaches, lifts, exhales. Appearance alone gives the model nothing to animate.

Tell it in order. Use connectors: At first... A moment later... As she turns... Finally... Events happen in the order you write them.

Pace it in the prompt. The clip is timed from your words. Write the pause if you want one: he waits, a beat of silence, then... Plan about one main action per two to three seconds; a 4.33-second clip carries one or two beats, a 10-second clip three or four.

Show emotion physically. Her shoulders drop and her jaw tightens, not she is sad.

One camera move per shot, with a destination. Say how the camera moves relative to the subject and what the frame looks like when it stops: The camera pushes in slowly until her face fills the frame. Two moves in one clip (pans left, then pushes in) usually get muddled. If the camera should not move, say the camera remains static; leaving the camera out invites drift.

Keep it focused. One scene, few characters, one consistent source of light. Start simple and add detail where it matters. Close-ups need more detail than wide shots.

Name the style early if it is not live action: A hand-drawn 2D animation..., A stop-motion scene..., A grainy 16 mm documentary shot...

Weave it into prose. No labels like Camera: or Audio:, no bullet lists, no tag strings.

Sound, music and speech #

Sound is generated from the prompt. If you do not describe it, the model chooses.

• Be specific: soft footsteps on tile, rain on a tin roof, a kettle beginning to whistle, not ambient sound.
• Tie each sound to something visible and put it where it happens in the action: the glass hits the floor with a sharp crack. Sounds with no visible cause may not sync.
• Start sound early. Audio can come in late; if it matters, say from the very first frame, ... and name the opening sound.
• Speech goes in quotation marks, with who says it and how: She says quietly, with a slight French accent, "We should go." Quoted words are spoken as written. Keep lines short: a few words per second of clip.
• Describe the voice: low and gravelly, bright and quick, a whisper, a shout.
• Music: name genre, instruments and mood: a slow solo piano in a minor key, a driving synth bassline.
• Silence and exclusions can be stated: No music. No one speaks. Only the wind is heard.
• Sound without any speech can come out thinner; a strong, concrete ambience description helps.

sound: false removes the audio track from the file. It does not change the price or the render time.

Camera vocabulary #

Use these terms as written; they are understood more reliably than synonyms.

A move does not slow down to fit a short clip; it is cut off. Give long moves a longer duration_seconds.

Text to video #

Nothing is given but the prompt, so describe everything: style, subject appearance, setting, light, action, camera and sound. Name the look in the first few words. Set the shape with aspect_ratio.

A wide shot of a lighthouse on a black rock at night in heavy rain, waves bursting white against the base. The lamp turns slowly and its beam sweeps from left to right across the water and through the rain, briefly lighting a small fishing boat rising on a swell. The boat's cabin light flickers and steadies. The camera remains static. From the very first frame, wind howls and rain drums on stone, a deep wave crash lands as the beam passes the boat, and a distant foghorn sounds once. No music.

Several shots in one clip are possible in this mode if you name each cut and re-establish the shot after it, but shots built as separate steps join more reliably and can be re-rendered one at a time.

The first frame of a text-to-video clip is saved as the step's start frame, so the result can be continued, edited or re-rendered like any other step.

Image to video #

The start frame already defines the look. Spend the prompt on what happens: motion, camera, sound.

• Do not re-describe the picture at length. A sentence that anchors the subject is enough: The woman in the yellow coat...
• Whatever you do say about the image must be true of it. A description that contradicts the frame can cause a jump to a different scene.
• Keep it one continuous take.
• The ending is open, so say where the action should go: ...and stops at the door, hand on the handle.

The woman turns from the window to face the camera and smiles, then tucks a strand of hair behind her ear as a breeze moves the curtain. The camera pushes in slowly to a close-up of her face. A soft piano melody plays, and a clock ticks quietly in the room.

Use image to video when you have a picture to bring to life and no fixed ending: portraits, product shots, illustrations, establishing shots.

First and last frame #

Both ends are fixed. The prompt describes what happens between them: how the scene gets from the start frame to the end frame, how the camera travels, and what is heard.

• Describe the transition, not the two pictures.
• Make sure the end frame is reachable in the clip's length. If much changes, use a longer clip.
• The two frames must agree on scene and props. Something present in only one of them will pop in or out. Make the end frame by editing the start frame (see Prompting edits and references).
• Motion in this mode tends to be measured and smooth. It suits transformations, time-lapses, changes of light and weather, reveals and camera moves with a defined destination better than fast action.
• Avoid large head turns and fast spins between the frames.

Daylight fades over the street as the sky deepens from pale grey to dark blue. The street lamps flicker on one after another from the far end toward the camera, and the shop window begins to glow. A cyclist passes from right to left and is gone. The camera remains static. Traffic hum thins out, a lamp buzzes as it lights, and footsteps pass close by.

Shots that continue each other #

From the second step on, a shot starts on the previous take's last frame.

• Keep the style words and the lighting description the same from step to step.
• Carry the sound across: describe the same ambience so it does not change at the join.
• Each step's prompt should describe only that step's action.

2D animation and stylised looks #

• Describe the style in both the start-frame prompt and the motion prompt, and say what it is not: flat hand-drawn cel animation with bold line art, no 3D shading.
• Keep faces large in frame; small stylised faces go soft in motion.
• Prefer small movements (blinks, hair, cloth, a hand) and avoid big head turns, which pull stylised characters toward a 3D look.

What to avoid #

• Readable on-screen text. Short, large words sometimes hold; spelling and stability are not reliable. Add titles afterwards.
• Chaotic physics: juggling, gymnastics, fast spinning, complex collisions. Walking, gesturing, dancing and vehicles are fine.
• Crowded scenes with many characters each doing something different.
• Contradictions: running quickly in slow motion, mixed light sources, two camera moves at once.
• Keyword lists and other models' prompt formats.
• Too much for the length. If a take feels rushed or cut off, lengthen the clip or split the action over two steps.

If a take misses #

A re-render with a different seed often fixes a small miss; each take is a new candidate and costs the same. If the same problem returns, change the prompt: simplify the action, state the camera explicitly, or move the troublesome part into the next step. To judge a take without watching it, an agent can call get_asset with preview: "contact_sheet" and look at six frames in order.