What the assistant can ask ShotWork for¶
An AI assistant works your library through the requests below. The wording is what the assistant itself reads before deciding what to do, taken straight from ShotWork, so it is exactly what your assistant was told.
They fall into three groups. Reading: the summary, a search, a thumbnail, a song's beats, the music library, the overlay templates, the reels, and the server's own status. Writing a reel: a draft, and a revision of one that exists. Producing from a reel: a timeline, and a video.
Reading¶
list_library_summary¶
What's in the artist's shot library: how many shots, which tags they actually use, which projects and companies and roles appear, how ratings are distributed, how many reels exist and how many shots no reel has ever used.
Call this first, before searching. The tag vocabulary is the artist's own and won't match the words a client uses -- a brief asking for "creature performance" has to be mapped onto whatever tags are really in here, and you can't do that without seeing them.
No parameters.
search_shots¶
Find shots to consider for a reel.
field_values filters on any preset field by id, matching any of the values given: {"company": ["Acme"], "materials": ["oak"]}. The field ids and their known values are in list_library_summary's field_values. Every metadata field goes through it, including the ones the shipped preset happens to define, because a shot selects file carries its own fields and an id is not fixed by this app.
project is separate on purpose: it matches the name of the shot selects document a shot came from, not the shot's Project field. For that field, use field_values.
Tags match ANY of the ones you give unless match_all_tags is set, so pass several related terms rather than guessing at one.
rating is the artist's own judgement of their work, 1-5, and is the single most reliable signal you have -- weight it heavily. Results come back best-rated first. An unrated shot isn't necessarily weak; it may not have been graded yet.
Each shot's interest_range, when present, is the part of the shot the artist marked as the good bit, as seconds into the shot. It is the artist's own note about where to look, so a hold shorter than the shot should usually cover it: give that shot trim: "interest" in draft_reel. Absent means the whole shot is fair game, not that nothing is known. has_interest_range filters on it.
reels_using says how many of the artist's reels already use a shot. never_used_in_a_reel keeps only the shots none of them use -- ask for it when the brief is "something they haven't seen", and consider it anyway: a reel made of the same five hero shots as the last three reels is a weaker reel than its ratings suggest.
has_thumbnail says whether a picture is already cached; either way get_shot_thumbnail can show you the shot.
Restricted shots are work the artist is NOT cleared to show publicly. They're excluded by default and you should leave them excluded unless the artist explicitly asks for them for a private showing.
Prefer searching more than once with different terms over one broad search: you're building a reel, so variety across the body of it matters as much as the strength of any single shot.
| Parameter | Type | Default |
|---|---|---|
tags |
list of text | none |
match_all_tags |
yes or no | no |
min_rating |
whole number | none |
project |
text | none |
max_duration_seconds |
number | none |
min_duration_seconds |
number | none |
include_restricted |
yes or no | no |
limit |
whole number | 40 |
field_values |
object | none |
has_interest_range |
yes or no | none |
never_used_in_a_reel |
yes or no | no |
get_shot_thumbnail¶
See a shot: one frame of it, or a strip of several.
With frames at 1 you get the same thumbnail ShotWork shows in its own lists, taken from the middle of the shot (or the middle of its marked interest range). Ask for up to 6 frames to see how the shot moves: they are spread evenly through it and come back with seconds_into_shot on each, so you can tell where the action is and where the shot goes flat.
Use it before deciding what leads a reel, what holds longest, and which of two similar shots to keep. Names, tags and ratings say what a shot is about; a frame says what it looks like, and reels are looked at. Don't look at every candidate: look at the ones the decision actually turns on.
The artist decides whether pictures may be sent at all: a checkbox in ShotWork, off until they turn it on. When it is off this tool answers with a refusal that says so; take it at its word, work from the text, and don't ask again unless the artist says they changed the setting. The pictures you do get are sent to the service running you, like anything else in the conversation, which is why the artist gets to decide.
A restricted shot is refused unless include_restricted is true, same as everywhere else. A shot whose video is missing from disk cannot be shown; the error says so.
| Parameter | Type | Default |
|---|---|---|
shot_id |
text | required |
frames |
whole number | 1 |
include_restricted |
yes or no | no |
get_beat_grid¶
The rhythm of a music track: tempo, every beat's own timestamp, how strong it is (0.0-1.0), and whether it's a downbeat.
Call this before proposing a beat-driven cut -- there's no way to guess tempo or beat timing from a file name alone.
audio_path is a file already on disk: one the artist named, or one from list_music, which holds the songs they have imported and says which are already analyzed. energy is how strong that beat's own onset is relative to the rest of the track -- useful for cut density: a quiet stretch can hold a shot longer, a loud one wants more cuts. downbeat is only meaningful when downbeats_available is true; when it's false, every beat comes back as a non-downbeat because no downbeat model is installed on this machine, not because the track doesn't have any -- don't read that as "this track has no downbeats."
You do not need this to cut on the beat: draft_reel lands every cut on the attached song's downbeats by itself. Read the grid when the artist asks for something specific (every beat, an accent, a section change) or to describe the song. When you cut to a song, attach it to the reel with draft_reel's music so the artist hears what you cut to.
| Parameter | Type | Default |
|---|---|---|
audio_path |
text | required |
list_music¶
The songs in the artist's music library: name, file path, length, whether the file is still where it was, whether its beats and downbeats are already analyzed, and how many reels use it.
Look here before asking the artist for a song: when they say "the usual track" or name a song without a path, this is where the path is. Pass audio_path to get_beat_grid and to draft_reel's music. A song used by many reels is probably the one they mean.
No parameters.
list_overlay_templates¶
The artist's overlay templates by name: the metadata fields each one shows (name, role, studio, year...), any fixed text, and whether it carries images such as logos.
These names are what draft_reel's default_template and a title card's template take; a name not in this list is an error. An artist with no templates has none, and you must not invent one: say the reel has no overlay and that they can add one themselves.
No parameters.
list_reels¶
Every reel in the artist's library, with its name, path, clip count, runtime, songs, overlay template, the assistant's notes if an assistant drafted it, and any problems (a shot that has since been deleted, for instance).
Call this when the brief refers to a reel that already exists -- "like the last one but shorter", "the reel I sent Acme" -- and when you want to know what the artist has already shown: a new reel that repeats an old one is worth less than its ratings say. Pass a reel_path from here to get_reel, revise_reel, export_timeline or render_reel.
No parameters.
get_reel¶
One reel in full: its clips in playing order with each shot's name and hold, its gaps and cards, its music, fades, overlay template, notes, length limit, and how it lays out in time.
reel_path comes from list_reels or draft_reel. Read a reel before revising it, so what you pass to revise_reel starts from what is actually there and not from memory of an earlier call.
| Parameter | Type | Default |
|---|---|---|
reel_path |
text | required |
server_info¶
Which ShotWork this is and what it can see: the version, whether it is the packaged app or a source checkout, the executable the client launched, the library file it reads and how many shot selects and reels that lists, the folder it writes to, and whether ffmpeg and the downbeat engine are available.
For checking the connection and reporting a problem, not for cutting reels. Call it when a summary comes back empty or a build seems wrong, or when the artist asks what they are connected to, and quote the answer to them: "which ShotWork is this reading" is the first question anyone helping them will ask.
Check tier and assistant_unlocked first. On the free tier every other tool answers with a refusal saying the assistant needs a license: take it at its word, tell the artist, and do not keep trying.
No parameters.
Writing a reel¶
draft_reel¶
Write a new reel for the artist to open in ShotWork, review, and export.
shots is your complete edit decision, in playing order. Each entry is an object. The usual one is a shot: {"shot_id": "...", "seconds": 2.5}. seconds is screen time -- leave it out to play the shot in full. Give hero shots room and weaker-but-relevant ones a shorter look; a reel where every shot is the same length reads as a contact sheet.
Asking for more time than a shot actually has is an error, not something that will be quietly shortened. Search results tell you each shot's real duration, so check before allocating.
trim decides which part of a shot survives being shortened, reel-wide: "head" keeps the start, "centre" (that exact value) keeps the middle, "interest" starts at the artist's own marked interest range when the shot has one and otherwise behaves like "head"; when the hold is longer than the footage after that point the clip starts earlier so it still fits, and the result's warnings say by how much. Keeping the middle is often better when the subject enters frame late; "interest" is the best choice whenever search results show an interest_range, because it is the artist telling you where to look. A shot entry can carry its own trim to override the reel-wide one.
Three other kinds of entry exist, each with a kind: {"kind": "gap", "seconds": 1} is a deliberate blank (black, silent, music continuing underneath): a breath between sections, never a placeholder. {"kind": "color", "seconds": 2, "color": "#000000"} is a solid card, for a fade-to-black feel at the head or tail. {"kind": "title", "template": "...", "seconds": 4} is a title card: one of the artist's overlay templates rendered full-frame over color. The template must exist; list_overlay_templates has the names. Don't open with a title card unless asked; most artists lead with their best shot and put the name at the end.
default_template puts the artist's overlay band (name, role, studio, whatever the template shows) on every shot. Only use a name from list_overlay_templates, and only when they have one they use; an unasked-for band is easy for them to remove but is a decision they did not make.
music attaches a song so the reel plays with it in ShotWork: {"audio_path": "...", "offset_seconds": 0, "start_seconds": 0, "end_seconds": null, "fade_out_seconds": 1.5}. audio_path is the file you gave get_beat_grid. offset_seconds is how far into the song the music starts (the good part is often 40 seconds in); start_seconds and end_seconds are where on the reel it plays. Attach the song whenever you cut to its beats: a reel reviewed silent tells the artist nothing about the cut.
fades adds a video fade to a clip's edge: [{"shot_id": "...", "edge": "in", "frames": 12}]. A fade-in on the first shot and a fade-out on the last is a common, safe choice; fades between shots inside the body are not, and cutting on the beat wants hard cuts.
With a song attached, every cut lands on the song's downbeats (its bars): each shot's seconds is taken as a rough length and its end is moved to the nearest bar, or to the bar before it when the nearest would run past the shot's footage. Give rough lengths and let it place the cuts; you do not need get_beat_grid for this. snap changes what it cuts on: "downbeats" (the default), "beats" for every beat (only when the artist asks for a faster cut), or "none" to keep your lengths exactly as given. A song with no downbeats to be had falls back to its beats, and the result's snapped_to says which was used. To cut on specific beats of your own choosing, pass beat_seconds (reel timestamps, from get_beat_grid); then only a cut already within a few frames of one of them is nudged onto it. Each clip in the result says whether it was snapped.
A snapped cut lands beat_offset_frames (-2.0 frames, so slightly before the beat) rather than exactly on it -- editors commonly cut a frame or two ahead so the incoming image has already settled by the time the beat actually hits, instead of trailing it. Pass 0 to land exactly on the beat instead, or a different value for more or less lead.
You do NOT choose where the file is written, and you should not try -- give a name and the path comes back in the result as reel_path. You also don't need to compute any frame numbers or timeline positions; that is all done here from the durations you give.
The result reports the runtime, any warnings, and where each clip landed. Nothing is written if the edit has problems -- you get the full list back so you can fix it and try again.
Use notes to say briefly why you ordered it the way you did -- which shot you led on and why, what you left out. The artist sees this while reviewing, so your reasoning reaches them with the reel.
This writes the reel only. When the artist wants a file for Resolve, Premiere or Final Cut, call export_timeline with the reel_path; for a video to watch, render_reel. To change this reel afterwards, call revise_reel with its reel_path rather than drafting another: one reel per conversation, revised, is what the artist wants to find in their folder.
This is a proposal. Aim for a strong starting point rather than agonizing over a perfect one -- they have the final say and the tools to use it.
| Parameter | Type | Default |
|---|---|---|
shots |
list of object | required |
name |
text | Reel |
timeline_fps |
number | none |
max_seconds |
number | none |
trim |
text | head |
include_restricted |
yes or no | no |
notes |
text | empty |
beat_seconds |
list of number | none |
beat_offset_frames |
number | -2.0 |
music |
object | none |
default_template |
text | empty |
fades |
list of object | none |
snap |
text | downbeats |
revise_reel¶
Change a reel that already exists, in place. This is how a conversation iterates: "swap the third shot", "twenty seconds shorter", "add the song". Call get_reel first if you don't already have the reel in front of you, and pass its reel_path.
Give only what changes. shots replaces the whole item list, so pass the complete new order in the same shape draft_reel takes (shots, gaps, color and title cards). Anything you leave out stays as the file has it, including changes the artist made in ShotWork since: per-shot template overrides, clip zoom, edge trims they dragged by hand. So when you do pass shots, expect their hand edits on the clips to be replaced by your entries; say so in notes if it matters.
music replaces the song; pass an empty object to remove it. timeline_fps pins the reel to a rate (24, 23.976, 25 and so on), the same as the Frame rate box in ShotWork; leave it out to keep what the file has. clear_max_seconds removes a length limit; max_seconds sets one. snap, beat_seconds and beat_offset_frames work exactly as in draft_reel and only apply when shots is given: with a song attached, the new cuts land on its downbeats unless you say otherwise.
The same checks as draft_reel run, and nothing is written when the edit has problems. A timeline or video made earlier from this reel does not follow the change: export or render again afterwards.
If the artist has this reel open in ShotWork while you write it, their next save wins over yours. Ask them to close it first when in doubt.
| Parameter | Type | Default |
|---|---|---|
reel_path |
text | required |
shots |
list of object | none |
name |
text | none |
notes |
text | none |
max_seconds |
number | none |
clear_max_seconds |
yes or no | no |
trim |
text | none |
include_restricted |
yes or no | none |
music |
object | none |
default_template |
text | none |
fades |
list of object | none |
beat_seconds |
list of number | none |
beat_offset_frames |
number | -2.0 |
snap |
text | downbeats |
timeline_fps |
number | none |
Producing from a reel¶
export_timeline¶
Write an editable timeline from a reel, for DaVinci Resolve, Premiere or Final Cut. reel_path comes from draft_reel, revise_reel or list_reels; the file lands in the artist's reels folder and the path comes back as written_to.
format_id is "fcp_xml" (the default: every NLE reads it) or "otio".
The timeline carries what the reel says: the edit, the music, the overlay band as an overlay track (one PNG per shot, drawn here from the artist's template and the shot's own metadata), and title cards as stills. The PNGs land in a <name>_assets folder next to the timeline; the result counts what was drawn, and warns if anything could not be.
| Parameter | Type | Default |
|---|---|---|
reel_path |
text | required |
format_id |
text | fcp_xml |
render_reel¶
Render a reel to a video file the artist can watch, from the reel's reel_path. The default preset, "h264_review", is a small H.264 file that plays anywhere; "prores_hq" and "dnxhr_hq" are editorial masters and only for when they ask.
This takes a while: expect a minute or more for a long reel. Progress is reported as it goes. The file lands in the artist's reels folder, and a same-named earlier render is replaced.
Music, fades, color cards, title cards, the overlay band and the length limit are all in the render, drawn the same way ShotWork draws them. A reel with a gap is refused: a gap is a hole in the edit, not black footage. The result counts the overlays and cards drawn, and warns if any could not be.
Render when the artist asks to see it, not after every draft: they can play the reel in ShotWork instantly, with everything on it.
| Parameter | Type | Default |
|---|---|---|
reel_path |
text | required |
preset_id |
text | h264_review |