Table of Contents Show
When people talk about AI in video editing, the conversation often jumps straight to automatic editing: give a model the footage, ask for a video, and hope that taste has somehow become an API call. That can be useful for fast social content, templates and some semi-automated workflows. It is not the problem I wanted to solve.
My starting point with Automation Agent for Adobe Premiere was the preparation work around the edit. Finding repeated takes, checking transcripts, mapping spoken content back to source clips, creating markers, preparing review sequences, finding possible teaser moments and turning a pile of project information into something an editor can actually work with. I wrote about the idea earlier as timeline-aware AI. The important word is not AI. It is context.
Automation Agent is my attempt to build that context layer around Adobe Premiere. It connects a general-purpose agent such as Codex or Claude to a live Premiere project through the Model Context Protocol, adds a permission system, lets successful workflows become reusable scripts, and can hand narrowly defined reasoning steps to a cloud or local model. It is currently in private beta, and there is no public price yet. That status matters. This is a look at how the system works, not a declaration that the robot editor has finally arrived and may now have your chair.
The hard problem is not controlling Premiere.
At first, the technical question sounds simple: can an AI agent control Premiere? By 2026, that is no longer the interesting part. Adobe itself put the Premiere AI Assistant into public beta in June. It can organise media, prepare footage, add markers and build rough assemblies from natural-language requests. So yes, an agent can operate an NLE.
The more difficult question is whether the agent understands what the project means. A language model can read a transcript. An editor needs more than that. Which transcript words belong to which source clip? Which part of that source clip is actually used? Where does it land in sequence time? What sits above and below it? Where are the cuts and markers? Is this clip in a source bin, a selects bin or a delivery sequence? A professional edit is a network of relationships, not a long text document with B-roll attached.
That distinction is why Automation Agent is not just another chat panel. The layer between the model and Premiere has to translate project state into useful context, expose the right operations, validate what the agent wants to do and restrict the damage when a confident model has misunderstood something. Confidence remains a presentation style, not evidence.
The alpha demo above shows the basic idea. The agent is not receiving a rendered video and guessing what might be useful. It can inspect the current project, ask for specific information, reason about the state it finds and execute controlled Premiere operations. That live loop is the most flexible mode, but it is only one of the ways Automation Agent can run.
Three ways to use the same automation layer
I deliberately separated the product into several execution models because not every job deserves a live AI session. If a task is already understood, having a model rediscover the implementation every morning is expensive improvisation dressed as convenience.
The simplest route is the Library. Automation Agent ships with ready-made workflows for tasks such as transcript proofreading and translation, markers, rendering utilities, publishing preparation and some FFmpeg-based finishing work. These are normal saved workflows. Some are completely deterministic. Some use external tools. Some call a configured model only for the small part that genuinely requires judgment.
The second route is the live MCP workflow. Here the connected agent remains involved while the job runs. It can inspect the project, perform read-only probes, validate actions, execute a step, inspect the result and decide what to do next. This is useful when the task is exploratory or depends heavily on the specific state of the open project.
The third route is really a combination of the first two, and it is the one I expect to matter most for repeated production work. A live MCP workflow is useful for exploring the problem first: what information is actually needed, where the difficult cases are, what the editor should review and what a useful interaction looks like. Once that works well, Automation Agent can turn the solution into a saved script. The workflow becomes visible logic in the block editor rather than a prompt fossil. You can inspect it, change it, test it with narrow permissions, store it in the Library and run it again without asking an agent to solve the plumbing from scratch.
Assistant and Workflow AI are different jobs.
This distinction is central to the architecture. I use the word Assistant for the live agent you are talking to while Premiere is open. Codex or Claude Code can fill that role. The Assistant sees the current problem, uses Automation Agent through MCP and can build or execute a project-specific workflow interactively.
Workflow AI is different. It is a provider called from inside an already saved workflow for a bounded reasoning task. Take transcript proofreading as a simple example. A script can deterministically collect transcript text, split it into safe chunks, validate IDs and prepare the Premiere write-back. Only the proofreading step needs semantic judgment, so only that step calls Workflow AI. The provider can be Codex CLI, Claude Code CLI or a local model through LM Studio.
This sounds like a small architectural detail. In practice, it changes cost, repeatability and safety. The model no longer needs to rediscover how Premiere transcript JSON is structured on every run. It receives the text it needs, returns a structured result, and the reviewed script handles the rest. AI becomes a component inside a workflow instead of the workflow itself.
Why fuss with the timeline?
Transcript-driven work is currently the easiest place to see the value because spoken content already has a semantic representation. Suppose a one-hour interview contains five versions of the same answer. A model can find similar passages in the transcript, but an editing workflow still needs to map those words back to exact source ranges and then build something useful in Premiere. A repeated-takes workflow can turn those ranges into a stacked comparison sequence. The editor still chooses the take. The agent removes the scavenger hunt.
Chapter markers are another example. A transcript can tell a model that the topic changes around a certain sentence. The timeline can tell it that a hard cut, title card or other edit event sits six frames away. A useful workflow can combine both and place a reviewable Premiere marker at the edit point rather than returning a plausible timestamp in a chat window.
The same applies to highlights and social cutdowns. The model can judge whether a passage sounds interesting, while the workflow maps the selected words into sequence time and creates duration markers with labels and explanations. This is the kind of preparation I mean by timeline-aware AI. It creates a review layer. It does not pretend that selecting the final quote is no longer an editorial decision.
Text is only part of the future. Reactions, product reveals, screen-recording steps, gestures and composition are visual information. Today, Automation Agent can use targeted frame export when a workflow needs visual evidence. Adobe Media Intelligence also shows where richer machine-readable visual context can go. The long-term problem is not producing an endless description of every frame. It is representing the right visual facts at the right source and sequence times for the task at hand.
An example: proofing a Premiere transcript
The transcript proofreader is an unglamorous example, which is one reason I like it. Speech recognition is usually good enough that the remaining mistakes are annoying rather than spectacular. But have you ever recorded a video with a Bavarian? Proper nouns, brand names and specialist terms are especially good at becoming real words that are completely wrong. Add dialect to the mix, and things get even more interesting. A spellchecker sees a legal word. A human sees the wrong technology, product or company.
In the walkthrough, Premiere has produced the wrong version of a difficult personal name. The useful part is not that a language model can propose another spelling. The useful part is the complete loop around that suggestion: select the relevant clips, retrieve their transcript data, use surrounding context and optional production terminology, explain likely corrections, ask for approval, write back only approved changes and verify the result inside Premiere.
How the transcript loop actually works
The workflow starts with selected clip project items or bins in Premiere. It filters out sequences, offline media and project objects that do not resolve to usable clips. It then exports the transcript data and verifies that the payload contains actual words and timings before treating it as something worth proofreading.
This is built on Adobe’s public Premiere UXP Transcript API. Transcript JSON export, JSON import and the action for importing text segments have been available since Premiere 25.6. Adobe added the explicit hasTranscript method in 26.3. Automation Agent does not treat a status flag as proof that useful transcript content exists. It exports the JSON and checks the data itself.
The transcript structure matters because the text is not the whole object. Timing, segment order, speaker metadata and identifiers must survive the round trip. The workflow therefore separates structural data from the words that need review. It is instructed not to summarise, translate, clean up the speaker’s style or magically improve an interview. Slang, fillers, false starts and dialect stay unless they are clearly transcription errors.
Context can be supplied before the proofread. That can be a list of people, product names, acronyms, places or client-approved terminology. Longer glossaries, speaker lists or event agendas can come from a text file. The context is guidance, not a license to invent background information. Without it, the workflow is meant to stay conservative and mark ambiguous cases as uncertain.
Before anything changes, the live recipe creates a correction report with the project item, transcript location, original text, proposed text, reason and confidence. The editor can approve high-confidence items, select individual corrections, provide new context or cancel. Names, quotations, numbers, dates, prices, and other sensitive statements are exactly where an approval step earns its keep.
The reusable proofreader is more interesting than the demo
The live MCP version is useful for demonstrating the full conversation, but once the workflow is understood, there is no reason to make the agent rediscover it every time. The Library therefore contains a saved workflow called Proofread Selected Transcripts With Workflow AI.
That script already knows how to collect the selected clips, export and batch the transcripts, validate every proposed change against the original text and import approved corrections. The Workflow AI provider does only the semantic proofreading. In this version, timing data is not sent through the model at all. The model receives transcript text and returns patch-style suggestions. The script validates and applies those suggestions after review.
The review mode can be conservative or fast. You can inspect every suggestion, automatically handle higher-confidence corrections while reviewing low-confidence ones, or apply a broader set after an additional warning. Even then, Premiere is only modified after a separate final import confirmation for each clip.
There is also a practical performance reason for freezing the plumbing. In our own mamoworld tests, a 45-minute interview took about 5 to 6 minutes as a live agent session and about 2.5 minutes with the ready-made Library workflow, using Codex in both cases. That is not a benchmark. Material, model and machine all change the result. It illustrates the architectural point: once the script knows how to collect and re-import the transcript, the model can spend its time on the part that actually needs a model.
Permissions before cleverness
An agent that can inspect a project is useful. An agent that can modify everything in it by default is a future incident report. Automation Agent therefore separates Premiere read access, Premiere write access, filesystem read access, and filesystem write access. Saved scripts and live agents can both run with restricted scopes.
For transcript review, read-only access is enough if you only want a QA report. If the workflow should write corrected transcripts back, a safer set-up is to put candidate clips in a dedicated bin and allow project writes only inside that bin. The same principle applies elsewhere. A marker workflow does not need permission to rearrange the entire edit. A report generator does not need permission to rename clips. Narrow permissions are boring until they aren’t.
The permission model does not solve every security question. If a live workflow uses a remote model, allowed project or transcript data still has to reach that provider. Permissions limit what the workflow may access and modify. They do not magically turn a cloud service into an offline one. That is a separate choice: workflow AIs can also run locally, which I will come back to later. Production teams still need to decide which providers are approved for which material.
From a successful chat to a production tool
The part I care about most is the handoff from exploration to repeatability. A live agent is excellent at discovering a workflow. You can describe a problem, let the Assistant inspect the project, test a solution and refine it. But a production workflow shouldn’t stay trapped in yesterday’s chat history.
Automation Agent can turn that solution into a saved script. You can inspect the generated logic in the block editor and edit it manually. This matters for both review and learning. The block editor is not mainly there because humans are better at repairing AI-generated code. Often they are not. Its value is that the workflow remains visible and editable: useful for reviewing what it actually does, checking security-sensitive steps, or turning a small utility into a family of slightly different versions.s
Saved scripts can also be launched without reopening the authoring conversation. The Library exposes local remote-execution URLs so that workflows can be triggered from keyboard tools, Stream Deck, Touch Portal, Keyboard Maestro, AutoHotkey, Raycast or Alfred. The URL points to the saved Library item, so the reviewed script remains the source of truth.
When Premiere is not enough: FFmpeg and Python
Some useful editing workflows require operations that Premiere’s API shouldn’t be forced to impersonate. Frame extraction, media probing and heavier data processing are obvious examples. Automation Agent therefore has Runtime Packs for external tools.
The current packs provide FFmpeg and a managed Python runtime. The FFmpeg pack can handle media analysis and frame extraction. The Python pack supports data-heavy tasks such as transcript batching and validation. Workflows declare the packs they require, and preflight checks stop the run before it gets halfway through a job and discovers a missing dependency.
The official packs are signed and verified. You can also import them from signed archives on machines without direct internet access. After installation, they run locally and are separate from the AI provider. That distinction matters in restricted environments because a deterministic FFmpeg or Python workflow doesn’t need a cloud model just because the product also supports AI.
Local models and restricted production machines
There are two very different meanings of “AI workflow” in a secure facility. One is a workflow that used AI while it was being authored but runs as reviewed local logic afterwards. The other needs a model to reason about current production data every time it runs. Those have completely different deployment requirements.
The first case is straightforward. Build and test the automation on an approved development machine, inspect the result, save it and deploy that script to a production machine. If it does not contain a Workflow AI step, execution can be deterministic. You can also import and use Runtime Packs locally.
If the saved workflow still needs semantic judgment, LM Studio can serve a local model as Workflow AI. A practical set-up is to use Codex or Claude while developing the workflow on non-sensitive material, reduce the model’s job to a small defined step, then run that step locally on the production machine. Smaller local models need more careful prompts, smaller batches and stricter structured output than stronger cloud agents, so “local” should not be mistaken for “equally capable”.
Live MCP work is different. If the agent must inspect new project state and make open-ended decisions during every run, the complete agent and model stack has to be available. If company policy forbids external model access and there is no approved local agent deployment, that workflow does not belong on that machine. Security policy remains stubbornly unimpressed by a clever demo.
What Automation Agent can already be used for
The current Workflow Recipes give a better picture of the product than a feature checklist. Spoken-content rough cuts can select one coherent story path from transcript-bearing material. Repeated-takes workflows create comparison timelines. Transcript-based assembly can locate requested passages in source recordings and build a sequence from them. Highlight workflows create review markers for teasers, pull quotes and social cutdowns. Chapter workflows map semantic boundaries back to the actual edited sequence.
There are also language and delivery workflows. Selected transcripts can be proofread or translated. An SRT toolbox exports, reviews, translates, condenses, reflows and analyses subtitle files. The use of SRT is partly pragmatic: Premiere’s current UXP API does not yet expose direct reading and writing of Caption Track text, so portable subtitle files provide a controllable route instead of pretending the missing API is there.
Other workflows move beyond transcripts. Reconstruct Edit From Reference can index selected source media, match shots from a rendered reference and rebuild source intervals into an editable Premiere timeline. Runtime tools can then help with additional media analysis. Interestingly, this kind of workflow does not need a model at runtime at all. AI may help create the workflow, while the resulting production tool can run entirely through deterministic local processing.
How this differs from Adobe’s own AI Assistant
Adobe’s AI Assistant arriving in Premiere is useful, but it doesn’t make Automation Agent less necessary. For quick Premiere-only requests, there is clear overlap. Adobe’s own tool can already organise media, prepare footage and create rough assemblies in a conversational workflow.
The distinction I am building around is openness at the workflow layer. Automation Agent lets you bring the agent or model environment you already use, including local models. A general-purpose agent can combine Premiere state with external scripts, spreadsheets, notes, files and research tools. A successful workflow can become a saved script rather than remaining a chat interaction. Saved scripts can call a different Workflow AI provider only where judgment is needed.
That does not make one approach “better”, just different. Adobe can integrate deeply with its own application and services. I am interested in a layer that lets production teams combine Premiere with the agent, local model, external context and the reviewed automation strategy they have chosen for the job. You know, Pipelines for actual productions and studios.
Where the edges still show
Automation Agent is still a beta. There is no public price, and I am not treating current performance measurements as product-wide benchmarks. Premiere APIs also continue to evolve. Transcript support has improved substantially since 25.6, while other areas, including direct Caption Track text access, still require workarounds. Host-version differences and incomplete project data remain things a workflow has to detect rather than wish away.
AI accuracy is another boundary, not a release-note item that disappears. A context-aware proofreader can still choose the wrong proper noun. A highlight detector can still prefer the wrong quote. A local model can still violate a requested schema. Automation Agent therefore validates responses against the specified schema and can send invalid results back to the model for correction.
Beyond that, robust workflows still benefit from narrow scopes, confidence handling, explicit approval points and verification after write-back. None of those is as photogenic as a prompt box, which is unfortunate because they are the part I would actually trust in production.
For new workflows, I still start with disposable or duplicate project material and keep the initial permissions narrow. The agent can of course implement the workflow, but it can also do much of the technical testing itself. Because it can inspect the project, run the workflow, and examine the resulting state, it can build test cases from the available material, check whether the expected result was produced, and iterate when it was not.
By the time I review a workflow manually, the remaining issues are often less about whether it works and more about user experience, test coverage, and whether I want to trust it with broader production access. Once that process is stable and repetitive, I save the workflow.
The workflow layer is the product.
Models will keep changing. The most useful model for one studio may be unacceptable in another. Adobe will keep expanding its own Assistant. Local models will get better. None of that removes the need to represent edit context correctly, map decisions back to source and sequence time, expose controlled operations and preserve a human review path. And once a solution works, it’s worth turning it into a reviewed, reusable workflow rather than asking the next model to rediscover it from scratch.
That is the part I am trying to build with Automation Agent. I do not need AI to have taste on my behalf. I need it to understand enough of the project to remove preparation work, surface useful choices and turn good solutions into tools that can be reviewed and repeated. If that gives the editor more time for the decisions that actually require an editor, the workflow has done its job.
Documentation and references
Automation Agent overview and private beta: mamoworld documentation.
Practical examples: Workflow Recipes.
Transcript workflow: Agent-Proofread Selected Transcripts.
Reusable script authoring: Create and Save Your Own Scripts.
External tooling: Runtime Packs.
Adobe’s underlying transcript API: Premiere UXP Transcript class.
Adobe AI Assistant: Premiere AI Assistant overview.
