Video Intelligence

StreamVibe: Proactive Video Agents and What They Mean for Footage

StreamVibe is a research paper on video agents that decide when to speak. See how it works, where it breaks on long video, and what it means for footage review.

Ceptory Team9 min read
Stacked model layers with one layer highlighted, representing a streaming video model with a separate memory layer

TL;DR

StreamVibe (arXiv, October 26, 2026) trains a streaming video model to decide every second whether to stay silent or help. It improves timing by 10.7% over baselines, but the authors and press coverage note it degrades on long videos, which matters for footage review.

On October 26, 2026, a 16-author team, described by The Decoder as led by Google, posted StreamVibe: Proactive Video Agents for Evolving User Intent to arXiv. The paper describes a video agent that watches a stream and decides, once per second, whether to stay silent or offer help. The Decoder covered it on November 1. It is the clearest recent example of a proactive video agent, and it also documents why such agents still struggle with long recordings.

This is research, not a product launch. But it points at where video understanding is heading, and its limits are useful to anyone who has to review footage at scale. Below we explain what the paper actually reports, what it does not claim, and what teams that search recorded video should take from it.

What is a proactive video agent?

A proactive video agent is a model that watches video as it plays and chooses when to speak, instead of waiting for a typed question. Most video assistants today work in a request and response loop: you ask, the model answers. The StreamVibe authors argue, as summarized by The Decoder, that this design means assistants rarely pick up on intentions the user never states, lose track of steps already completed, and keep following an old instruction after the user changes their mind.

The paper frames this around three kinds of intent:

  • Explicit intent: the user gives a clear instruction ("tell me when the glue has dried").
  • Implicit intent: the user has a vague goal and the assistant must infer what would help.
  • Dynamic intent: the request changes over time. The Decoder's example is a user taking apart a toy excavator who asks for progress updates, then switches to a quality check after three minutes.

Think of a workshop assistant that sees you stuck on a software step and suggests the next menu, rather than a search box that returns matching clips. That is a different job from searching a video archive, and it is worth keeping the two apart.

What exactly did the StreamVibe paper release?

The paper reports three components. According to the arXiv abstract, they are:

  1. StreamVibe-Bench: a benchmark with 4.1K evaluation samples and 85K training samples covering explicit, implicit and dynamic intents.
  2. StreamVibe-VL: a streaming model that infers implicit intents from continuous video and adapts as those intents change.
  3. StreamVibe-Harness: a component that tracks intents as persistent states using hierarchical memory.

The Decoder adds detail on how the data was built: roughly 4,100 test examples drawn from more than 183 hours of everyday-life and software recordings. Gemini 2.5 Flash suggested when to offer help and what to say, and four human evaluators had to unanimously approve each example. The Decoder reports that StreamVibe-VL is based on Alibaba's Qwen3.5-9B, and that a second, reinforcement-learning stage makes the model more reserved so it does not constantly interrupt.

The design choice that matters most is the separation of roles. A distinct memory system tracks pending requests and completed steps, while the model alone decides whether to speak and what to say. That split targets the failure the authors describe: assistants that repeat advice or claim something is done when it is not.

What results does the paper report?

The authors report that StreamVibe-VL improves response timing and answer quality by 10.7% over existing baselines, and that the harness adds a further 12.3% gain in proactive interaction performance. The Decoder reports that the model beats its base model and other open-source models by a wide margin, while a proprietary Google Gemini model scores higher in several tests. It also says the specialized training does not appear to harm general video comprehension.

Two case studies from the paper, as described by The Decoder, show the kind of error the work targets:

ScenarioWhat a comparison model didWhat StreamVibe-VL did
GIMP black-and-white tutorialJoyAI claimed the finished image was already saved, though the user had not opened itGave the correct next step: "Colors," then "Desaturate"
Rolled jam toastOther models described on-screen actions, such as the bread being flattenedAdvised cutting off the crust first

Both are cases where a good answer depends on what has already happened in the video, not only on the current frame. That is the same property that matters when you review recorded footage: the meaning of a moment often depends on what came before.

Where does it break? The long-video problem

The most useful result for footage teams is a negative one. The Decoder reports that performance across all tested systems drops sharply on longer videos because errors compound, with a key content metric falling by up to 61.5% when the system relies on its own history. In videos longer than five minutes, it says, models rarely pick the right moment to speak and struggle to track changed instructions.

The researchers also acknowledge, per The Decoder, that real-world situations are likely more complex than their test setup, and they list how user preferences and the environment influence each other over time as future work.

Put that next to the benchmark scale. The test material is everyday-life and software recordings, in clips measured in minutes. A single camera produces 24 hours of video a day. A warehouse, a construction site or a newsroom archive can hold thousands of hours. A live agent that degrades after five minutes of history is not a substitute for indexed, searchable video. It is a different tool, and the paper does not claim otherwise.

Why does this matter for security, safety and archive teams?

It matters because the same model family will probably reach those teams, and the paper shows which requirements to ask about before it does. Proactive agents are attractive in three settings:

  • Workplace safety: an assistant that notices a worker skipping a step in a procedure and says so in the moment. Our guide to workplace safety monitoring and compliance covers the review side of this today.
  • Operations and training: step tracking on a production line or in a software workflow, close to what video process monitoring already tries to measure from recordings.
  • Robotics and field work: agents that follow a long sequence of actions and flag errors. We covered a related release in Gemini Robotics ER 2 on Google Cloud.

Each of those still has a recorded-video half. When an alert fires, someone has to find the footage before and after it, confirm what happened, and keep it as a record. A proactive agent speaks in the moment, and recorded-video search answers questions afterwards. Real deployments need both, and the second does not go away because the first improves.

There is also a cost and privacy dimension the paper does not address. A model that reasons over a continuous stream every second implies continuous inference on live video, and for people on camera that raises consent and retention questions. Set retention and consent rules before you add any always-on analysis.

How is this different from searching recorded video?

Recorded-video search is retrieval over an index built in advance, while a proactive agent is generation over a live stream. In search, video is processed once, and every later question is a lookup. You can ask "forklift near the loading dock after 10 pm" a month later and get timestamps. A streaming agent has no such index: it carries a rolling memory, and the paper's long-video result shows what happens to that memory as the stream grows.

Proactive streaming agent (StreamVibe-style)Indexed video search
TriggerThe model decides when to speakYou ask a question
InputLive stream, second by secondRecorded video, indexed once
MemoryRolling state kept by a harnessA persistent index of scenes, speech and on-screen text
StrengthTimely, in-the-moment guidanceFinding any past moment across hours of footage
Reported weak spotLong videos, where errors compoundDepends on what was indexed and the query

Neither is better in general. They answer different questions, and the fair summary is that the research frontier is on the left column while most enterprise footage problems live in the right one. For the basics of the right column, read enterprise video search in natural language.

Where does Ceptory fit?

Ceptory is on the recorded-video side. It indexes video you upload or add by URL through the API, lets you search it in natural language across scenes, speech and on-screen text, and runs multimodal analysis from the dashboard or the REST API. It does not offer a live proactive assistant, and we are not claiming StreamVibe-style capabilities.

What the paper does change is how we would describe a good review workflow:

  • Search first, then ask. Ceptory Search finds the moments; Ceptory Understand is where you ask what happened in them.
  • Keep history in the index, not in a model's context. The paper shows rolling memory degrading on long streams. A persistent index does not have that failure mode, though it has its own limits: search quality depends on the footage and the query, so verify results against the video.
  • Price it before you commit. Indexing is $0.042 per minute plus $0.0015 per minute of infrastructure, search is $4 per 1,000 queries, and the first 10 minutes of indexing are free. See pricing.

Caveats: Ceptory publishes no accuracy figures, and we make no claim that it beats any research model on any benchmark. If you need live, in-the-moment guidance for a person at work, a product built for that is the better fit. For a broader view of how the categories split, see video intelligence platforms vs traditional VMS.

What should teams do now?

Treat StreamVibe as a signal about direction, not a purchase decision. A practical sequence:

  1. Separate the two jobs. List which of your video questions are "tell me now" (live) and which are "find me later" (recorded). Most compliance, investigation and archive work is the second kind.
  2. Test on your own length of video. The paper's weak spot is long video. If you pilot any proactive agent, run it on your real shift lengths, not on short clips.
  3. Ask vendors how memory works. Is state kept in a rolling context, or in a persistent store? What happens after an hour? After a day?
  4. Measure interruptions, not only accuracy. The authors needed a reinforcement-learning stage specifically to stop the model from interrupting too often. A noisy assistant gets switched off.
  5. Keep the recording. Whatever speaks live, retain the footage under a written policy so a human can verify what the agent said.
  6. Watch for a release. The paper reports a benchmark, a model and a harness. We have not seen a product, pricing or an API, so do not plan around one yet.

A concept of stacked timeline tracks for video, audio and speech with the matching span highlighted

What are the limits of this coverage?

Everything above comes from the arXiv abstract and The Decoder's summary of the paper. We have not run StreamVibe, the benchmark numbers are the authors' own, and The Decoder's article does not name the individual authors or independently reproduce the results. The reported gains are relative to the baselines the authors chose, and real-world footage is messier than the benchmark, as the authors themselves note.

Conclusion

StreamVibe is a credible step toward video agents that know when to speak, and a useful reminder of why long video remains hard. For teams whose problem is finding what happened in hours of recorded footage, the practical answer today is still an index you can query. To see that workflow on your own video, start with Ceptory: the first 10 minutes of indexing are free.

Frequently asked questions

What is StreamVibe?

StreamVibe is a research system for proactive video agents. The paper introduces a benchmark (StreamVibe-Bench), a streaming model (StreamVibe-VL) and a memory component (StreamVibe-Harness) that decide when an assistant should speak while watching video.

Is StreamVibe a product I can use for video search?

No. It is a research paper with a benchmark and model, aimed at live assistants that watch a user work. It is not a product or API for searching recorded footage, and the paper is not a deployment guide.

How much does StreamVibe improve over other models?

The authors report a 10.7% gain in response timing and answer quality over existing baselines for StreamVibe-VL, and a further 12.3% gain in proactive interaction from the memory harness. These are benchmark results, not field results.

Why does long video matter here?

The Decoder reports that performance drops sharply on videos longer than five minutes, as errors compound. Anyone reviewing hours of recorded footage should treat live proactive agents as unproven for that job.

Does Ceptory use proactive agents?

Ceptory indexes recorded video and lets you search it in natural language and run analysis from the dashboard or REST API. It does not offer a live proactive assistant that interrupts a user.

Ceptory

Search your own footage in plain language. The free tier indexes 60 minutes of video.