Skip to content

Native local video ingestion #927

Description

@goolamabbas

Summary

space-bunny-alpha advertises native video understanding, but local MP4 analysis currently requires a custom AVFoundation/Swift frame extractor because read_file only reports the video as binary. Please consider adding a native read_video path for video-capable models, with metadata, bounded sampling, and a clear fallback for unsupported models.

Expected Behavior

There should be a read_video tool or read_file can be extended to natively understand video if selected model has native video undersanding

Actual Behavior

harness built a swift frame extractor

Steps to reproduce the issue

  1. Gave the following prompt which was derived from here. You can also extract the video from the tweet
    https://x.com/yusufg/status/2061429526585106486
    Load the video in kyle-geocities.mp4 and turn it into an Apple-style website from 2026. Generate a plan and write it in space-bunny-alpha.md

I thought space-bunny-alpha would natively understand the video

Command Code Version

1.65.0

Operating System

macOS

Terminal/IDE

ghostty

Shell

zsh

Session file (optional)

No response

Fix prompt (optional)

No response

Additional context

OS: macOS Tahoe 26.5.2

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions