Optional integration recipe · AI + media

Track objects. Transcode the result.

Connect a separately supplied tracking plugin and model to CoreMF’s decode, annotation and encoding pipeline. This integration recipe shows how the media stages fit together.

Prepare the plugin ↗ See the command ↗

For a first run without a tracking package or model, use the CoreMF evaluation guide ↗.

What CoreMF connects

From video frames to annotated output.

1. Decode

The source reads compressed media; the decoder supplies video frames to the tracking plugin.

2. Track & highlight

The plugin returns timestamped boxes, labels, confidence and track IDs. CoreMF renders the returned detections onto frames.

3. Encode & mux

The encoder converts annotated frames to the selected output size and frame rate. The sink writes video and available audio to MP4.

The tracking model provides the detections; CoreMF provides the media graph, plugin integration, highlighting and output pipeline. Detection quality and throughput depend on the selected model and hardware.

01 · Prepare

Use your tracking plugin and model.

  • A matching CoreMF runtime with runAPI, Python plugin support and usable H.264/AAC encoders. Keep the package's runtime dependency layout intact.
  • A separately supplied, compatible tracking package registering CoreAIProcessor with the object_track algorithm and AI processor profile. Configure its licensed model and inference dependencies according to that package's instructions.
  • A short video containing objects the model can recognize, and new output filenames in a writable directory.

The tracking package, model and Python dependencies are supplied separately. The base SDK includes no AI model pack. Plugin access and delivery ↗

02 · Run

Track, annotate and transcode in one command.

Replace the plugin path and input filename. The plugin must already be configured with its model. This example requests 1280×720 at 25 fps and keeps in-process GPL encoders disabled. 

runAPI --script-plugin /absolute/path/to/tracker/plugin.json \
  -i input.mp4 -o tracked.mp4 -p objecttrack \
  -s 1280x720 -r 25 -gpl off \
  -filter-config '{"sample_every":1}' -j tracked.graph.json

sample_every: 1 runs inference on every frame reaching the processor. Model-specific settings belong to the selected package. On Windows, use runAPI.exe and your shell's quoting and line-continuation syntax.

-gpl off controls encoder selection; it does not change the license of the linked media runtime. If no allowed H.264 encoder is available, configure a supported encoder/device before running.

Download the command and integration notes ↗

03 · Inspect

Check the video, tracking results and graph.

  • tracked.mp4: play the full output. Boxes and labels should follow actual detections; frames without detections need no boxes. Check audio, synchronization and final-frame drain.
  • tracked_all_results.csv: inspect timestamps, track IDs, confidence and bounding boxes emitted through the AI profile. Compare events with the corresponding video frames.
  • tracked.graph.json: inspect the exported source, decoder, AI processor, highlight, encoder and sink nodes.

Artifact names follow the compatible plugin package. Validate output, resource use and per-frame latency on your target hardware.

Bring the same graph into your application.

runAPI builds this pipeline through CoreMF_API.h. Your application can create and link nodes in code or load a configured Graph JSON pipeline; language bindings expose the same application boundary. Custom frame processing stays in the selected plugin.

Application APIs and Graph JSON ↗

Discuss your AI processing needs.

Discuss the plugin, platform and output requirements for your application.

Discuss AI integration ↗