To Vector, or Not To Vector?

‍

Actually that’s not the only question facing you, dear MAM vendor and media managers. Indeed there are too many questions. You know them all.

What’s the ingest and sync workflow? How long does the indexing take? What’s the pipeline for adding models?

Will it be on prem and air-gapped? Or on cloud and connected to a suite of LLMs?

Can it scale to hundreds terabytes? Petabytes?

And we need to proactively query daily users of these systems at this critical juncture to find out exactly what their search use cases look like today and what they’d like to see tomorrow, given the tsunami of opportunities AI presents us with today.

What do editors, producers, coaches, marketers, programmers, media managers and journalists see when they look into their own video search crystal balls?

What are they impressed by – and what not – with the new generation of LLM capabilities?  These user insights combined with vendor innovation are two keys to unlock the true full value of your content, whether it was shot yesterday or last century.

Above all, how should MAM vendors approach the question of future-proofing their architecture? Or is it already too late?

Should you be ingesting someone else’s embeddings, for example Open AI, and then simply use your existing database’s vector add-on offer? Or do you need to buy a new vector database like Pinecone?

Or should you instead use a service to create the embeddings, search index, multimodal rankings and simply utilize their unified response?

How clunky and slow is too clunky and slow?

Will it be satisfactory for users to continually swivel chair across experiences? Or will they need a unified search and discover canvas?

And finally, what happens next year when we are able to have natural conversations about our archives with superintelligent AI librarians?

For all this uncertainty, why not kick the decision down the road?

‍

Time to Act: Why the Fork is Here Now

Sure, I’m being cute with Elon Musk’s now famous “Fork in the Road” memos, both at Twitter and DOGE.

But as a guy that spent time building a MAM for Ericsson on top of Vidispine over 12 years ago, I’d like to make the case for why it’s imperative to act now.

‍

Why now? Three reasons. Progress. Power. And product.

First, we’ve seen critical progress in this field that people are just now becoming aware of. ‘Old school’ computer vision & hearing techniques like Face Recognition, OCR, and speech-to-text have gotten cheaper and faster to implement. Face Recognition is radically different today from a cost and workflow POV than it was just a few years back. And yes, it’s more accurate as well.

Second, we all sense the emerging power of LLM systems to be precisely the tool we might need to surf, search and browse the sheer overwhelming volume of video content media companies have today. As I’ve written about previously in the Omnisearch e-book, LLMs bring context and understanding to video search that we’ve never had before.

We can also use them for tagging, metadata creation, and more. They have revealed new buzzwords like Retrieval Augmented Generation which is – at its core – a “similarity search” that indeed requires a vector database, as do all deep learning algorithms.

And finally, the third reason is what I’ll call the game-changer in this new era of video understanding, which is that the product category itself is going through a massive transformation. Previously, search was simple. Now, with so many more tools at our disposal, the table stakes feature set for video search has rapidly moved to multimodality. This is a feature that is both high concept & common sense.

Just like we humans use vision and hearing to make sense of the world, so too can today’s most advanced search systems bring various modes of information processing into their algorithms detecting objects, scenes, logos, faces, text, and other dimensions of vision and sound.

At Omnisearch we are going deep into these realms. We are passionate about the importance of user experience and interface in this new era of video search, given so many new tools available for us to coordinate & integrate. And we’re maniacal about speed and accuracy.

These are all mission critical domains that you’ll need to master even if you choose to run with a traditional database vendor’s vector add-on and grab your embeddings from Open AI.

I need to repeat that.

Even if you use an AI partner to generate your embeddings and dump those into your database vendor’s add-on product, you will still not avoid having to build a product around that search experience.

Product. Progress. And Power. Time to act. Here’s how.

‍

How to Integrate Multimodality, LLMs & Vector Search into your MAM without a Complete Code Rewrite

Ok, you’ve gotten this far and maybe you’re ready to take some action. You’re planning to make a clarion call internally to integrate vector search capabilities into your Media Asset Management (MAM) platform.

Can you do it without completely overhauling existing pipelines?

Several strategies can leverage existing technologies while minimizing disruption to current workflows.

1. Omnisearch: A One-Stop Solution for Next-gen Video Search

Instead of building and maintaining multimodal, vector and LLM-integrated search capabilities in-house, one option is to partner with a vendor like Omnisearch and allow us to be your complete search service.

It’s easy to integrate Omnisearch via API which delivers cutting-edge multimodal video search with an exciting roadmap that will be tackling some of the most challenging unlocks with LLMs in the next two years.

How It Works

1. We support any workflow. Small libraries are simple. For larger archives, we’d set a workflow to index your files in-place.
2. We create optimized indices including pointers to your content.
3. Your team deploys a search box and delegates search requests to our API. (Or you can utilize our beautiful web interface.)
4. Our platform returns lightning-fast, highly accurate multimodal search results, seamlessly optimized as between text, semantic, similarity and multimodal responses.
5. Sleep worry-free. No tech debt. No backlog. No dev overhead.

Pros

✅  No investment needed in product.

✅ Fast indexing no infrastructure to manage.

✅ High-precision search across face, OCR, logo and text search.
✅ Competitive pricing with a very low cost model.
✅ Future-proof—benefits from ongoing model & LLM improvements.

Cons

❌ Reliance on an external provider.
❌ Generically, latency considerations are top of mind. We invite you to test the responsiveness of real-time search applications & compare to native as well as alternative competitors.

Cost Factors

Omnisearch is built on a platform that requires only consumer-level GPUs for core computer vision features. We can get creative with you in deploying LLM use cases as we see tremendous growth opportunities there. And we are incredibly efficient, proud of our cost leadership.

2. Hybrid Search: Combining with Existing Keyword Search

Tempting! Many MAMs already have robust keyword search (e.g., Elasticsearch, Solr), and have mastered the art of optimizing text searches. It might be tempting to therefore move to a simple hybrid approach that can merge the results surfaced in two different systems without requiring a full system overhaul.

But there is more engineering required here than meets the eye. While we bet this is the most common approach, it has downsides in two key important areas.

First, speed, efficiency and human computer interaction are going to be continued challenges with this hybrid approach. While it’s true that some LLM activities are increasingly presenting users with multiple seconds of waiting for a response, that is not the ideal way to sift through terabytes of video.

Second, while this path doesn’t require nearly as much product investment as a roll your own solution would, (see number 4 below), you’re not escaping the need for some development.

And as every MAM vendor knows, some development can easily turn into more development. Leading to an internal quagmire fairly quickly.

‍How It Works

1. The MAM maintains its existing keyword-based search capability.
2. The search is also sent to a second vector search service, like Omnisearch. Or the user can pull up two browsers at once.
3. The MAM will have to merge results from both sources or present both result sets side-by-side, with no multimodal, no contextual, and no algorithmic logic to help the user sift through dirt to find gold.

Pros

✅ No need to abandon the existing search stack.
✅ Immediate improvements to search accuracy.
✅ No additional infrastructure required—only API calls.

Cons

❌ Merging and ranking results will of course require additional engineering effort. But what’s more: It’s a slow process that is impossible to accelerate.
❌ Duplicating queries is inefficient.

❌ Unknown which query types will perform better under keyword-based search vs multimodal vs other methods.

Cost Factors

Omnisearch pricing for vector-based search queries.

Existing MAM licensing costs. Plus extra engineering costs to implement even a simplified results merge, which could easily be $50K+ in dev costs if not more.

3. Either of the Above with a Crawl Walk Run process

A third path is a subset of the previous options, which essentially focuses on the process and workflow by which a content owner and MAM user, at least, can begin to explore and learn about next-generation video search. It’s unclear whether this option would apply to a MAM vendor itself. We’d certainly love to engage in dialog.

But the key is to attack the problem by shrinking it down to size.

Omnisearch is proud to have active archive customers with hundreds of Terabytes of video on prem and air-gapped from the internet. But decisions to deploy video search across such gargantuan volumes of data can be intimidating.

So, simply think about gradually introducing vector search by processing smaller amounts of content with narrower search use cases and goals.

This could be by focusing on frequently searched assets. Content is often prioritized in workflows and such prioritizations can apply here as well. Or we can also support on-demand embeddings for user queries. Omnisearch systems are lightning-fast and can work at the speed of the news cycle.

And above all, we can focus our efforts and ideas on end-viewer experiences that could drive dollars to your bottom line.

Pros

✅ Low upfront cost—only pay for active assets in a small sandbox.
✅ No need to manage embedding storage or indexing.
✅ Scales with content growth.

Cons

❌ Portions of content will remain unsearchable via vector methods, giving users the sense of using a flashlight as opposed to having strong overhead lighting.

Cost Factors

These are probably less relevant in this scenario since the core idea is to start with a subset of content to minimize costs.

4. Build Your Own: Use of Open-Source or Commercial Vector Database

While I’ve put this at number four, I’m a veteran of our community and I’m sure most readers that have gotten this far probably came to read this one first and foremost.

“Why wouldn’t I build this myself?”

I know, it’s tempting. We are all such innovators in this space. But it’s a bad idea. Why? For the same reason you don’t put money under a mattress. Not only is it less safe, but you lose the benefit of compounding interest.

Just like compounded interest in your savings can add up over the years, so too does the complexity of multimodal search tasks in software engineering quickly compound when indices get moderately large. These create massive, system level challenges that are way beyond simple vector similarity search.

Even selecting the appropriate vector database foundation is challenging.

Milvus offers powerful distributed vector search capabilities but comes with significant deployment complexities. Users struggle with distributed configurations and data inconsistencies like malformed JSON.

Weaviate provides an accessible platform for AI applications but faces challenges with multi-node clusters. Also, startup behavior is often inconsistent across instances.

pgvector for PostgreSQL leverages familiar infrastructure but creates significant resource demands that impact overall database performance. Even small indices of tens of millions of embeddings may require hundreds of gigabytes on disk and aggressive vacuuming to maintain performance.

Added to the above are issues like GPU compatibility, performance degradation, specialized memory optimizations, and careful partitioning strategies you’ll need to implement.

Even beyond these challenges, the core difficulty still lies in meaningfully combining different modalities into a search result. You’ll need to develop a sophisticated ranking system that intelligently weighs signals across all these vectors, whether faces, logos, speech or text.

This becomes especially critical when handling large result sets where simple vector similarity falls short.

The bottom line? Obtaining quality embeddings is just the starting point. Significant engineering effort is required to transform these raw embeddings into high-quality search results that are semantically relevant.

Good luck! Keep us posted. Self-hosting comes with infrastructure, maintenance, and engineering overhead that we in the media business simply can’t afford right now.

Pros of Open-Source or DIY

✅ No vendor lock-in.
✅ Fully customizable.

‍
Cons of Open-Source or DIY

❌ Requires significant engineering effort.
❌ Ongoing infrastructure costs.
❌ No automatic model improvements.

Cost Factors

The engineering development costs for establishing your very own LLM-integrated AI video search platforms would be quite large, even with the fantastic tools available today to work with.

Summary & Conclusion? Call Omnisearch Today

So, there you have it.

You have a decision to make. It’s time to act. And unless you want to brave the cold without proper attire, we’d strongly suggest exploring partner engagements with leading multimodal video search and understanding platforms, like Omnisearch.

Omnisearch eliminates the known and hidden costs of the options above by offering a fully managed search platform, user interface and API that gives you the results you need, the roadmap you want, and the user experience users and  media managers will be delighted by.

→ Omnisearch provides an immediate upgrade with semantic and similarity search without infrastructure investments.

→ Omnisearch can seamlessly work alongside existing keyword-based search, if you decide to move ahead with a hybrid search solution.

→ And Omnisearch eliminates the need for self-hosted vector search frameworks if cost control, speed to market and zero maintenance are your needs.

Omnisearch is the best choice for MAM solutions seeking world-class vector search with low cost, no infrastructure burden, and great usability.

Alternative approaches (open-source, cloud-managed services) involve higher costs, complexity and won’t match our performance.