Introducing Atlas Vector Search: Build Intelligent Applications with Semantic Search and AI Over Any Type of Data

We’re excited to announce Atlas Vector Search now in General Availability. Vector Search now supports production workloads allowing you to continue to build intelligent applications powered by semantic search and generative AI, while optimizing resource consumption and improving performance with Search Nodes.

Read the blog below for the full announcement and list of benefits.

The moment has finally come. Artificial Intelligence has shifted left. What was once built and often trapped inside enterprise-wide data science and machine learning teams is now readily available to builders everywhere. But to harness the incredible power of these new tools, you need to build on top of a reliable, composable, and elegant data platform. At the same time, as we’ve all seen, these new capabilities are only as good as the data or “ground truth” they have access to. That’s why we’re thrilled to be adding yet another capability to the MongoDB Atlas Developer Data Platform to unlock the full potential of your data and power AI Applications. Today, MongoDB is thrilled to announce our exciting new Vector Search capability designed to meet the demands of data in all forms and allow our partners to harness these incredible new capabilities.

What is the capability?

For those of you unfamiliar, Vector Search is a capability that allows you to query your data based on semantics or the meaning of the data rather than the data itself. This is made possible by being able to represent any form of data numerically as a Vector which can then be compared to one another through sophisticated algorithms. The first step is to take source data whether it be text, audio, image, or video, and convert them into “Vectors” or “Embeddings” using an “encoding model.” With recent advances in Artificial Intelligence, these vectors are now better able to capture the meaning of data by projecting lower dimensional data into a higher dimensional space which contains more context about the data. Once this data has been transformed into these numeric representations you can query to find similar values using an Approximate Nearest Neighbors algorithm which allows your queries to very quickly find data with similar vectors. This enables you to satisfy queries like “give me movies with the feeling of sorrow” or “give me images that look like…”. This capability unlocks a whole new class of capabilities.

How does it relate to our platform?

With this functionality natively built into MongoDB Atlas you don’t need to copy and transform your data, learn some new stack and syntax, or manage a whole new set of infrastructure. With MongoDB’s Atlas Vector Search none of this is necessary, you can utilize these powerful new capabilities all within a world class and battle tested platform to build applications faster than ever before. Many of the challenges inherent in harnessing AI and Vector Search stem from the complexity involved in safely and securely exposing your application data. These tasks add layers of friction to the developer experience and make your applications harder to build, debug, and maintain. MongoDB erases all of these challenges while bringing the power of Vector Search to a platform that organically scales vertically and horizontally to support any workload you throw at it. Finally, none of this matters without guarantees around security and availability, and MongoDB’s commitment to a secure data management solution along with high availability through redundancy and automatic failover ensure that your application will never miss a beat.

New at MongoDB.local London

As of .Local London we’re excited to announce the introduction of a dedicated Vector Search aggregation stage that can be invoked via $vectorSearch. This new aggregation stage introduces a few new concepts that add new power and make it easier than ever to utilize Vector Search. With $vectorSearch you can also utilize a pre-filter with MQL syntax (e.g. $gte, $eq, etc…) which filters out documents as you traverse the index leading to consistent results and high performance. Any developer who understands MongoDB will be able to take advantage of this filtering capability with ease! Finally we’re also introducing two ways to tune your results inside the aggregation stage, both a “numCandidates” and “limit” parameter, with these parameters you can tune how many documents should be candidates for the approximate nearest neighbor search and then limit how many results you want with the “limit.”

How does it interact with the ecosystem?

The amount of innovation happening around Artificial Intelligence is astounding, and it’s amazing to see the advances the Open Source community is quickly making. There are huge gains being made in open source Language Models as well as the various methods they can be integrated into applications. With raw power exposed by Artificial Intelligence, it’s never been more important to have a solid abstraction over the capability to give developers the flexibility they need. With this in mind we are thrilled to share that we have several capabilities supported in LangChain and LlamaIndex, from Vector Search support all the way to Chat Logging and document indexing. We’re moving fast here and will continue to release new functionality for the premier providers.

Wrap up

With all of this said, things are just getting started, we’re committed at MongoDB to helping developers power the next generation of AI-enabled Applications with the best Developer Data Platform in the market. We’re also going to be looking into more frameworks and plugin architectures that we can support. But as always, the most important part of this equation is you, the developer. We’re going to be talking to the community and finding the ways we can serve you best and ensure we’re meeting your needs every step of the way. Go forth and build!