Docs Menu
Docs Home
/
MongoDB Atlas
/
Atlas Vector Search
/
Tutorials

How to Perform Hybrid Search

On this page

  • Prerequisites
  • Create the Atlas Vector Search and Atlas Search Indexes
  • Required Access
  • Procedure
  • Run a Combined Semantic Search and Full-Text Search Query
  • About the Query
  • Procedure
  • Learn by Watching

This tutorial demonstrates how to combine the semantic search results from a $vectorSearch query with full-text search results from a $search query by using reciprocal rank fusion. Reciprocal rank fusion is a way to combine results from different types of searches into a single result. This tutorial takes you through the following steps:

  1. Create an Atlas Vector Search index on the plot_embeddings field in the sample_mflix.embedded_movies collection.

  2. Create an Atlas Search index on the title field in the sample_mflix.embedded_movies collection.

  3. Combine and run a $vectorSearch query against the plot_embeddings field in the sample_mflix.embedded_movies collection with a $search query against the title field by using reciprocal rank fusion.

Prerequisites

Before you begin, ensure that your Atlas cluster meets the requirements described in the Prerequisites.

Note

Ensure that your Atlas cluster has enough memory to store both Atlas Search and Atlas Vector Search indexes and run performant queries.

Create the Atlas Vector Search and Atlas Search Indexes

This section demonstrates how to create the following indexes on the fields in the sample_mflix.embedded_movies collection:

  • An Atlas Vector Search index on the plot_embeddings field for running vector queries against that field.

  • An Atlas Search index on the title field for running full-text search against that field.

Required Access

To create Atlas Vector Search and Atlas Search indexes, you must have at least Project Data Access Admin access to the project.

Procedure

1

In Atlas, go to the Clusters page for your project.

  1. If it is not already displayed, select the organization that contains your desired project from the Organizations menu in the navigation bar.

  2. If it is not already displayed, select your desired project from the Projects menu in the navigation bar.

  3. If the Clusters page is not already displayed, click Database in the sidebar.

2

Go to the Atlas Search page for your cluster.

You can go the Atlas Search page from the sidebar, the Data Explorer, or your cluster details page.

  1. In the sidebar, click Atlas Search under the Services heading.

  2. From the Select data source dropdown, select your cluster and click Go to Atlas Search.

  1. Click the Browse Collections button for your cluster.

  2. Expand the database and select the collection.

  3. Click the Search Indexes tab for the collection.

  1. Click the cluster's name.

  2. Click the Atlas Search tab.

3

Click Create Index.

4

Define the Atlas Vector Search index.

  1. Click Create Search Index.

  2. Under Atlas Vector Search, select JSON Editor and then click Next.

  3. In the Database and Collection section, find the sample_mflix database, and select the embedded_movies collection.

  4. In the Index Name field, enter rrf-vector-search.

  5. Replace the default definition with the following index definition and then click Next.

5

Define the Atlas Vector Search index.

  1. Replace the default definition with the following index definition.

    This index definition indexes the plot_embedding field as the vector type. The plot_embedding field contains embeddings created using OpenAI's text-embedding-ada-002 embedding model. The index definition specifies 1536 vector dimensions and measures similarity using euclidean.

    1{
    2  "fields": [
    3    {
    4      "type": "vector",
    5      "path": "plot_embedding",
    6      "numDimensions": 1536,
    7      "similarity": "euclidean"
    8    }
    9  ]
    10}
6

Review the index definition and then click Create Search Index.

A modal window displays to let you know that your index is building.

7

Click Close to close the You're All Set! modal window and wait for the index to finish building.

The index should take about one minute to build. While it builds, the Status column reads Initial Sync. When it finishes building, the Status column reads Active.

8

Click Create Index to define the Atlas Search index.

9

Select an Atlas Search Configuration Method and click Next.

  • For a guided experience, select the Atlas Search Visual Editor.

  • To edit the raw index definition, select the Atlas Search JSON Editor.

10

Enter the Index Name, and set the Database and Collection.

  1. In the Index Name field, enter rrf-full-text-search.

    Note

    If you name your index default, you don't need to specify an index parameter when using the $search pipeline stage. Otherwise, you must specify the index name using the index parameter.

  2. In the Database and Collection section, find the sample_mflix database, and select the embedded_movies collection.

11

Define the Atlas Search index.

The following index definition indexes the title field as the string type for querying the field.

You can use the Atlas Search Visual Editor or the Atlas Search JSON Editor in the Atlas user interface to create the index.

  1. Click Next.

  2. Click Refine Your Index.

  3. In the Index Configurations section, toggle to disable Dynamic Mapping.

  4. In the Field Mappings section, click Add Field to display the Add Field Mapping window.

  5. Select title from the Field Name dropdown.

  6. Select String from the Data Type dropdown.

  7. Click Add.

  1. Replace the default index definition with the following definition.

    1{
    2  "mappings": {
    3    "dynamic": false,
    4    "fields": {
    5      "title": [{
    6        "type": "string"
    7      }]
    8    }
    9  }
    10}

  2. Click Next.

12

Click Create Search Index.

A modal window displays to let you know that your index is building.

13

Close the You're All Set! modal window by clicking Close and wait for the index to finish building.

The index should take about one minute to build. While it is building, the Status column reads Initial Sync. When it is finished building, the Status column reads Active.

Run a Combined Semantic Search and Full-Text Search Query

This section demonstrates how to query the data in the sample_mflix.embedded_movies collection by using the $vectorSearch and $search pipeline stages and combine each document's scores from both stages to re-sort the documents in the results. This ensures that documents appearing in both searches appear at the top of the combined results.

About the Query

The following query retrieves the sorted search results from the semantic search and the full-text search, and assigns a reciprocal rank score to the documents in the results based on their position in the results array. The reciprocal rank score is calculated by using the following formula:

1.0/{document position in the results + vector or full-text penalty + constant value}

The query then adds the scores from both the searches for each document, ranks the documents based on the combined score, and sorts the documents to return a single result.

Query Variables

The sample query defines the following variables to add weight to the score, with a lower number providing higher weight:

  • vector_penalty

  • full_text_penalty

Query Stages

The sample query uses the following pipeline stages:

  • $vectorSearch stage to search the plot_embeddings field for the string new york specified as vector embeddings in the queryVector field of the query. The query uses ada-002-text embedding, which is the same as the vector embedding in the plot_embedding field. The query also specifies a search for up to 100 nearest neighbors and limit the results to 20 documents only. This stage returns the sorted documents from the semantic search in the results.

  • $group stage to group all the documents in the results from the semantic search in a field named docs.

  • $unwind stage to unwind the array of documents in the docs field and store the position of the document in the results array in a field named rank.

  • $addFields stage to add a new field named vs_score that contains the reciprocal rank score for each document in the results. Here, reciprocal rank score is calculated by dividing 1.0 by the sum of rank, the vector_penalty weight, and a constant value of 1.

  • $project stage to include only the following fields in the results:

    • vs_score

    • _id

    • title

  • $unionWith stage to combine the results from the preceding stages with the results of the following stages in the sub-pipeline:

    • $search stage to search for movies that contain the term new york in the title field. This stage returns the sorted documents from the full-text search in the results.

    • $limit stage to limit the output to 20 results only.

    • $group stage to group all the documents from the full-text search in a field named docs.

    • $unwind stage to unwind the array of documents in the docs field and store the position of the document in the results array in a field named rank.

    • $addFields stage to add a new field named fts_score that contains the reciprocal rank score for each document in the results. Here, reciprocal rank score is calculated by dividing 1.0 by the sum of the value of rank, the full_text penalty weight, and a constant value of 1.

    • $project stage to include only the following fields in the results:

      • fts_score

      • _id

      • title

  • $group stage to group the documents in the results from the preceding stages by title, vs_score, and fts_score.

  • $project stage to include only the following fields in the results:

    • vs_score

    • fts_score

    • _id

    • title

  • $project stage to add a field named score that contains the sum of vs_score and fts_score to the results.

  • $sort stage to sort the results by score in descending order.

  • $limit stage to limit the output to 10 results only.

Procedure

1

Connect to your cluster using mongosh.

Open mongosh in a terminal window and connect to your cluster. For detailed instructions on connecting, see Connect via mongosh.

2

Use the sample_mflix database.

Run the following command at mongosh prompt:

use sample_mflix
switched to db sample_mflix
3

Run the following Atlas Search queries against the embedded_movies collection.

1var vector_penalty = 1;
2var full_text_penalty = 10;
3db.embedded_movies.aggregate([
4  {
5    "$vectorSearch": {
6      "index": "rrf-vector-search",
7      "path": "plot_embedding",
8      "queryVector": [-0.0105516575,-0.014830452,0.008882049,-0.04193625,-0.025848519,-0.0006020224,-0.026200015,0.013458264,-0.022779683,-0.036366377,0.021887423,0.0011533482,0.0037211322,0.005840251,-0.010822039,-0.012261824,0.020400321,-0.019994749,0.017236853,-0.011132979,-0.008084422,0.028200842,-0.009963577,-0.014384322,0.00086311,0.0014938605,-0.0019332313,-0.01080852,0.0029556132,-0.01561456,0.013803001,-0.017223334,-0.028714567,-0.02417215,0.006715613,-0.007165123,-0.010727406,-0.012491648,0.007300314,-0.00210053,0.010531379,-0.03139135,-0.0036569166,-0.006577042,-0.026957085,0.010260996,-0.0135393785,-0.006401294,-0.024239747,0.0108896345,0.017223334,0.021508887,-0.013938192,0.0022576896,-0.03187804,0.005735478,-0.02148185,-0.0045559364,-0.011396601,0.00010197415,-0.007435505,-0.018426534,-0.018859144,0.029687943,0.0009699954,-0.024253266,-0.005272449,0.012667396,0.003501447,0.01593902,0.032256573,-0.005002067,0.0037650694,-0.00025200448,0.013525859,-0.00007509437,-0.013613733,0.012620079,-0.0013265617,0.010173122,0.012795828,-0.043423347,-0.025159044,0.026172977,0.018318381,0.011937365,-0.032986604,0.018588763,-0.00857111,-0.005691541,0.021252025,0.02146833,0.026091862,0.0016290515,-0.035987843,0.03228361,-0.012511927,0.034906317,-0.0150062,-0.005752377,0.0064181928,-0.01126141,-0.0030079996,-0.023834173,-0.0045086197,0.00857111,-0.018047998,-0.02898495,0.013444745,-0.017250372,-0.01321492,0.016912393,0.009395774,-0.0405573,-0.007976269,-0.025726847,0.009943298,-0.010686848,-0.028552338,-0.037934594,0.008152017,0.008381842,0.014925086,-0.024753472,0.041828096,0.00078875496,0.00059991004,-0.006746031,-0.013688088,-0.005225132,0.019021373,-0.0012057347,0.004718166,0.006803487,-0.020292168,0.019170083,-0.03139135,0.009679675,-0.02359083,-0.03455482,0.0046640895,0.029796096,-0.004910813,0.01666905,-0.006272862,0.015357697,0.015830865,0.0130864885,0.008577869,-0.018710434,0.007415226,-0.018642839,0.008915846,0.0051947143,-0.007530139,0.024712915,0.0025449705,0.004640431,0.0078005204,0.011748098,-0.006262723,-0.010855837,0.027551925,-0.03122912,0.014384322,0.017764097,0.016209401,0.005542831,0.014911567,-0.0060768356,0.0015090695,0.030769471,-0.0141544975,0.014938605,0.015236026,0.017669464,0.014046345,0.02146833,-0.016642012,0.00039923593,0.0043362514,0.015587522,0.015519926,0.0381509,-0.0014980852,0.0055056536,0.020630145,-0.017399082,-0.00039754604,-0.018075036,-0.0077261655,0.01816967,0.02041384,-0.008422399,-0.67098,-0.015722713,-0.004549177,-0.006590561,0.018507648,0.012951298,0.019143045,0.007117806,-0.023158219,-0.011356044,-0.013573176,0.0037312715,0.013458264,-0.01785873,-0.01457359,-0.0046235323,0.021671116,-0.008131739,-0.020927567,0.009395774,-0.017155739,0.0016712988,-0.009389015,-0.0019264717,0.017412601,0.008848251,-0.0009801347,-0.008679262,-0.011957644,0.0022948673,-0.043612614,0.012342938,0.023212295,-0.01729093,0.04853357,-0.0012657257,-0.0342574,-0.00240133,0.029011989,0.015952539,-0.0018605661,-0.0021596763,0.012762031,-0.0061917477,0.0020768717,0.0056983004,0.020616626,-0.004532278,0.005718579,0.005752377,0.01996771,0.0026379144,-0.022117248,-0.0073679094,0.01623644,-0.003994894,0.009253824,-0.01638515,0.01757483,0.0017946605,-0.0018487369,-0.008550831,-0.030715395,-0.0131743625,-0.015533445,0.0061816084,-0.007820799,0.0009480269,-0.00051710557,-0.011876529,-0.0007722786,0.034960393,-0.017182777,0.0049953074,0.012430812,0.027430253,0.011139738,-0.00083142467,0.005076422,-0.009956817,0.019994749,0.010490822,-0.017926326,0.017385563,0.01157235,-0.019332312,-0.02928237,0.0062694824,-0.00058385613,-0.0024892043,0.00840888,0.015411774,-0.017966883,-0.014708781,-0.0070840083,0.0053400444,0.009585042,0.010037932,0.015425293,-0.007070489,-0.0039272984,-0.022563377,-0.0069893748,0.0020025168,0.0013054381,0.0038225255,0.009618839,0.010416467,0.029093103,-0.038394243,0.009247065,-0.015695674,-0.04307185,-0.01396523,0.025794443,-0.031769883,0.028714567,0.0016324313,0.01246461,-0.011971163,0.0060903546,-0.013106767,0.01742612,-0.018805068,0.010734165,0.017155739,0.014492475,-0.021197949,-0.018467091,0.005089941,-0.010423226,0.020846453,0.017358525,-0.008138498,0.01757483,0.0057084397,0.009733752,-0.010923433,-0.004157123,-0.024658838,-0.025253678,-0.0015673706,-0.00871982,-0.0155469645,-0.0049513704,-0.024658838,-0.01665553,-0.016871836,-0.041584753,0.012667396,-0.01800744,-0.021252025,-0.036771953,0.011018067,-0.011132979,-0.019494543,-0.0009497168,-0.024848105,0.012180709,0.00105111,0.01849413,0.026930047,-0.03290549,-0.00886177,0.018210227,-0.011626426,0.023658425,0.019819,-0.0022188223,-0.017061103,0.003680575,-0.005360323,0.006688575,0.006928539,0.023090623,0.019170083,-0.004975029,0.0027849346,-0.002884638,-0.00033164042,-0.00069031905,-0.015100835,-0.013992269,0.0069893748,-0.0011372942,-0.00006242022,0.008300727,0.02252282,-0.021454811,0.0026801615,-0.021846866,0.01065981,-0.02148185,0.009618839,-0.010943712,-0.009558003,0.00780728,0.012160431,0.0018842246,0.022198362,0.033121794,-0.022414668,0.004042211,-0.0019535099,-0.020859972,-0.029498676,0.010132565,-0.018994335,0.0146817425,0.0028694288,0.016777202,-0.023793615,0.008577869,0.0062694824,0.027714156,0.049209524,-0.019102488,0.0042111995,0.008280449,0.015668636,-0.010795001,-0.027565444,-0.0004302876,0.009639118,-0.01396523,0.013884116,0.01051786,0.029796096,0.0072529973,-0.021062758,-0.021873904,0.0051947143,0.0053265253,0.0074287453,-0.018561725,-0.0062356847,0.002435128,-0.029660905,0.02223892,-0.007888394,0.015263064,-0.008882049,0.03531189,0.00062990555,0.002029555,0.0035927007,0.03406813,0.006472269,-0.02118443,0.028173804,-0.012843145,-0.010423226,-0.01232266,-0.014560071,0.010632772,-0.011396601,0.019318793,-0.01576327,0.015384736,0.030174632,0.003364566,-0.006550004,0.017939845,-0.0012961437,0.0022002335,-0.012059037,0.027565444,-0.0069488175,-0.0030536267,0.007719406,-0.003707613,-0.00097084034,0.034446668,-0.013681329,0.01320816,-0.034527782,-0.0060159992,-0.016398668,-0.0010604043,0.0000769955,-0.013154084,-0.02928237,0.016317554,0.015857903,-0.015141392,-0.01680424,0.003829285,-0.004326112,-0.0034811683,0.060187034,0.008503513,0.026808375,-0.00065863365,-0.008996961,-0.0013899325,0.00043641345,0.03587969,-0.015330659,0.004988548,-0.026078343,0.0077532036,-0.006249204,-0.025632214,0.0039712354,0.03471705,0.008530552,0.013715127,-0.0089564035,-0.0022188223,-0.008604907,-0.001997447,0.0010468853,-0.006789968,-0.0007300314,-0.011748098,-0.018210227,-0.009287622,0.009672916,0.008740098,-0.010544898,-0.005786175,-0.03204027,-0.018223746,-0.01653386,0.091929875,-0.0019585795,-0.00009811909,0.015709193,0.015830865,-0.0034591996,-0.010315073,0.0025280716,0.01622292,-0.010429986,0.01607421,-0.029471638,0.021441292,0.0054955143,-0.012532205,-0.0016417257,-0.0033814649,-0.020062344,0.014762857,-0.009895981,-0.0018250785,-0.005029105,0.015817346,0.039692078,-0.004549177,0.004910813,0.038799815,0.007415226,0.010443505,-0.016425706,-0.011470956,0.017250372,0.008043865,0.016439226,-0.0024266783,-0.013397428,0.022603935,0.0026649525,0.008976682,-0.007029932,0.022630973,-0.0013341662,0.0014820313,-0.02822788,0.0195351,-0.004988548,-0.0285253,0.008300727,-0.01785873,-0.007043451,0.049155448,-0.0036535368,-0.008043865,-0.0060497974,-0.004897294,0.012329419,-0.015871422,-0.035690423,0.002342184,-0.002359083,-0.012822866,-0.033256985,-0.00021609435,-0.005752377,-0.018223746,-0.028714567,-0.016006615,0.0034913076,-0.037474945,-0.014032826,0.010828799,-0.019994749,-0.0047114063,-0.0043193526,0.021089796,-0.022482263,0.012410534,0.01320816,0.020332726,0.02117091,0.001982238,-0.031742845,0.005363703,0.008104701,0.004086148,0.017615387,0.026470397,-0.033743672,-0.0021360177,0.022265958,0.0060430374,0.00040155952,0.016195882,0.014330246,0.005887568,-0.028795682,0.011227612,0.022252439,0.0020768717,-0.03360848,0.016696088,-0.017547792,0.006357357,-0.00705697,0.016628493,0.025172563,0.008523792,0.021400735,-0.032499917,-0.0069893748,0.034635935,-0.028038613,0.011484475,0.019737886,-0.015114354,0.023036547,0.005904467,0.015479369,-0.006144431,-0.0057794154,0.021887423,-0.04323408,0.0037346513,0.03049909,0.0060295183,0.008625186,-0.0036873345,0.004413986,-0.012593041,-0.003322319,-0.0066243587,0.00570506,-0.013478543,-0.015060278,-0.028714567,-0.011450677,-0.0061038737,0.008212853,-0.0040760087,-0.027416734,0.0034017435,-0.018413015,0.013654291,-0.005329905,-0.026794856,-0.019521581,0.015411774,0.0013924673,-0.012478129,0.008469716,-0.01680424,0.0029268852,-0.02433438,-0.008239891,-0.004988548,-0.020224573,-0.023536753,-0.008942884,0.018075036,0.018859144,0.026794856,0.010727406,-0.0013502201,0.004116566,-0.012572763,0.010862596,-0.019426946,0.030255746,-0.0073543903,0.018467091,0.025334792,0.006833905,-0.011051864,0.005992341,-0.018142631,0.018805068,0.0075368984,-0.013106767,-0.018899702,-0.032121383,-0.010139325,0.011504754,-0.018845625,-0.00061216176,-0.025618695,-0.008665743,0.018264303,-0.0113290055,0.009828386,0.00078326283,0.015290102,-0.016479783,0.01457359,0.008679262,-0.004971649,-0.007104287,-0.01608773,-0.004160503,0.005512413,0.025726847,0.021955019,0.00021630559,-0.0133636305,0.003974615,-0.0050054467,0.012538965,-0.0047857612,-0.015411774,0.016141806,-0.03033686,-0.008631946,0.0042416174,-0.0020194156,-0.0267543,-0.010423226,0.018115593,-0.02763304,0.019940672,-0.030553166,-0.01574975,-0.010017653,0.0051879548,0.011132979,0.0037549299,0.010531379,0.0005052764,-0.0041199457,-0.014384322,-0.000210391,0.006844044,-0.013241958,0.014384322,0.008983442,-0.014478956,-0.023509715,0.0013637393,-0.011673743,0.016020134,-0.014708781,0.027213948,0.01907545,0.00225262,-0.02868753,-0.024280304,0.010166363,0.013390669,-0.001786211,0.013356871,0.000743128,-0.008882049,-0.014208574,0.018534686,-0.03301364,0.025578137,0.014451918,-0.025294235,-0.011633186,-0.013323073,-0.0011744718,-0.0005673797,0.010585455,0.019602695,0.011335765,0.0070096534,-0.0059112264,-0.0056070467,-0.02400992,0.015438812,-0.012248305,0.0237125,-0.02626761,-0.00210222,-0.001411901,0.0006020224,0.016141806,0.021684635,-0.0016417257,-0.021346658,-0.016398668,-0.0034439906,0.025104968,0.011220853,-0.012342938,-0.0026243953,-0.021684635,-0.011484475,-0.0046066334,0.014438398,0.01051786,-0.0036704356,-0.0039881347,-0.010315073,-0.0052656895,0.015844384,-0.0031009435,-0.006830525,0.03617711,0.019007854,-0.0011753167,-0.007002894,-0.0071245655,0.008638705,-0.03590673,-0.0044815815,-0.01772354,-0.015736232,0.012721473,-0.01576327,-0.00690826,-0.009341698,0.013505581,0.009977096,0.016939431,0.00090155494,0.043774847,-0.005029105,0.0012302381,-0.027159872,-0.006353977,0.018710434,-0.014032826,0.016128287,0.00871982,-0.000052861793,0.034960393,0.009699954,0.012092835,0.010470543,-0.03214842,-0.019278236,-0.012701195,0.024131592,-0.008814453,0.0030316582,-0.004860116,-0.018467091,-0.009138912,0.02479403,0.00525555,-0.018980816,-0.015087316,0.039421696,0.033148833,0.010024413,-0.010977509,-0.020440878,-0.006245824,0.0017219953,-0.04223367,-0.012451091,0.0005977977,0.029904248,0.008699541,-0.024212709,-0.033554405,0.0033459773,-0.023023028,-0.0090442775,0.0017084762,0.024145111,0.018534686,-0.008476475,-0.020954605,-0.005042624,0.01141012,0.020075863,0.009192988,0.007543658,0.027295062,-0.028741606,0.027957499,0.005167676,-0.015790308,0.010240718,0.015790308,-0.0074084667,0.0018064898,0.00780728,-0.011457437,-0.017628906,-0.014627666,0.00067933474,-0.0074084667,0.024942739,0.029471638,-0.04372077,0.0029995502,0.017669464,-0.013431226,-0.0029116762,0.012505167,-0.0034355412,-0.0042923144,0.0022458604,-0.025902595,-0.005928125,0.0074693025,-0.023212295,0.021671116,0.013721886,-0.01638515,0.0270382,0.0038867411,0.007090768,-0.014384322,-0.013323073,-0.031012814,-0.0019889975,0.025469985,-0.028849758,-0.01607421,-0.011139738,0.020292168,-0.025605176,0.008199334,0.0049953074,-0.009618839,-0.011227612,0.002614256,0.017696502,-0.0017794515,0.00059399544,0.0071516037,-0.0077870013,-0.005181195,0.00031495278,0.010220439,-0.0073341117,0.026483916,0.010936952,-0.0056847814,-0.050967008,-0.017682983,0.01576327,0.008976682,0.004234858,0.20451695,0.025267197,0.006205267,0.04012469,-0.020292168,0.02268505,0.025524061,0.004025312,-0.0034980671,-0.010720646,-0.016006615,0.0019906876,-0.014032826,-0.0064959275,-0.008388601,-0.019318793,-0.02027865,-0.0333381,-0.017899288,-0.012789069,0.0041807815,0.00077565835,-0.0074422644,-0.014181536,0.035365965,0.0062120263,-0.010923433,0.012566003,0.023766577,0.011187055,-0.0037042333,-0.0037278917,0.020603107,0.00720568,-0.013471783,-0.0054921345,-0.014127459,-0.0092335455,0.01457359,0.013654291,0.0034034334,-0.0037954873,0.0037109929,-0.003036728,-0.014073383,0.037204564,0.009341698,-0.009963577,0.020968124,0.018737473,-0.030120555,-0.017980402,0.020008268,0.04277443,0.0091456715,0.0054008802,0.0146817425,0.0018723953,0.00025137077,0.00003131573,-0.020521993,0.022630973,-0.020549031,0.0048398376,-0.011558831,0.0023540133,-0.018115593,0.014019307,0.0003785348,-0.0022255818,0.010294794,-0.010957231,-0.013674569,-0.0074219857,-0.015236026,-0.031012814,0.03247288,0.042882584,-0.0022019234,0.0123091405,-0.009956817,0.010092008,0.0285253,-0.010774722,0.005928125,-0.036717877,0.0011279999,0.0030603863,0.009301141,0.008483235,0.0070231725,0.0026666424,-0.016682569,-0.009841905,0.004295694,0.0067696893,0.0037887278,0.011842731,-0.018196708,0.0074219857,-0.009740512,-0.03487928,0.019724367,0.009084835,-0.0015496268,0.0030248987,0.0075909747,0.0049074334,-0.027443772,-0.035555232,-0.0034406108,-0.0183319,-0.0035994602,0.0047553433,0.026970604,0.0037684492,0.029633867,-0.013127046,0.010315073,-0.020075863,0.0052927276,-0.022320034,-0.018696915,-0.00048373028,-0.0014507684,-0.009713473,-0.012261824,0.0007921347,0.0067933477,-0.032391764,-0.0019383009,0.0064621298,0.0016772134,-0.014492475,-0.0021850246,0.028417148,0.0021900942,-0.029039027,-0.004545797,0.0017946605,0.017196296,0.011322246,0.008841491,0.021211468,-0.0077261655,-0.0023776717,0.01666905,0.008293968,0.0035318648,-0.008199334,-0.009983855,-0.01607421,0.0021731954,-0.020900529,0.015344178,-0.030472051,-0.035771538,-0.025321273,-0.022644492,0.036609724,-0.011072143,0.009524206,0.026510954,-0.015952539,-0.021968538,-0.00976755,-0.17564015,0.025483504,0.04163883,-0.0074828216,0.016858317,0.018886182,-0.005948404,-0.012511927,-0.0004121213,0.012146912,0.0366638,-0.0046674693,-0.031607654,-0.016912393,0.0022053032,-0.01681776,-0.012491648,-0.000077734825,0.0044984804,0.004741824,0.017196296,-0.011497995,0.037934594,-0.034014054,-0.01321492,0.014195055,0.004315973,0.019116007,-0.002867739,-0.0101866415,-0.026848933,-0.011639945,0.0059382645,0.0044241254,0.027416734,0.020981643,-0.010936952,-0.018656358,-0.014492475,0.017182777,0.0017372044,0.024699396,0.016763683,0.013309554,-0.0030705256,0.029390523,0.0300124,-0.0018098695,-0.009267343,-0.004370049,0.014749338,-0.017953364,-0.0014490786,0.009652637,0.0034591996,0.0030434874,0.0017101661,0.0031938874,0.007192161,-0.0195351,-0.027092276,-0.006874462,0.009138912,-0.0002828449,-0.01816967,-0.014046345,-0.018372457,0.0024114694,-0.028930875,0.024739953,0.013106767,-0.036609724,0.0150197195,-0.02148185,0.031715807,0.018372457,-0.017088141,0.011497995,-0.007219199,-0.010558417,-0.012822866,0.033446252,-0.018980816,-0.014465437,0.007983029,0.003873222,-0.0055631096,-0.0029927907,-0.011173536,-0.015087316,-0.0076112533,-0.027957499,0.00885501,-0.025361832,0.005647604,0.0038495637,-0.011788655,-0.0074557834,-0.02207669,0.012444331,0.029633867,0.007759963,-0.030472051,0.035041507,0.030282784,0.017899288,-0.0031905076,0.019413427,0.011545312,-0.015425293,-0.025294235,-0.008104701,0.026402801,0.013620493,-0.002717339,0.0037312715,0.0129242595,-0.007557177,0.0003821258,-0.00049724936,0.045234907,0.00063708756,-0.011470956,0.036528606,0.0016391908,-0.016128287,-0.0859274,-0.02026513,0.02702468,0.016777202,-0.017182777,0.0420444,0.0032952805,0.013012134,-0.00076678646,0.029850172,0.008415639,-0.016560897,0.00720568,-0.010842318,0.033121794,0.013992269,0.004772242,-0.027254505,-0.010578696,0.017507235,-0.015641598,0.0033628761,-0.010754444,-0.016425706,-0.00012293931,0.0064959275,-0.018602282,0.01546585,0.010166363,0.016709607,-0.009476889,-0.01756131,-0.015871422,-0.010004134,-0.015100835,0.0056104264,-0.031445425,0.010071729,0.010544898,-0.043774847,-0.00020711683,0.010078489,-0.004373429,-0.02056255,-0.010423226,-0.011335765,0.015330659,-0.0021546066,0.012829626,-0.0063810153,-0.005289348,-0.0059991004,-0.004562696,0.014614147,-0.009348458,-0.02238763,-0.015411774,0.015276583,-0.017317967,-0.0056543634,-0.012843145,-0.0070840083,-0.019873077,0.017682983,0.0033138692,0.011308727,-0.022171324,-0.031742845,0.015114354,-0.020954605,-0.021603521,0.010085248,-0.0300124,-0.00285253,-0.0390702,-0.011903567,-0.014289688,-0.018413015,0.019819,-0.035933767,-0.03439259,-0.027768232,0.009929779,0.0071110465,0.031310234,0.020968124,0.0091456715,-0.022130767,0.0020802515,-0.023901768,0.008631946,0.03136431,0.002223892,-0.04023284,-0.0075504174,0.017588349,0.0036873345,-0.014140978,0.012856664,0.0020616627,-0.010605734,-0.01036915,-0.07100231,0.012545724,-0.0143167265,-0.017304448,0.040638413,-0.0063945344,-0.0042145792,-0.021671116,-0.0014592179,0.0075909747,-0.042152554,-0.010132565,0.006671676,0.01172106,-0.019751405,-0.030715395,0.025835,0.018791549,0.010653051,0.021211468,0.023388043,0.004025312,0.016763683,-0.009862184,-0.02373954,0.000056558423,-0.015263064,0.016398668,0.005546211,-0.010930193,0.020102901,-0.035798576,0.009057797,0.043801885,0.00039712357,-0.015087316,0.02131962,0.026497435,0.034473706,0.0089564035,-0.009787829,-0.0075639365,0.003815766,-0.016979989,-0.009943298,0.004072629,0.007530139,-0.0028829481,0.021279063,0.018129112,0.019291755,0.030526128,0.0016434155,-0.026145939,-0.012329419,-0.030363899,0.024077516,0.008800934,-0.006742651,0.011808934,0.02506441,0.012491648,0.007834318,0.022252439,0.014695262,-0.014330246,-0.021441292,-0.0027088895,0.026835414,-0.026686704,-0.0018808448,0.0011474336,0.02569981,-0.001518364,0.023834173,-0.017642425,0.013032412,-0.00630328,-0.029552752,0.023969363,0.008787415,-0.0086184265,-0.004610013,0.022590416,0.004931092,0.0007553797,-0.018967297,-0.0019889975,-0.0037008536,0.015668636,-0.005529312,-0.015371216,0.00645875,-0.00031579772,0.009977096,0.011815693,0.005569869,-0.009476889,0.01426265,0.023509715,0.018669877,-0.017777616,-0.014641185,-0.019913634,-0.011802174,0.025510542,0.0026091863,-0.011315486,-0.013992269,0.012342938,0.031932116,0.021211468,0.00871982,0.026105382,-0.01787225,0.020616626,-0.013857078,-0.033554405,-0.033824787,0.0135393785,0.016777202,0.02175223,0.012674156,-0.007841078,0.018210227,0.0076044938,-0.0074287453,-0.0068541835,0.02898495,-0.0045221387,0.025916114,0.0050561433,-0.023117661,-0.0012031998,-0.020346245,0.009334939,-0.0054279184,0.0031736088,-0.010653051,0.040151726,-0.001487101,0.008544071,0.004386948,0.015601041,0.037501983,0.013647531,0.010092008,-0.027876385,-0.018696915,0.017493716,0.013296035,0.012200988,-0.018886182,-0.013992269,-0.0021579864,0.0017811414,0.02207669,-0.019183602,0.011585869,0.026565032,0.010747684,0.024280304,-0.016114768,-0.01922416,0.004444404,0.01574975,0.0092335455,-0.008368323,-0.029309409,0.011173536,-0.00067173026,-0.012856664,-0.027146352,0.009111873,-0.014898048,-0.02822788,0.00811822,0.02087349,0.03244584,-0.00024080896,0.022603935,-0.019994749,-0.0014609077,0.018196708,-0.018075036,-0.0022728986,0.0028711187,-0.022739125],
9      "numCandidates": 100,
10      "limit": 20
11    }
12  }, {
13    "$group": {
14      "_id": null,
15      "docs": {"$push": "$$ROOT"}
16    }
17  }, {
18    "$unwind": {
19      "path": "$docs", 
20      "includeArrayIndex": "rank"
21    }
22  }, {
23    "$addFields": {
24      "vs_score": {
25        "$divide": [1.0, {"$add": ["$rank", vector_penalty, 1]}]
26      }
27    }
28  }, {
29    "$project": {
30      "vs_score": 1, 
31      "_id": "$docs._id", 
32      "title": "$docs.title"
33    }
34  },
35  {
36    "$unionWith": {
37      "coll": "movies",
38      "pipeline": [
39        {
40          "$search": {
41            "index": "rrf-full-text-search",
42            "phrase": {
43              "query": "new york",
44              "path": "title"
45            }
46          }
47        }, {
48          "$limit": 20
49        }, {
50          "$group": {
51            "_id": null,
52            "docs": {"$push": "$$ROOT"}
53          }
54        }, {
55          "$unwind": {
56            "path": "$docs", 
57            "includeArrayIndex": "rank"
58          }
59        }, {
60          "$addFields": {
61            "fts_score": {
62              "$divide": [
63                1.0,
64                {"$add": ["$rank", full_text_penalty, 1]}
65              ]
66            }
67          }
68        },
69        {
70          "$project": {
71            "fts_score": 1,
72            "_id": "$docs._id",
73            "title": "$docs.title"
74          }
75        }
76      ]
77    }
78  },
79  {
80    "$group": {
81      "_id": "$title",
82      "vs_score": {"$max": "$vs_score"},
83      "fts_score": {"$max": "$fts_score"}
84    }
85  },
86  {
87    "$project": {
88      "_id": 1,
89      "title": 1,
90      "vs_score": {"$ifNull": ["$vs_score", 0]},
91      "fts_score": {"$ifNull": ["$fts_score", 0]}
92    }
93  },
94  {
95    "$project": {
96      "score": {"$add": ["$fts_score", "$vs_score"]},
97      "_id": 1,
98      "title": 1,
99      "vs_score": 1,
100      "fts_score": 1
101    }
102  },
103  {"$sort": {"score": -1}},
104  {"$limit": 10}
105])
[
  { _id: 'Pixels', vs_score: 0.5, fts_score: 0, score: 0.5 },
  {
    _id: 'Fighting',
    vs_score: 0.3333333333333333,
    fts_score: 0,
    score: 0.3333333333333333
  },
  {
    _id: 'A Most Violent Year',
    vs_score: 0.25,
    fts_score: 0,
    score: 0.25
  },
  {
    _id: 'Mistletoe Over Manhattan',
    vs_score: 0.2,
    fts_score: 0,
    score: 0.2
  },
  {
    _id: 'End of Days',
    vs_score: 0.16666666666666666,
    fts_score: 0,
    score: 0.16666666666666666
  },
  { _id: 'Sam', vs_score: 0.125, fts_score: 0, score: 0.125 },
  {
    _id: 'Rumble in the Bronx',
    vs_score: 0.1111111111111111,
    fts_score: 0,
    score: 0.1111111111111111
  },
  {
    _id: 'Escape from New York',
    vs_score: 0.047619047619047616,
    fts_score: 0.05555555555555555,
    score: 0.10317460317460317
  },
  { _id: 'The Other Guys', vs_score: 0.1, fts_score: 0, score: 0.1 },
  {
    _id: 'Sky Captain and the World of Tomorrow',
    vs_score: 0.09090909090909091,
    fts_score: 0,
    score: 0.09090909090909091
  }
]

If you sort the results in ascending order by replacing the value of score on line 103 with 1, Atlas Vector Search returns the following results:

[
  {
    _id: '2 Days in New York',
    vs_score: 0,
    fts_score: 0.03333333333333333,
    score: 0.03333333333333333
  },
  {
    _id: 'An Englishman in New York',
    vs_score: 0,
    fts_score: 0.034482758620689655,
    score: 0.034482758620689655
  },
  {
    _id: 'New York, I Love You',
    vs_score: 0,
    fts_score: 0.03571428571428571,
    score: 0.03571428571428571
  },
  {
    _id: 'New York: A Documentary Film',
    vs_score: 0,
    fts_score: 0.037037037037037035,
    score: 0.037037037037037035
  },
  {
    _id: 'A Couch in New York',
    vs_score: 0,
    fts_score: 0.038461538461538464,
    score: 0.038461538461538464
  },
  {
    _id: 'Sherlock Holmes in New York',
    vs_score: 0,
    fts_score: 0.04,
    score: 0.04
  },
  {
    _id: 'A King in New York',
    vs_score: 0,
    fts_score: 0.041666666666666664,
    score: 0.041666666666666664
  },
  {
    _id: 'Live from New York!',
    vs_score: 0,
    fts_score: 0.043478260869565216,
    score: 0.043478260869565216
  },
  {
    _id: 'Sleepless in New York',
    vs_score: 0,
    fts_score: 0.045454545454545456,
    score: 0.045454545454545456
  },
  {
    _id: 'Gangs of New York',
    vs_score: 0,
    fts_score: 0.047619047619047616,
    score: 0.047619047619047616
  }
]

Learn by Watching

Watch a demonstration of an application that showcases hybrid search queries combining Atlas Search full-text and vector search to return a single merged result set. The application implements Relative Score Fusion (RSF) and Reciprocal Rank Fusion (RRF) to return a merged set created by using a rank fusion algorithm.

Duration: 2.43 Minutes

Back

Semantic Search for Text

Next

AI Integrations

On this page