Key-Value Databases

Key value databases, also known as key value stores, are database types where data is stored in a “key-value” format and optimized for reading and writing that data. The data is fetched by a unique key or a number of unique keys to retrieve the associated value with each key. The values can be simple data types like strings and numbers or complex objects.

MongoDB covers a wide range of database examples and use cases, supporting key-value pair data concepts. With its flexible schema and rich query language with secondary indexes, MongoDB is a compelling store for “key-value” data. Learn more in this article and try it with MongoDB Atlas, MongoDB’s Database-as-a-Service platform.

What is a key-value database?

Over the years, database systems have evolved from legacy relational databases storing data in rows and columns to NoSQL distributed databases allowing a solution per use case. Key-value pair stores are not a new concept and were already with us for the last few decades. One of the known stores is the old Windows Registry allowing the system/applications to store data in a “key-value” structure, where a key can be represented as a unique identifier or a unique path to the value.

Data is written (inserted, updated, and deleted) and queried based on the key to store/retrieve its value.

How do key-value databases work?

A key-value database, AKA key-value store, associates a value (which can be anything from a number or simple string to a complex object) with a key, which is used to keep track of the object. In its simplest form, a key-value store is like a dictionary/array/map object as it exists in most programming paradigms, but which is stored in a persistent way and managed by a Database Management System (DBMS).

Key-value databases use compact, efficient index structures to be able to quickly and reliably locate a value by its key, making them ideal for systems that need to be able to find and retrieve data in constant time. Redis, for instance, is a key-value database that is optimized for tracking relatively simple data structures (primitive types, lists, heaps, and maps) in a persistent database. By only supporting a limited number of value types, Redis is able to expose an extremely simple interface to querying and manipulating them, and when configured optimally is capable of high throughput.

What are the features of a key-value database?

A key-value database is defined by the fact that it allows programs or users of programs to retrieve data by keys, which are essentially names, or identifiers, that point to some stored value. Because key-value databases are defined so simply, but can be extended and optimized in numerous ways, there is no global list of features, but there are a few common ones:

• Retrieving a value (if there is one) stored and associated with a given key
• Deleting the value (if there is one) stored and associated with a given key
• Setting, updating, and replacing the value (if there is one) associated with a given key

Modern applications will probably require more than the above, but this is the bare minimum for a key-value store.

When to use a key-value database

There are several use-cases where choosing a key value store approach is an optimal solution:

• Real time random data access, e.g., user session attributes in an online application such as gaming or finance.
• Caching mechanism for frequently accessed data or configuration based on keys.
• Application is designed on simple key-based queries.

MongoDB as a key-value store

MongoDB stores data in collections, which are a group of BSON (Binary JSON) documents where each document is essentially built from a field-value structure. The ability of MongoDB to efficiently store flexible schema documents and perform an index on any of the additional fields for random seeks makes it a compelling key-value store.

{
    session_id : "ueyrt-jshdt-6783-utyrts",
      create_time : 1122384666000
}

Further, MongoDB’s document values allow nested key-value structures, allowing not only for accessing data by key in a global sense, but accessing and manipulating data associated with keys within documents, and even creating indexes that allow fast retrieval by these secondary kinds of keys.

{
    name: "John",
      age : 35,
      dob : ISODate("01-05-1990"),
      profile_pic : "https://example.com/john.jpg",
      social : {
             twitter : "@mongojohn",
                 linkedin : "https://linkedin.com/abcd_mongojohn"
               }
}

MongoDB’s native drivers support multiple top used languages like Python, C#, C++, and Node.js, allowing you to store the key value data in your language of choice.

Secondary indexes to support key value

Each one of the fields can be Indexed based on your query patterns. For example, if we seek for a specific sessionid as the key and the createtime as a value, we can index db.sessions.createIndex({session_id : 1}) and query on that key:

db.sessions.find({session_id : "ueyrt-jshdt-6783-utyrts" },{create_time : 1}).create_time;

Wild card indexes to support key value

Wild card indexing allows users to index every field or a subset of fields in a MongoDB collection. Therefore, if we have a set of field-value types stored in a single document and queries could come dynamically for each identifier, we can create a single index for those field value sets.

db.profiles.createIndex({"$**" : 1 });

As a result, our queries will have a full per field-value query supported by this index. Having said that, wild card indexing should only be used in use cases when we cannot predict the field names upfront and the variety of the queries predicates require so. See wild card restrictions for more information.

Schema design to support key value

Since MongoDB documents can be complex objects, applications can use a schema design to minimize index footprints and optimize access for a “key-value” approach. This design pattern is called the Attribute Pattern and it utilizes arrays of documents to store a “key-value” structure.

attributes: [
        {
        key: "USA",
        value: ISODate("1977-05-20T01:00:00+01:00")
        },
        {
        key: "France",
        value: ISODate("1977-10-19T01:00:00+01:00")
        },
        {
        key: "Italy",
        value: ISODate("1977-10-20T01:00:00+01:00")
        },
        {
        key: "UK",
        value: ISODate("1977-12-27T01:00:00+01:00")
        },
        ... 
    ]

Indexing {attributes.key : 1 , attributes.value : 1} will allow us to search on any key with just one index.

Key-value database vs cache

Databases supporting key-value stores persist the data to a disk serving the database files, while a key-value cache implementation will mostly keep the data loaded in memory. In case of a server fault or restart, the data needs to be preloaded into the cache as it was not persistent.

MongoDB uses the cache of its WiredTiger engine to optimize data access and read performance together with strong consistency and high availability across replica sets. This allows for more resilient and available field-value stores while still using the best performance of cached data.

Advantages of key-value databases

A key-value approach allows defining efficient and compact data structure to access data in a simple form of a key-value fetch/update/remove.

MongoDB documents can form compact flexible structures to support fast indexing for your key-value stores. On the other hand, MongoDB documents may consist of rich objects which can contain entire hierarchies and sub-values, and sophisticated indexing allows documents to be retrieved by any number of different keys.

Summary

Key-value stores are used for use cases where applications will require values to be retrieved fast via keys, like maps or dictionaries in programming languages. The compact structure and straightforward indexing and seeking through those indexes makes this database concept a win for specific application workloads.

However, modern applications will probably require more than just a key-value retrieval and this is where MongoDB and MongoDB Atlas offer the optimal solution. MongoDB can support the field-value store solution while allowing complex objects to be formed and multiple ways to query the data: Full-Text Search, Aggregation Framework, Atlas Data Tiering, or Scaling it across multiple shards.

Try MongoDB Atlas as your key-value database and reveal new possibilities to innovate your applications.