Online Archive.
Tier your MongoDB Atlas data, query it in place.

The amount of data that applications generate is growing exponentially, introducing cost and infrastructure complexity for many companies today. MongoDB Atlas makes it easy to manage your entire data lifecycle without replicating or migrating it across multiple systems.

With MongoDB Atlas Online Archive, you can seamlessly tier your data across fully managed databases and cloud object storage, all while retaining the ability to query it through a single endpoint. Create a rule to automatically archive infrequently accessed data from your live MongoDB Atlas clusters to fully managed cloud object storage and save on operational and transactional data storage costs.

Online Archive allows customers to automatically tier data across Atlas clusters and MongoDB-managed cloud object storage and use a unified endpoint to query that data from their applications.

Here’s how it works:

1. First, you need to write an archiving rule by providing a namespace (database and collection) and one of the following options:
   1. Date Match: Date field (within the documents, can be nested), and age limit (number of days past the date field when archival should begin)
   2. Custom Filter: Write a custom query (e.g. archive = true)
2. Next, you can choose up to two commonly queried fields in addition to the date field which will allow us to partition archived documents for optimal query performance.
3. Once you confirm the details of the online archive, Atlas begins archiving documents that match the rule to fully-managed cloud object storage.
4. You can then use the new connection string you are provided with, to query both your Atlas cluster and their Online Archive simultaneously. You are also provided with an archive-only connection string to query archival data independent from live cluster data.

Archived documents are deleted from the Atlas cluster and cannot be updated or deleted once archived. You can pause, edit rules, or delete the archive at any time.

Atlas customers can use Online Archive on all dedicated (M10 or higher) clusters. Archived data is stored on MongoDB’s fully managed cloud object storage and is encrypted by MongoDB’s encryption key. To query an Online Archive, customers can use a public connection or AWS PrivateLink.

Online Archive archival jobs run every five minutes. Documents that match the customer’s archive rule are stored in a temporary collection on the Atlas cluster, then archived off in files up to 100MB, and archiving up to 2GB total per 5 minute interval. This is done so that archival jobs don’t overwhelm the cluster’s resources.

2GBs every 5 minutes is the fastest that archiving will happen, if the archival job is consistently archiving less than 2GB of data every 5 minutes then the interval will gradually decrease to further reduce demand on the cluster.

With the data expiration feature, you can specify for how many days data should be stored in the online archive before being deleted. You can set an expiration from the archive as low as seven days and as high as 9,125 days; you can set the archive expiration time through either the Atlas UI or the Admin API. Expiration rules can be edited after creation, if needed.

Yes, with Scheduled Archiving, you can configure the scheduled window by setting rules. The window can be scheduled to repeat every day, every week, or every month, depending on your preference.

No, Online Archive is not an alternative to backup. It is a utility to enable a data tiering strategy. Online Archive should be used in conjunction with your backup strategy to ensure that the data that has moved off their Atlas cluster into their Online Archive has already been captured by your backups.

Please read the documentation on more details to get started with archiving your MongoDB Atlas data with Online Archive.