Elevating Database Performance: Introducing Query Insights in MongoDB Atlas

Frank Sun and Sabina Friden
May 2, 2024 | Updated: May 29, 2024
#MongoDB.local NYC '24

Today, at .local NYC, MongoDB Atlas introduced the new Query Insights tab, enhancing how users monitor, manage, and optimize their database performance directly within the Atlas UI. This new feature offers developers deeper insights into their database’s performance, with a more powerful query analysis tool and detailed namespace-level metrics for faster issue resolution and enhanced performance.

Applications and workloads change over time, making it increasingly difficult to track inefficient queries that strain a database's resources. Metrics can spike for various reasons, and developers need the right tooling to determine the source of the problem so they can quickly identify and resolve the issue. MongoDB Atlas's Query Insights directly tackles these challenges by enhancing MongoDB's observability capabilities with two crucial features: Namespace Insights and an upgraded Query Profiler.

This post is also available in: Deutsch, Français, Español, Português, Italiano, 한국어, 简体中文.


Query Insights delivers performance optimization through actionable intelligence

The introduction of MongoDB Atlas Query Insights demonstrates MongoDB’s commitment to advanced database management. This feature enhances our platform’s observability capabilities with detailed and actionable insights. This feature integrates Namespace Insights and an upgraded Query Profiler within a new dynamic interface, helping boost database performance by streamlining diagnostics and reducing troubleshooting times.

The newly added Namespace Insights provides users with collection-level latency statistics and a comprehensive view of how the hottest collections on a cluster perform over time. This enables developers to answer "Who or what is causing the problem?” which is instrumental in identifying performance trends and prioritizing query optimizations.

The enhanced cluster-centric Query Profiler introduces a more comprehensive view of slow and inefficient queries over a broader period. Having an overall view of data across the entire cluster facilitates more straightforward navigation between nodes and a longer lookback period to identify trends. This ultimately reduces troubleshooting time, thereby enhancing developer productivity and improving overall database performance.

Key benefits of Query Insights

Query Insights brings MongoDB Atlas users several new benefits, including:

  • Granular telemetry: Faster identification and resolution of database issues with namespace-level latency statistics

  • Improved observability: It is easier to spot performance trends, identify root causes, and debug applications

  • Enhanced productivity: Reduced troubleshooting time thanks to a more comprehensive view of slow operations

Try it out!

The Query Insights page provides more granular insights into database performance by providing collection and operation-level details. The Namespace Insights page provides metrics for the top 20 collections by total latency.

Screenshot of the Query Insights dashboard with the caption: The Query Insights page gives more granular insights into database performance by providing collection and operation level details. The Namespace Insights page provides metrics for the top 20 collections by total latency.

Hover over the charts to see how collections perform relative to each other over time. This information makes it easier to answer the question: “who/what is causing the problem?”

Use the Query Profiler to view specific slow operations. Click on a point in the scatter plot to bring up additional metadata about each slow operation.

Click on View More Details to see more metrics and metadata about each slow operation, including the app name, the operation, the plan summary, execution stats, etc.

Empowering users for peak performance

The launch of Query Insights in MongoDB Atlas underscores MongoDB’s commitment to enhancing our platform's observability capabilities. By providing users with the necessary tools and insights for optimal database performance, MongoDB enables developers to spend less time debugging and more time creating—lowering the total cost of ownership and maximizing efficiency, adding significant value to our users' operations.

Sign up for MongoDB Atlas, our cloud database service, to see Query Insights in action, and for more information, see Monitor Query Performance.

← Previous

Atlas Edge Server is Now in Public Preview

We’re excited to announce that Atlas Edge Server is now in public preview! Any developer on Atlas can now deploy Edge Server for their connected infrastructure. Learn more in our docs or get started today. Developers value MongoDB’s developer data platform for the flexibility and ease of use of the document model, as well as for helpful tools like search and charts that simplify data management. As a crucial component of our Atlas for the Edge solution, Atlas Edge Server extends these capabilities to remote and network-constrained environments. First announced at MongoDB.local London 2023, Atlas for the Edge enables local data processing and management within edge environments and across edge devices, reducing latency, enhancing performance, and allowing for disconnection resilience. This post is also available in: Deutsch , Français , Español , Português , Italiano , 한국어 , 简体中文 . What's new in public preview? One of our top priorities is providing developers with a seamless experience when managing their data and applications. We continuously seek to enhance this experience, which is why, starting today, Atlas Edge Server can be directly downloaded, configured, and managed through the Atlas UI. Developers who deploy from the Atlas UI will be able to choose between two onboarding flows to ensure that their configuration is tailored to their needs. This includes both developers who want to connect their edge server with a MongoDB driver or client, and those who want to support connecting to the Edge Server via Device Sync. Why Atlas Edge Server? While edge computing brings data processing closer to end-users and offers substantial benefits, such as network resilience and increased security, a number of challenges inherent to edge computing can make it difficult to fully leverage. Edge computing challenges include managing complex networks, handling large volumes of data, and addressing security concerns, any of which can deter organizations from adopting edge computing. Additionally, the costs associated with building, maintaining, and scaling edge computing systems can be significant. Atlas for the Edge and Atlas Edge Server alleviate these challenges. Atlas Edge Server provides a MongoDB instance equipped with a synchronization server that can be deployed on local or remote infrastructure. It enables real-time synchronization, conflict resolution, and disconnection tolerance. This ensures that mission-critical applications and devices operate seamlessly, even with intermittent connectivity. Edge Server allows for selective synchronization of only modified fields, conserving bandwidth and prioritizing crucial data transfers to Atlas. It also maintains edge client functionality even with intermittent cloud connectivity, preventing disruptions to essential operations like inventory management and point-of-sale systems. Processing data locally reduces latency and enables rapid data insights, reducing dependency on central databases. We'll meet you at the edge The Public Preview of Atlas Edge Server underscores MongoDB’s ongoing commitment to enhancing our developer data platform for distributed infrastructures. As we continue to invest in Atlas for the Edge, MongoDB’s goal is to equip teams with a robust data solution that not only offers an exceptional developer experience but also empowers them to drive innovative solutions for their businesses and customers. Get started today , or visit the Atlas for the Edge web page to learn more about how companies are benefiting from our edge solution.

May 2, 2024
Next →

The Converged AI and Application Datastore for Insurance

In the inherently information-driven insurance industry, companies ingest, analyze, and process massive amounts of data, requiring extensive decision-making. To manage this, they rely on a myriad of technologies and IT support staff to keep operations running smoothly but often lack effectiveness due to their outdated nature. Artificial intelligence (AI) holds great promise for insurers by streamlining processes, enhancing decision-making, and improving customer experiences with significantly less time, resources, and staff compared with traditional IT systems. The convergence of AI and innovative application datastores is transforming how insurers work with data. In this post, we’ll look at how these elements are reshaping the insurance industry and offering greater potential for AI-powered applications, with MongoDB at the heart of the converged AI and application datastore. Scenario planning and flexible data layers One of the primary concerns for IT leaders and decision-makers in the insurance industry is making smart technology investments. The goal is to consolidate existing technology portfolios, which often include a variety of systems like SQL Server, Oracle, and IBM IMS. Consolidation helps reduce inventory and prepare for the future. But what does future-proofing really look like? Scenario planning is an effective strategy for future-proofing. This involves imagining different plausible futures and investing in the common elements that remain beneficial across all scenarios. For insurance companies, a crucial common thread is the data layer. By making data easier to work with, companies can ensure that their technology investments remain valuable regardless of how future scenarios unfold. MongoDB’s flexible developer data platform offers a distinct architectural advantage by making data easier to work with, regardless of the cloud vendor or AI application in use. This flexibility is vital for preparing for disruptive future scenarios, whether they involve regulatory changes, market shifts, or technological advancements. Watch now: The Converged AI and Application Datastore: How API's, AI & Data are Reshaping Insurance The role of AI and data in insurance Generative AI is revolutionizing the insurance sector, offering new ways to manage and utilize data. According to Celent's 2023 Technology Insight and Strategy Survey, 33% of companies across different industries have AI projects in planning, 29% in development, and 19% in production (shown in Figure 1 below). This indicates a significant shift towards AI-driven solutions by insurers actively experimenting with gen AI. Figure 1: Celent Technology Insight and Strategy Survey 2023 However, there's tension between maintaining existing enterprise systems and innovating with AI. Insurance companies must balance keeping the lights on with investing in AI to meet the expectations of boards and stakeholders. The solution lies in integrating AI in a way that enhances operational efficiency without overwhelming existing systems. However, data challenges need to be addressed to achieve this, specifically around access to data. According to a Workday Global Survey , only 4% of respondents said their data is fully accessible, and 59% say their enterprise data is somewhat or completely siloed. Without a solid data foundation, insurers will struggle to achieve the benefits they are looking for from AI. Data architectures and unstructured data When adopting advanced technologies like AI and ML, which require data as the foundation, organizations often grapple with the challenge of integrating these innovations into legacy systems due to their inflexibility and resistance to modification. A robust data architecture is essential for future-proofing and consolidating technology investments. Insurance companies often deal with a vast amount of unstructured data, such as claim images and videos, which can be challenging to manage. By leveraging AI, specifically through vector search and large language models, companies can efficiently process and analyze this data. MongoDB is ideal for managing unstructured data due to its flexible, JSON-like document model, which accommodates a wide variety of data types and structures without requiring a predefined schema. Additionally, MongoDB’s flexibility enables insurers to integrate seamlessly with various technologies, making it a versatile and powerful solution for unstructured data management. For example, consider an insurance adjuster assessing damage from claim photos. Traditionally, this would require manually reviewing each image. With AI, the photos can be converted into vector embeddings and matched against a database of similar claims, drastically speeding up the process. This not only improves efficiency but also enhances the accuracy of assessments. The converged AI and application datastore with MongoDB Building a single view of data across various systems is a game-changer for the insurance industry. Data warehouses and data lakes have long provided single views of customer and claim data, but they often rely on historical data, which may be outdated. The next step is integrating real-time data with these views to make them more dynamic and actionable. A versatile database platform plays a crucial role in this integration. By consolidating data into a single, easily accessible view, insurance companies can ensure that various personas, from underwriters to data scientists, can interact with the data effectively. This integration allows for more responsive and informed decision-making, which is crucial for staying competitive in a rapidly evolving market. This can be achieved with a converged AI and application datastore, as shown in Figure 2 below. This is where operational data, analytics insights, and unstructured data become operationally ready for the applications that leverage AI. Figure 2: Converged AI and application datastore reference architecture The convergence of AI, data, and application datastores is reshaping the insurance industry. By making smart technology investments, leveraging AI to manage unstructured data, and building robust data architectures, insurance companies can future-proof their operations and embrace innovation. A versatile and flexible data platform provides the foundation for these advancements, enabling companies to make their data more accessible, actionable, and valuable. The MongoDB Atlas developer data platform puts powerful AI and analytics capabilities directly in the hands of developers and offers the capabilities to enrich applications by consolidating, ingesting, and acting on any data type instantly. Because MongoDB serves as the operational data store (ODS)—with its flexible document model—insurers can efficiently handle large volumes of data in real-time. By integrating MongoDB with AI/ML platforms, insurers can develop models trained on the most accurate and up-to-date data, thereby addressing the critical need for adaptability and agility in the face of evolving technologies. With built-in security controls across all data, whether managed in a customer environment or through MongoDB Atlas, a fully managed cloud service, MongoDB ensures robust security with features such as authentication (single sign-on and multi-factor authentication), role-based access controls, and comprehensive data encryption. These security measures act as a safeguard for sensitive data, mitigating the risk of unauthorized access from external parties and providing organizations with the confidence to embrace AI and ML technologies. If you would like to learn more about the convergence of AI and application datastores, visit the following resources: Video: The Converged AI and Application Datastore: How API's, AI & Data are Reshaping Insurance Paper: Innovation in Insurance with Artificial Intelligence The MongoDB Solutions Library is curated with tailored solutions to help developers kick-start their projects

July 18, 2024