Applications
Customer stories, use cases, and experiences of MongoDB
Teach & Learn with MongoDB: Professor Abdussalam Alawini, University of Illinois at Urbana-Champaign
In this series of interviews, we talk to students and educators around the world who are using MongoDB to make their classes more engaging and relevant. By exploring their stories, we uncover how MongoDB’s innovative platform and resources are transforming educational landscapes and empowering the next generation of tech-savvy professionals. From creative teaching approaches to advanced classroom solutions, the MongoDB for Educators program can help you transform your classroom with cutting-edge technology and free resources. It can help you provide students with an interactive and dynamic learning environment that bridges the gap between theoretical knowledge and practical application. The program includes a variety of free resources for educators crafted by MongoDB experts to prepare learners with in-demand database skills and knowledge. Program participants have access to MongoDB Atlas credits, curriculum materials, certifications, and membership in a global community of educators from over 700 universities. From theory to practice: Hands-on MongoDB Teaching Professor Abdussalam Alawini is known for his creative use of MongoDB in his courses. He heavily uses MongoDB's free cluster to demonstrate MongoDB concepts during classes, and his students also use the free cluster for their projects, giving them hands-on experience with real-world applications. Currently, a Teaching Associate Professor at the University of Illinois Urbana-Champaign, Professor Alawini’s research interests span databases, applied machine learning, and education. He is particularly focused on applying machine learning methods to enhance classroom experiences and education. His work also includes developing next-generation data management systems, such as data provenance, citation, and scientific management systems. He recently received the U of I’s 2024 Campus Excellence in Undergraduate Education award, which highlights his commitment to teaching and the impact he’s had on his students. Professor Alawini is currently collaborating with colleagues on research to map how databases, data systems, data management, and related courses are taught in introductory computer science undergraduate courses worldwide. Professor Alawini’s story offers valuable insights for educators eager to enhance their teaching and prepare students for a tech-driven future. Check out how MongoDB Atlas has revolutionized his teaching by simplifying database deployment, management, and scaling, allowing students to focus more on learning MongoDB concepts. Tell us about your educational journey and what sparked your interest in databases. My educational journey began with a bachelor's degree in Computer Science from the University of Tripoli in 2002. I then spent over six years in the industry as a database administrator, lead software developer, and IT Manager. In 2011, I returned to academia and earned two master's degrees in Computer Science and Engineering and Technology Management from Portland State University, followed by a Ph.D. in Computer Science in 2016. Subsequently, I joined the University of Pennsylvania for a two-year postdoctoral training. My interest in databases was sparked during my time as a database administrator at PepsiCo, where I enjoyed maintaining the company's databases and building specialized reports to improve business operations. I was particularly fascinated by database systems’ ability to optimize queries and handle millions of concurrent user requests seamlessly. This experience led me to focus my doctoral studies on building data management systems for scientific applications. What courses are you currently teaching at the University of Illinois Urbana-Champaign? Currently, I teach Database Systems and Data Management in the Cloud courses at the University of Illinois Urbana-Champaign. In addition, I also teach a course to University High School students to introduce them to data management and database basics. My intention with teaching databases to high schoolers is to use data management as a gateway to lower entry barriers into computing fields for non-computer science students and to recruit underrepresented minorities to computing. What inspired you to start teaching MongoDB? I was inspired to start teaching MongoDB after seeing several surveys indicating that it is the most used database in web development and one of the leading document-oriented databases. MongoDB offers several unique features that set it apart from other databases, including the aggregation pipeline, which simplifies data processing and transformation. Additionally, MongoDB's flexible schema design allows for easier handling of unstructured data, and its horizontal scalability ensures robust performance as data volumes grow. These features make MongoDB an essential tool for modern web development, and I wanted to equip my students with the skills to leverage this powerful technology. How do you design your course content to effectively integrate MongoDB and engage students in practical learning? In all my data management courses, I focus on teaching students the concept of data models, including relational, document, key-value, and graph. In my Database Systems course, I teach MongoDB alongside SQL and Neo4J to highlight the unique features and capabilities of each data model. This comparative approach helps students appreciate the importance and applications of different databases, ultimately making them better data engineers. In my Data Management in the Cloud course, I emphasize the system's side of MongoDB, particularly its scalability. Understanding how MongoDB is built to handle large volumes of data efficiently provides students with practical insights into managing data in a cloud environment. To effectively integrate MongoDB and engage students in practical learning, I use a hybrid flipped-classroom approach. Students watch recorded lectures before class, allowing us to dedicate class time to working through examples together. Additionally, students form teams to work on various data management scenarios using a collaborative online assessment tool called PrairieLearn. This model fosters peer learning and collaboration, enhancing the overall educational experience. How has MongoDB supported you in enhancing your teaching methods and upskilling your students? I would like to sincerely thank MongoDB for Academia for the amazing support and material they provided to enhance my course design. The free courses offered at MongoDB University have significantly improved my course delivery, allowing me to provide more in-depth and practical knowledge to my students. I heavily use MongoDB's free cluster to demonstrate MongoDB concepts during classes, and my students also use the free cluster for their projects, which gives them hands-on experience with real-world applications. MongoDB Atlas has been a game-changer in my teaching methods. As a fully managed cloud database, it simplifies the process of deploying, managing, and scaling databases, allowing students to focus on learning and applying MongoDB concepts without getting bogged down by administrative tasks. The flexibility and reliability of MongoDB Atlas make it an invaluable tool for both educators and students in the field of data management. Could you elaborate on the key findings from your ITiCSE paper on students' experiences with MongoDB and how these insights can help other educators? In my ITiCSE paper, we conducted an in-depth analysis of students' submissions to MongoDB homework assignments to understand their learning experiences and challenges. The study revealed that as students use more advanced MongoDB operators, they tend to make more reference errors, indicating a need for a better conceptual understanding of these operators. Additionally, when students encounter new functionalities, such as the $group operator, they initially struggle but generally do not repeat the same mistakes in subsequent problems. These insights suggest that educators should allocate more time and effort to teaching advanced MongoDB concepts and provide additional support during the initial learning phases. By understanding these common difficulties, instructors can better tailor their teaching strategies to improve student outcomes and enhance their learning experience. What advice would you give to fellow educators who are considering implementing MongoDB in their own courses to ensure a successful and impactful experience for their students? Implementing MongoDB in your courses can be highly rewarding. Here’s some advice to ensure success: Foundation in Data Models: Teach MongoDB alongside other database types to highlight unique features and applications, making students better data engineers. Utilize MongoDB Resources: Leverage support from MongoDB for Academia, free courses from MongoDB University, and free clusters for hands-on projects. Practical Learning: Use MongoDB Atlas to simplify database management and focus on practical applications. Focus on Challenges: Allocate more time for advanced MongoDB concepts. Address common errors and use tools like PrairieLearn that capture students' interactions and learning progress to identify learning patterns and adjust instruction. Encourage Real-World Projects: Incorporate practical projects to enhance skills and relevance. Continuous Improvement: Gather feedback to iteratively improve course content and share successful strategies with peers. MongoDB is always evolving so make sure to stay tuned with their updates and new features. These steps will help create an engaging learning environment, preparing students for real-world data management. Apply to MongoDB for Educators program and explore free resources for educators crafted by MongoDB experts to prepare learners with in-demand database skills and knowledge.
Nokia Corteca Scales Wi-Fi Connectivity to Millions of Devices With MongoDB Atlas
Nokia’s home Wi-Fi connectivity cloud platform was launched in 2019 as the Nokia WiFi Cloud Controller (NWCC). In 2023, it was renamed and relaunched as the Corteca Home Controller, becoming part of the Corteca software suite that delivers smarter broadband for a better experience. The Corteca Home Controller can be hosted on Amazon Web Services, Google Cloud, or Microsoft Azure, and is the industry’s first platform to support three management services—device management, Wi-Fi management, and application management. Supporting TR-369 (a standardized remote device management protocol) also allows the Home Controller to work in a multi-vendor environment, managing both Nokia broadband devices and third-party broadband devices. By solving connectivity issues before the end-user detects them, and by automatically optimizing Wi-Fi performance, the Home Controller helps deliver excellent customer experiences to millions of users, 24/7. During the five years that Nokia Corteca has been a MongoDB Atlas customer, the Home Controller has successfully scaled from 500,000 devices to over 4.5 million. There are now 75 telecommunications customers of Home Controller spread across all regions of the globe. Having the stability, efficiency, and performance to scale Nokia Corteca's solution is end-to-end, from applications embedded in the device, through the home, and into the cloud. Algorithms assess data extracted from home networks, based on which performance parameters automatically adjust as needed—changing Wi-Fi channels to avoid network interference, for example—thereby ensuring zero downtime. The Home Controller processes real-time data sent from millions of devices, generating massive volumes of data. With a cloud optimization team tasked with deploying the solution across the globe to ever more customers, the Home Controller needed to store and manage its vast dataset and to onboard new telecommunication organizations more easily without incurring any downtime. Prior to Nokia Corteca moving to MongoDB Atlas, its legacy relational database lacked stability and required both admin and application teams to manage operations. A flexible model with time series capabilities That's where MongoDB Atlas came in. Nokia was familiar with the MongoDB Atlas database platform, having already worked with it as part of a previous company acquisition and solution integration. As Nokia's development team had direct experience with the scalability, manageability, and ease of use offered by MongoDB Atlas, they knew it had the potential to address the Home Controller’s technical and business requirements. There was another key element: Nokia wanted to store time-series data—a sequence of data points in which insights are gained by analyzing changes over time. MongoDB Atlas has the unique ability to store operational and time series data in parallel and provides robust querying capabilities on that data. Other advantages include MongoDB's flexible schema, which helps developers store data to match the application's needs and adapt as data changes over time. MongoDB Atlas also provides features such as Performance Advisor that monitors the performance of the database and makes intelligent recommendations to optimize and improve the performance and resource consumption Fast real time data browsing and scalability made easy Previously, scaling the database had been time-consuming and manual. With MongoDB Atlas, the team can easily scale up as demand increases with very little effort and no downtime. This also means it is much more straightforward to add new clients, such as large telecommunications companies. Having started with 100GB of data, the team now has more than 1.3 terabytes, and can increase the disc space in a fraction of a second, positioning the team to be able to scale with the business. As the Home Controller grows and onboards more telcos, the team anticipates a strengthening relationship with MongoDB. “We have a very good relationship with the MongoDB team,” said Jaisankar Gunasekaran, Head of Cloud Hosting and Operations at Nokia. “One of the main advantages is their local presence—they’re accessible, they’re friendly, and they’re experts. It makes our lives easier and lets us concentrate on our products and solutions.” To learn more about how MongoDB can help drive innovation and capture customer imaginations, check out our MongoDB for Telecommunications page.
Unlock PDF Search in Insurance with MongoDB & SuperDuperDB
As industries go, the insurance industry is particularly document-driven. Insurance professionals, including claim adjusters and underwriters, spend considerable time handling documentation with a significant portion of their workday consumed by paperwork and administrative tasks. This makes solutions that speed up the process of reviewing documents all the more important. Retrieval-augmented generation (RAG) applications are a game-changer for insurance companies, enabling them to harness the power of unstructured data while promoting accessibility and flexibility. This is especially true for PDFs, which despite their prevalence are difficult to search, leading claim adjusters and underwriters to spend hours reviewing contracts, claims, and guidelines in this common format. By combining MongoDB and SuperDuperDB you can build a RAG-powered system for PDF search, thus bringing efficiency and accuracy to this cumbersome task. With a PDF search application, users can simply type a question in natural language and the app will sift through company data, provide an answer, summarize the content of the documents, and indicate the source of the information, including the page and paragraph where it was found. In this blog, we will dive into the architecture of how this PDF search application can be created and what it looks like in practice. Why should insurance companies care about PDF Search? Insurance firms rely heavily on data processing. To make investment decisions or handle claims, they leverage vast amounts of data, mostly unstructured. As previously mentioned, underwriters and claim adjusters need to comb through numerous pages of guidelines, contracts, and reports, typically in PDF format. Manually finding and reviewing every piece of information is time-consuming and can easily lead to expensive mistakes, such as incorrect risk estimations. Quickly finding and accessing relevant content is key. Combining Atlas Vector Search and LLMs to build RAG apps can directly impact the bottom line of an insurance company. Behind the scenes: System architecture and flow As mentioned, MongoDB and SuperDuperDB underpin our information retrieval system. Let’s break down the process of building it: The user adds the PDFs that need to be searched. A script scans them, creates the chunks, and vectorizes them (see Figure 1). The chunking step is carried out using a sliding window methodology, which ensures that potentially important transitional data between chunks is not lost, helping to preserve continuity of context. Vectors and chunk metadata are stored in MongoDB, and an Atlas Vector Search index is created (see Figure 3). The PDFs are now ready to be queried. The user selects a customer, asks a question, and the system returns an answer, where it was found and highlights the section with a red frame (see Figure 3). Figure 1: PDF chunking, embedding creation, and storage orchestrated with SuperDuperDB Each customer has a guidelines PDF associated with their account based on their residency. When the user selects a customer and asks a question, the system runs a Vector Search query on that particular document, seamlessly filtering out the non-relevant ones. This is made possible by the pre-filtering field included in the search query. Atlas Vector Search also takes advantage of MongoDB’s new Search Nodes dedicated architecture, enabling better optimization for the right level of resourcing for specific workload needs. Search Nodes provide dedicated infrastructure for Atlas Search and Vector Search workloads, allowing you to optimize your compute resources and fully scale your search needs independent of the database. Search Nodes provide better performance at scale, delivering workload isolation, higher availability, and the ability to optimize resource usage. Figure 2: PDF querying flow, orchestrated with SuperDuperDB SuperDuperDB SuperDuperDB is an open-source Python framework for integrating AI models and workflows directly with and across major databases for more flexible and scalable custom enterprise AI solutions. It enables developers to build, deploy, and manage AI on their existing data infrastructure and data, while using their preferred tools, eliminating data migration and duplication. With SuperDuperDB, developers can: Bring AI to their databases, eliminate data pipelines and moving data, and minimize engineering efforts, time to production, and computation resources. Implement AI workflows with any open and closed source AI models and APIs, on any type of data, with any AI and Python framework, package, class or function. Safeguard their data by switching from APIs to hosting and fine-tuning your own models, on your own existing infrastructure, whether on-premises or in the cloud. Easily switch between embedding models and LLMs, to other API providers as well as hosting your own models, on HuggingFace, or elsewhere just by changing a small configuration. Build next-generation AI apps on your existing database SuperDuperDB provides an array of sample use cases and notebooks that developers can use to get started, including vector search with MongoDB, embedding generation, multimodal search, retrieval-augmented generation (RAG), transfer learning, and many more. The demo showcased in this post is adapted from an app previously developed by SuperDuperDB. Let's put it into practice To show you how this could work in practice, let’s look at, an underwriter handling a specific case. The underwriter is seeking to identify the risk control measures as shown in Figure 3 below but needs to look through documentation. Analyzing the guidelines PDF associated with a specific customer helps determine the loss in the event of an accident or the new premium in the case of a policy renewal. The app assists by answering questions and displaying relevant sections of the document. Figure 3: Screenshot of the UI of the application, showing the question asked, the LLM’s answer, and the reference document where the information is found By integrating MongoDB and SuperDuperDB, you can create a RAG-powered system for efficient and accurate PDF search. This application allows users to type questions in natural language, enabling the app to search through company data, provide answers, summarize document content, and pinpoint the exact source of the information, including the specific page and paragraph. If you would like to learn more about Vector Search powered apps and SuperDuperDB, visit the following resources: PDF Search in Insurance Github repository Search PDFs at Scale with MongoDB and Nomic SuperDuperDB Github, includes notebooks and examples
Unified Namespace Implementation with MongoDB and MaestroHub
In the complex world of modern manufacturing, a crucial challenge has long persisted: how to seamlessly connect the physical realm of industrial control systems with the digital landscape of enterprise operations. The International Society of Automation's ISA-95 standard, often visualized as the automation pyramid, has emerged as a guiding light. As shown below, this five-level hierarchical model empowers manufacturers to bridge the gap between these worlds, unlocking a path toward smarter, more integrated operations. Figure 1: In the automation pyramid, data moves up or down one layer at a time, using point-to-point connections. Manufacturing organizations face a number of challenges when implementing smart manufacturing applications due to the sheer volume and variety of data generated. An average factory produces terabytes of data daily, including time series data from machines stored in process historians and accessed by supervisory control and data acquisition (or SCADA) systems. Additionally, manufacturing execution systems (MES), enterprise resource planning (ERP) systems, and other operations software generate vast amounts of structured and unstructured data. Globally, the manufacturing industry generates an estimated 1.9 petabytes of data annually . Manufacturing leaders are eager to leverage their data for AI and generative AI projects, but a Workday Global Survey reveals that only 4% of the survey’s respondents believe their data is fully accessible for such applications. Data silos are a significant hurdle, with data workers spending an average of 48% of their time on data search and preparation. A popular approach to making data accessible is consolidating it in a cloud data warehouse and then adding context. However, this can be costly and inefficient, as dumping data without context makes it difficult for AI developers to understand its meaning and origin, especially for operational technology time series data. Figure 2: Pushing uncontextualized data to a data warehouse and then adding context is expensive and inefficient. All these issues underscore the need for a new approach—one that not only standardizes data across disparate shop floor systems, but also seamlessly weaves context into the fabric of this data. This is where the Unified Namespace (UNS) comes in. Figure 3: Unified Namespace provides the right data and context to all the applications connected to it. Unified Namespace is a centralized, real-time repository for all production data. It provides a single, comprehensive view of the business's current state. Using an event-driven architecture, applications publish real-time updates to a central message broker, which subscribers can consume asynchronously. This creates a flexible, decoupled ecosystem where applications can both produce and consume data as needed. Figure 4: UNS enables all the enterprise systems to have one centralized location to get the data they need for what they want to accomplish. MaestroHub and MongoDB: Solving the UNS challenge Initially introduced in 2011 at the Hannover Fair of Industrial Technologies, the core idea behind Industry 4.0 is to establish seamless connectivity and interoperability between disparate systems used in manufacturing. And UNS aims to solve this. Over the past five years, we have seen interest in UNS ramping up steadily, and now manufacturers are looking for practical ways to implement it. In particular, a question we’re frequently asked is where does UNS actually live. To answer that question, we need to look at popular architecture patterns, and the pros and cons of each. The most common pattern is implementing UNS in an MQTT broker. An MQTT broker will act as an intermediary entity that receives messages published by clients, filters the messages by topic, and distributes them to subscribers. The reason most manufacturers choose MQTT is it is an open architecture that is easy to implement. However, the challenge with just using the MQTT broker is that the clients don't get historical data access (which will be required to build the analytical and AI applications). Another approach can be to just dump all the data in a data warehouse and then add context to it. This can solve the problem of historical data access but it is an inefficient approach to standardize messages after they have been landed in the data warehouse in the cloud. A superior solution for comprehensive, real-time data access is combining a single source of truth (SSoT) Unified Namespace platform like MaestroHub with a flexible multi-cloud data platform like MongoDB Atlas. MaestroHub creates SSoT for industrial data, resulting in an up to 80% reduction in integration effort for brownfield facilities. Figure 5: MaestroHub SSoT creates a unified data integration layer, saving up to 50% of time in data contextualization (Source: MaestroHub). MaestroHub provides the connectivity layer to all data sources on the factory floor, along with contextualization and data orchestration. This makes it easy to connect the data needed for the UNS, enrich it with more context, and then publish it to consumers using the protocol that works best for them. Under the hood, MaestroHub stores metadata of connections, instances, and flows, and uses MongoDB as the database to store all this data. MongoDB’s flexible data modeling patterns reduce the complexity of mapping and transforming data when it's shared across different clients in the UNS. Additionally, scalable data indexing overcomes performance concerns as the UNS grows over time. Figure 6: MaestroHub and MongoDB together enable a real-time UNS plus long-term storage. MongoDB: The foundation for intelligent industrial UNS In the quest to build a unified namespace system (UNS) for the modern industrial landscape, the choice of database becomes paramount. So why turn to MongoDB? Scalability and high availability: It scales effortlessly, both vertically and horizontally (sharding), to handle the torrent of data from sensors, machines, and processes. Operational Technology (OT) systems generate vast amounts of data from these sources, and MongoDB ensures seamless management of that information. Document data model: Its adaptable document model accommodates diverse data structures, ensuring a harmonious flow of information. A Unified Namespace (UNS) must handle data from any factory source, accommodating structure variations. MongoDB's flexible schema design allows different data models to coexist in a single database, with schema extensibility at runtime. This flexibility facilitates the seamless integration of new data sources and types into the UNS. Real-time data processing: MongoDB Change Streams and Atlas Device Sync empower third-party applications to access real-time data updates. This is essential for monitoring, alerting, and real-time analysis within a UNS, enabling prompt responses to critical events. Gen AI application development with ease: Atlas Vector Search efficiently performs semantic searches on vector embeddings stored in MongoDB Atlas. This capability seamlessly integrates with large language models (LLMs) to provide relevant context in retrieval-augmented generation (RAG) systems. Given that the Universal Name Service (UNS) functions as a single source of truth for industrial applications, connecting gen AI apps to retrieve context from the UNS database ensures accurate and reliable information retrieval for these applications. With the foundational database established, let's explore MaestroHub, a platform designed to leverage the power of MongoDB in industrial settings. The MaestroHub platform MaestroHub is a provider of a SSoT for industrial data, specifically tailored for manufacturers. It achieves this through: Data connectors: MaestroHub connects to diverse data sources using 38 different industrial communication protocols, encompassing OT drivers, files, databases (SQL, NoSQL, time series), message brokers, web services, cloud systems, historians, and data warehouses. The bi-directional nature of 90% of these protocols ensures comprehensive data integration, leaving no data siloed. Data contextualization based on ISA-95: Leveraging ISA-95 Part 2, MaestroHub employs a semantic hierarchy and a clear naming convention for easy navigation and understanding of data topics. The contextualization of the payload is not just limited to the unit of measure AND definitional but also contains Enterprise/Site/Area/Line/Cell details, which are invaluable for analytics studies. Data contextualization is an important feature of a UNS platform. Logic flows/rule engine: Adhering to the UNS principle "Do not make any assumptions on how the data will be consumed," the data should flow flexibly from sources to brokers and from brokers to consumers in terms of rules, frequencies, and multiple endpoints. MaestroHub allows you to set rules such as Always, OnChange, OnTrue, and WhileTrue, where you can dynamically determine the conditions using events and inputs via JavaScript. Insights created by MaestroHub: MaestroHub provides real-time diagnostics of data health by leveraging Prometheus, Elasticsearch, Fluentd, and Kibana. Network problems, changed endpoints, and changed data types are automatically diagnosed and reported as insights. Additionally, MaestroHub uses NATS for queue management and stream analytics, buffering data in the event of a network outage. This allows IT and OT teams to monitor, debug, and audit logs with full data lineage. Conclusion The ISA-95 automation pyramid presents significant challenges for the manufacturing industry, including a lack of flexibility, limited scalability, and difficulty integrating new technologies. By adopting a Unified Namespace architecture with MaestroHub and MongoDB, manufacturers can overcome these challenges and achieve real-time visibility and control over their operations, leading to increased efficiency and improved business outcomes. Read more on how MongoDB enables Unified Namespace via its multi-cloud developer data platform. We are actively working with our clients on solving Unified Namespace challenges. Take a look at our Manufacturing and Industrial IoT page for more stories or contact us through the web form in the link.
Microservices: Realizing the Benefits Without the Complexity
The microservice architecture has emerged as the preferred, modern approach for developers to build and deploy applications on the cloud. It can help you deliver more reliable applications, and address the scale and latency concerns for System Reliability Engineers (SREs) and operations. But microservices aren't without their hangups. For developers, microservices can lead to additional complexity and cognitive overhead, such as cross-service coordination, shared states across multiple services, and coding and testing failure logic across disconnected services. While the monolith was suboptimal for compute and scale efficiencies, the programming model was simple. So the question is, can we get the best of both worlds? In addition, how do we make the individual services easier to build and adapt to changing requirements? Since, at their core, microservices provide access to and perform operations on data, how do we architect services so that developers can easily work with data? How can we make it easier for developers to add new types of data and data sources and perform a wide variety of data operations without the complexity of managing caches and using multiple query languages (SQL, full-text and vector search, time series, geospatial, etc.) The development complexity associated with microservice architectures occurs at two levels: service orchestration and service data management. The diagram below depicts this complexity. At the orchestration level, a typical application may support tens or hundreds of processes, and each may have thousands or millions of executions. To make this work, services are often connected by a patchwork of queues. Developers spend quite a bit of time tracking and managing all the various workflows. The sheer scale necessitates a central mechanism to manage concurrent tasks and sharded databases to manage the state of millions of concurrent workflow instances. To add more complexity, each microservice is deployed using a set of data platforms including RDBMS, caches, search engines, and vector and NoSQL databases. Developers must work with multiple query languages, write code to keep data in sync among these platforms and write code to deal with edge cases when invariably data or indexes are not in sync. Finally, developer productivity is inhibited by the brittleness of RDBMS, which lacks flexibility when trying to incorporate new or changing data types. As a result, microservice applications often end up with complex architectures that are difficult to develop against and maintain in terms of both the individual microservices and the service orchestration. Realizing the benefits without the complexity One approach to address these microservice challenges is to combine two technologies: Temporal and MongoDB. Both give you the benefits of microservices while simplifying the implementation of service orchestration. Together, they allow developers to build services that can easily handle a wide variety of data, eliminate the need to code for complex infrastructure and reduce the likelihood of failure. They simplify the data model and your code. In one real-world example, open-source indexing company Mixpeek leverages the combination of MongoDB and Temporal to provide a platform enabling organizations to easily incorporate multi-modal data sources in AI applications. Mixpeek’s CEO Ethan Steininger states, “Temporal’s durable execution guarantees and MongoDB's flexible data model are core components of Mixpeek’s multimodal data processing and storage. Combined, they enable our users to run high volume ML on commodity hardware without worrying about dropped jobs.” MongoDB and Temporal: Build like a monolith with durable microservices Both MongoDB and Temporal were built by developers, for developers. They both use a code-first approach to solving the complex infrastructure needs of our modern applications within our application code and empower developers to be more productive. They are part of an emerging development stack that greatly simplifies data and all the cross-functional coordination we need in our cloud applications. Ultimately, the combination of these two developer-focused platforms allows you to simplify design, development, and testing of microservice-based applications. With the document model of MongoDB, you model data as real world objects and not tables, rows, and columns. With Temporal, you design your end-to-end service flows as workflows as described by domain experts without having to explicitly identify every edge case and exception (Temporal handles those implicitly). Temporal and MongoDB provide the same benefits that, when combined, multiply in value. You become more agile, as not only can everyone understand your code better, but you are no longer challenged by the cognitive overload of trying to coordinate, comprehend, and test a web of disconnected and complex services. Together, they allow us to reliably orchestrate business processes within apps that are all speaking the language of the data itself. Combining Temporal and MongoDB results in the simplified architecture shown below. Temporal enables orchestration to be implemented at a higher level of abstraction, eliminating much of the event management and queuing complexity. MongoDB, in turn, provides a single integrated data platform with a unified query language thereby eliminating much of the data management complexity. Let’s examine MongoDB and Temporal in more depth to better understand their capabilities and why they facilitate the rapid development of microservices-based applications. MongoDB: Simplifying microservice data MongoDB's features align well with the principles of microservices architectures. It reduces the need for niche databases and the associated costs of deploying and maintaining a complicated sprawl of data technologies. More explicitly, MongoDB delivers key benefits for microservice development: Flexible schema, flexible services: Unlike relational databases with rigid schemas, MongoDB's document model allows microservices to easily evolve as data requirements change. Distributed scale for data-heavy, distributed services: MongoDB scales horizontally by adding more partitions to distribute the load. This aligns with the modular nature of microservices, where individual services can scale based on their specific needs. Unified query language reduces microservice sprawl: MongoDB supports a diverse set of data operations without requiring multiple data platforms (caches, vector, and text search engines, time series, geospatial, etc.) Operational efficiency: MongoDB Atlas, the cloud-based version of MongoDB, simplifies managing databases for microservices. It handles provisioning, backups, and patching, freeing developers to focus on core responsibilities. Integrated developer data platform: The integrated developer data platform delivers an intuitive set of tools to build services that support mobile clients, real-time analytics, data visualization, and historical analysis across many service databases. With MongoDB, development teams use one interface for all their services and run it anywhere, even across clouds. Also, it provides a data foundation for your microservices that is highly available, scalable, and secure. It greatly simplifies microservices development so that you can focus on your business problems and not data. Temporal: Don't coordinate services, orchestrate them Temporal delivers an open-source, durable execution solution that removes the complexity of building scalable distributed microservices. It presents a development abstraction that preserves the complete application state so that in the case of a host or software failure, it can seamlessly migrate execution to another machine. This means you can develop applications as if failures—like network outages or server crashes—do not exist. Temporal handles these issues, allowing you to focus on implementing business logic rather than coding complex failure detection and recovery routines. Here's how Temporal simplifies application development: Durable workflows: Temporal maintains the state and progress of a defined workflow across multiple services, even in the face of server crashes, network partitions, and other types of failures. This durability ensures that your application logic can resume where it left off, making your overall application more resilient. Simplifies failure handling: Temporal abstracts away the complex error handling and retry logic that developers typically have to implement in distributed systems. This abstraction allows developers to focus on business logic rather than the intricacies of ensuring their end-to-end services can handle failures gracefully. Scale: Temporal applications are inherently scalable and capable of handling billions of workflow executions. Long-running services: Temporal supports long-running operations, from seconds to years, with the same level of reliability and scalability. By providing a platform that handles the complexities of distributed systems, Temporal allows developers to concentrate on implementing business logic in their services. This focus can lead to faster development times and more reliable applications, as developers are not bogged down by the intricacies of state management, retries, and error handling. The next generation of microservices development is here Developers want to code. They want to solve business problems. They do not want to be bogged down by the complexity of infrastructure failures. They want to model their apps and data so that it is aligned with the real-world entities and domains they are solving for. Using MongoDB and Temporal together solves these complexities. Together, they simplify design, development, and testing of microservice-based applications so that you can focus on business problems and deliver more features faster. Getting started with Temporal and MongoDB Atlas We can help you design the best architecture for your organization’s needs. Feel free to connect with your MongoDB and Temporal account teams or contact us to schedule a collaborative session and explore how Temporal and MongoDB can optimize your AI development process.
Payments Modernization and the Role of the Operational Data Layer
To stay relevant and competitive, payment solution providers must enhance their payment processes to adapt to changing customer expectations, regulatory demands, and advancing technologies. The imperative for modernization is clear: payment systems must become faster, more secure, and seamlessly integrated across platforms. Driven by multiple factors—real-time payments, regulatory shifts like Payment Services Directive 2 (PSD2), heightened customer expectations, the power of open banking, and the disruptive force of fintech startups—the need for payment modernization has never been more pressing. But transformation is not without its challenges. Complex systems, industry reliance on outdated technology, high upgrade costs, and technical debt all pose formidable obstacles. This article will explore modernization approaches and how MongoDB helps smooth transformations. Approaches to modernization As businesses work to modernize their payment systems, they need to overcome the complexities inherent in updating legacy systems. Forward-thinking organizations embrace innovative strategies to streamline their operations, enhance scalability, and facilitate agile responses to evolving market demands. Two such approaches gaining prominence in the realm of payment system modernization are domain-driven design and microservices architecture : Domain-driven design: This approach focuses on a business's core operations to develop scalable and easier-to-manage systems. Domain-driven design ensures that technology serves strategic business goals by aligning system development with business needs. At its core, this approach seeks to break down complex business domains into manageable components, or "domains," each representing a distinct area of business functionality. Microservices architecture: Unlike traditional monolithic architectures, characterized by tightly coupled and interdependent components, a microservices architecture decomposes applications into a collection of loosely coupled services, each of which is responsible for a specific business function or capability. It introduces more flexibility and allows for quicker updates, facilitating agile responses to changing business requirements. Discover how Wells Fargo launched their next-generation card payments by building an operational data store with MongoDB . Modernizing with an operational data layer In the payments modernization process, the significance of an operational data layer (ODL) cannot be overstated. An ODL is an architectural pattern that centrally integrates and organizes siloed enterprise data, making it available to consuming applications. The simplest representation of this pattern looks something like the sample reference architecture below. Figure 1: Operational Data Layer structure An ODL is deployed in front of legacy systems to enable new business initiatives and to meet new requirements that the existing architecture can’t handle—without the difficulty and risk of fully replacing legacy systems. It can reduce the workload on source systems, improve availability, reduce end-user response times, combine data from multiple systems into a single repository, serve as a foundation for re-architecting a monolithic application into a suite of microservices, and more. The ODL becomes a system of innovation, allowing the business to take an iterative approach to digital transformation. Here's why an ODL is considered ideal for payment operations: Unified data management: Payment systems involve handling a vast amount of diverse data, including transaction details, customer information, and regulatory compliance data. An ODL provides a centralized repository for storing and managing this data, eliminating silos and ensuring data integrity. Real-time processing: An ODL enables real-time processing of transactions, allowing businesses to handle high numbers of transactions swiftly and efficiently. This capability is essential for meeting customer expectations for instant payments and facilitating seamless transactions across various channels. Scalability and flexibility: Payment systems must accommodate fluctuating transaction volumes and evolving business needs. An ODL offers scalability and flexibility, allowing businesses to scale their infrastructure as demand grows. Enhanced security: An ODL incorporates robust security features —such as encryption, access controls, and auditing capabilities—to safeguard data integrity and confidentiality. By centralizing security measures within the ODL, businesses can ensure compliance with regulatory requirements and mitigate security risks effectively. Support for payments data monetization: Payment systems generate a wealth of data that can provide valuable insights into customer behavior, transaction trends, and business performance. An ODL facilitates real-time analytics and reporting by providing a unified platform for collecting, storing, and analyzing this data. Transform with MongoDB MongoDB’s fundamental technology principles ensure companies can reap the advantages of microservices and domain-driven design—specifically, our flexible data model and built-in redundancy, automation, and scalability. Indeed, the document model is tailor-made for the intricacies of payment data, ensuring adaptability and scalability as market demands evolve. Here’s how MongoDB helps with domain-driven design and microservice implementation to adopt industry best practices: Ease of use: MongoDB’s document model makes it simple to model or remodel data to fit the needs of payment applications. Documents are a natural way of describing data. They present a single data structure, with related data embedded as sub-documents and arrays, making it simpler and faster for developers to model how data in the application will be mapped to data stored in the database. In addition, MongoDB guarantees the multi-record ACID transactional semantics that developers are familiar with, making it easier to reason about data. Flexibility: MongoDB’s dynamic schema is ideal for handling the requirements of microservices and a domain-driven design. Domain-driven design emphasizes modeling the domain to reflect the business requirements, which may evolve over time. MongoDB's flexible schema allows you to store domain objects as documents without rigid schema constraints, facilitating agile development and evolution of the domain model. Speed: Using MongoDB for an ODL means you can get better performance when accessing data, and write less code to do so. A document is a single place for the database to read and write data for an entity. This locality of data ensures the complete document can be accessed in a single database operation that avoids the need internally to pull data from many different tables and rows. Data access and microservice-based APIs: MongoDB integrates seamlessly with modern technologies and frameworks commonly used in microservices architectures. MongoDB's flexible data model and ability to handle various data types, including structured and unstructured data, is a great fit for orchestrating your open API ecosystem to make data flow between banks, third parties, and consumers possible. Scalability: Even if an ODL starts at a small scale, you need to be prepared for growth as new source systems are integrated, adding data volume, and new consuming systems are developed, increasing workload. MongoDB provides horizontal scale-out on low-cost, commodity hardware or cloud infrastructure using sharding to meet the needs of an ODL with large data sets and high throughput requirements. High availability: Microservices architectures require high availability to ensure that individual services remain accessible even in the event of failures. MongoDB provides built-in replication and failover capabilities, ensuring data availability and minimal downtime in case of server failures. Payment modernization is not merely a trend but a strategic imperative. By embracing modern payment solutions and leveraging the power of an ODL with MongoDB, organizations can unlock new growth opportunities, enhance operational efficiency, and deliver superior customer experiences. Learn how to build an operational data layer with MongoDB using this Payments Modernization Solution Accelerator . Learn more about how MongoDB is powering industries on our solution library .
Five Languages, One Goal: A Developer's Path to Certification Mastery
MongoDB Community Creator Markandey Pathak has become a certified developer in five different programming languages: C#, Java, Node.JS, PHP, and Python. Pursuing multiple certifications equips developers with a diverse skill set, making them invaluable team members. Fluency across different programming languages enables them to foster platform-agnostic solutions and promote adaptability, collaboration, and informed decision-making, which are crucial for success in the global tech landscape. To understand what led Markandey to take on so many certifications while managing a busy and successful career, we spoke with him to gain insights into the challenges and triumphs he faced. What motivated you to pursue certification in multiple programming languages, and how has achieving such a diverse set of skills impacted your career? C was the first programming language I learned, followed by C# and the .NET ecosystem a few years later. Transitioning to a new language like C# after knowing one was straightforward. I then delved into ASP.NET, JAVA, and subsequently PHP. Despite the differing syntax of these languages, I found that fundamental programming concepts remained consistent. This enlightening realization led me to explore JavaScript and, later, Python. Such a diverse skill set made me a go-to resource for many senior leaders seeking insights. This versatility allowed me to transcend categorization based on programming ecosystems in the workplace, evolving my mindset to develop platform-agnostic solutions. I believe in the adage of being a jack of all trades while still mastering one or more. I took on the challenge of discovering MongoDB drivers available for various platforms. I created sample applications to practice basic MongoDB concepts using specific drivers, and soon, everything fell into place effortlessly. What tips or advice would you share with someone who looks up to your achievement and aspires to become a certified developer in multiple languages like C#, Java, Node.JS, PHP, and Python? How can they effectively approach learning and mastering these languages? Before attempting proficiency in MongoDB across multiple languages, it's crucial to prioritize understanding fundamental concepts such as data modeling practices, CRUD operations, and indexes. Mastering MongoDB's shell, MongoSh, is essential to grasp the workings of MongoDB's read and write operations. Following this, individuals should select a programming environment they're most adept in and practice executing MongoDB operations within that ecosystem. Constructing a personal project can aid in practically observing various MongoDB concepts in action. Utilizing resources such as MongoDB Certification Learning Paths , practice tests, and MongoDB Documentation is vital for excelling in certification exams. Additionally, it's advisable to undertake the initial certification in the programming language one feels most comfortable with. Reflection is key; saving or emailing exam scores enables individuals to identify areas needing improvement for future attempts. With proficiency in C#, Java, Node.JS, PHP, and Python, how do you perceive the role of versatility in today's tech industry, especially regarding job opportunities and project flexibility? Programming languages, very much like spoken languages, are merely a medium. The most important thing is knowing what to say. The tech industry depends on problems, and developers seek solutions to them. Once they have a solution, programming languages help make those solutions a reality. It’s not hard to learn different programming languages or even to master them. Knowing the basics of different programming ecosystems can give developers an edge regarding job opportunities. It makes them flexible and enables them to make crucial and informed decisions in choosing the correct tech stack or defining good architecture for solutions. In your experience, how does fluency in multiple languages enhance collaboration and innovation within development teams, particularly in today's globalized tech landscape? Fluency or even practical awareness about programming languages or ecosystems promotes versatility in problem-solving, facilitates cross-functional collaboration, supports agile development, enables integration with legacy systems, fosters global collaboration, reduces dependency, and empowers informed decision-making, all of which are crucial for staying competitive in today's globalized tech landscape. As a MongoDB Community Creator, how do you leverage your expertise in these five languages to contribute to and engage with the broader tech community? What advice would you offer aspiring developers seeking to expand their skill set? I aim to open-source my MongoDB-focused projects across various ecosystems, accompanied by detailed articles outlining their construction. Since these projects were designed with exams in mind, they serve as skill-testing tools for developers and comprehensive guides to the various components comprising certification exams. I advocate for developers to choose a favorite language and compare others to it, as this approach facilitates a quicker and more efficient understanding of concepts. Relating new information to familiar concepts makes learning easier and more effective. The MongoDB Community Advocacy Program is a vibrant global community designed for MongoDB enthusiasts who are passionate about advocating for the platform. Our Community Creators Program welcomes members of all skill levels eager to deepen their involvement in advancing MongoDB's community and technology. We empower our members to expand their expertise, visibility, and leadership by actively engaging with and advocating for MongoDB technologies among users worldwide. Join us and amplify your impact within the MongoDB community! Elevate your career with MongoDB University 's 1,000+ learning assets. Access free courses and hands-on labs, and earn certifications to boost your skills and stand out in tech.
The Journey of MongoDB with COVESA in the Connected Vehicle Landscape
There’s a popular saying: “If you want to go fast, go alone; if you want to go far, go together.” I would argue The Connected Vehicle Systems Alliance (COVESA) in partnership with their extensive member network, turns this saying on its head. They have found a way to go fast, together and also go far, together. COVESA is an industry alliance focused on enabling the widespread adoption of connected vehicle systems. This group aims to accelerate the development of these technologies through collaboration and standardization. It's made up of various stakeholders in the automotive and technology sectors, including car manufacturers, suppliers, and tech companies. COVESA’s collaborative approach allows members to accelerate progress. Shared solutions eliminate the need for individual members to reinvent the wheel. This frees up their resources to tackle new challenges, as the community collectively builds, tests, and refines foundational components. As vehicles become more connected, the data they generate explodes in volume, variety, and velocity. Cars are no longer just a mode of transportation, but a platform for advanced technology and data-driven services. This is where MongoDB steps in. MongoDB and COVESA As the database trusted for mission-critical systems by enterprises such as Cathay Pacific , Volvo Connect , or Cox Automotive ; MongoDB has gained expertise in automotive, along with many other industries, building cross-industry knowledge in handling large-scale, diverse data sets. This in turn enables us to contribute significantly to vehicle applications and provide a unique view, especially in the data architecture discussions within COVESA. MongoDB solutions support these kinds of innovations, enabling automotive companies to leverage data for advanced features. One of the main features we provide is Atlas Device SDKs : a low-footprint, embedded database directly living on ECUs. It can synchronize data automatically with the cloud using Atlas Device Sync , our data transfer protocol that compresses the data handles conflict resolution, and only syncs delta changes, making it extremely efficient in terms of operations and maintenance. VSS: The backbone of connected vehicle data An important area of COVESA’s work is the Vehicle Signal Specification (VSS). VSS is a standardized framework used to describe data of a vehicle, such as speed, location, and diagnostic information. This standardization is essential for interoperability between different systems and components within a vehicle, as well as for external communication with other vehicles and infrastructure. VSS has been gaining more and more adoption, and it’s backed by ongoing contributions from BMW, Volvo Cars, Jaguar LR, Robert Bosch and Geotab, among others. MongoDB’s BSON and our Object-oriented Device SDKs uniquely position us to contribute to VSS implementation. The VSS data structured maps 1 to 1 to documents in MongoDB and objects in Atlas Device SDKs , which simplifies development, and speeds up applications by completely skipping any Relational Mapper layer. For every read or write, there is no need to transform the data between relational and VSS. Our insights into data structuring, querying, and management can help optimize the way data is stored and accessed in connected vehicles, making it more efficient and robust. Where MongoDB contributes MongoDB, within COVESA, finds its most meaningful contributions in areas where data complexities and community collaboration intersect. First, we can share insights into managing vast and varied data emerging from connected vehicles generating data on everything from engine performance to driver behavior. Second, we have an important role in supporting the standardization efforts, crucial for ensuring different systems within vehicles can communicate seamlessly. Our inputs can help ensure these standards are robust and practical, considering the real-world scenarios of data usage in vehicles. Some of our contributions include an Over the Air update architectural review presented at Troy COVESA’s AMM in October 2023; sharing insights about the Data Middleware PoC with BMW; and weekly contributions at the Data Expert Group. You can find some of our contributions on COVESA’s Wiki page . In essence, MongoDB's role in COVESA is about providing a unique perspective from the database management point of view, offering our understanding from different industries and use cases to support the developments towards more connected and intelligent vehicles. MongoDB, COVESA, and AWS together at CES2024 MongoDB’s most recent collaboration with COVESA was at the Consumer Electronics Show CES 2024 during which MongoDB’s Connected Vehicle solution was showcased. This solution leverages Atlas Device SDKs, such as the SDK for C++ , which enables local data storage, in-vehicle data synchronization, and also uni and bi-directional data transfer with the cloud. Below is a schematic illustrating the integration of MongoDB within the software-defined vehicle: Schema 1: End to end integration for the connected vehicle At CES 2024, MongoDB also teamed up with AWS for a compelling presentation, " AI-powered Connected Vehicles with MongoDB and AWS " led by Dr. Humza Akhtar and Mohan Yellapantula, Head of Automotive Solutions & Go To Market at AWS. The session delved into the intricacies of building connected vehicle user experiences using MongoDB Atlas. It showcased the combined strengths of MongoDB's expertise and AWS's generative AI tools, emphasizing how Atlas Vector Search unlocks the full lifecycle value of connected vehicle data. During the event, MongoDB also engaged in a conversation with The Six Five, exploring various aspects of mobility, self-driving vehicles (SDVs), and the MongoDB and AWS partnership. This discussion extended to merging IT and OT, GenAI, Atlas Edger Server, and Atlas Device SDK. Going forward At the end of the road, it’s all about enhancing the end-user experience and providing unique value propositions. Defect diagnosis based on the acoustics of the engine, improved crash assistance with mobile and vehicle telemetry data, just-in-time food ordering while on the road, in-vehicle payments, and much, much more. What all these experiences have in common is the combination of interconnected data from different systems. At MongoDB, we are laser-focused on empowering OEMs to create, transform, and disrupt the automotive industry by unleashing the power of software and data. We enable this by: Partnering with alliances such as COVESA to build a strong ecosystem or collaboration. Having one single API for In-vehicle Data Storage, Edge to Cloud Synchronization, Time Series storage, and more, improves the developer experience. Focusing on having a robust, scalable, and secure suite of services trusted by tens of thousands of customers in more than 100 countries. Together with COVESA’s vision for connected vehicles, we’re driving a future where this industry is safer, more efficient, and seamlessly integrated into the digital world. The journey is just beginning. To learn more about MongoDB-connected mobility solutions, visit the MongoDB for Manufacturing & Motion webpage . Achieving fast, reliable and compressed data exchange is one of the pillars of Software Defined Vehicles, learn how MongoDB Atlas and Edge Server can help in this short demo .
Enabling Commerce Innovation with the Power of MongoDB and Google Cloud
Across all industries, business leaders are grappling with economic uncertainty, cost concerns, disruption to supply chains, and pressure to embrace new technologies like generative AI. In this dynamic landscape, having a performant and future-proofed technology foundation is critical to your business’s success. Kin + Carta, a Premier Google Cloud Partner and MongoDB Systems Integrator Partner, recently launched the Integrated Commerce Network . The Integrated Commerce Network is an Accelerator that enables clients to modernize to a composable commerce platform and create value with their commerce data on Google Cloud with a pre-integrated solution in as little as six weeks. This article explains the concept of composable commerce and explores how MongoDB and Google Cloud form a powerful combination that enables innovation in commerce. Finally, it explains how Kin + Carta can help you navigate the complexity facing businesses today with their approach to digital decoupling. Unraveling the complexity: What is composable commerce? Why microservices and APIs? The evolution of commerce architecture Traditional monolithic architectures, once the cornerstone of commerce platforms, are facing challenges in meeting the demands of today's fast-paced digital environment. Microservices, a paradigm that breaks down applications into small, independent services, offer a solution to the limitations of monoliths. This architectural shift allows for improved agility, scalability, and maintainability. Defining composable commerce Composable commerce is a component-based, API-driven design approach that gives businesses the flexibility to build and run outstanding buying experiences free of constraints found in legacy platforms. To be truly composable, the platform must support key tenets: Support continuous delivery without downtime at the component level Have API as the contract of implementation between services, with open, industry-standard protocols providing the glue between components Be SaaS based, or portable to run on any modern public cloud environment Allow the open egress and ingress of data — no black-boxes of vendor data ownership Defining APIs and microservices APIs play a pivotal role in connecting microservices, enabling seamless communication and data exchange. This modular approach empowers businesses to adapt quickly to market changes, launch new features efficiently, and scale resources as needed. Enhanced scalability, resilience, and agility Taking a microservices approach provides businesses with options and now represents a mature and battle-tested approach with commoditized architectures, infrastructure-as-code, and open-source design patterns to enable robust, resilient, and scalable commerce workloads at lower cost and risk. Additionally, the decoupled nature of microservices facilitates faster development cycles. Development teams can work on isolated components, allowing for parallel development and quicker releases. This agility is a game-changer in the competitive e-commerce landscape, where rapid innovation is essential for staying ahead. Microservices and API-based commerce solutions (like commercetools, which is powered by MongoDB) have begun to dominate the market with their composable approach, and for good reason. These solutions remove the dead-end of legacy commerce suite software and enable a brand to pick and choose to enhance its environment on its own terms and schedule. MongoDB Atlas: The backbone of intelligent, generative AI-driven experiences As e-commerce has developed, customers are expecting more from their interactions — flat, unsophisticated experiences just don’t cut it anymore and brands need to deliver on the expectation of immediacy and contextual relevance. Taking a microservices approach enables richer and more granular data to be surfaced, analyzed, and fed back into the loop, perhaps leveraging generative AI to synthesize information that previously would have been difficult or impossible without huge computing capabilities. However, to do this well you need core data infrastructure that underpins the platform and provides the performance, resilience, and advanced features required. MongoDB Atlas on Google Cloud can play a pivotal role in this enablement. Flexible data models: Microservices often require diverse data models. MongoDB Atlas, a fully managed database service, accommodates these varying needs with its flexible schema design, which allows businesses to adapt their data structures without compromising performance. Horizontal scalability: Modern commerce moves a lot of data. MongoDB Atlas excels in distributing data across multiple nodes, ensuring that the database can handle increased loads effortlessly. Real-time data access: Delivering on expectations relies on real-time data access. MongoDB Atlas supports real-time, event-driven data updates, ensuring you are using the most up-to-date information about your customers. Serverless deployment: Rather than spend time and money managing complex database infrastructure, MongoDB Atlas can leverage serverless deployment, allowing developers to focus on building features that delight customers and impact the bottom line. Unleashing generative AI with MongoDB and Google Cloud Generative AI applications thrive on massive datasets and require robust data management. MongoDB effortlessly handles the complex and ever-evolving nature of gen AI data. This includes text, code, images, and more, allowing you to train your models on a richer data tapestry. MongoDB Atlas: Streamlined gen AI development on Google Cloud MongoDB Atlas, the cloud-based deployment option for MongoDB, integrates seamlessly with Google Cloud. Atlas offers scalability and manageability, letting you focus on building groundbreaking gen AI applications. Here's how this powerful duo functions together: Data ingestion and storage: Effortlessly ingest your training data, regardless of format, into MongoDB Atlas on Google Cloud. This data can include text for natural language processing, code for programming tasks, or images for creative generation. AI model training: Leverage Google Cloud's AI services like Vertex AI to train your gen AI models using the data stored in MongoDB Atlas. Vertex AI provides pre-built algorithms and tools to streamline model development. Operationalization and serving: Once trained, deploy your gen AI model seamlessly within your application. MongoDB Atlas ensures the smooth data flow to and from your model, enabling real-time generation. Vector search with MongoDB Atlas: MongoDB Atlas Vector Search allows for efficient retrieval of similar data points within your gen AI dataset. This is crucial for tasks like image generation or recommendation systems. Advantages of this open approach By leveraging a microservices architecture, APIs, and the scalability and flexibility of Atlas, businesses can build agile and adaptable composable platforms. Atlas seamlessly integrates with Google Cloud, providing a streamlined environment for developing and deploying generative AI models. This integrated approach offers several benefits: Simplified development: The combined power of MongoDB Atlas and Google Cloud streamlines the development process, allowing you to focus on core gen AI functionalities. Scalability and flexibility: Both MongoDB Atlas and Google Cloud offer on-demand scalability, ensuring your infrastructure adapts to your gen AI application's growing needs. Faster time to market: The ease of integration and development offered by this combination helps you get your gen AI applications to market quickly. Cost-effectiveness: Both MongoDB Atlas and Google Cloud offer flexible pricing models, allowing you to optimize costs based on your specific gen AI project requirements. Digital decoupling, a legacy modernization approach With so much digital disruption, technology leaders are constantly being challenged. Existing legacy architectures and infrastructure can be extremely rigid and hard to unravel. Over 94% of senior leaders reported experiencing tech anxiety . So how do you manage this noise, meet the needs of the business, stay relevant, and evolve your technology so that you can deliver the kinds of experiences audiences expect? Digital decoupling is a legacy modernization approach that enables large, often well-established organizations to present a unified online experience to their users, take full advantage of their data, innovate safely, and compete effectively with digital natives. Technology evolves rapidly, and an effective microservices solution should be designed with future scalability and adaptability in mind. Kin + Carta helps to ensure that your solution is not only robust for current requirements but also capable of evolving with emerging technologies and business needs. It all starts with a clear modernization strategy that allows you to iteratively untangle from legacy systems, while also meeting the needs of business stakeholders seeking innovation. Navigating commerce complexity with Kin + Carta on Google Cloud Commerce is undergoing a significant transformation, and businesses need a future-proof technology foundation to handle the demands of complex models and massive datasets. That’s why Kin + Carta launched their Integrated Commerce Network , the first commerce-related solution that’s part of Google’s Industry Value Network . With the right tools and partners, your business can be at the forefront of innovation with generative AI, through automating tasks in revolutionary new ways, creating entirely new content formats, and delivering more personalized customer experiences. The complexities of commerce transformation can be daunting. But you can master the art of digital decoupling and leverage the strengths of the Integrated Commerce Network to unlock limitless possibilities and gain an edge over your competition. Check out Kin + Carta’s guide: Flipping the script — A new vision of legacy modernization enabled by digital decoupling . Get started with MongoDB Atlas on Google Cloud today.
A Smarter Factory Floor with MongoDB Atlas and Google Cloud's Manufacturing Data Engine
The manufacturing industry is undergoing a transformative shift from traditional to digital, propelled by data-driven insights, intelligent automation, and artificial intelligence. Traditional methods of data collection and analysis are no longer sufficient to keep pace with the demands of today's competitive landscape. This is precisely where Google Cloud’s Manufacturing Data Engine (MDE) and MongoDB Atlas come into play, offering a powerful combination for optimizing your factory floor. Unlock the power of your factory data MDE is positioned to transform the factory floor with the power of cloud-driven insights. MDE simplifies communication with your factory floor, regardless of the diverse protocols your machines might use. It effortlessly connects legacy equipment with modern systems, ensuring a comprehensive data stream. MDE doesn't just collect data, it transforms it. By intelligently processing and contextualizing the information, you gain a clearer picture of your production processes in real-time with a historical pretext. It offers pre-built analytics and AI tools directly addressing common manufacturing pain points. This means you can start finding solutions faster, whether it's identifying bottlenecks, reducing downtime, or optimizing resource utilization. Conveniently, it also offers great support for integrations that can further enhance the potential of the data (e.g. additional data sinks). The MongoDB Atlas developer data platform enhances MDE by providing scalability and flexibility through automated scaling to adapt to evolving data requirements. This makes it particularly suitable for dynamic manufacturing environments. MongoDB’s document model can handle diverse data types and structures effortlessly while being incredibly flexible because of its native JSON format. This allows for enriching MDE data with other relevant data, such as supply chain logistics, for a deeper understanding of the factory business. You can gain immediate insights into your operations through real-time analytics, enabling informed decision-making based on up-to-date data. While MDE offers a robust solution for collecting, contextualizing, and managing industrial data, leveraging it alongside MongoDB Atlas offers tremendous advantages Inside the MDE integration Google Cloud’s Manufacturing Data Engine (MDE) acts as a central hub for your factory data. It not only processes and enriches the data with context, but also offers flexible storage options like BigQuery and Cloud Storage. Now, customers already using MongoDB Atlas can skip the hassle of application re-integration and make this data readily accessible for applications. Through this joint solution developed by Google Cloud and MongoDB, you can seamlessly move the processed streaming data from MDE to MongoDB Atlas using Dataflow jobs. MDE publishes the data via a Pub/Sub subscription, which is then picked up by a custom Dataflow job built by MongoDB. This job transforms the data into the desired format and writes it to your MongoDB Atlas cluster. Google Cloud’s MDE and MongoDB Atlas utilize compatible data structures, simplifying data integration through a shared semantic configuration. Once the data resides in MongoDB Atlas, your existing applications can access it seamlessly without any code modifications, saving you time and effort. The flexibility of MDE, combined with the scalability and ease of use of MongoDB Atlas, makes this a powerful and versatile solution for various data-driven use cases such as predictive maintenance and quality control, while still providing factory ownership of the data. Instructions on how to set up the dataflow job are available in the MongoDB github repository. Conclusion If you want to level up your manufacturing data analytics, pairing MDE with MongoDB Atlas provides a proven, easy-to-implement solution. It's easy to get started with MDE and MongoDB Atlas .
Unleashing Developer Potential–and Managing Costs–with MongoDB Atlas
In today's business landscape, where unpredictability has become the norm, engineering leaders have to balance the dual challenges of managing uncertainty while optimizing IT costs. Indeed, the 2024 MarginPLUS Deloitte survey—which draws on insights from over 300 business leaders—emphasizes a collective pivot towards growth initiatives and cost transformations amidst the fluctuating global economic climate. MongoDB Atlas: A developer's ally for cost-effective productivity Executives across industries want to cut costs without impacting innovation; based on the Deloitte survey , 83% of companies are looking to change how they run their business margin improvement efforts. This is where MongoDB Atlas , the most advanced cloud database service on the market, comes in. An integrated suite of data services that simplify how developers build with data, MongoDB Atlas helps teams enhance their productivity without compromising on cost efficiency by offering visibility and control over spending—balancing developer freedom with data governance and cost management. This helps organizations escape the modernization hamster wheel—or the vicious cycle of continuously updating technology without making real progress, and draining resources while failing to deliver meaningful improvements. Put another way, MongoDB gives teams more time to innovate, instead of just maintaining the status quo. Outlined below are the built-in features of MongoDB Atlas, which enable customers to get the most out of their data while also focusing on budget optimization. Strategic features for cost optimization with MongoDB Atlas Right-sizing your cluster Use MongoDB Atlas’s Cluster Sizing Guide or auto-scalability to match your cluster with your workload, optimizing resource use with options for every requirement, including Low CPU options for lighter workloads. Pausing clusters and global distribution Save costs by pausing your cluster , and securely storing data for up to 30 days with auto-resume. Furthermore, Global Clusters improve performance across regions while maintaining cost efficiency and compliance. Index and storage management Enhance performance and reduce costs with MongoDB Atlas’s Performance Advisor , which provides tailored index and schema optimizations for better query execution and potential reductions in cluster size. Strategic data management Reduce storage expenses using Online Archive for infrequently accessed data and TTL indexes for efficient Time Series data management, ensuring only essential data is stored. Securely backup data before deletion with mongodump. Enhanced spend management Use spend analysis, billing alerts , and usage insights via the Billing Cost Explorer for detailed financial management and optimization. Resource tagging and customizable dashboards provide in-depth financial reporting and visual expense tracking, supporting effective budgeting and cost optimization. Additionally, Opt for serverless instances to adjust to your workload's scale, offering a pay-for-what-you-use model that eliminates overprovisioning concerns. Transforming uncertainty into advancement MongoDB Atlas equips IT decision-makers and developers with the features and tools to balance developer productivity with strategic cost management, transforming economic uncertainty into a platform for strategic advancement. MongoDB Atlas is more than a database management solution; it’s a strategic partner in optimizing your IT spending, ensuring that your organization remains agile, efficient, and cost-effective in the face of change. Need expert assistance in taking control of your MongoDB Atlas costs? MongoDB’s Professional Services team can provide a deep-dive assessment of your environment to build a tailored optimization plan—and to help you execute. Reach out to learn how we can support your cost optimization goals! If you haven't yet set up your free cluster on MongoDB Atlas , now is a great time to do so. You have all the instructions in this DevCenter article .
From Relational Databases to AI: An Insurance Data Modernization Journey
Imagine you’re a data architect, a developer, or a data engineer at an insurance company. Management has asked you and your team to build a new AI claim adjustment system, a customer-facing LLM-powered chatbot, and an application to streamline the underwriting process. However, doing so is far from straightforward due to the challenges you face on a daily basis. The bulk of your time is spent navigating your company’s outdated legacy systems, which were built in the 1970s and 1980s. Some of these legacy platforms were written in COBOL and CICS, and today very few people on your team know how to develop and maintain those technologies. Moreover, the data models you work with are another source of frustration. Every interaction with them is a reminder of the intricate structures that have evolved over time, making data manipulation and analysis a nightmare. In sum, legacy systems are preventing your team—and your company—from innovating and keeping up with both your industry and customer demands. Whether you’re trying to modernize your legacy systems to improve operational efficiency, boost developer productivity, or if you want to build AI-powered apps that integrate with large language models (LLMs), MongoDB has a solution for that. In this post, we’ll walk you through a journey that starts with a relational data model refactored into MongoDB collections, vectorization and querying of unstructured data and, finally, retrieval augmented generation (RAG) : asking large language models (LLMs) questions about data in natural language. Identifying, modernizing, and storing the data Our journey starts with an assessment of the data sources we want to work with. As shown below, we can bucket the data into three different categories: Structured legacy data: Tables of claims, coverages, billings, and more. Is your data locked in rigid relations schemas? This tutorial is a step-by-step guide on how to migrate a real-life insurance relational model with the help of MongoDB Relational Migrator , refactoring 21 tables to only five MongoDB collections. Structured data (JSON): You might have files of policies, insurance products, or forms in JSON format. Check out our docs to learn how to insert those into a MongoDB collection. Unstructured data (PDFs, Audios, Images, etc.): If you need to create and store a numerical representation (vector embedding) of, for instance, claim-related photos of accidents or PDFs of policy guidelines, you can have a look at this blog that will walk you through the process of generating embeddings of pictures of car crashes and persisting them alongside existing fields in a MongoDB collection. Figure 1: Storing different types of data into MongoDB Regardless of the original format or source, our data has finally landed into MongoDB Atlas into what we call a Converged AI Data Store, which is a platform that centrally integrates and organizes enterprise data, including vectors, that enable the development of ML- and AI-powered applications. Accessing, experimenting, and interacting with the data It’s time to put the data to work. The Converged AI Data Store unlocks a plethora of use cases and efficiency gains, both for the business and for developers. The next step of the journey is about the different ways we can interact with our data: Database and Full Text Search: Learn how to run database queries, start from the basics and move up to advanced features such as facets, fuzzy search, autocomplete, highlighting, and more with Atlas Search . Vector Search: We can finally leverage unstructured data. The Image Search blog we mentioned earlier also explains how to create a Vector Search index and run vector queries against embeddings of photos. RAG: Combining Vector Search and the power of LLMs, it is possible to interact in natural language with our data (see Figure 2 below), asking complex questions and getting detailed answers. Follow this tutorial to become a RAG expert. Figure 2: Retrieval augmented generation (RAG) diagram where we dynamically combine our custom data with the LLM to generate reliable and relevant outputs Having explored all the different ways we can ask questions of the data, we made it to the end of our journey. You are now ready to modernize your company’s systems and finally be able to keep up with the business’ demands. What will you build next? If you would like to discover more about Converged AI and Application Data Stores with MongoDB, take a look at the following resources: AI, Vectors, and the Future of Claims Processing: Why Insurance Needs to Understand The Power of Vector Databases Build a ML-Powered Underwriting Engine in 20 Minutes with MongoDB and Databricks