Docs Menu
Docs Home
/
MongoDB Atlas
/
Atlas Stream Processing

Manage Connection Registries

On this page

  • Prerequisites
  • View Connections in the Connection Registry
  • Add a Connection to the Connection Registry
  • Modify a Connection from the Connection Registry
  • Delete a Connection from the Connection Registry

Each Stream Processing Instance stores every configured connection to a streaming data source or sink in its connection registry. The procedures on this page walk you through creating and configuring your Atlas Stream Processing connections.

Prerequisites

To create and configure connections, you must have the following:

  • An Atlas project

  • An Atlas user with the Project Owner or the Project Stream Processing Owner role to manage a connection registry

    Note

    The Project Owner role allows you to create database deployments, manage project access and project settings, manage IP Access List entries, and more.

    The Project Stream Processing Owner role enables Atlas Stream Processing actions such as viewing, creating, deleting, and editing stream processing instances, and viewing, adding, modifying, and deleting connections in the connection registry.

    See Project Roles to learn more about the differences between the two roles.

  • An Atlas cluster

View Connections in the Connection Registry

To view a connection to your stream processing instance:

To list all connections from one stream processing instance you specify using the Atlas CLI, run the following command:

atlas streams connections list [options]

To return the details for one stream processing connection you specify using the Atlas CLI, run the following command:

atlas streams connections describe <streamConnectionName> [options]

To learn more about the syntax and parameters for the previous commands, see the Atlas CLI documentation for atlas streams connections list and atlas streams connections describe.

Tip

See: Related Links

1

In Atlas, go to the Stream Processing page for your project.

  1. If it's not already displayed, select the organization that contains your project from the Organizations menu in the navigation bar.

  2. If it's not already displayed, select your project from the Projects menu in the navigation bar.

  3. In the sidebar, click Stream Processing under the Services heading.

2

Go to the Connection Registry page.

  1. Locate the overview panel of the stream processing instance containing the connection registry you want to view.

  2. Click Configure.

  3. Select the Connection Registry tab.

Atlas Stream Processing displays all of the connections available in your instance connection registry.

The Atlas Administration API provides different endpoints for retrieving one or all connections in a connection registry.

View One Connection in a Connection Registry

List All Connections in a Connection Registry

To list all available connections using mongosh, use the sp.listConnections() method. It returns a list of documents describing each connection in the stream processing instance's connection registry. It has the following syntax:

sp.listConnections()

Add a Connection to the Connection Registry

Atlas Stream Processing supports the following connection types:

Each type has distinct configuration requirements.

Add an Apache Kafka Connection

To add a Kafka connection to your stream processing instance:

To create one connection for the stream processing instance you specify using the Atlas CLI, run the following command:

atlas streams connections create [connectionName] [options]

To learn more about the command syntax and parameters, see the Atlas CLI documentation for atlas streams connections create.

Tip

See: Related Links

When you create a stream processing instance using the Atlas CLI, you must provide a .json configuration file to define the new connection's parameters. The format of this file depends on the type of connection you define.

For an Apache Kafka connection, provide a configuration file with the following syntax:

{
  "name": "<name>",
  "type": "Kafka",
  "bootstrapServers": "<address:port>, ...",
  "security": {
    "protocol": "<protocol>"
  },
  "authentication": {
    "mechanism": "<auth-mechanism>",
    "username": "<username>",
    "password": "<password>"
  }
}
1

In Atlas, go to the Stream Processing page for your project.

  1. If it's not already displayed, select the organization that contains your project from the Organizations menu in the navigation bar.

  2. If it's not already displayed, select your project from the Projects menu in the navigation bar.

  3. In the sidebar, click Stream Processing under the Services heading.

2

Go to the Connection Registry.

  1. Locate the overview panel of the stream processing instance you want to modify and click Configure.

  2. Select the Connection Registry tab.

3

Click + Add connection.

4

Add a new connection.

  1. Select a Kafka connection.

  2. Provide a Connection Name. Each connection name must be unique within a stream processing instance. This is the name used to reference the connection in Atlas Stream Processing aggregations.

  3. Select a Network Access type. Atlas Stream Processing supports Public IP or VPC Peering connections.

    VPC peering for Atlas Stream Processing is currently in private preview. To learn more, contact your account team.

    Click the Public IP button. No further configuration is needed for this network access type.

    1. Click the VPC Peering button.

    2. Toggle Enable VPC Peering on.

    3. From the dropdown menu, select one of your existing VPC Peering connections. If you do not have a VPC peering connection, Configure an Atlas Network Peering Connection.

  4. Specify an IP address for one or more bootstrap servers for your Apache Kafka system.

  5. From the dropdown menu, select a Security Protocol Method.

    Atlas Stream Processing supports SASL_PLAINTEXT or SASL_SSL.

    SASL_PLAINTEXT is incompatible with VPC peering. To use VPC peering, you must select the SASL_SSL method.

    1. From the dropdown menu, select a SASL Mechanism.

      Atlas Stream Processing supports:

      • PLAIN

      • SCRAM-SHA-256

      • SCRAM-SHA-512

    2. Provide a Username for authentication.

    3. Provide a password for authentication.

    4. Click Add connection.

    1. From the dropdown menu, select a SASL Mechanism.

      Atlas Stream Processing supports:

      • PLAIN

      • SCRAM-SHA-256

      • SCRAM-SHA-512

    2. Click Upload to upload your Certificate Authority PEM file

    3. Provide a Username for authentication.

    4. Provide a password for authentication.

    5. Click Add connection.

The Atlas Administration API provides an endpoint for adding a connection to a connection registry.

Add a Connection to the Connection Registry

Important

After adding an external connection such as an Apache Kafka cluster to your connection registry, you must add Atlas IP addresses to an access list for that external connection. For more information, see Allow Access to or from the Atlas Control Plane.

Add an Atlas Connection

To add an Atlas database connection to your stream processing instance:

To create one connection for the stream processing instance you specify using the Atlas CLI, run the following command:

atlas streams connections create [connectionName] [options]

To learn more about the command syntax and parameters, see the Atlas CLI documentation for atlas streams connections create.

Tip

See: Related Links

When you create a stream processing instance using the Atlas CLI, you must provide a .json configuration file to define the new connection's parameters. The format of this file depends on the type of connection you define.

For an Atlas database connection, provide a configuration file with the following syntax:

{
  "name": "<name>",
  "type": "Cluster",
  "clusterName": "<clusterName>"
}
1

In Atlas, go to the Stream Processing page for your project.

  1. If it's not already displayed, select the organization that contains your project from the Organizations menu in the navigation bar.

  2. If it's not already displayed, select your project from the Projects menu in the navigation bar.

  3. In the sidebar, click Stream Processing under the Services heading.

2

Go to the Connection Registry.

  1. Locate the overview panel of the stream processing instance you want to modify and click Configure.

  2. Select the Connection Registry tab.

3

Click + Add connection.

4

Add a new connection.

To create a new connection to an Atlas change stream:

  1. Select an Atlas Database connection.

  2. Provide a Connection Name. Each connection name must be unique within an stream processing instance. This is the name used to reference the connection in Atlas Stream Processing aggregations.

  3. From the dropdown menu, select an Atlas Cluster. Atlas Stream Processing is only available on dedicated-tier clusters.

  4. Click Add connection.

The Atlas Administration API provides an endpoint for adding a connection to a connection registry.

Add a Connection to the Connection Registry

Add a Built-in Sample Connection

For a connection to the built-in sample_stream_solar, provide a configuration file with the following syntax:

To create one connection for the stream processing instance you specify using the Atlas CLI, run the following command:

atlas streams connections create [connectionName] [options]

To learn more about the command syntax and parameters, see the Atlas CLI documentation for atlas streams connections create.

Tip

See: Related Links

When you create a stream processing instance using the Atlas CLI, you must provide a .json configuration file to define the new connection's parameters. The format of this file depends on the type of connection you define.

For an sample connection, provide a configuration file with the following contents:

{
  "name": "sample_stream_solar",
  "type": "Sample"
}
1

In Atlas, go to the Stream Processing page for your project.

  1. If it's not already displayed, select the organization that contains your project from the Organizations menu in the navigation bar.

  2. If it's not already displayed, select your project from the Projects menu in the navigation bar.

  3. In the sidebar, click Stream Processing under the Services heading.

2

Go to the Connection Registry.

  1. Locate the overview panel of the stream processing instance you want to modify and click Configure.

  2. Select the Connection Registry tab.

3

Click + Add connection.

4

Add a new connection.

You can use a sample connection built into Atlas Stream Processing to develop and test new stream processors. The sample connection acts as a source of streaming data, but cannot be used as a sink.

To create a new connection to the sample stream:

  1. Select a Sample Stream connection.

  2. From the dropdown menu, select sample_stream_solar.

  3. Click Add connection.

The Atlas Administration API provides an endpoint for adding a connection to a connection registry.

Add a Connection to the Connection Registry

Modify a Connection from the Connection Registry

To modify a connection from a connection registry:

To modify the details of one connection on the stream processing instance you specify using the Atlas CLI, run the following command:

atlas streams connections update <connectionName> [options]

To learn more about the command syntax and parameters, see the Atlas CLI documentation for atlas streams connections update.

Tip

See: Related Links

1

In Atlas, go to the Stream Processing page for your project.

  1. If it's not already displayed, select the organization that contains your project from the Organizations menu in the navigation bar.

  2. If it's not already displayed, select your project from the Projects menu in the navigation bar.

  3. In the sidebar, click Stream Processing under the Services heading.

2

Go to the Connection Registry.

Locate the overview panel of the stream processing instance that contains the connection registry that you want to modify and click Configure in the upper right.

3

Select the Connection Registry tab.

4

Modify the connection.

  1. For the connection that you want to modify in the table, click the pencil icon in the Actions column.

    You can edit a connection only if it is not currently in use by a stream processor. If you want to edit a connection in use, stop all stream processors that use the connection first.

  2. Modify the fields you want to modify.

  3. Click Save changes.

The Atlas Administration API provides an endpoint for editing a stream processing instance.

Delete One Stream Instance

Delete a Connection from the Connection Registry

To delete a connection from a connection registry:

To remove one connection from the stream processing instance you specify using the Atlas CLI, run the following command:

atlas streams connections delete <connectionName> [options]

To learn more about the command syntax and parameters, see the Atlas CLI documentation for atlas streams connections delete.

Tip

See: Related Links

1

In Atlas, go to the Stream Processing page for your project.

  1. If it's not already displayed, select the organization that contains your project from the Organizations menu in the navigation bar.

  2. If it's not already displayed, select your project from the Projects menu in the navigation bar.

  3. In the sidebar, click Stream Processing under the Services heading.

2

Go to the Connection Registry.

  1. Locate the overview panel of the stream processing instance that contains the connection registry that you want to modify

  2. Click Configure in the upper right.

  3. Select the Connection Registry tab.

3

Delete the connection.

  1. For the connection you want to modify in the table, click the trash can icon in the Actions column of that entry.

  2. When prompted, click Delete.

The Atlas Administration API provides an endpoint for editing a stream processing instance.

Delete One Stream Instance

Back

Manage Stream Processing Instances

Next

Manage Stream Processors

On this page