Step 1: Create a Kafka user
Last updated
Was this helpful?
Last updated
Was this helpful?
Superstream requires a Kafka user with the below configuration to be able to communicate and analyze connected clusters.
For effective functioning, a user or token requires the following permissions:
Cluster-level:
Describe all topics
, List all topics, Describe configs, Describe cluster
Topic-level:
Read: All topics
Alter: All topics
Delete: All topics
Describe: All topics
Alter: All topics
AlterConfigs: All topics
DescribeConfigs: All topics
Consumer group-level:
Describe
List Consumer Groups
ACL statements examples:
The following information will be required for each cluster:
Bootstrap servers (Kafka URL)
Authentication security protocol (No auth / SSL / SASL_SSL)
SSL with validation "on" would require a key.pem
,cert.pem
, and ca.pem
JMX port and token
For connecting Confluent Cloud clusters to Superstream, two types of API keys are required to be created:
In Confluent Console: Top-right menu -> Accounts & access -> Accounts -> Service Accounts -> "Add service account"
In the "Add service account" wizard:
Name the service account "Superstream
"
Permissions ("+ Add role assignment"):
For each organization: BillingAdmin
, ResourceKeyAdmin
, and MetricsViewer
For each environment: MetricsViewer
,DataDiscovery
, Operator
For environment -> Schema Registry
Select resource: All schema subjects
Select role: ResourceOwner
For each cluster: CloudClusterAdmin
, MetricsViewer
For each designated cluster -> Topics
DeveloperRead
: All topics
DeveloperManage
: All topics
For each designated cluster -> Consumer Groups
Read all Consumer group
In Confluent Console: Top-right menu -> API Keys -> + Add API key
Follow the following steps:
Create and save the newly created credentials using the cluster name.
In Confluent Console: Left menu -> Home -> Environments -> <environment name>
-> <cluster name>
-> API Keys
Click on "+ Add key"
Choose "Service account" -> "Superstream
" (The one we created in Step 1)
ACLs:
Cluster
ALTER_CONFIGS
: ALLOW
DESCRIBE
: ALLOW
DESCRIBE_CONFIGS
: ALLOW
Consumer Group
Rule 1:
Consumer group ID: *
Pattern type: LITERAL
Operation: Delete
Permission: ALLOW
Rule 2:
Consumer group ID: *
Pattern type: LITERAL
Operation: Describe
Permission: ALLOW
Rule 3:
Consumer group ID: *
Pattern type: LITERAL
Operation: Read
Permission: ALLOW
Topic
Rule 1:
Topic name: *
Pattern type: LITERAL
Operation: ALTER
Permission: ALLOW
Rule 2:
Topic name: *
Pattern type: LITERAL
Operation: ALTER_CONFIGS
Permission: ALLOW
Rule 3:
Topic name: *
Pattern type: LITERAL
Operation: DELETE
Permission: ALLOW
Rule 4:
Topic name: *
Pattern type: LITERAL
Operation: DESCRIBE
Permission: ALLOW
Rule 5:
Topic name: *
Pattern type: LITERAL
Operation: DESCRIBE_CONFIGS
Permission: ALLOW
Rule 6:
Topic name: superstream
Pattern type: LITERAL
Operation: Create
Permission: ALLOW
Rule 7:
Topic name: *
Pattern type: LITERAL
Operation: READ
Permission: ALLOW
Create and save the newly created credentials using the cluster name.
Be sure you’re signed in to the AWS Console with your default browser, then click here to:
Enter required parameters (e.g., NodeGroupRoleArn).
Acknowledge IAM resource creation.
Click Create Stack or Update Stack (choose Update Stack if the Superstream IAM role already exists).
Confirm status: CREATE_COMPLETE or UPDATE_COMPLETE.
Click on Outputs to get IAM Role details:
Be sure you’re signed in to the AWS Console with your default browser, then click here to:
Acknowledge IAM resource creation.
Click Create Stack or Update Stack (choose Update Stack if the Superstream IAM user already exists).
Confirm status: CREATE_COMPLETE or UPDATE_COMPLETE.
Click on Outputs to get the programmatic user details.
Create a new Access secret key for the user and use it in SSM Console to connect the new cluster.