Add event streaming to AWS (#180)

This commit is contained in:
pascal-fischer
2024-04-19 19:33:18 +02:00
committed by GitHub
parent d74dfcbf2c
commit fd209d9486
8 changed files with 122 additions and 2 deletions

Binary file not shown.

Before

Width:  |  Height:  |  Size: 264 KiB

After

Width:  |  Height:  |  Size: 158 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 256 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 126 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 234 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 184 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 258 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 131 KiB

View File

@@ -8,7 +8,7 @@ This document provides step-by-step instructions and best practices for setting
## Datadog
Before you start creating and configuring a Datadog event-streaming integration, ensure that you have the following:
Before you start creating and configuring a Datadog event streaming integration, ensure that you have the following:
- A Datadog account with the permissions to create and manage API keys. If you don't have the required permissions, ask your Datadog administrator to grant them to you.
### Step 1: Create a Datdog API key
@@ -17,7 +17,7 @@ Before you start creating and configuring a Datadog event-streaming integration,
- Give it a descriptive name like `NetBird Event Streaming`
- Click `Create Key`
- Copy the key. You will need this key when configuring an integration in NetBird.
### Step 2: Create an event-streaming integration in NetBird
### Step 2: Create an event streaming integration in NetBird
- Navigate to the [Integrations » Event Streaming](https://preview.netbird.io/integrations) tab in the NetBird dashboard
<p>
<img src="/docs-static/img/how-to-guides/event-streaming-integration.png" alt="event-streaming-integration" className="imagewrapper-big"/>
@@ -31,4 +31,124 @@ Before you start creating and configuring a Datadog event-streaming integration,
- Then enter the API key you created in [Step 1](#step-1-create-a-datdog-api-key) and click `Connect`
<p>
<img src="/docs-static/img/how-to-guides/datadog-api-key.png" alt="datadog-api-key" className="imagewrapper" width="400" style={{ display: 'block', marginLeft: 'auto', marginRight: 'auto' }} />
</p>
## Amazon S3
Before you start creating and configuring an Amazon S3 event streaming integration, ensure that you have the following:
- An AWS account with the permissions to create and manage S3 buckets.
- Permissions to create and manage IAM users, roles and policies.
If you don't have the required permissions, ask your AWS administrator to grant them to you.
### Step 1: Create an S3 bucket
- Navigate to the [S3 dashboard](https://console.aws.amazon.com/s3/home)
- Select the correct region in the top menu
- Click `Create bucket`
- Give it a descriptive name like `netbird-activity-events`
- (Optional) Change bucket configurations to your needs
- Click `Create bucket`
### Step 2: Create an IAM user
- Navigate to the [IAM Dashboard](https://console.aws.amazon.com/iam/home)
- Create an IAM User (for details see the [Amazon Docs](https://docs.aws.amazon.com/IAM/latest/UserGuide/id_users_create.html))
- Create a custom policy with the following permissions:
```json
{
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Action": [
"s3:PutObject",
"s3:PutObjectAcl"
],
"Resource": "arn:aws:s3:::netbird-activity-events/*"
}
]
}
```
- Attach the policy to the IAM user
- Select the user and navigate to the `Security credentials` tab
- Click `Create access key`
- Select `Third-party service` and click `Next`
- Give it a description
- Store `Access key` and `Secret access key` in a secure place. You will need these when configuring an integration in NetBird.
### Step 3: Create an event streaming integration in NetBird
- Navigate to the [Integrations &raquo; Event Streaming](https://preview.netbird.io/integrations) tab in the NetBird dashboard
<p>
<img src="/docs-static/img/how-to-guides/event-streaming-integration.png" alt="event-streaming-integration" className="imagewrapper-big"/>
</p>
- Enable and configure the `Amazon S3` integration
- First select the region your S3 bucket is created in
<p>
<img src="/docs-static/img/how-to-guides/s3-region-select.png" alt="s3-region-select" className="imagewrapper" width="400" style={{ display: 'block', marginLeft: 'auto', marginRight: 'auto' }} />
</p>
- Then enter the S3 bucket name you created in [Step 1](#step-1-create-an-s3-bucket) and click `Next`
<p>
<img src="/docs-static/img/how-to-guides/s3-bucket-name.png" alt="s3-bucket-name" className="imagewrapper" width="400" style={{ display: 'block', marginLeft: 'auto', marginRight: 'auto' }} />
</p>
- Enter the `Access key` and `Secret access key` you created in [Step 2](#step-2-create-an-iam-user) and click `Connect`
<p>
<img src="/docs-static/img/how-to-guides/s3-iam-credentials.png" alt="s3-iam-credentials" className="imagewrapper" width="400" style={{ display: 'block', marginLeft: 'auto', marginRight: 'auto' }} />
</p>
## Amazon Data Firehose
Before you start creating and configuring an Amazon Data Firehose event streaming integration, ensure that you have the following:
- An AWS account with the permissions to create and manage data firehose delivery streams.
- Permissions to create and manage IAM users, roles and policies.
If you don't have the required permissions, ask your AWS administrator to grant them to you.
### Step 1: Create a Data Firehose stream
- Navigate to the [Data Firehose Dashboard](https://console.aws.amazon.com/firehose/home)
- Click `Create Firehose stream`
- As source select `Direct PUT` and the desired destination
- Give it a descriptive name like `netbird-activity-events` and configure the stream to your needs
### Step 2: Create an IAM user
- Navigate to the [IAM Dashboard](https://console.aws.amazon.com/iam/home)
- Create an IAM User (for details see the [Amazon Docs](https://docs.aws.amazon.com/IAM/latest/UserGuide/id_users_create.html))
- Create a custom policy with the following permissions (replace the resource with the ARN of your delivery stream):
```json
{
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Action": [
"firehose:PutRecord",
"firehose:PutRecordBatch"
],
"Resource": "arn:aws:firehose:region:accountID:deliverystream/netbird-event-streaming"
}
]
}
```
- Attach the policy to the IAM user
- Select the user and navigate to the `Security credentials` tab
- Click `Create access key`
- Select `Third-party service` and click `Next`
- Give it a description
- Store `Access key` and `Secret access key` in a secure place. You will need these when configuring an integration in NetBird.
### Step 3: Create an event streaming integration in NetBird
- Navigate to the [Integrations &raquo; Event Streaming](https://preview.netbird.io/integrations) tab in the NetBird dashboard
<p>
<img src="/docs-static/img/how-to-guides/event-streaming-integration.png" alt="event-streaming-integration" className="imagewrapper-big"/>
</p>
- Enable and configure the `Amazon Data Firehose` integration
- First select the region your Firehose stream is created in
<p>
<img src="/docs-static/img/how-to-guides/firehose-region-select.png" alt="firehose-region-select" className="imagewrapper" width="400" style={{ display: 'block', marginLeft: 'auto', marginRight: 'auto' }} />
</p>
- Then enter the Firehose stream name you created in [Step 1](#step-1-create-a-data-firehose-stream) and click `Next`
<p>
<img src="/docs-static/img/how-to-guides/firehose-stream-name.png" alt="firehose-stream-name" className="imagewrapper" width="400" style={{ display: 'block', marginLeft: 'auto', marginRight: 'auto' }} />
</p>
- Enter the `Access key` and `Secret access key` you created in [Step 2](#step-2-create-an-iam-user-2) and click `Connect`
<p>
<img src="/docs-static/img/how-to-guides/firehose-iam-credentials.png" alt="firehose-iam-credentials" className="imagewrapper" width="400" style={{ display: 'block', marginLeft: 'auto', marginRight: 'auto' }} />
</p>