sendnrw/netbird-docs
2
0
Fork 0
mirror of https://github.com/netbirdio/docs.git synced 2026-05-04 08:16:35 +00:00
Code Issues Packages Projects Releases Wiki Activity

Traffic events (#290)

Browse Source 
* Add Network Flow Logs documentation and navigation entry

* Enhance Network Flow Logs documentation with updated retention policy and troubleshooting examples

* Enhance NetFlow documentation with detailed examples for traffic verification and troubleshooting

* Update section title in NetFlow documentation for clarity

* Enhance NetFlow documentation with additional examples and clarifications for flow log analysis

* Enhance NetFlow documentation with detailed troubleshooting guidance and insights on flow log utility for network visibility

* Enhance NetFlow documentation with updates on log retention policy, log shipping to DataDog

* Update NetFlow documentation to clarify use cases and enhance examples for TCP, UDP, and ICMP connections

* Enhance NetFlow documentation with new examples and clarifications for flow log analysis

* Enhance NetFlow documentation with improved formatting, additional details, and examples for flow log entries

* Enhance NetFlow documentation with additional images and improved clarity for peer-to-host and peer-to-subnet connection scenarios

* Update NetFlow documentation to include flow direction and clarify data volume measurement

* fix doc

* Update src/pages/how-to/netflow.mdx

Co-authored-by: Viktor Liu <17948409+lixmal@users.noreply.github.com>

* Update src/pages/how-to/netflow.mdx

Co-authored-by: Viktor Liu <17948409+lixmal@users.noreply.github.com>

* Update src/pages/how-to/netflow.mdx

Co-authored-by: Viktor Liu <17948409+lixmal@users.noreply.github.com>

* Update src/pages/how-to/netflow.mdx

Co-authored-by: Viktor Liu <17948409+lixmal@users.noreply.github.com>

* Update src/pages/how-to/netflow.mdx

Co-authored-by: Viktor Liu <17948409+lixmal@users.noreply.github.com>

* Update src/pages/how-to/netflow.mdx

Co-authored-by: Viktor Liu <17948409+lixmal@users.noreply.github.com>

* Update src/pages/how-to/netflow.mdx

Co-authored-by: Viktor Liu <17948409+lixmal@users.noreply.github.com>

* Clarify data volume measurement in NetFlow documentation and remove redundant peer-to-peer log entry explanations

* Remove redundant explanations in NetFlow documentation for clarity

* Replace instances of "flow log" with "traffic event" for consistency in NetFlow documentation

* add update image to reflect correct info and added retention info.

* add more information to the traffic events page and temporarily disable netflow page from navigation

* correct wording

---------

Co-authored-by: Viktor Liu <17948409+lixmal@users.noreply.github.com>
Co-authored-by: Maycon Santos <mlsmaycon@gmail.com>
This commit is contained in:
hakansa
2025-03-24 03:40:57 +08:00
committed by GitHub
parent 7507b931d5
commit dd9f4e24e6
17 changed files with 419 additions and 2 deletions
Download Patch File Download Diff File Expand all files Collapse all files

208
src/pages/how-to/traffic-events-logging.mdx
View File

@@ -20,7 +20,7 @@ you to better manage and secure your environment.
## Enabling Traffic Events Logging
Traffic events logging is disabled by default. To enable it on the NetBird dashboard, navigate to `Settings > Networks`.
Under the Experimental section, you’ll find the `Enable Traffic Events` option. Toggle the switch to enable traffic event logging.
Under the Experimental section, you'll find the `Enable Traffic Events` option. Toggle the switch to enable traffic event logging.
By default, traffic reporting in userspace is always enabled, providing basic logging of network interactions.
However, packet size reporting at the kernel level is disabled by default to minimize CPU usage.
@@ -35,6 +35,14 @@ at the kernel level. Be aware that enabling this option may lead to higher CPU u
<img src="/docs-static/img/how-to-guides/traffic-events-logging-settings.png" alt="traffic-events-logging-settings" className="imagewrapper-big"/>
</p>
## Log Retention
While in experimental mode, logs are retained for **48 hours**.
Additionally, please note that the current API returns a maximum of **50,000 events**.
We are actively working on expanding this limit in the coming days to support larger datasets and increased usage.
## Report rate
The events might take up to 10 minutes to become available via API and Dashboard. For some TCP connections, the full event cycle might take longer, depending on OS settings and connection termination.
## Enable Traffic Events Streaming to SIEM Systems
@@ -42,4 +50,200 @@ NetBird allows you to stream traffic events directly to your Security Informatio
By enabling this feature, you can seamlessly monitor and analyze NetBird network flow events within your existing SIEM infrastructure,
enhancing your ability to detect and respond to security events.
For detailed instructions on supported integrations and how to set them up, refer to the [integrations guide](/how-to/activity-event-streaming).
For detailed instructions on supported integrations and how to set them up, refer to the [integrations guide](/how-to/activity-event-streaming).
## Traffic Events Data
When enabled, a NetBird peer will record metadata for each network flow that it participates in. The data collected by peers includes:
* **Timestamp**: When the flow started and ended.
* **Flow ID**: A unique identifier for the traffic event flow.
* **Type**: The type of traffic event, such as Start, End, or Blocked.
* **Source and Destination IP Addresses**: The IP of the peer (source) and the IP of the remote endpoint (destination). For peer-to-peer traffic,
these will be the NetBird network IPs (e.g. 100.x.x.x addresses of each peer). For traffic to an external resource (like a private server or subnet), the destination might be an IP in that remote network.
* **Source and Destination Ports**: The network ports used by the connection (for TCP/UDP flows).
* **ICMP Code and Type**: For ICMP traffic, the ICMP code and type.
* **Protocol**: The protocol of the traffic, such as TCP, UDP, or ICMP.
* **Direction**: Whether the flow was inbound or outbound. This takes into consideration the perspective of the **peer reporting the traffic** and the NetBird interface.
* **Volume of Data**: The amount of data transferred, measured in number of packets and bytes sent/received for the duration of the flow.
* **Resource ID**: Network route or Networks resource ID that the flow is associated with. This is useful for identifying the routing configuration that allowed the flow. DNS route information is **available only** on the routing client.
* **Rule ID**: The ID of the policy that allowed the flow. This is useful for identifying the access control policy that allowed the flow. This information is **available only** on the receiving side of the traffic.
In addition to the data collected by the peers, the NetBird API provides additional context about the peers and resources involved in the traffic event. These details include:
* **Peer Name**: The name of the peer.
* **Peer ID**: The unique identifier of the peer.
* **Resource name**: The name of the resource or network route.
* **Policy Name**: The name of the policy that allowed the flow.
* **User ID, name, and email**: The name and email of the user associated with the source peer.
* **Reporter ID**: The unique identifier of the peer that reported the traffic event.
* **Received timestamp**: The timestamp when the event was received by the NetBird servers.
<details>
<summary>API sample response</summary>
```json
{
"destination": {
"address": "142.250.185.206:443",
"dns_label": "*.google.com",
"geo_location": {
"city_name": "",
"country_code": ""
},
"id": "cvco2st9q2cs73btphmg",
"name": "Any google.com domain",
"os": "",
"type": "DOMAIN_RESOURCE"
},
"direction": "EGRESS",
"flow_id": "9682d060-3b28-4fa3-8b47-98595a51bbda",
"icmp_code": 0,
"icmp_type": 0,
"id": "c94e398c-dbfb-4344-8c47-a731b984d86e",
"policy_id": "ndkslcanlksncl",
"policy_name": "Allow google access",
"protocol": 6,
"receive_timestamp": "2025-03-22T20:26:19.491144Z",
"reporter_id": "ldkfnwklenfklernl",
"rx_bytes": 0,
"rx_packets": 0,
"source": {
"address": "100.89.67.186:50229",
"dns_label": "macbook-pro-10-2",
"geo_location": {
"city_name": "Frankfurt",
"country_code": "DE"
},
"id": "ldkfnwklenfklernl",
"name": "MacBook-Pro-10.local",
"os": "Darwin",
"type": "PEER"
},
"timestamp": "2025-03-22T20:26:16.937522Z",
"tx_bytes": 64,
"tx_packets": 1,
"type": "TYPE_START",
"user_email": "john@example.com",
"user_id": "google-oauth2|xyz0123",
"user_name": "John Doe"
}
```
</details>
## Viewing Traffic Events on the Dashboard
There are two places where you can see the traffic events on the NetBird dashboard:
1. **Traffic events**: Under Activity, you will find the Traffic events menu. This view shows the traffic events in a table format for all peers in your network.
2. **Peer details**: When you click on a peer, you will see the traffic events for that peer in the Peer details view.
### Filters
You can use various filters to search and filter received events. The filters include:
- **Peer name**: Name of the peer that is the source or destination of the traffic event
- **Resource name**: Name of the resource or network route
- **IP address**: Source or destination IP addresses
- **Ports**: Source or destination ports
- **User**: User from the peer that initiated the connection
- **Timestamp**: Time range of the event
- **Protocol**: ICMP, TCP, UDP
- **Type of event**:
- **P2P connection started (inbound/outbound)**: Events with started status and initiated by peers to other peers
- **P2P connection stopped (inbound/outbound)**: Events with stopped status and initiated by peers to other peers
- **P2P connection blocked (inbound/outbound)**: Events with blocked status and initiated by peers to other peers
- **Routed connection started (inbound/outbound)**: Events with started status and with a remote resource destination
- **Routed connection stopped (inbound/outbound)**: Events with stopped status and with a remote resource destination
- **Routed connection blocked (inbound/outbound)**: Events with blocked status and with a remote resource destination
## Correlating events
Events can be correlated by observing the traffic from both peers involved in a traffic session. Let's say you have two peers, Peer A and Peer B, and Peer A initiates a connection to Peer B. In the traffic events,
you will see up to 4 events from these peers. If the connection was successful, you will see a started and a stopped event from Peer A and Peer B. But, if one peer blocks the connection, then you will see a started and stopped events from the initiator and
a blocked event from the responder.
<Note>
The blocked event will be available only if the destination uses the NetBird client in userspace mode.
This is the case for Windows and MacOs nodes. For Linux, you can enable userspace mode with the environment variable `NB_WG_KERNEL_DISABLED=true` and `NB_FORCE_USERSPACE_ROUTER=true`
</Note>
### Viewing TCP and UDP connections
You can use source ports to correlate TCP and UDP. Below we will analyze a few examples for a connection between a user computer and a Web and FTP servers.
The peer Maycons-MacBook-Pro.local initiates a connection to the Web server. The source port is `51997` and the destination port is TCP/80. The connection is successful, and the event is marked as started and stopped. See screenshot below:
<p>
<img src="/docs-static/img/how-to-guides/traffic-events/p2p-tcp-allow.png" alt="P2P TCP Allowed" className="imagewrapper-big"/>
</p>
Besides the ports and protocol information, we can review the event description in the screenshot above to understand what happened. For all four events we have the following:
- **Peer `Maycons-MacBook-Pro-2.local` requested P2P connection to Peer `webserver`**: This is the event from the perspective of the peer that initiated the connection. The sources and destination provide the IPs and ports used in the connection.
- **Peer `webserver` received P2P connection from Peer `Maycons-MacBook-Pro-2.local`**: This is the event from the perspective of the peer that received the connection.
- **Peer `Maycons-MacBook-Pro-2.local` stopped P2P connection to Peer `webserver`**: This is the event from the perspective of the peer that initiated the connection when the connection ended on its side.
- **Peer `webserver` stopped P2P connection from Peer `Maycons-MacBook-Pro-2.local`**: This is the event from the perspective of the peer that received the connection when the connection has ended on its side.
In case of the peer receiving the connection, the stopped status might arrive several minutes later, due to TCP sessions.
The UDP connection is very similar:
<p>
<img src="/docs-static/img/how-to-guides/traffic-events/p2p-udp-allow.png" alt="P2P UDP Allowed" className="imagewrapper-big"/>
</p>
<Note>
The UDP connection is stateless, so the stopped event will be generated right after a certain period of inactivity.
</Note>
When a connection is blocked, you may see similar entries to the following events but with a few differences:
**TCP**:
<p>
<img src="/docs-static/img/how-to-guides/traffic-events/p2p-tcp-blocked.png" alt="P2P TCP Blocked" className="imagewrapper-big"/>
</p>
**UDP**:
<p>
<img src="/docs-static/img/how-to-guides/traffic-events/p2p-udp-blocked.png" alt="P2P UDP Blocked" className="imagewrapper-big"/>
</p>
Key differences:
- There are no events that have started or stopped on the refusing side. The connection is blocked right after the request.
- Depending on the application making a request in the initiator, you may see multiple blocked events from the receiving side of the connection due to retries:
<Note>
The blocked event will be available only if the destination is using the NetBird client in userspace mode.
This is the case for Windows and MacOs nodes. For Linux, you can enable userspace mode with the environment variable `NB_WG_KERNEL_DISABLED=true` and `NB_FORCE_USERSPACE_ROUTER=true`
</Note>
### Viewing ICMP connections
ICMP events are similar to TCP and UDP events. The main difference is that ICMP doesn't have ports:
<p>
<img src="/docs-static/img/how-to-guides/traffic-events/p2p-icmp-allowed.png" alt="P2P ICMP Allowed" className="imagewrapper-big"/>
</p>
<Note>
The ICMP connection is stateless, so the stopped event will be generated right after a certain period of inactivity.
</Note>
### Routed events
Routed events follow the same pattern as P2P events. The main difference is that the destination or source can be a resource or network route. Below, we have a few examples of a connection from a peer to a resource:
**ICMP**:
<p>
<img src="/docs-static/img/how-to-guides/traffic-events/routed-icmp-allowed.png" alt="Routed ICMP Allowed" className="imagewrapper-big"/>
</p>
**TCP**:
<p>
<img src="/docs-static/img/how-to-guides/traffic-events/routed-tcp-allowed.png" alt="Routed TCP Allowed" className="imagewrapper-big"/>
</p>
Key differences:
- The source or destination is a resource or network route.
- On the receiver side, we will have an indication of which routing peer reported the event.
- The source or destination identifier will be unknown on the routing peer side if the route or resource is of a DNS type.
For site-2-site connections, the events will be similar to the above examples, but you will see a routing peer for each event:
<p>
<img src="/docs-static/img/how-to-guides/traffic-events/s2s-tcp-allowed.png" alt="S2S TCP Allowed" className="imagewrapper-big"/>
</p>
## Limitations
There are a few differences between the different Wireguard modes NetBird supports and the data captured by the NetBird agent.
| Feature | Kernel Mode | Userspace Mode | Netstack Mode |
|:---------:|:-------------:|:----------------:|:---------------:|
| Blocked traffic event | No | Yes | Yes |
| Site-2-Site events | No | Yes | Yes |
| Rule IDs | No | Yes | Yes |
| Allowed rule ID for routed events | Yes | No | No |
| Byte counters for routed events | Yes | No | No |
We are actively working to improve the data captured by the NetBird agent in Kernel and userspace modes to align with customers' expectations.
## Conclusion
Traffic events logging provides a powerful tool for monitoring and analyzing network traffic across your infrastructure.
Enabling this feature can provide valuable insights into network activity, enhance security measures, and improve operational efficiency.
The ability to correlate events, view detailed traffic data, and stream events to SIEM systems empowers you to make informed decisions
and take proactive steps to protect your network environment.