mirror of
https://github.com/netbirdio/docs.git
synced 2026-04-16 07:26:35 +00:00
Refer to the new config.yaml file (#628)
This commit is contained in:
@@ -47,7 +47,7 @@ Changing servers erases the device's current NetBird config, so you'll need to c
|
|||||||
|
|
||||||
<img src="/docs-static/img/get-started/android-tv/change-server.png" alt="confirm-erase-cponfig" className="imagewrapper-large"/>
|
<img src="/docs-static/img/get-started/android-tv/change-server.png" alt="confirm-erase-cponfig" className="imagewrapper-large"/>
|
||||||
|
|
||||||
Enter your management server endpoint. For cloud users, this is `https://api.netbird.io:443`. For self-hosted users, it's usually `https://your_management_server_url:443`, but you can refer to your management.json configuration file if you're unsure.
|
Enter your management server endpoint. For cloud users, this is `https://api.netbird.io:443`. For self-hosted users, it's usually `https://your_management_server_url:443`, but you can refer to the `exposedAddress` field in your `config.yaml` (or `management.json` for older multi-container setups) if you're unsure.
|
||||||
|
|
||||||
If enrolling the device with a setup key, select '+ Add this device with a setup key' and enter your setup key. Select 'change' to apply your new management server config, and if successful you'll see the following:
|
If enrolling the device with a setup key, select '+ Add this device with a setup key' and enter your setup key. Select 'change' to apply your new management server config, and if successful you'll see the following:
|
||||||
|
|
||||||
|
|||||||
@@ -43,7 +43,7 @@ Changing servers erases the device's current NetBird config, so you'll need to c
|
|||||||
|
|
||||||
<img src="/docs-static/img/get-started/android/confirm-erase-config.png" alt="confirmeraseconfig" className="imagewrapper-medium"/>
|
<img src="/docs-static/img/get-started/android/confirm-erase-config.png" alt="confirmeraseconfig" className="imagewrapper-medium"/>
|
||||||
|
|
||||||
Enter your management server endpoint. For cloud users, this is `https://api.netbird.io:443`. For self-hosted users, it's usually `https://your_management_server_url:443`, but you can refer to your management.json configuration file if you're unsure.
|
Enter your management server endpoint. For cloud users, this is `https://api.netbird.io:443`. For self-hosted users, it's usually `https://your_management_server_url:443`, but you can refer to the `exposedAddress` field in your `config.yaml` (or `management.json` for older multi-container setups) if you're unsure.
|
||||||
|
|
||||||
If enrolling the device with a setup key, select '+ Add this device with a setup key' and enter your setup key. Select 'change' to apply your new management server config, and if successful you'll see the following:
|
If enrolling the device with a setup key, select '+ Add this device with a setup key' and enter your setup key. Select 'change' to apply your new management server config, and if successful you'll see the following:
|
||||||
|
|
||||||
|
|||||||
@@ -30,7 +30,7 @@ Navigate to **'Settings > Change Server'**, and the app will warn you that chang
|
|||||||
<img src="/docs-static/img/get-started/tvos/confirm-change-server.png" alt="tvos-confirm-change-server" className="imagewrapper-large"/>
|
<img src="/docs-static/img/get-started/tvos/confirm-change-server.png" alt="tvos-confirm-change-server" className="imagewrapper-large"/>
|
||||||
|
|
||||||
After confirming, you'll be presented with a dialog to enter your management server URL. Cloud users who want to enroll the device with a setup key should enter the URL For cloud users, this is `https://api.netbird.io:443`.
|
After confirming, you'll be presented with a dialog to enter your management server URL. Cloud users who want to enroll the device with a setup key should enter the URL For cloud users, this is `https://api.netbird.io:443`.
|
||||||
For self-hosted users, it's usually `https://your_management_server_url:443`, but you can refer to your management.json configuration file if you're unsure.
|
For self-hosted users, it's usually `https://your_management_server_url:443`, but you can refer to the `exposedAddress` field in your `config.yaml` (or `management.json` for older multi-container setups) if you're unsure.
|
||||||
|
|
||||||
If enrolling the device with a setup key, select '+ Add this device with a setup key' and enter your setup key. Select 'Change' to apply your new management server config. NetBird will run a brief verification step
|
If enrolling the device with a setup key, select '+ Add this device with a setup key' and enter your setup key. Select 'Change' to apply your new management server config. NetBird will run a brief verification step
|
||||||
on the details you've entered, after which you can move on to connecting.
|
on the details you've entered, after which you can move on to connecting.
|
||||||
|
|||||||
@@ -120,7 +120,7 @@ Future versions will also support connection events that occur in NetBird agents
|
|||||||
<Note>
|
<Note>
|
||||||
The `unknown` name or `unknown@unknown.com` email address may be displayed in the activity event store if the encryption key has been corrupted or lost. This issue is most relevant for self-hosted setups. In this case, the events returned by the API could show `unknown@unknown.com` for the email address field and `unknown` for the name field.
|
The `unknown` name or `unknown@unknown.com` email address may be displayed in the activity event store if the encryption key has been corrupted or lost. This issue is most relevant for self-hosted setups. In this case, the events returned by the API could show `unknown@unknown.com` for the email address field and `unknown` for the name field.
|
||||||
|
|
||||||
If the configuration files have been generated by the `configure.sh` script, you can find the previous encryption key in the backup files in the same folder as the script. Look for the <b>DataStoreEncryptionKey</b> field in the `management.json` backup file.
|
For the combined setup, the encryption key is the <b>server.store.encryptionKey</b> field in <b>config.yaml</b>. For older multi-container setups generated by the <b>configure.sh</b> script, look for the <b>DataStoreEncryptionKey</b> field in the <b>management.json</b> backup file.
|
||||||
</Note>
|
</Note>
|
||||||
|
|
||||||
## Enable Audit Events Streaming to SIEM Systems
|
## Enable Audit Events Streaming to SIEM Systems
|
||||||
|
|||||||
@@ -267,7 +267,7 @@ environment:
|
|||||||
Configuration values are applied in the following order (later values override earlier):
|
Configuration values are applied in the following order (later values override earlier):
|
||||||
|
|
||||||
1. Default values
|
1. Default values
|
||||||
2. Configuration file (`management.json`)
|
2. Configuration file (`config.yaml` for combined setup, `management.json` for older multi-container setup)
|
||||||
3. Environment variables
|
3. Environment variables
|
||||||
4. Command-line flags
|
4. Command-line flags
|
||||||
|
|
||||||
@@ -275,6 +275,6 @@ Configuration values are applied in the following order (later values override e
|
|||||||
|
|
||||||
- [Self-hosting Quickstart Guide](/selfhosted/selfhosted-quickstart)
|
- [Self-hosting Quickstart Guide](/selfhosted/selfhosted-quickstart)
|
||||||
- [Advanced Self-hosting Guide](/selfhosted/selfhosted-guide)
|
- [Advanced Self-hosting Guide](/selfhosted/selfhosted-guide)
|
||||||
- [Configuration Files Reference](/selfhosted/configuration-files) - Detailed documentation for docker-compose.yml, management.json, and other config files
|
- [Configuration Files Reference](/selfhosted/configuration-files) - Detailed documentation for docker-compose.yml, config.yaml, and other config files
|
||||||
- [Management SQLite Store](/selfhosted/sqlite-store)
|
- [Management SQLite Store](/selfhosted/sqlite-store)
|
||||||
- [Management Postgres Store](/selfhosted/postgres-store)
|
- [Management Postgres Store](/selfhosted/postgres-store)
|
||||||
|
|||||||
@@ -38,7 +38,32 @@ NetBird will refuse to disable local authentication if no external identity prov
|
|||||||
|
|
||||||
## Configuration
|
## Configuration
|
||||||
|
|
||||||
Update your `management.json` to disable local authentication:
|
### Combined setup (config.yaml)
|
||||||
|
|
||||||
|
Set `localAuthDisabled` to `true` in your `config.yaml`:
|
||||||
|
|
||||||
|
```yaml
|
||||||
|
server:
|
||||||
|
auth:
|
||||||
|
localAuthDisabled: true
|
||||||
|
```
|
||||||
|
|
||||||
|
Restart the server:
|
||||||
|
```bash
|
||||||
|
docker compose restart netbird-server
|
||||||
|
```
|
||||||
|
|
||||||
|
<Note>
|
||||||
|
For a full list of available configuration options, see the [config.yaml.example](https://github.com/netbirdio/netbird/blob/main/combined/config.yaml.example) reference file.
|
||||||
|
</Note>
|
||||||
|
|
||||||
|
### Older multi-container setup (management.json)
|
||||||
|
|
||||||
|
<Note>
|
||||||
|
This section applies to deployments using the older multi-container architecture. If you deployed using [`getting-started.sh`](/selfhosted/selfhosted-quickstart), you are on the combined setup and should use the `config.yaml` instructions above. See the [migration guide](/selfhosted/migration/combined-container) to upgrade.
|
||||||
|
</Note>
|
||||||
|
|
||||||
|
Update your `management.json`:
|
||||||
|
|
||||||
```json
|
```json
|
||||||
{
|
{
|
||||||
@@ -49,12 +74,18 @@ Update your `management.json` to disable local authentication:
|
|||||||
}
|
}
|
||||||
```
|
```
|
||||||
|
|
||||||
After updating the configuration, restart the Management service.
|
Restart the Management service:
|
||||||
The local login option will no longer appear on the login page—users will only see the configured external identity providers.
|
```bash
|
||||||
|
docker compose restart management
|
||||||
|
```
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
After updating the configuration, the local login option will no longer appear on the login page — users will only see the configured external identity providers.
|
||||||
|
|
||||||
## Re-enabling Local Authentication
|
## Re-enabling Local Authentication
|
||||||
|
|
||||||
To restore local authentication, set `LocalAuthDisabled` to `false` (or remove the option) and restart the Management service.
|
To restore local authentication, set `localAuthDisabled` back to `false` in `config.yaml` (or `LocalAuthDisabled` in `management.json`) and restart the server.
|
||||||
All previously created local users will be able to log in again with their existing passwords.
|
All previously created local users will be able to log in again with their existing passwords.
|
||||||
|
|
||||||
|
|
||||||
|
|||||||
@@ -41,7 +41,39 @@ Consider a [standalone external IdP](/selfhosted/selfhosted-guide#step-3-configu
|
|||||||
|
|
||||||
### Enabling Embedded IdP
|
### Enabling Embedded IdP
|
||||||
|
|
||||||
The embedded IdP is enabled by default when using the new [`getting-started.sh`](/selfhosted/selfhosted-quickstart) quickstart script. For manual configuration, update your `management.json`:
|
The embedded IdP is enabled by default when using the [`getting-started.sh`](/selfhosted/selfhosted-quickstart) quickstart script and requires no manual configuration.
|
||||||
|
|
||||||
|
#### Combined setup (config.yaml)
|
||||||
|
|
||||||
|
The embedded IdP is always enabled in the combined setup and cannot be disabled. The relevant settings in `config.yaml` are:
|
||||||
|
|
||||||
|
```yaml
|
||||||
|
server:
|
||||||
|
auth:
|
||||||
|
issuer: "https://netbird.example.com/oauth2"
|
||||||
|
localAuthDisabled: false
|
||||||
|
signKeyRefreshEnabled: true
|
||||||
|
dashboardRedirectURIs:
|
||||||
|
- "https://netbird.example.com/nb-auth"
|
||||||
|
- "https://netbird.example.com/nb-silent-auth"
|
||||||
|
cliRedirectURIs:
|
||||||
|
- "http://localhost:53000/"
|
||||||
|
|
||||||
|
store:
|
||||||
|
encryptionKey: "<auto-generated-key>"
|
||||||
|
```
|
||||||
|
|
||||||
|
<Note>
|
||||||
|
For a full list of available configuration options, see the [config.yaml.example](https://github.com/netbirdio/netbird/blob/main/combined/config.yaml.example) reference file.
|
||||||
|
</Note>
|
||||||
|
|
||||||
|
#### Older multi-container setup (management.json)
|
||||||
|
|
||||||
|
<Note>
|
||||||
|
This section applies to deployments using the older multi-container architecture. If you deployed using `getting-started.sh`, you are on the combined setup and should use the `config.yaml` instructions above. See the [migration guide](/selfhosted/migration/combined-container) to upgrade.
|
||||||
|
</Note>
|
||||||
|
|
||||||
|
Update your `management.json`:
|
||||||
|
|
||||||
```json
|
```json
|
||||||
{
|
{
|
||||||
@@ -53,17 +85,7 @@ The embedded IdP is enabled by default when using the new [`getting-started.sh`]
|
|||||||
}
|
}
|
||||||
```
|
```
|
||||||
|
|
||||||
### Configuration Options
|
Or configure via environment variables on the `management` container:
|
||||||
|
|
||||||
| Option | Description | Default |
|
|
||||||
|--------|-------------|---------|
|
|
||||||
| `EmbeddedIdP.Enabled` | Enable/disable the embedded IdP | `true` (quickstart) |
|
|
||||||
| `EmbeddedIdP.DataDir` | Directory for IdP data storage | `/var/lib/netbird/idp` |
|
|
||||||
| `EncryptionKey` | Base64-encoded AES-256 key for encrypting user data | Auto-generated |
|
|
||||||
|
|
||||||
### Environment Variables
|
|
||||||
|
|
||||||
When using docker-compose, you can configure these via environment variables:
|
|
||||||
|
|
||||||
```yaml
|
```yaml
|
||||||
environment:
|
environment:
|
||||||
@@ -330,7 +352,7 @@ The embedded IdP encrypts sensitive user data at rest:
|
|||||||
| Name | AES-256-GCM |
|
| Name | AES-256-GCM |
|
||||||
| Password | bcrypt hash (via Dex) |
|
| Password | bcrypt hash (via Dex) |
|
||||||
|
|
||||||
The encryption key is configured in `management.json` and should be:
|
The encryption key is configured in `config.yaml` (`server.store.encryptionKey`) or `management.json` (`EncryptionKey`) for older multi-container setups. It should be:
|
||||||
|
|
||||||
- Generated using a cryptographically secure random generator
|
- Generated using a cryptographically secure random generator
|
||||||
- Stored securely (not in version control)
|
- Stored securely (not in version control)
|
||||||
@@ -364,17 +386,23 @@ User authentication events are logged in the activity log:
|
|||||||
|
|
||||||
### "Embedded IdP not available" error
|
### "Embedded IdP not available" error
|
||||||
|
|
||||||
Ensure `EmbeddedIdP.Enabled` is `true` in `management.json` and the Management service has been restarted.
|
For the combined setup, the embedded IdP is always enabled. For older multi-container setups, ensure `EmbeddedIdP.Enabled` is `true` in `management.json` and restart the Management service.
|
||||||
|
|
||||||
### Users can't log in
|
### Users can't log in
|
||||||
|
|
||||||
1. Check Management service logs: `docker compose logs management`
|
1. Check server logs: `docker compose logs netbird-server` (or `docker compose logs management` for older multi-container setups)
|
||||||
2. Verify the encryption key hasn't changed
|
2. Verify the encryption key hasn't changed
|
||||||
3. Confirm the user exists: Check **Team** → **Users** in Dashboard
|
3. Confirm the user exists: Check **Team** → **Users** in Dashboard
|
||||||
|
|
||||||
## Disabling Embedded IdP
|
## Disabling Embedded IdP
|
||||||
|
|
||||||
To switch from embedded IdP to a (standalone) external IdP:
|
<Note>
|
||||||
|
The embedded IdP cannot be disabled in the combined setup. If you want to use only external identity providers, you can [disable local authentication](/selfhosted/identity-providers/disable-local-authentication) instead and add your external IdP through the Dashboard UI.
|
||||||
|
</Note>
|
||||||
|
|
||||||
|
### Older multi-container setup (management.json)
|
||||||
|
|
||||||
|
To switch from embedded IdP to a standalone external IdP on the older multi-container setup:
|
||||||
|
|
||||||
1. Configure your external IdP following the [Advanced guide](/selfhosted/selfhosted-guide)
|
1. Configure your external IdP following the [Advanced guide](/selfhosted/selfhosted-guide)
|
||||||
2. Update `management.json`:
|
2. Update `management.json`:
|
||||||
|
|||||||
@@ -2,6 +2,10 @@
|
|||||||
|
|
||||||
This guide helps you migrate your existing NetBird self-hosted installation from using an external Coturn TURN server to the new embedded STUN server integrated into the relay service.
|
This guide helps you migrate your existing NetBird self-hosted installation from using an external Coturn TURN server to the new embedded STUN server integrated into the relay service.
|
||||||
|
|
||||||
|
<Warning>
|
||||||
|
This guide applies only to the **older multi-container setup** with separate `management`, `signal`, `relay`, and `coturn` containers configured via `management.json`. If you deployed using the [`getting-started.sh`](/selfhosted/selfhosted-quickstart) script (combined setup with `config.yaml`), STUN is already embedded — no migration is needed. If you are still on the older setup, consider migrating to the [combined container](/selfhosted/migration/combined-container) instead, which handles this automatically.
|
||||||
|
</Warning>
|
||||||
|
|
||||||
<Note>
|
<Note>
|
||||||
**Who is this guide for?** This migration guide is intended for users who:
|
**Who is this guide for?** This migration guide is intended for users who:
|
||||||
- Have an existing NetBird self-hosted deployment with **Coturn configured for STUN** (NAT traversal)
|
- Have an existing NetBird self-hosted deployment with **Coturn configured for STUN** (NAT traversal)
|
||||||
|
|||||||
@@ -244,7 +244,7 @@ The reverse proxy SSO feature authenticates users through an OAuth2/OIDC flow th
|
|||||||
|
|
||||||
If you want to keep using your current external identity provider, follow these three steps:
|
If you want to keep using your current external identity provider, follow these three steps:
|
||||||
|
|
||||||
#### Step 1: Add callback URL to management.json
|
#### Step 1: Add callback URL to management.json (multi-container only)
|
||||||
|
|
||||||
Add the `AuthCallbackURL` and `AuthClientID` fields to the `HttpConfig` section of your `management.json`:
|
Add the `AuthCallbackURL` and `AuthClientID` fields to the `HttpConfig` section of your `management.json`:
|
||||||
|
|
||||||
@@ -307,7 +307,7 @@ After configuring SSO for your external identity provider, verify it works:
|
|||||||
3. Confirm you are redirected to your IdP login page
|
3. Confirm you are redirected to your IdP login page
|
||||||
4. After authenticating, confirm you are redirected back to the service and can access it
|
4. After authenticating, confirm you are redirected back to the service and can access it
|
||||||
|
|
||||||
If the redirect fails or you see an error from your IdP, double-check that the callback URL is correctly registered in both `management.json` and your identity provider's settings.
|
If the redirect fails or you see an error from your IdP, double-check that the callback URL is correctly registered in both your configuration (`management.json` for multi-container setups) and your identity provider's settings.
|
||||||
|
|
||||||
## Connecting through Traefik instead of Docker network
|
## Connecting through Traefik instead of Docker network
|
||||||
|
|
||||||
|
|||||||
@@ -1,25 +1,75 @@
|
|||||||
# Management Postgresql store
|
# Management PostgreSQL store
|
||||||
|
|
||||||
## Using Postgres for fresh installations
|
PostgreSQL is recommended for production deployments. It supports concurrent access, enabling multiple management instances for high availability.
|
||||||
|
|
||||||
As of version 0.26.0, the default configuration for fresh installations is SQLite storage.
|
## Configuration
|
||||||
However, users have the option to leverage the benefits of Postgres for new instances beginning from version `0.27.8`.
|
|
||||||
To enable Postgres, add to your `setup.env` the following variable:
|
### Combined setup (config.yaml)
|
||||||
|
|
||||||
|
To use PostgreSQL, update your `config.yaml`:
|
||||||
|
|
||||||
|
```yaml
|
||||||
|
server:
|
||||||
|
store:
|
||||||
|
engine: "postgres"
|
||||||
|
dsn: "host=<PG_HOST> user=<PG_USER> password=<PG_PASSWORD> dbname=<PG_DB_NAME> port=<PG_PORT>"
|
||||||
|
```
|
||||||
|
|
||||||
|
You can also pass the DSN as an `NB_` environment variable on the `netbird-server` container in your `docker-compose.yml`:
|
||||||
|
|
||||||
|
```yaml
|
||||||
|
environment:
|
||||||
|
- NB_STORE_ENGINE_POSTGRES_DSN=host=<PG_HOST> user=<PG_USER> password=<PG_PASSWORD> dbname=<PG_DB_NAME> port=<PG_PORT>
|
||||||
|
```
|
||||||
|
|
||||||
|
Restart the server and confirm:
|
||||||
|
```bash
|
||||||
|
docker compose restart netbird-server
|
||||||
|
docker compose logs netbird-server
|
||||||
|
```
|
||||||
|
You should see:
|
||||||
|
```
|
||||||
|
INFO management/server/store.go:109: using Postgres store engine
|
||||||
|
```
|
||||||
|
|
||||||
|
<Note>
|
||||||
|
For a full list of available configuration options, see the [config.yaml.example](https://github.com/netbirdio/netbird/blob/main/combined/config.yaml.example) reference file.
|
||||||
|
</Note>
|
||||||
|
|
||||||
|
### Older multi-container setup (management.json)
|
||||||
|
|
||||||
|
<Note>
|
||||||
|
This section applies to deployments using the older multi-container architecture with separate `management`, `signal`, `relay`, and `coturn` containers. If you deployed using the [`getting-started.sh`](/selfhosted/selfhosted-quickstart) script, you are on the combined setup and should use the `config.yaml` instructions above. See the [migration guide](/selfhosted/migration/combined-container) to upgrade.
|
||||||
|
</Note>
|
||||||
|
|
||||||
|
To enable Postgres, add to your `setup.env`:
|
||||||
```bash
|
```bash
|
||||||
NETBIRD_STORE_CONFIG_ENGINE=postgres
|
NETBIRD_STORE_CONFIG_ENGINE=postgres
|
||||||
```
|
```
|
||||||
This will result in a configuration similar to the following in your `management.json` file:
|
This sets the following in your `management.json`:
|
||||||
```json
|
```json
|
||||||
"StoreConfig": {
|
"StoreConfig": {
|
||||||
"Engine": "postgres"
|
"Engine": "postgres"
|
||||||
}
|
}
|
||||||
```
|
```
|
||||||
You can switch back to sqlite storage by setting the `NETBIRD_STORE_CONFIG_ENGINE` variable to `sqlite`.
|
|
||||||
|
Create a `.env` file with the connection string:
|
||||||
|
```bash
|
||||||
|
NETBIRD_STORE_ENGINE_POSTGRES_DSN="host=<PG_HOST> user=<PG_USER> password=<PG_PASSWORD> dbname=<PG_DB_NAME> port=<PG_PORT>"
|
||||||
|
```
|
||||||
|
|
||||||
|
Update `docker-compose.yml` to pass the config to the `management` container:
|
||||||
|
```yaml
|
||||||
|
environment:
|
||||||
|
- NETBIRD_STORE_ENGINE_POSTGRES_DSN=${NETBIRD_STORE_ENGINE_POSTGRES_DSN}
|
||||||
|
env_file:
|
||||||
|
- .env
|
||||||
|
```
|
||||||
|
|
||||||
<Note>
|
<Note>
|
||||||
Switching between storage options requires migration steps to prevent data loss.
|
Switching between storage options requires migration steps to prevent data loss.
|
||||||
</Note>
|
</Note>
|
||||||
|
|
||||||
|
|
||||||
## Migrating from SQLite store to Postgres store
|
## Migrating from SQLite store to Postgres store
|
||||||
This migration process allows users to seamlessly transition between storage options while maintaining data integrity.
|
This migration process allows users to seamlessly transition between storage options while maintaining data integrity.
|
||||||
|
|
||||||
@@ -28,14 +78,22 @@ This migration process allows users to seamlessly transition between storage opt
|
|||||||
</Note>
|
</Note>
|
||||||
|
|
||||||
1. Backup your data store (`store.db` in `datadir` - default `/var/lib/netbird/`)
|
1. Backup your data store (`store.db` in `datadir` - default `/var/lib/netbird/`)
|
||||||
|
|
||||||
|
For the combined setup:
|
||||||
|
```bash
|
||||||
|
mkdir backup
|
||||||
|
docker compose cp -a netbird-server:/var/lib/netbird/. backup/
|
||||||
|
```
|
||||||
|
|
||||||
|
For the older multi-container setup:
|
||||||
```bash
|
```bash
|
||||||
mkdir backup
|
mkdir backup
|
||||||
docker compose cp -a management:/var/lib/netbird/. backup/
|
docker compose cp -a management:/var/lib/netbird/. backup/
|
||||||
```
|
```
|
||||||
|
|
||||||
2. Import Sqlite data to Postgres
|
2. Import SQLite data to Postgres
|
||||||
|
|
||||||
For migrating the Sqlite data we rely on the [pgloader](https://github.com/dimitri/pgloader) tool. You can install it by running
|
For migrating the SQLite data we rely on the [pgloader](https://github.com/dimitri/pgloader) tool. You can install it by running
|
||||||
`sudo apt-get install pgloader` on debian or `brew install pgloader` on MacOS.
|
`sudo apt-get install pgloader` on debian or `brew install pgloader` on MacOS.
|
||||||
|
|
||||||
```bash
|
```bash
|
||||||
@@ -68,74 +126,94 @@ DELETE FROM name_server_groups WHERE name_server_groups.account_id NOT IN (SELEC
|
|||||||
DROP TABLE IF EXISTS rules;
|
DROP TABLE IF EXISTS rules;
|
||||||
```
|
```
|
||||||
|
|
||||||
4. Enable Postgres by updating the `management.json` file and setting the `Engine` field to `postgres` as the following example:
|
4. Enable Postgres in your configuration.
|
||||||
|
|
||||||
|
For the combined setup, update `config.yaml`:
|
||||||
|
```yaml
|
||||||
|
server:
|
||||||
|
store:
|
||||||
|
engine: "postgres"
|
||||||
|
dsn: "host=<PG_HOST> user=<PG_USER> password=<PG_PASSWORD> dbname=<PG_DB_NAME> port=<PG_PORT>"
|
||||||
|
```
|
||||||
|
|
||||||
|
For the older multi-container setup, update `management.json`:
|
||||||
```json
|
```json
|
||||||
"StoreConfig": {
|
"StoreConfig": {
|
||||||
"Engine": "postgres"
|
"Engine": "postgres"
|
||||||
}
|
}
|
||||||
```
|
```
|
||||||
|
And pass the DSN via environment variable as described in the [older multi-container setup](#older-multi-container-setup-managementjson) section above.
|
||||||
|
|
||||||
5. Create `.env` file with the following content:
|
5. Restart the server and confirm:
|
||||||
|
|
||||||
|
For the combined setup:
|
||||||
```bash
|
```bash
|
||||||
NETBIRD_STORE_ENGINE_POSTGRES_DSN="host=<PG_HOST> user=<PG_USER> password=<PG_PASSWORD> dbname=<PG_DB_NAME> port=<PG_PORT>"
|
docker compose restart netbird-server
|
||||||
|
docker compose logs netbird-server
|
||||||
```
|
```
|
||||||
|
|
||||||
6. Update `docker-compose.yml` file to pass the postgres configuration to the `management` container
|
For the older multi-container setup:
|
||||||
```yaml
|
|
||||||
environment:
|
|
||||||
- NETBIRD_STORE_ENGINE_POSTGRES_DSN=${NETBIRD_STORE_ENGINE_POSTGRES_DSN}
|
|
||||||
env_file:
|
|
||||||
- .env
|
|
||||||
```
|
|
||||||
|
|
||||||
7. Restart the management service
|
|
||||||
```bash
|
```bash
|
||||||
docker compose restart management
|
docker compose restart management
|
||||||
```
|
|
||||||
|
|
||||||
8. Check logs to confirm the store switch:
|
|
||||||
```bash
|
|
||||||
docker compose logs management
|
docker compose logs management
|
||||||
```
|
```
|
||||||
|
|
||||||
You should see an entry similar to:
|
You should see an entry similar to:
|
||||||
```
|
```
|
||||||
2024-05-10T15:09:34Z INFO management/server/store.go:109: using Postgres store engine
|
INFO management/server/store.go:109: using Postgres store engine
|
||||||
```
|
```
|
||||||
|
|
||||||
## Rollback to Sqlite store
|
## Rollback to SQLite store
|
||||||
To rollback to the Sqlite store, follow these steps:
|
To rollback to the SQLite store, follow these steps:
|
||||||
<Note>
|
|
||||||
The following commands assume you use the latest docker version with the compose plugin. If you have docker-compose installed as a standalone, please use docker-compose as a command.
|
|
||||||
</Note>
|
|
||||||
|
|
||||||
1. Restore `store.db` backup
|
1. Restore `store.db` backup
|
||||||
|
|
||||||
|
For the combined setup:
|
||||||
|
```bash
|
||||||
|
docker compose cp backup/. netbird-server:/var/lib/netbird/
|
||||||
|
```
|
||||||
|
|
||||||
|
For the older multi-container setup:
|
||||||
```bash
|
```bash
|
||||||
docker compose cp backup/. management:/var/lib/netbird/
|
docker compose cp backup/. management:/var/lib/netbird/
|
||||||
```
|
```
|
||||||
|
|
||||||
2. Enable SQLite by updating the `management.json` file and setting the `Engine` field to `sqlite` as the following example:
|
2. Switch the store engine back to SQLite.
|
||||||
|
|
||||||
|
For the combined setup, update `config.yaml`:
|
||||||
|
```yaml
|
||||||
|
server:
|
||||||
|
store:
|
||||||
|
engine: "sqlite"
|
||||||
|
```
|
||||||
|
|
||||||
|
For the older multi-container setup, update `management.json`:
|
||||||
```json
|
```json
|
||||||
"StoreConfig": {
|
"StoreConfig": {
|
||||||
"Engine": "sqlite"
|
"Engine": "sqlite"
|
||||||
}
|
}
|
||||||
```
|
```
|
||||||
|
|
||||||
3. Restart the Management service.
|
3. Restart the server and confirm:
|
||||||
|
|
||||||
|
For the combined setup:
|
||||||
```bash
|
```bash
|
||||||
docker compose restart management
|
docker compose restart netbird-server
|
||||||
|
docker compose logs netbird-server
|
||||||
```
|
```
|
||||||
|
|
||||||
4. Check logs to confirm the store switch:
|
For the older multi-container setup:
|
||||||
```bash
|
```bash
|
||||||
|
docker compose restart management
|
||||||
docker compose logs management
|
docker compose logs management
|
||||||
```
|
```
|
||||||
|
|
||||||
You should see an entry similar to:
|
You should see an entry similar to:
|
||||||
```
|
```
|
||||||
2024-05-10T15:09:34Z INFO management/server/store.go:109: using SQLite file store engine
|
INFO management/server/store.go:109: using SQLite file store engine
|
||||||
```
|
```
|
||||||
|
|
||||||
## Optional Rollback Postgres data to Sqlite
|
## Optional Rollback Postgres data to SQLite
|
||||||
|
|
||||||
This is optional and should be used only if you want to rollback the data from Postgres to SQLite while running the same NetBird version.
|
This is optional and should be used only if you want to rollback the data from Postgres to SQLite while running the same NetBird version.
|
||||||
For migrating the Postgres data, we rely on the `pg_dump`, `sed`, and `sqlite3` tools. Make sure these are installed before proceeding
|
For migrating the Postgres data, we rely on the `pg_dump`, `sed`, and `sqlite3` tools. Make sure these are installed before proceeding
|
||||||
@@ -145,7 +223,7 @@ For migrating the Postgres data, we rely on the `pg_dump`, `sed`, and `sqlite3`
|
|||||||
pg_dump --data-only --column-inserts "postgresql://<PG_USER>:<PG_PASSWORD>@<PG_HOST>:<PG_PORT>/<PG_DB_NAME>" > data.sql
|
pg_dump --data-only --column-inserts "postgresql://<PG_USER>:<PG_PASSWORD>@<PG_HOST>:<PG_PORT>/<PG_DB_NAME>" > data.sql
|
||||||
```
|
```
|
||||||
|
|
||||||
2. Convert exported Postgres data sql to Sqlite format
|
2. Convert exported Postgres data sql to SQLite format
|
||||||
```bash
|
```bash
|
||||||
sed \
|
sed \
|
||||||
-e 's/\\\\:/\:/g' \
|
-e 's/\\\\:/\:/g' \
|
||||||
@@ -159,39 +237,26 @@ sed \
|
|||||||
-e '/^[[:space:]]*SELECT/d' data.sql > data.sql
|
-e '/^[[:space:]]*SELECT/d' data.sql > data.sql
|
||||||
```
|
```
|
||||||
|
|
||||||
3. Generate database schema from Sqlite backup
|
3. Generate database schema from SQLite backup
|
||||||
```bash
|
```bash
|
||||||
sqlite3 backup/store.db '.schema' > schema.sql
|
sqlite3 backup/store.db '.schema' > schema.sql
|
||||||
````
|
````
|
||||||
|
|
||||||
4. Create Sqlite database with Postgres exported data
|
4. Create SQLite database with Postgres exported data
|
||||||
```bash
|
```bash
|
||||||
sqlite3 store.db '.read schema.sql' && sqlite3 store.db '.read data.sql'
|
sqlite3 store.db '.read schema.sql' && sqlite3 store.db '.read data.sql'
|
||||||
```
|
```
|
||||||
|
|
||||||
5. Copy db to the management container
|
5. Copy db to the container
|
||||||
|
|
||||||
|
For the combined setup:
|
||||||
|
```bash
|
||||||
|
docker compose cp store.db netbird-server:/var/lib/netbird/store.db
|
||||||
|
```
|
||||||
|
|
||||||
|
For the older multi-container setup:
|
||||||
```bash
|
```bash
|
||||||
docker compose cp store.db management:/var/lib/netbird/store.db
|
docker compose cp store.db management:/var/lib/netbird/store.db
|
||||||
```
|
```
|
||||||
|
|
||||||
6. Enable SQLite by updating the `management.json` file and setting the `Engine` field to `sqlite` as the following example:
|
6. Switch the store engine back to SQLite using the instructions in the [Rollback to SQLite store](#rollback-to-sqlite-store) section above.
|
||||||
```json
|
|
||||||
"StoreConfig": {
|
|
||||||
"Engine": "sqlite"
|
|
||||||
}
|
|
||||||
```
|
|
||||||
|
|
||||||
7. Restart the Management service.
|
|
||||||
```bash
|
|
||||||
docker compose restart management
|
|
||||||
```
|
|
||||||
|
|
||||||
8. Check logs to confirm the store switch:
|
|
||||||
```bash
|
|
||||||
docker compose logs management
|
|
||||||
```
|
|
||||||
|
|
||||||
You should see an entry similar to:
|
|
||||||
```
|
|
||||||
2024-05-10T15:09:34Z INFO management/server/store.go:109: using SQLite file store engine
|
|
||||||
```
|
|
||||||
@@ -7,6 +7,10 @@ a 3rd party open-source STUN/TURN service [Coturn](https://github.com/coturn/cot
|
|||||||
|
|
||||||
If you would like to learn more about the architecture please refer to the [architecture section](/about-netbird/how-netbird-works).
|
If you would like to learn more about the architecture please refer to the [architecture section](/about-netbird/how-netbird-works).
|
||||||
|
|
||||||
|
<Warning>
|
||||||
|
This guide uses the **older multi-container architecture** with separate `management`, `signal`, `relay`, and `coturn` containers, configured via `management.json`. For new deployments, the [Quickstart guide](/selfhosted/selfhosted-quickstart) uses the **combined setup** with a single `netbird-server` container and `config.yaml`. If you are on the older setup, see the [migration guide](/selfhosted/migration/combined-container) to upgrade.
|
||||||
|
</Warning>
|
||||||
|
|
||||||
<Note>
|
<Note>
|
||||||
**New to self-hosting?** The [Quickstart guide](/selfhosted/selfhosted-quickstart) uses the built-in identity provider and is the fastest way to get started. This advanced guide is for users who need to integrate with an existing IdP or have specific enterprise requirements.
|
**New to self-hosting?** The [Quickstart guide](/selfhosted/selfhosted-quickstart) uses the built-in identity provider and is the fastest way to get started. This advanced guide is for users who need to integrate with an existing IdP or have specific enterprise requirements.
|
||||||
</Note>
|
</Note>
|
||||||
|
|||||||
@@ -1,21 +1,9 @@
|
|||||||
# Management SQLite store
|
# Management SQLite store
|
||||||
|
|
||||||
In NetBird, the Management system traditionally relies on JSON file storage.
|
SQLite is the default storage engine for NetBird Management. It requires no external dependencies and works out of the box for new installations.
|
||||||
|
|
||||||
<Note>
|
<Note>
|
||||||
Starting from version [0.24.0](https://github.com/netbirdio/netbird/releases/tag/v0.24.0),
|
If you need concurrent access or high availability with multiple management instances, consider using [PostgreSQL](/selfhosted/postgres-store) instead.
|
||||||
we have introduced support for SQLite.
|
|
||||||
This addition aims to provide users with enhanced performance and scalability options.
|
|
||||||
</Note>
|
|
||||||
|
|
||||||
<Note>
|
|
||||||
Starting from version [0.26.0](https://github.com/netbirdio/netbird/releases/tag/v0.26.0),
|
|
||||||
SQLite is **the default** storage engine for new installations.
|
|
||||||
Please follow this manual if you insist on migrating from JSON file storage.
|
|
||||||
If you have new installation you already have SQLite. To confirm please check the `management` logs on startup:
|
|
||||||
```
|
|
||||||
2024-02-06T15:28:04Z INFO management/server/store.go:77: using SQLite store engine
|
|
||||||
```
|
|
||||||
</Note>
|
</Note>
|
||||||
|
|
||||||
## SQLite entity relationship diagram
|
## SQLite entity relationship diagram
|
||||||
@@ -26,27 +14,59 @@ For a high-level overview of the SQLite schema, refer to the Entity Relationship
|
|||||||
<img src="/docs-static/img/selfhosted/sqlite-erd.png" alt="high-level-dia" className="imagewrapper-big"/>
|
<img src="/docs-static/img/selfhosted/sqlite-erd.png" alt="high-level-dia" className="imagewrapper-big"/>
|
||||||
</p>
|
</p>
|
||||||
|
|
||||||
## Using SQLite for fresh installations
|
## Configuration
|
||||||
|
|
||||||
As of version 0.24.0, the default configuration for fresh installations is JSON file storage.
|
### Combined setup (config.yaml)
|
||||||
However, users have the option to leverage the benefits of SQLite for new instances.
|
|
||||||
To enable SQLite, add to your setup.env the following variable:
|
SQLite is the default engine. If you need to explicitly set it in your `config.yaml`:
|
||||||
|
|
||||||
|
```yaml
|
||||||
|
server:
|
||||||
|
store:
|
||||||
|
engine: "sqlite"
|
||||||
|
```
|
||||||
|
|
||||||
|
Confirm SQLite is active by checking the server logs on startup:
|
||||||
|
```bash
|
||||||
|
docker compose logs netbird-server
|
||||||
|
```
|
||||||
|
You should see:
|
||||||
|
```
|
||||||
|
INFO management/server/store.go:77: using SQLite store engine
|
||||||
|
```
|
||||||
|
|
||||||
|
<Note>
|
||||||
|
For a full list of available configuration options, see the [config.yaml.example](https://github.com/netbirdio/netbird/blob/main/combined/config.yaml.example) reference file.
|
||||||
|
</Note>
|
||||||
|
|
||||||
|
### Older multi-container setup (management.json)
|
||||||
|
|
||||||
|
<Note>
|
||||||
|
This section applies to deployments using the older multi-container architecture with separate `management`, `signal`, `relay`, and `coturn` containers. If you deployed using the [`getting-started.sh`](/selfhosted/selfhosted-quickstart) script, you are on the combined setup and should use the `config.yaml` instructions above. See the [migration guide](/selfhosted/migration/combined-container) to upgrade.
|
||||||
|
</Note>
|
||||||
|
|
||||||
|
To enable SQLite, add to your `setup.env`:
|
||||||
```bash
|
```bash
|
||||||
NETBIRD_STORE_CONFIG_ENGINE=sqlite
|
NETBIRD_STORE_CONFIG_ENGINE=sqlite
|
||||||
```
|
```
|
||||||
This will result in a configuration similar to the following in your management.json file:
|
This sets the following in your `management.json`:
|
||||||
```json
|
```json
|
||||||
"StoreConfig": {
|
"StoreConfig": {
|
||||||
"Engine": "sqlite"
|
"Engine": "sqlite"
|
||||||
}
|
}
|
||||||
```
|
```
|
||||||
You can switch back to JSON file storage by setting the `NETBIRD_STORE_CONFIG_ENGINE` variable to `jsonfile`.
|
|
||||||
<Note>
|
|
||||||
Switching between storage options requires migration steps to prevent data loss.
|
|
||||||
</Note>
|
|
||||||
|
|
||||||
## Migrating from JSON store to SQLite store
|
## Migrating from JSON store to SQLite store
|
||||||
This migration process allows users to seamlessly transition between storage options while maintaining data integrity.
|
|
||||||
|
<Note>
|
||||||
|
This migration only applies to the older multi-container setup. The combined setup has always used SQLite by default and does not support JSON file storage.
|
||||||
|
</Note>
|
||||||
|
|
||||||
|
<Note>
|
||||||
|
Since version [0.28.0](https://github.com/netbirdio/netbird/releases/tag/v0.28.0), support for JSON file storage has been removed. All installations should be on SQLite or PostgreSQL.
|
||||||
|
</Note>
|
||||||
|
|
||||||
|
This migration process allows users to seamlessly transition from JSON file storage to SQLite while maintaining data integrity.
|
||||||
Ensure that you are running the commands described below after upgrading to version 0.24.0 or later.
|
Ensure that you are running the commands described below after upgrading to version 0.24.0 or later.
|
||||||
|
|
||||||
### Migration commands
|
### Migration commands
|
||||||
@@ -102,7 +122,7 @@ To migrate from JSON file store to SQLite, follow these steps:
|
|||||||
```bash
|
```bash
|
||||||
docker exec -it netbird-management /go/bin/netbird-mgmt sqlite-migration upgrade --log-file console
|
docker exec -it netbird-management /go/bin/netbird-mgmt sqlite-migration upgrade --log-file console
|
||||||
```
|
```
|
||||||
5. Enable SQLite by updating the management.json file and setting the `Engine` field to `sqlite` as the following example:
|
5. Enable SQLite by updating the `management.json` file and setting the `Engine` field to `sqlite`:
|
||||||
```json
|
```json
|
||||||
"StoreConfig": {
|
"StoreConfig": {
|
||||||
"Engine": "sqlite"
|
"Engine": "sqlite"
|
||||||
@@ -120,48 +140,3 @@ You should see an entry similar to:
|
|||||||
```
|
```
|
||||||
2023-10-19T18:55:29Z INFO management/server/store.go:75: using SQLite store engine
|
2023-10-19T18:55:29Z INFO management/server/store.go:75: using SQLite store engine
|
||||||
```
|
```
|
||||||
|
|
||||||
### Rollback to JSON file store
|
|
||||||
<Note>
|
|
||||||
Since version [0.28.0](https://github.com/netbirdio/netbird/releases/tag/v0.28.0), support for JSON file storage has been removed, and the option to roll back to JSON store is no longer supported.
|
|
||||||
</Note>
|
|
||||||
|
|
||||||
To rollback to the JSON file store, follow these steps:
|
|
||||||
<Note>
|
|
||||||
The following commands assume you use the latest docker version with the compose plugin. If you have docker-compose installed as a standalone, please use docker-compose as a command.
|
|
||||||
</Note>
|
|
||||||
1. After accessing the path that contains your docker-compose.yml and management.json files, stop the NetBird Management service.
|
|
||||||
```bash
|
|
||||||
docker compose stop management
|
|
||||||
```
|
|
||||||
2. Backup your data store (`store.db` in `datadir` - default "/var/lib/netbird/").
|
|
||||||
```bash
|
|
||||||
mkdir backup
|
|
||||||
docker compose cp -a management:/var/lib/netbird/ backup/
|
|
||||||
```
|
|
||||||
3. Start NetBird Management service:
|
|
||||||
```bash
|
|
||||||
docker compose start management
|
|
||||||
```
|
|
||||||
4. Run the migration to SQLite subcommand:
|
|
||||||
```bash
|
|
||||||
docker exec -it netbird-management /go/bin/netbird-mgmt sqlite-migration downgrade --log-file console
|
|
||||||
```
|
|
||||||
5. Enable JSON file by updating the management.json file and setting the `Engine` field to `jsonfile` as the following example:
|
|
||||||
```json
|
|
||||||
"StoreConfig": {
|
|
||||||
"Engine": "jsonfile"
|
|
||||||
}
|
|
||||||
```
|
|
||||||
6. Restart the Management service.
|
|
||||||
```bash
|
|
||||||
docker compose restart management
|
|
||||||
```
|
|
||||||
7. Check logs to confirm the store switch:
|
|
||||||
```bash
|
|
||||||
docker compose logs management
|
|
||||||
```
|
|
||||||
You should see an entry similar to:
|
|
||||||
```
|
|
||||||
2023-10-19T18:55:29Z INFO management/server/store.go:72: using JSON file store engine
|
|
||||||
```
|
|
||||||
|
|||||||
@@ -54,7 +54,8 @@ This page will help with various issues when self-hosting NetBird.
|
|||||||
|
|
||||||
**Solution**: This tab is only visible when the embedded IdP is enabled. Check your deployment configuration:
|
**Solution**: This tab is only visible when the embedded IdP is enabled. Check your deployment configuration:
|
||||||
- For quickstart deployments, the embedded IdP should be enabled by default
|
- For quickstart deployments, the embedded IdP should be enabled by default
|
||||||
- For advanced deployments, ensure `EmbeddedIdP.Enabled` is set to `true` in management.json
|
- For the combined setup, the embedded IdP is always enabled
|
||||||
|
- For older multi-container deployments, ensure `EmbeddedIdP.Enabled` is set to `true` in `management.json`
|
||||||
|
|
||||||
### Users not syncing from SSO provider
|
### Users not syncing from SSO provider
|
||||||
|
|
||||||
@@ -74,7 +75,7 @@ To test your TURN configuration you can access the [online tester](https://webrt
|
|||||||
There you will find a ICE servers input box, where you can select and remove the existing server, then add your turn server
|
There you will find a ICE servers input box, where you can select and remove the existing server, then add your turn server
|
||||||
configuration as follows:
|
configuration as follows:
|
||||||
|
|
||||||
Please replace <b>netbird.DOMAIN.com</b> and <b>PASSWORD</b> with the information from the <b>management.json</b> TURNConfig, then click on <b>Add server</b>.
|
Please replace <b>netbird.DOMAIN.com</b> and <b>PASSWORD</b> with your STUN/TURN server details from your configuration (<b>config.yaml</b> for the combined setup, or the <b>TURNConfig</b> section in <b>management.json</b> for older multi-container setups), then click on <b>Add server</b>.
|
||||||
|
|
||||||
<p>
|
<p>
|
||||||
<img src="/docs-static/img/selfhosted/troubleshooting/turn.png" alt="turn" width="700" className="imagewrapper"/>
|
<img src="/docs-static/img/selfhosted/troubleshooting/turn.png" alt="turn" width="700" className="imagewrapper"/>
|
||||||
|
|||||||
Reference in New Issue
Block a user