Refer to the new config.yaml file (#628)

This commit is contained in:
Misha Bragin
2026-02-19 12:03:32 +01:00
committed by GitHub
parent 243c1af0c1
commit 5c059fa7b8
13 changed files with 275 additions and 167 deletions

View File

@@ -47,7 +47,7 @@ Changing servers erases the device's current NetBird config, so you'll need to c
<img src="/docs-static/img/get-started/android-tv/change-server.png" alt="confirm-erase-cponfig" className="imagewrapper-large"/> <img src="/docs-static/img/get-started/android-tv/change-server.png" alt="confirm-erase-cponfig" className="imagewrapper-large"/>
Enter your management server endpoint. For cloud users, this is `https://api.netbird.io:443`. For self-hosted users, it's usually `https://your_management_server_url:443`, but you can refer to your management.json configuration file if you're unsure. Enter your management server endpoint. For cloud users, this is `https://api.netbird.io:443`. For self-hosted users, it's usually `https://your_management_server_url:443`, but you can refer to the `exposedAddress` field in your `config.yaml` (or `management.json` for older multi-container setups) if you're unsure.
If enrolling the device with a setup key, select '+ Add this device with a setup key' and enter your setup key. Select 'change' to apply your new management server config, and if successful you'll see the following: If enrolling the device with a setup key, select '+ Add this device with a setup key' and enter your setup key. Select 'change' to apply your new management server config, and if successful you'll see the following:

View File

@@ -43,7 +43,7 @@ Changing servers erases the device's current NetBird config, so you'll need to c
<img src="/docs-static/img/get-started/android/confirm-erase-config.png" alt="confirmeraseconfig" className="imagewrapper-medium"/> <img src="/docs-static/img/get-started/android/confirm-erase-config.png" alt="confirmeraseconfig" className="imagewrapper-medium"/>
Enter your management server endpoint. For cloud users, this is `https://api.netbird.io:443`. For self-hosted users, it's usually `https://your_management_server_url:443`, but you can refer to your management.json configuration file if you're unsure. Enter your management server endpoint. For cloud users, this is `https://api.netbird.io:443`. For self-hosted users, it's usually `https://your_management_server_url:443`, but you can refer to the `exposedAddress` field in your `config.yaml` (or `management.json` for older multi-container setups) if you're unsure.
If enrolling the device with a setup key, select '+ Add this device with a setup key' and enter your setup key. Select 'change' to apply your new management server config, and if successful you'll see the following: If enrolling the device with a setup key, select '+ Add this device with a setup key' and enter your setup key. Select 'change' to apply your new management server config, and if successful you'll see the following:

View File

@@ -30,7 +30,7 @@ Navigate to **'Settings > Change Server'**, and the app will warn you that chang
<img src="/docs-static/img/get-started/tvos/confirm-change-server.png" alt="tvos-confirm-change-server" className="imagewrapper-large"/> <img src="/docs-static/img/get-started/tvos/confirm-change-server.png" alt="tvos-confirm-change-server" className="imagewrapper-large"/>
After confirming, you'll be presented with a dialog to enter your management server URL. Cloud users who want to enroll the device with a setup key should enter the URL For cloud users, this is `https://api.netbird.io:443`. After confirming, you'll be presented with a dialog to enter your management server URL. Cloud users who want to enroll the device with a setup key should enter the URL For cloud users, this is `https://api.netbird.io:443`.
For self-hosted users, it's usually `https://your_management_server_url:443`, but you can refer to your management.json configuration file if you're unsure. For self-hosted users, it's usually `https://your_management_server_url:443`, but you can refer to the `exposedAddress` field in your `config.yaml` (or `management.json` for older multi-container setups) if you're unsure.
If enrolling the device with a setup key, select '+ Add this device with a setup key' and enter your setup key. Select 'Change' to apply your new management server config. NetBird will run a brief verification step If enrolling the device with a setup key, select '+ Add this device with a setup key' and enter your setup key. Select 'Change' to apply your new management server config. NetBird will run a brief verification step
on the details you've entered, after which you can move on to connecting. on the details you've entered, after which you can move on to connecting.

View File

@@ -120,7 +120,7 @@ Future versions will also support connection events that occur in NetBird agents
<Note> <Note>
The `unknown` name or `unknown@unknown.com` email address may be displayed in the activity event store if the encryption key has been corrupted or lost. This issue is most relevant for self-hosted setups. In this case, the events returned by the API could show `unknown@unknown.com` for the email address field and `unknown` for the name field. The `unknown` name or `unknown@unknown.com` email address may be displayed in the activity event store if the encryption key has been corrupted or lost. This issue is most relevant for self-hosted setups. In this case, the events returned by the API could show `unknown@unknown.com` for the email address field and `unknown` for the name field.
If the configuration files have been generated by the `configure.sh` script, you can find the previous encryption key in the backup files in the same folder as the script. Look for the <b>DataStoreEncryptionKey</b> field in the `management.json` backup file. For the combined setup, the encryption key is the <b>server.store.encryptionKey</b> field in <b>config.yaml</b>. For older multi-container setups generated by the <b>configure.sh</b> script, look for the <b>DataStoreEncryptionKey</b> field in the <b>management.json</b> backup file.
</Note> </Note>
## Enable Audit Events Streaming to SIEM Systems ## Enable Audit Events Streaming to SIEM Systems

View File

@@ -267,7 +267,7 @@ environment:
Configuration values are applied in the following order (later values override earlier): Configuration values are applied in the following order (later values override earlier):
1. Default values 1. Default values
2. Configuration file (`management.json`) 2. Configuration file (`config.yaml` for combined setup, `management.json` for older multi-container setup)
3. Environment variables 3. Environment variables
4. Command-line flags 4. Command-line flags
@@ -275,6 +275,6 @@ Configuration values are applied in the following order (later values override e
- [Self-hosting Quickstart Guide](/selfhosted/selfhosted-quickstart) - [Self-hosting Quickstart Guide](/selfhosted/selfhosted-quickstart)
- [Advanced Self-hosting Guide](/selfhosted/selfhosted-guide) - [Advanced Self-hosting Guide](/selfhosted/selfhosted-guide)
- [Configuration Files Reference](/selfhosted/configuration-files) - Detailed documentation for docker-compose.yml, management.json, and other config files - [Configuration Files Reference](/selfhosted/configuration-files) - Detailed documentation for docker-compose.yml, config.yaml, and other config files
- [Management SQLite Store](/selfhosted/sqlite-store) - [Management SQLite Store](/selfhosted/sqlite-store)
- [Management Postgres Store](/selfhosted/postgres-store) - [Management Postgres Store](/selfhosted/postgres-store)

View File

@@ -38,7 +38,32 @@ NetBird will refuse to disable local authentication if no external identity prov
## Configuration ## Configuration
Update your `management.json` to disable local authentication: ### Combined setup (config.yaml)
Set `localAuthDisabled` to `true` in your `config.yaml`:
```yaml
server:
auth:
localAuthDisabled: true
```
Restart the server:
```bash
docker compose restart netbird-server
```
<Note>
For a full list of available configuration options, see the [config.yaml.example](https://github.com/netbirdio/netbird/blob/main/combined/config.yaml.example) reference file.
</Note>
### Older multi-container setup (management.json)
<Note>
This section applies to deployments using the older multi-container architecture. If you deployed using [`getting-started.sh`](/selfhosted/selfhosted-quickstart), you are on the combined setup and should use the `config.yaml` instructions above. See the [migration guide](/selfhosted/migration/combined-container) to upgrade.
</Note>
Update your `management.json`:
```json ```json
{ {
@@ -49,12 +74,18 @@ Update your `management.json` to disable local authentication:
} }
``` ```
After updating the configuration, restart the Management service. Restart the Management service:
The local login option will no longer appear on the login page—users will only see the configured external identity providers. ```bash
docker compose restart management
```
---
After updating the configuration, the local login option will no longer appear on the login page — users will only see the configured external identity providers.
## Re-enabling Local Authentication ## Re-enabling Local Authentication
To restore local authentication, set `LocalAuthDisabled` to `false` (or remove the option) and restart the Management service. To restore local authentication, set `localAuthDisabled` back to `false` in `config.yaml` (or `LocalAuthDisabled` in `management.json`) and restart the server.
All previously created local users will be able to log in again with their existing passwords. All previously created local users will be able to log in again with their existing passwords.

View File

@@ -41,7 +41,39 @@ Consider a [standalone external IdP](/selfhosted/selfhosted-guide#step-3-configu
### Enabling Embedded IdP ### Enabling Embedded IdP
The embedded IdP is enabled by default when using the new [`getting-started.sh`](/selfhosted/selfhosted-quickstart) quickstart script. For manual configuration, update your `management.json`: The embedded IdP is enabled by default when using the [`getting-started.sh`](/selfhosted/selfhosted-quickstart) quickstart script and requires no manual configuration.
#### Combined setup (config.yaml)
The embedded IdP is always enabled in the combined setup and cannot be disabled. The relevant settings in `config.yaml` are:
```yaml
server:
auth:
issuer: "https://netbird.example.com/oauth2"
localAuthDisabled: false
signKeyRefreshEnabled: true
dashboardRedirectURIs:
- "https://netbird.example.com/nb-auth"
- "https://netbird.example.com/nb-silent-auth"
cliRedirectURIs:
- "http://localhost:53000/"
store:
encryptionKey: "<auto-generated-key>"
```
<Note>
For a full list of available configuration options, see the [config.yaml.example](https://github.com/netbirdio/netbird/blob/main/combined/config.yaml.example) reference file.
</Note>
#### Older multi-container setup (management.json)
<Note>
This section applies to deployments using the older multi-container architecture. If you deployed using `getting-started.sh`, you are on the combined setup and should use the `config.yaml` instructions above. See the [migration guide](/selfhosted/migration/combined-container) to upgrade.
</Note>
Update your `management.json`:
```json ```json
{ {
@@ -53,17 +85,7 @@ The embedded IdP is enabled by default when using the new [`getting-started.sh`]
} }
``` ```
### Configuration Options Or configure via environment variables on the `management` container:
| Option | Description | Default |
|--------|-------------|---------|
| `EmbeddedIdP.Enabled` | Enable/disable the embedded IdP | `true` (quickstart) |
| `EmbeddedIdP.DataDir` | Directory for IdP data storage | `/var/lib/netbird/idp` |
| `EncryptionKey` | Base64-encoded AES-256 key for encrypting user data | Auto-generated |
### Environment Variables
When using docker-compose, you can configure these via environment variables:
```yaml ```yaml
environment: environment:
@@ -330,7 +352,7 @@ The embedded IdP encrypts sensitive user data at rest:
| Name | AES-256-GCM | | Name | AES-256-GCM |
| Password | bcrypt hash (via Dex) | | Password | bcrypt hash (via Dex) |
The encryption key is configured in `management.json` and should be: The encryption key is configured in `config.yaml` (`server.store.encryptionKey`) or `management.json` (`EncryptionKey`) for older multi-container setups. It should be:
- Generated using a cryptographically secure random generator - Generated using a cryptographically secure random generator
- Stored securely (not in version control) - Stored securely (not in version control)
@@ -364,17 +386,23 @@ User authentication events are logged in the activity log:
### "Embedded IdP not available" error ### "Embedded IdP not available" error
Ensure `EmbeddedIdP.Enabled` is `true` in `management.json` and the Management service has been restarted. For the combined setup, the embedded IdP is always enabled. For older multi-container setups, ensure `EmbeddedIdP.Enabled` is `true` in `management.json` and restart the Management service.
### Users can't log in ### Users can't log in
1. Check Management service logs: `docker compose logs management` 1. Check server logs: `docker compose logs netbird-server` (or `docker compose logs management` for older multi-container setups)
2. Verify the encryption key hasn't changed 2. Verify the encryption key hasn't changed
3. Confirm the user exists: Check **Team** → **Users** in Dashboard 3. Confirm the user exists: Check **Team** → **Users** in Dashboard
## Disabling Embedded IdP ## Disabling Embedded IdP
To switch from embedded IdP to a (standalone) external IdP: <Note>
The embedded IdP cannot be disabled in the combined setup. If you want to use only external identity providers, you can [disable local authentication](/selfhosted/identity-providers/disable-local-authentication) instead and add your external IdP through the Dashboard UI.
</Note>
### Older multi-container setup (management.json)
To switch from embedded IdP to a standalone external IdP on the older multi-container setup:
1. Configure your external IdP following the [Advanced guide](/selfhosted/selfhosted-guide) 1. Configure your external IdP following the [Advanced guide](/selfhosted/selfhosted-guide)
2. Update `management.json`: 2. Update `management.json`:

View File

@@ -2,6 +2,10 @@
This guide helps you migrate your existing NetBird self-hosted installation from using an external Coturn TURN server to the new embedded STUN server integrated into the relay service. This guide helps you migrate your existing NetBird self-hosted installation from using an external Coturn TURN server to the new embedded STUN server integrated into the relay service.
<Warning>
This guide applies only to the **older multi-container setup** with separate `management`, `signal`, `relay`, and `coturn` containers configured via `management.json`. If you deployed using the [`getting-started.sh`](/selfhosted/selfhosted-quickstart) script (combined setup with `config.yaml`), STUN is already embedded — no migration is needed. If you are still on the older setup, consider migrating to the [combined container](/selfhosted/migration/combined-container) instead, which handles this automatically.
</Warning>
<Note> <Note>
**Who is this guide for?** This migration guide is intended for users who: **Who is this guide for?** This migration guide is intended for users who:
- Have an existing NetBird self-hosted deployment with **Coturn configured for STUN** (NAT traversal) - Have an existing NetBird self-hosted deployment with **Coturn configured for STUN** (NAT traversal)

View File

@@ -244,7 +244,7 @@ The reverse proxy SSO feature authenticates users through an OAuth2/OIDC flow th
If you want to keep using your current external identity provider, follow these three steps: If you want to keep using your current external identity provider, follow these three steps:
#### Step 1: Add callback URL to management.json #### Step 1: Add callback URL to management.json (multi-container only)
Add the `AuthCallbackURL` and `AuthClientID` fields to the `HttpConfig` section of your `management.json`: Add the `AuthCallbackURL` and `AuthClientID` fields to the `HttpConfig` section of your `management.json`:
@@ -307,7 +307,7 @@ After configuring SSO for your external identity provider, verify it works:
3. Confirm you are redirected to your IdP login page 3. Confirm you are redirected to your IdP login page
4. After authenticating, confirm you are redirected back to the service and can access it 4. After authenticating, confirm you are redirected back to the service and can access it
If the redirect fails or you see an error from your IdP, double-check that the callback URL is correctly registered in both `management.json` and your identity provider's settings. If the redirect fails or you see an error from your IdP, double-check that the callback URL is correctly registered in both your configuration (`management.json` for multi-container setups) and your identity provider's settings.
## Connecting through Traefik instead of Docker network ## Connecting through Traefik instead of Docker network

View File

@@ -1,25 +1,75 @@
# Management Postgresql store # Management PostgreSQL store
## Using Postgres for fresh installations PostgreSQL is recommended for production deployments. It supports concurrent access, enabling multiple management instances for high availability.
As of version 0.26.0, the default configuration for fresh installations is SQLite storage. ## Configuration
However, users have the option to leverage the benefits of Postgres for new instances beginning from version `0.27.8`.
To enable Postgres, add to your `setup.env` the following variable: ### Combined setup (config.yaml)
To use PostgreSQL, update your `config.yaml`:
```yaml
server:
store:
engine: "postgres"
dsn: "host=<PG_HOST> user=<PG_USER> password=<PG_PASSWORD> dbname=<PG_DB_NAME> port=<PG_PORT>"
```
You can also pass the DSN as an `NB_` environment variable on the `netbird-server` container in your `docker-compose.yml`:
```yaml
environment:
- NB_STORE_ENGINE_POSTGRES_DSN=host=<PG_HOST> user=<PG_USER> password=<PG_PASSWORD> dbname=<PG_DB_NAME> port=<PG_PORT>
```
Restart the server and confirm:
```bash
docker compose restart netbird-server
docker compose logs netbird-server
```
You should see:
```
INFO management/server/store.go:109: using Postgres store engine
```
<Note>
For a full list of available configuration options, see the [config.yaml.example](https://github.com/netbirdio/netbird/blob/main/combined/config.yaml.example) reference file.
</Note>
### Older multi-container setup (management.json)
<Note>
This section applies to deployments using the older multi-container architecture with separate `management`, `signal`, `relay`, and `coturn` containers. If you deployed using the [`getting-started.sh`](/selfhosted/selfhosted-quickstart) script, you are on the combined setup and should use the `config.yaml` instructions above. See the [migration guide](/selfhosted/migration/combined-container) to upgrade.
</Note>
To enable Postgres, add to your `setup.env`:
```bash ```bash
NETBIRD_STORE_CONFIG_ENGINE=postgres NETBIRD_STORE_CONFIG_ENGINE=postgres
``` ```
This will result in a configuration similar to the following in your `management.json` file: This sets the following in your `management.json`:
```json ```json
"StoreConfig": { "StoreConfig": {
"Engine": "postgres" "Engine": "postgres"
} }
``` ```
You can switch back to sqlite storage by setting the `NETBIRD_STORE_CONFIG_ENGINE` variable to `sqlite`.
Create a `.env` file with the connection string:
```bash
NETBIRD_STORE_ENGINE_POSTGRES_DSN="host=<PG_HOST> user=<PG_USER> password=<PG_PASSWORD> dbname=<PG_DB_NAME> port=<PG_PORT>"
```
Update `docker-compose.yml` to pass the config to the `management` container:
```yaml
environment:
- NETBIRD_STORE_ENGINE_POSTGRES_DSN=${NETBIRD_STORE_ENGINE_POSTGRES_DSN}
env_file:
- .env
```
<Note> <Note>
Switching between storage options requires migration steps to prevent data loss. Switching between storage options requires migration steps to prevent data loss.
</Note> </Note>
## Migrating from SQLite store to Postgres store ## Migrating from SQLite store to Postgres store
This migration process allows users to seamlessly transition between storage options while maintaining data integrity. This migration process allows users to seamlessly transition between storage options while maintaining data integrity.
@@ -28,14 +78,22 @@ This migration process allows users to seamlessly transition between storage opt
</Note> </Note>
1. Backup your data store (`store.db` in `datadir` - default `/var/lib/netbird/`) 1. Backup your data store (`store.db` in `datadir` - default `/var/lib/netbird/`)
For the combined setup:
```bash
mkdir backup
docker compose cp -a netbird-server:/var/lib/netbird/. backup/
```
For the older multi-container setup:
```bash ```bash
mkdir backup mkdir backup
docker compose cp -a management:/var/lib/netbird/. backup/ docker compose cp -a management:/var/lib/netbird/. backup/
``` ```
2. Import Sqlite data to Postgres 2. Import SQLite data to Postgres
For migrating the Sqlite data we rely on the [pgloader](https://github.com/dimitri/pgloader) tool. You can install it by running For migrating the SQLite data we rely on the [pgloader](https://github.com/dimitri/pgloader) tool. You can install it by running
`sudo apt-get install pgloader` on debian or `brew install pgloader` on MacOS. `sudo apt-get install pgloader` on debian or `brew install pgloader` on MacOS.
```bash ```bash
@@ -68,74 +126,94 @@ DELETE FROM name_server_groups WHERE name_server_groups.account_id NOT IN (SELEC
DROP TABLE IF EXISTS rules; DROP TABLE IF EXISTS rules;
``` ```
4. Enable Postgres by updating the `management.json` file and setting the `Engine` field to `postgres` as the following example: 4. Enable Postgres in your configuration.
For the combined setup, update `config.yaml`:
```yaml
server:
store:
engine: "postgres"
dsn: "host=<PG_HOST> user=<PG_USER> password=<PG_PASSWORD> dbname=<PG_DB_NAME> port=<PG_PORT>"
```
For the older multi-container setup, update `management.json`:
```json ```json
"StoreConfig": { "StoreConfig": {
"Engine": "postgres" "Engine": "postgres"
} }
``` ```
And pass the DSN via environment variable as described in the [older multi-container setup](#older-multi-container-setup-managementjson) section above.
5. Create `.env` file with the following content: 5. Restart the server and confirm:
For the combined setup:
```bash ```bash
NETBIRD_STORE_ENGINE_POSTGRES_DSN="host=<PG_HOST> user=<PG_USER> password=<PG_PASSWORD> dbname=<PG_DB_NAME> port=<PG_PORT>" docker compose restart netbird-server
docker compose logs netbird-server
``` ```
6. Update `docker-compose.yml` file to pass the postgres configuration to the `management` container For the older multi-container setup:
```yaml
environment:
- NETBIRD_STORE_ENGINE_POSTGRES_DSN=${NETBIRD_STORE_ENGINE_POSTGRES_DSN}
env_file:
- .env
```
7. Restart the management service
```bash ```bash
docker compose restart management docker compose restart management
```
8. Check logs to confirm the store switch:
```bash
docker compose logs management docker compose logs management
``` ```
You should see an entry similar to: You should see an entry similar to:
``` ```
2024-05-10T15:09:34Z INFO management/server/store.go:109: using Postgres store engine INFO management/server/store.go:109: using Postgres store engine
``` ```
## Rollback to Sqlite store ## Rollback to SQLite store
To rollback to the Sqlite store, follow these steps: To rollback to the SQLite store, follow these steps:
<Note>
The following commands assume you use the latest docker version with the compose plugin. If you have docker-compose installed as a standalone, please use docker-compose as a command.
</Note>
1. Restore `store.db` backup 1. Restore `store.db` backup
For the combined setup:
```bash
docker compose cp backup/. netbird-server:/var/lib/netbird/
```
For the older multi-container setup:
```bash ```bash
docker compose cp backup/. management:/var/lib/netbird/ docker compose cp backup/. management:/var/lib/netbird/
``` ```
2. Enable SQLite by updating the `management.json` file and setting the `Engine` field to `sqlite` as the following example: 2. Switch the store engine back to SQLite.
For the combined setup, update `config.yaml`:
```yaml
server:
store:
engine: "sqlite"
```
For the older multi-container setup, update `management.json`:
```json ```json
"StoreConfig": { "StoreConfig": {
"Engine": "sqlite" "Engine": "sqlite"
} }
``` ```
3. Restart the Management service. 3. Restart the server and confirm:
For the combined setup:
```bash ```bash
docker compose restart management docker compose restart netbird-server
docker compose logs netbird-server
``` ```
4. Check logs to confirm the store switch: For the older multi-container setup:
```bash ```bash
docker compose restart management
docker compose logs management docker compose logs management
``` ```
You should see an entry similar to: You should see an entry similar to:
``` ```
2024-05-10T15:09:34Z INFO management/server/store.go:109: using SQLite file store engine INFO management/server/store.go:109: using SQLite file store engine
``` ```
## Optional Rollback Postgres data to Sqlite ## Optional Rollback Postgres data to SQLite
This is optional and should be used only if you want to rollback the data from Postgres to SQLite while running the same NetBird version. This is optional and should be used only if you want to rollback the data from Postgres to SQLite while running the same NetBird version.
For migrating the Postgres data, we rely on the `pg_dump`, `sed`, and `sqlite3` tools. Make sure these are installed before proceeding For migrating the Postgres data, we rely on the `pg_dump`, `sed`, and `sqlite3` tools. Make sure these are installed before proceeding
@@ -145,7 +223,7 @@ For migrating the Postgres data, we rely on the `pg_dump`, `sed`, and `sqlite3`
pg_dump --data-only --column-inserts "postgresql://<PG_USER>:<PG_PASSWORD>@<PG_HOST>:<PG_PORT>/<PG_DB_NAME>" > data.sql pg_dump --data-only --column-inserts "postgresql://<PG_USER>:<PG_PASSWORD>@<PG_HOST>:<PG_PORT>/<PG_DB_NAME>" > data.sql
``` ```
2. Convert exported Postgres data sql to Sqlite format 2. Convert exported Postgres data sql to SQLite format
```bash ```bash
sed \ sed \
-e 's/\\\\:/\:/g' \ -e 's/\\\\:/\:/g' \
@@ -159,39 +237,26 @@ sed \
-e '/^[[:space:]]*SELECT/d' data.sql > data.sql -e '/^[[:space:]]*SELECT/d' data.sql > data.sql
``` ```
3. Generate database schema from Sqlite backup 3. Generate database schema from SQLite backup
```bash ```bash
sqlite3 backup/store.db '.schema' > schema.sql sqlite3 backup/store.db '.schema' > schema.sql
```` ````
4. Create Sqlite database with Postgres exported data 4. Create SQLite database with Postgres exported data
```bash ```bash
sqlite3 store.db '.read schema.sql' && sqlite3 store.db '.read data.sql' sqlite3 store.db '.read schema.sql' && sqlite3 store.db '.read data.sql'
``` ```
5. Copy db to the management container 5. Copy db to the container
For the combined setup:
```bash
docker compose cp store.db netbird-server:/var/lib/netbird/store.db
```
For the older multi-container setup:
```bash ```bash
docker compose cp store.db management:/var/lib/netbird/store.db docker compose cp store.db management:/var/lib/netbird/store.db
``` ```
6. Enable SQLite by updating the `management.json` file and setting the `Engine` field to `sqlite` as the following example: 6. Switch the store engine back to SQLite using the instructions in the [Rollback to SQLite store](#rollback-to-sqlite-store) section above.
```json
"StoreConfig": {
"Engine": "sqlite"
}
```
7. Restart the Management service.
```bash
docker compose restart management
```
8. Check logs to confirm the store switch:
```bash
docker compose logs management
```
You should see an entry similar to:
```
2024-05-10T15:09:34Z INFO management/server/store.go:109: using SQLite file store engine
```

View File

@@ -7,6 +7,10 @@ a 3rd party open-source STUN/TURN service [Coturn](https://github.com/coturn/cot
If you would like to learn more about the architecture please refer to the [architecture section](/about-netbird/how-netbird-works). If you would like to learn more about the architecture please refer to the [architecture section](/about-netbird/how-netbird-works).
<Warning>
This guide uses the **older multi-container architecture** with separate `management`, `signal`, `relay`, and `coturn` containers, configured via `management.json`. For new deployments, the [Quickstart guide](/selfhosted/selfhosted-quickstart) uses the **combined setup** with a single `netbird-server` container and `config.yaml`. If you are on the older setup, see the [migration guide](/selfhosted/migration/combined-container) to upgrade.
</Warning>
<Note> <Note>
**New to self-hosting?** The [Quickstart guide](/selfhosted/selfhosted-quickstart) uses the built-in identity provider and is the fastest way to get started. This advanced guide is for users who need to integrate with an existing IdP or have specific enterprise requirements. **New to self-hosting?** The [Quickstart guide](/selfhosted/selfhosted-quickstart) uses the built-in identity provider and is the fastest way to get started. This advanced guide is for users who need to integrate with an existing IdP or have specific enterprise requirements.
</Note> </Note>

View File

@@ -1,21 +1,9 @@
# Management SQLite store # Management SQLite store
In NetBird, the Management system traditionally relies on JSON file storage. SQLite is the default storage engine for NetBird Management. It requires no external dependencies and works out of the box for new installations.
<Note> <Note>
Starting from version [0.24.0](https://github.com/netbirdio/netbird/releases/tag/v0.24.0), If you need concurrent access or high availability with multiple management instances, consider using [PostgreSQL](/selfhosted/postgres-store) instead.
we have introduced support for SQLite.
This addition aims to provide users with enhanced performance and scalability options.
</Note>
<Note>
Starting from version [0.26.0](https://github.com/netbirdio/netbird/releases/tag/v0.26.0),
SQLite is **the default** storage engine for new installations.
Please follow this manual if you insist on migrating from JSON file storage.
If you have new installation you already have SQLite. To confirm please check the `management` logs on startup:
```
2024-02-06T15:28:04Z INFO management/server/store.go:77: using SQLite store engine
```
</Note> </Note>
## SQLite entity relationship diagram ## SQLite entity relationship diagram
@@ -26,27 +14,59 @@ For a high-level overview of the SQLite schema, refer to the Entity Relationship
<img src="/docs-static/img/selfhosted/sqlite-erd.png" alt="high-level-dia" className="imagewrapper-big"/> <img src="/docs-static/img/selfhosted/sqlite-erd.png" alt="high-level-dia" className="imagewrapper-big"/>
</p> </p>
## Using SQLite for fresh installations ## Configuration
As of version 0.24.0, the default configuration for fresh installations is JSON file storage. ### Combined setup (config.yaml)
However, users have the option to leverage the benefits of SQLite for new instances.
To enable SQLite, add to your setup.env the following variable: SQLite is the default engine. If you need to explicitly set it in your `config.yaml`:
```yaml
server:
store:
engine: "sqlite"
```
Confirm SQLite is active by checking the server logs on startup:
```bash
docker compose logs netbird-server
```
You should see:
```
INFO management/server/store.go:77: using SQLite store engine
```
<Note>
For a full list of available configuration options, see the [config.yaml.example](https://github.com/netbirdio/netbird/blob/main/combined/config.yaml.example) reference file.
</Note>
### Older multi-container setup (management.json)
<Note>
This section applies to deployments using the older multi-container architecture with separate `management`, `signal`, `relay`, and `coturn` containers. If you deployed using the [`getting-started.sh`](/selfhosted/selfhosted-quickstart) script, you are on the combined setup and should use the `config.yaml` instructions above. See the [migration guide](/selfhosted/migration/combined-container) to upgrade.
</Note>
To enable SQLite, add to your `setup.env`:
```bash ```bash
NETBIRD_STORE_CONFIG_ENGINE=sqlite NETBIRD_STORE_CONFIG_ENGINE=sqlite
``` ```
This will result in a configuration similar to the following in your management.json file: This sets the following in your `management.json`:
```json ```json
"StoreConfig": { "StoreConfig": {
"Engine": "sqlite" "Engine": "sqlite"
} }
``` ```
You can switch back to JSON file storage by setting the `NETBIRD_STORE_CONFIG_ENGINE` variable to `jsonfile`.
<Note>
Switching between storage options requires migration steps to prevent data loss.
</Note>
## Migrating from JSON store to SQLite store ## Migrating from JSON store to SQLite store
This migration process allows users to seamlessly transition between storage options while maintaining data integrity.
<Note>
This migration only applies to the older multi-container setup. The combined setup has always used SQLite by default and does not support JSON file storage.
</Note>
<Note>
Since version [0.28.0](https://github.com/netbirdio/netbird/releases/tag/v0.28.0), support for JSON file storage has been removed. All installations should be on SQLite or PostgreSQL.
</Note>
This migration process allows users to seamlessly transition from JSON file storage to SQLite while maintaining data integrity.
Ensure that you are running the commands described below after upgrading to version 0.24.0 or later. Ensure that you are running the commands described below after upgrading to version 0.24.0 or later.
### Migration commands ### Migration commands
@@ -102,7 +122,7 @@ To migrate from JSON file store to SQLite, follow these steps:
```bash ```bash
docker exec -it netbird-management /go/bin/netbird-mgmt sqlite-migration upgrade --log-file console docker exec -it netbird-management /go/bin/netbird-mgmt sqlite-migration upgrade --log-file console
``` ```
5. Enable SQLite by updating the management.json file and setting the `Engine` field to `sqlite` as the following example: 5. Enable SQLite by updating the `management.json` file and setting the `Engine` field to `sqlite`:
```json ```json
"StoreConfig": { "StoreConfig": {
"Engine": "sqlite" "Engine": "sqlite"
@@ -120,48 +140,3 @@ You should see an entry similar to:
``` ```
2023-10-19T18:55:29Z INFO management/server/store.go:75: using SQLite store engine 2023-10-19T18:55:29Z INFO management/server/store.go:75: using SQLite store engine
``` ```
### Rollback to JSON file store
<Note>
Since version [0.28.0](https://github.com/netbirdio/netbird/releases/tag/v0.28.0), support for JSON file storage has been removed, and the option to roll back to JSON store is no longer supported.
</Note>
To rollback to the JSON file store, follow these steps:
<Note>
The following commands assume you use the latest docker version with the compose plugin. If you have docker-compose installed as a standalone, please use docker-compose as a command.
</Note>
1. After accessing the path that contains your docker-compose.yml and management.json files, stop the NetBird Management service.
```bash
docker compose stop management
```
2. Backup your data store (`store.db` in `datadir` - default "/var/lib/netbird/").
```bash
mkdir backup
docker compose cp -a management:/var/lib/netbird/ backup/
```
3. Start NetBird Management service:
```bash
docker compose start management
```
4. Run the migration to SQLite subcommand:
```bash
docker exec -it netbird-management /go/bin/netbird-mgmt sqlite-migration downgrade --log-file console
```
5. Enable JSON file by updating the management.json file and setting the `Engine` field to `jsonfile` as the following example:
```json
"StoreConfig": {
"Engine": "jsonfile"
}
```
6. Restart the Management service.
```bash
docker compose restart management
```
7. Check logs to confirm the store switch:
```bash
docker compose logs management
```
You should see an entry similar to:
```
2023-10-19T18:55:29Z INFO management/server/store.go:72: using JSON file store engine
```

View File

@@ -54,7 +54,8 @@ This page will help with various issues when self-hosting NetBird.
**Solution**: This tab is only visible when the embedded IdP is enabled. Check your deployment configuration: **Solution**: This tab is only visible when the embedded IdP is enabled. Check your deployment configuration:
- For quickstart deployments, the embedded IdP should be enabled by default - For quickstart deployments, the embedded IdP should be enabled by default
- For advanced deployments, ensure `EmbeddedIdP.Enabled` is set to `true` in management.json - For the combined setup, the embedded IdP is always enabled
- For older multi-container deployments, ensure `EmbeddedIdP.Enabled` is set to `true` in `management.json`
### Users not syncing from SSO provider ### Users not syncing from SSO provider
@@ -74,7 +75,7 @@ To test your TURN configuration you can access the [online tester](https://webrt
There you will find a ICE servers input box, where you can select and remove the existing server, then add your turn server There you will find a ICE servers input box, where you can select and remove the existing server, then add your turn server
configuration as follows: configuration as follows:
Please replace <b>netbird.DOMAIN.com</b> and <b>PASSWORD</b> with the information from the <b>management.json</b> TURNConfig, then click on <b>Add server</b>. Please replace <b>netbird.DOMAIN.com</b> and <b>PASSWORD</b> with your STUN/TURN server details from your configuration (<b>config.yaml</b> for the combined setup, or the <b>TURNConfig</b> section in <b>management.json</b> for older multi-container setups), then click on <b>Add server</b>.
<p> <p>
<img src="/docs-static/img/selfhosted/troubleshooting/turn.png" alt="turn" width="700" className="imagewrapper"/> <img src="/docs-static/img/selfhosted/troubleshooting/turn.png" alt="turn" width="700" className="imagewrapper"/>