mirror of
https://github.com/netbirdio/docs.git
synced 2026-09-28 17:59:05 +02:00
262 lines
7.3 KiB
Plaintext
262 lines
7.3 KiB
Plaintext
# Management PostgreSQL store
|
|
|
|
PostgreSQL is recommended for production deployments. It supports concurrent access, enabling multiple management instances for high availability.
|
|
|
|
## Configuration
|
|
|
|
### Combined setup (config.yaml)
|
|
|
|
To use PostgreSQL, update your `config.yaml`:
|
|
|
|
```yaml
|
|
server:
|
|
store:
|
|
engine: "postgres"
|
|
dsn: "host=<PG_HOST> user=<PG_USER> password=<PG_PASSWORD> dbname=<PG_DB_NAME> port=<PG_PORT>"
|
|
```
|
|
|
|
You can also pass the DSN as an `NB_` environment variable on the `netbird-server` container in your `docker-compose.yml`:
|
|
|
|
```yaml
|
|
environment:
|
|
- NB_STORE_ENGINE_POSTGRES_DSN=host=<PG_HOST> user=<PG_USER> password=<PG_PASSWORD> dbname=<PG_DB_NAME> port=<PG_PORT>
|
|
```
|
|
|
|
Restart the server and confirm:
|
|
```bash
|
|
docker compose restart netbird-server
|
|
docker compose logs netbird-server
|
|
```
|
|
You should see:
|
|
```
|
|
INFO management/server/store.go:109: using Postgres store engine
|
|
```
|
|
|
|
<Note>
|
|
For a full list of available configuration options, see the [config.yaml.example](https://github.com/netbirdio/netbird/blob/main/combined/config.yaml.example) reference file.
|
|
</Note>
|
|
|
|
### Older multi-container setup (management.json)
|
|
|
|
<Note>
|
|
This section applies to deployments using the older multi-container architecture with separate `management`, `signal`, `relay`, and `coturn` containers. If you deployed using the [`getting-started.sh`](/selfhosted/selfhosted-quickstart) script, you are on the combined setup and should use the `config.yaml` instructions above. See the [migration guide](/selfhosted/migration/combined-container) to upgrade.
|
|
</Note>
|
|
|
|
To enable Postgres, add to your `setup.env`:
|
|
```bash
|
|
NETBIRD_STORE_CONFIG_ENGINE=postgres
|
|
```
|
|
This sets the following in your `management.json`:
|
|
```json
|
|
"StoreConfig": {
|
|
"Engine": "postgres"
|
|
}
|
|
```
|
|
|
|
Create a `.env` file with the connection string:
|
|
```bash
|
|
NETBIRD_STORE_ENGINE_POSTGRES_DSN="host=<PG_HOST> user=<PG_USER> password=<PG_PASSWORD> dbname=<PG_DB_NAME> port=<PG_PORT>"
|
|
```
|
|
|
|
Update `docker-compose.yml` to pass the config to the `management` container:
|
|
```yaml
|
|
environment:
|
|
- NETBIRD_STORE_ENGINE_POSTGRES_DSN=${NETBIRD_STORE_ENGINE_POSTGRES_DSN}
|
|
env_file:
|
|
- .env
|
|
```
|
|
|
|
<Note>
|
|
Switching between storage options requires migration steps to prevent data loss.
|
|
</Note>
|
|
|
|
## Migrating from SQLite store to Postgres store
|
|
This migration process allows users to seamlessly transition between storage options while maintaining data integrity.
|
|
|
|
<Note>
|
|
The following commands assume you use the latest docker version with the compose plugin. If you have docker-compose installed as a standalone, please use docker-compose as a command.
|
|
</Note>
|
|
|
|
1. Backup your data store (`store.db` in `datadir` - default `/var/lib/netbird/`)
|
|
|
|
For the combined setup:
|
|
```bash
|
|
mkdir backup
|
|
docker compose cp -a netbird-server:/var/lib/netbird/. backup/
|
|
```
|
|
|
|
For the older multi-container setup:
|
|
```bash
|
|
mkdir backup
|
|
docker compose cp -a management:/var/lib/netbird/. backup/
|
|
```
|
|
|
|
2. Import SQLite data to Postgres
|
|
|
|
For migrating the SQLite data we rely on the [pgloader](https://github.com/dimitri/pgloader) tool. You can install it by running
|
|
`sudo apt-get install pgloader` on debian or `brew install pgloader` on MacOS.
|
|
|
|
```bash
|
|
pgloader --type sqlite backup/store.db "postgresql://<PG_USER>:<PG_PASSWORD>@<PG_HOST>:<PG_PORT>/<PG_DB_NAME>"
|
|
```
|
|
|
|
3. Resolve any compatibility issues that may arise due to the migration process.
|
|
|
|
<Note>
|
|
As of version 0.26.0, NetBird used SQLite as its default database store. However, SQLite lacked support for cascading deletions,
|
|
which means that related entries were not automatically removed when their parent entries were deleted.
|
|
With the migration to PostgreSQL, foreign key constraints are enforced, including cascading deletes, to ensure referential integrity.
|
|
</Note>
|
|
|
|
To address this, you can execute the following SQL commands in your PostgreSQL database:
|
|
|
|
```sql
|
|
DELETE FROM policy_rules WHERE policy_rules.policy_id NOT IN (SELECT id FROM policies);
|
|
```
|
|
|
|
```sql
|
|
DELETE FROM routes WHERE routes.account_id NOT IN (SELECT id FROM accounts);
|
|
```
|
|
|
|
```sql
|
|
DELETE FROM name_server_groups WHERE name_server_groups.account_id NOT IN (SELECT id FROM accounts);
|
|
```
|
|
|
|
```sql
|
|
DROP TABLE IF EXISTS rules;
|
|
```
|
|
|
|
4. Enable Postgres in your configuration.
|
|
|
|
For the combined setup, update `config.yaml`:
|
|
```yaml
|
|
server:
|
|
store:
|
|
engine: "postgres"
|
|
dsn: "host=<PG_HOST> user=<PG_USER> password=<PG_PASSWORD> dbname=<PG_DB_NAME> port=<PG_PORT>"
|
|
```
|
|
|
|
For the older multi-container setup, update `management.json`:
|
|
```json
|
|
"StoreConfig": {
|
|
"Engine": "postgres"
|
|
}
|
|
```
|
|
And pass the DSN via environment variable as described in the [older multi-container setup](#older-multi-container-setup-managementjson) section above.
|
|
|
|
5. Restart the server and confirm:
|
|
|
|
For the combined setup:
|
|
```bash
|
|
docker compose restart netbird-server
|
|
docker compose logs netbird-server
|
|
```
|
|
|
|
For the older multi-container setup:
|
|
```bash
|
|
docker compose restart management
|
|
docker compose logs management
|
|
```
|
|
|
|
You should see an entry similar to:
|
|
```
|
|
INFO management/server/store.go:109: using Postgres store engine
|
|
```
|
|
|
|
## Rollback to SQLite store
|
|
To rollback to the SQLite store, follow these steps:
|
|
|
|
1. Restore `store.db` backup
|
|
|
|
For the combined setup:
|
|
```bash
|
|
docker compose cp backup/. netbird-server:/var/lib/netbird/
|
|
```
|
|
|
|
For the older multi-container setup:
|
|
```bash
|
|
docker compose cp backup/. management:/var/lib/netbird/
|
|
```
|
|
|
|
2. Switch the store engine back to SQLite.
|
|
|
|
For the combined setup, update `config.yaml`:
|
|
```yaml
|
|
server:
|
|
store:
|
|
engine: "sqlite"
|
|
```
|
|
|
|
For the older multi-container setup, update `management.json`:
|
|
```json
|
|
"StoreConfig": {
|
|
"Engine": "sqlite"
|
|
}
|
|
```
|
|
|
|
3. Restart the server and confirm:
|
|
|
|
For the combined setup:
|
|
```bash
|
|
docker compose restart netbird-server
|
|
docker compose logs netbird-server
|
|
```
|
|
|
|
For the older multi-container setup:
|
|
```bash
|
|
docker compose restart management
|
|
docker compose logs management
|
|
```
|
|
|
|
You should see an entry similar to:
|
|
```
|
|
INFO management/server/store.go:109: using SQLite file store engine
|
|
```
|
|
|
|
## Optional Rollback Postgres data to SQLite
|
|
|
|
This is optional and should be used only if you want to rollback the data from Postgres to SQLite while running the same NetBird version.
|
|
For migrating the Postgres data, we rely on the `pg_dump`, `sed`, and `sqlite3` tools. Make sure these are installed before proceeding
|
|
|
|
1. Export Postgres data
|
|
```bash
|
|
pg_dump --data-only --column-inserts "postgresql://<PG_USER>:<PG_PASSWORD>@<PG_HOST>:<PG_PORT>/<PG_DB_NAME>" > data.sql
|
|
```
|
|
|
|
2. Convert exported Postgres data sql to SQLite format
|
|
```bash
|
|
sed \
|
|
-e 's/\\\\:/\:/g' \
|
|
-e 's/\\\\//g' \
|
|
-e 's/\\\\;/;/g' \
|
|
-e '/^SET /d' \
|
|
-e '/setval/d' \
|
|
-e "s/'true'/1/g" \
|
|
-e "s/'false'/0/g" \
|
|
-e 's/public\.//' \
|
|
-e '/^[[:space:]]*SELECT/d' data.sql > data.sql
|
|
```
|
|
|
|
3. Generate database schema from SQLite backup
|
|
```bash
|
|
sqlite3 backup/store.db '.schema' > schema.sql
|
|
````
|
|
|
|
4. Create SQLite database with Postgres exported data
|
|
```bash
|
|
sqlite3 store.db '.read schema.sql' && sqlite3 store.db '.read data.sql'
|
|
```
|
|
|
|
5. Copy db to the container
|
|
|
|
For the combined setup:
|
|
```bash
|
|
docker compose cp store.db netbird-server:/var/lib/netbird/store.db
|
|
```
|
|
|
|
For the older multi-container setup:
|
|
```bash
|
|
docker compose cp store.db management:/var/lib/netbird/store.db
|
|
```
|
|
|
|
6. Switch the store engine back to SQLite using the instructions in the [Rollback to SQLite store](#rollback-to-sqlite-store) section above. |