.. | ||
elastic_migrations | ||
migrations | ||
templates | ||
.dockerignore | ||
.env.example | ||
Dockerfile | ||
migrate.ts | ||
package.json | ||
plopfile.js | ||
README.md | ||
setup.sh | ||
tsconfig.json |
Database management
This workspace is used for database schema definitions and migrations.
Project currently uses PostgreSQL 11. Make sure you use the correct version.
Migrations usage
❗ It is important to understand that migrations that had already been performed are locked and are not a subject to change. Therefore for every alteration of schema you need to generate a new migration and perform alterations there.
Never commit changed files for migrations that already had been performed elsewhere, - it will break the migrations.
To migrate to latest version: yarn migrate
To migrate up/down to specific version: yarn migrate <version>
To migrate down to empty state: yarn migrate 0000
To generate new migration: yarn generate
* At the moment, version numbers expect to be padded to 4 digits
Policies and roles
We use Row Level Security when accessing the database from the application. In order to create a correct schema for new tables please study migration files for the schemas of previous tables (including possible later changes).
In order to use Row Level Security every transaction with the database must set the correct role via omnivore.set_claims
function.
Current roles
omnivore_user
- a user role that is intended for a regular user to access the data. Currently, this is the primary user of the database.
Database users on GCP
postgres
- administrator of the database, used for migrations.
app_user
- a user that the app uses to login to database.
❗ Do not issue any extra grants to app_user
other than that are needed to assume a certain internal role, i.e. GRANT omnivore_user TO app_user
Installing and Using locally
- Install and run the postgresql service on your machine.
Configure access to the database
On some systems, Postgres will only allow the local postgres user to connect to the database. One way around this is to
set the authentication method in your pg_hba.conf file. The trust
method allows any user who can connect to
database to make changes. The default is peer
which means the user must be logged into the system as the postgres
user. Another option is md5
or password
which allows access to the postgres user with a password instead of
needing to be logged in locally as postgres
. For simplicity, set the method to trust
. You
will need to restart the postgresql service after making this change.
-
Verify you can connect to postgres
psql -U postgres
. -
Quit psql (command is
\q
) -
Create database using handy CLI tool (which needs the
-U <user>
flag for now):$ createdb -U postgres omnivore
-
Copy .env.example file to .env file:
cp .env.example .env
-
Modify
.env
and setPG_USER
topostgres
-
Run migration:
yarn migrate
Accessing the database locally
Instead of using the superuser to access, create a user with the omnivore_user
role. You can choose your local
username instead of app_user
here to avoid needing the -U app_user
flag in the psql
command below.
- Create a user named
app_user
in Postgres - Allow
app_user
to assume the roles necessary for the application. Do not manually grant any other role toapp_user
$ psql -U postgres
# CREATE USER app_user WITH ENCRYPTED PASSWORD 'app_pass';
# GRANT omnivore_user to app_user;
-
Update the
PG_USER
andPG_PASSWORD
values in.env
files (packages/db, pkg/api) toapp_user
andapp_pass
, respectively -
You can now use psql to login to your database:
psql -U app_user -d omnivore
Gotchas
Postgres Row-Level Security can at times catch us off guard: there are policies limiting select/update operations on
tables based on active user role/ID in a transaction block. So at times, when working on the local database, one must
make sure to login via postgres
user to view all rows in the tables or perform updates.