Data and migrations

This page explains where Behemoth keeps its data, where the tables come from, and how a change to your declarations reaches your database. Read How Behemoth fits together first.

Behemoth does not create tables when it starts. You declare tables in code, Behemoth writes a migration for them, and you apply it before the first Boot.

The database layer #

Behemoth stores users, sessions, and tokens in your database, through an adapter. An adapter is a small module for one database that you pass to Boot. Each adapter is a separate Go module, so you download only the ones you use.

The following table lists the adapters. Each module path starts with github.com/go-behemoth/behemoth/storage/adapters/:

DatabaseModuleMinimum release
PostgreSQLpostgres15
MySQLmysql8.4
SQL Serversqlserver2019
SQLitesqlite3.53, and it needs cgo
MongoDBmongo7.0, as a replica set
Bun and GORMbun and gormFor an application that already uses one of these libraries

The Supported databases reference lists the releases that each adapter is tested on.

Build the adapter with app.Resolver, the value that Prepare returns. It makes your queries and your migrations agree on table and column names.

go
db := postgres.NewPostgresAdapter(sqlDB, app.Resolver)

Optional key-value storage #

By default, sessions and rate-limit counters live in your database. To keep them in a key-value store instead, pass a Redis adapter in BootConfig.KV. A session is then served from the store without a database read.

The store and the adapter #

Your code reaches data in two ways, both on the AuthContext:

  • ac.Store reads and writes Behemoth's own tables. It adds what a bare write skips: ids, timestamps, hooks, and a transaction around the write.
  • ac.DB is the adapter. Use it for your own tables.

Where tables come from #

Every table is declared in code during Prepare, in a fixed order. A later declarer can add columns to a table that an earlier one declared.

OrderDeclared byExamples
1Behemoth itselfusers, accounts, sessions, tokens, verification_codes, reserved_values, rate_limits, audit_log. See Core tables.
2Each plugin, in dependency orderA plugin's own tables, or extra columns on users
3Your applicationYour own tables, or extra columns on any table above

Declare your own tables #

Pass a function to PrepareConfig.Schema. It declares a model, a Go type that implements behemoth.Model, together with its table:

go
func declareSchema(reg schema.Registry) error {
    return reg.Declare(
        &Todo{},
        schema.Table{
            Name: "todos",
            Columns: []schema.Column{
                {Name: "id", Type: schema.ColTypeUuid, PrimaryKey: true},
                {Name: "title", Type: schema.ColTypeString, Length: 255},
                {Name: "done", Type: schema.ColTypeBoolean, Default: false},
            },
        },
    )
}

app, err := engine.Prepare(
    plugins,
    engine.PrepareConfig{
        Schema: declareSchema,
    },
)
if err != nil {
    log.Fatal(err)
}

The examples/init directory in the repository has the complete Todo model. A table can also set a PhysicalName, so that the name in your code differs from the name in the database. The resolver translates between them.

Clients see public columns only

A column that you add to users stays private until you mark it public. Routes that return a user send only its public columns. See Core tables.

Migrations #

A migration is the list of schema operations that makes your database match your declarations. Behemoth compares what you declared with what the database holds, and writes the difference to your migrations folder as a numbered migration.

Because Prepare needs no connection, the behemoth command can read your declarations without starting your application.

Choose a migration path #

Set the path in PrepareConfig.Migration. The two paths differ in who applies the migration:

PathGenerateOnly (default)PathManaged
BehemothWrites a .json migration and a .sql scriptWrites the same files, then applies the migration itself
YouApply the script with your own tool, such as psqlRun behemoth migrate
Compared withThe live databaseA snapshot that Behemoth keeps
Extra tablesNoneTwo tables for its own record of applied migrations
Use it whenYou already have a migration tool, or a team reviews every changeThe service has no other migration tool

The default never touches your database. behemoth generate writes files and changes nothing else.

The workflow #

You repeat the same steps whenever your declarations change, for example when you add a plugin or a table:

  1. Change your declarations.
  2. Run behemoth generate to preview the next migration. It writes nothing but a draft file.
  3. Answer any questions. See Questions.
  4. Run behemoth generate -confirm to write the migration to your migrations folder.
  5. Review the .sql script.
  6. Apply it. On the default path, run the script with your own tool. On the managed path, run behemoth migrate -confirm.
  7. Run behemoth generate again. When the database matches your declarations, it answers No schema changes detected.

Commit the whole migrations folder. Behemoth takes the number of the next migration from the files in it.

Questions #

Some differences have more than one answer. If a column becomes shorter, Behemoth can alter it, which may reject existing rows, or leave it as it is. generate asks you in the terminal, and saves each answer in draft.json in your migrations folder.

In CI, where there is no terminal, a question without an answer stops the command with exit code 1. Commit the draft file, or give the answer with the -answer flag.

Set up the command #

The command reads your plugins and schema from your own code, so you don't describe them in a second place. Put two functions in a file named behemoth.go, in a package other than main:

internal/auth/behemoth.go
package auth

func Prepare() (*engine.PreparedApp, error) {
    return engine.Prepare(
        []core.Plugin{emailpassword.New(emailpassword.Options{})},
        engine.PrepareConfig{
            Migration: migration.MigrationConfig{
                FolderPath: "migrations",
            },
            Schema: declareSchema,
        },
    )
}

func MigrationBackend(
    ctx context.Context,
    app *engine.PreparedApp,
) (migration.Backend, error) {
    db, err := OpenDB(ctx)
    if err != nil {
        return migration.Backend{}, err
    }
    return postgres.MigrationBackend(db, app.Resolver), nil
}

In this example, OpenDB is your own function that opens a *sql.DB. For MySQL, return mysql.MigrationBackend instead. Your server calls the same Prepare and passes the result to Boot, so the tables a migration creates are the tables your server reads.

Then install the launcher and run it from your application's directory:

bash
go install github.com/go-behemoth/behemoth/cmd/behemoth@latest

cd my-app
behemoth generate
Experimental

The command line is a first version. It has ready-made setups for PostgreSQL and MySQL, and the launcher needs the Go toolchain. The Migration CLI reference describes the other databases, running without the launcher, and exit codes.

Hand-written migrations #

For a change that Behemoth cannot derive, such as a data backfill, pass a migration.CustomMigration in PrepareConfig.Migrations. Behemoth places it among the generated operations by its DependsOn list. The name of a custom migration is permanent, so never reuse one.

Limits #

  • Apply the first migration before Boot. Boot does not create tables, and a missing table fails the first query.
  • SQLite needs foreign keys switched on. Boot refuses a SQLite database that does not enforce them. Add _foreign_keys=on to the DSN.
  • MongoDB has no migrations. It needs a replica set, and you call EnsureIndexes on the adapter to create the unique indexes your schema declares. Boot refuses to start without them. See MongoDB.
  • Other SQL databases need a hand-built backend. The command has ready-made backends for PostgreSQL and MySQL only. For SQLite or SQL Server, you fill migration.Backend from the adapter module's driver.

Next steps #

To go deeper, read the following pages: