golang/ent
1
0
Fork 0
mirror of https://github.com/ent/ent.git synced 2026-05-24 09:31:56 +03:00
Code Issues Packages Projects Releases Wiki Activity

dialect/sql/schema: atlas engine is now default (#2698)

Browse Source 
* atlas engine is default, enabled diff by replay

* Apply suggestions from code review

* docs

* apply CR
This commit is contained in:
Jannik Clausen
2022-07-05 12:29:15 +02:00
committed by GitHub
parent 91b643091f
commit 5b67bdab4f
19 changed files with 828 additions and 1475 deletions
Download Patch File Download Diff File Expand all files Collapse all files

296
doc/md/versioned-migrations.md
View File

@@ -8,18 +8,27 @@ applying the computed changes directly to the database, it will generate a set o
necessary SQL statements to migrate the database. These files can then be edited to your needs and be applied by any
tool you like (like golang-migrate, Flyway, liquibase).
![atlas-versioned-migration-process](https://entgo.io/images/assets/migrate-atlas-versioned.png)
## Generating Versioned Migration Files
### From Client
As mentioned above, versioned migrations do only work, if the new Atlas based migration engine is used. Migration files
are generated by computing the difference between two **states**. We call the state reflected by your Ent schema the **
desired** state, and the last state before your most recent changes the **current** state. There are two ways for Ent to
determine the current state:
If you want to use an instantiated Ent client to create new migration files, you have to enable the versioned
migrations feature flag in order to have Ent make the necessary changes to the generated code. Depending on how you
execute the Ent code generator, you have to use one of the two options:
1. Replay the existing migration directory and inspect the schema (default)
2. Connect to an existing database and inspect the schema
1. If you are using the default go generate configuration, simply add the `--feature sql/versioned-migration` to
the `ent/generate.go` file as follows:
We emphasize to use the first option, as it has the advantage for you to not have to connect to a production database to
create a diff. In addition, this approach also works, if you have multiple deployments in different migration states.
The first step is to enable the versioned migration feature by passing in the `sql/versioned-migration` feature flag.
Depending on how you execute the Ent code generator, you have to use one of the two options:
![atlas-versioned-migration-process](https://entgo.io/images/assets/migrate-atlas-replay.png)
#### With Ent CLI
If you are using the default go generate configuration, simply add the `--feature sql/versioned-migration` to
the `ent/generate.go` file as follows:
```go
package ent
@@ -27,8 +36,10 @@ package ent
//go:generate go run -mod=mod entgo.io/ent/cmd/ent generate --feature sql/versioned-migration ./schema
```
2. If you are using the code generation package (e.g. if you are using an Ent extension), add the feature flag as
follows:
#### With entc package
If you are using the code generation package (e.g. if you are using an Ent extension), add the feature flag as
follows:
```go
//go:build ignore
@@ -43,108 +54,59 @@ import (
)
func main() {
err := entc.Generate("./schema", &gen.Config{}, entc.FeatureNames("sql/versioned-migration"))
err := entc.Generate("./schema", &gen.Config{}, entc.FeatureNames(gen.FeatureVersionedMigration.Name))
if err != nil {
log.Fatalf("running ent codegen: %v", err)
}
}
```
After regenerating the project, there will be an extra `Diff` method on the Ent client that you can use to inspect the
connected database, compare it with the schema definitions and create SQL statements needed to migrate the database to
the graph.
After regenerating the project, you have two ways to generate new migration files. First, there will be a new top-level
method in the generated `migrate` package: `NamedDiff`. All you have to do is provide an Atlas
[dev database](https://atlasgo.io/dev-database) URL for the desired database dialect.
```go
package main
import (
"context"
"log"
"context"
"log"
"<project>/ent"
"<project>/ent/migrate"
"ariga.io/atlas/sql/migrate"
"entgo.io/ent/dialect/sql/schema"
_ "github.com/go-sql-driver/mysql"
atlas "ariga.io/atlas/sql/migrate"
"entgo.io/ent/dialect"
"entgo.io/ent/dialect/sql/schema"
_ "github.com/go-sql-driver/mysql"
)
func main() {
client, err := ent.Open("mysql", "root:pass@tcp(localhost:3306)/test")
if err != nil {
log.Fatalf("failed connecting to mysql: %v", err)
}
defer client.Close()
ctx := context.Background()
// Create a local migration directory.
dir, err := migrate.NewLocalDir("migrations")
dir, err := atlas.NewLocalDir("migrations")
if err != nil {
log.Fatalf("failed creating atlas migration directory: %v", err)
}
// Write migration diff.
err = client.Schema.Diff(ctx, schema.WithDir(dir))
// You can use the following method to give the migration files a name.
// err = client.Schema.NamedDiff(ctx, "migration_name", schema.WithDir(dir))
opts := []schema.MigrateOption{
schema.WithDir(dir), // provide migration directory
schema.WithMigrationMode(schema.ModeReplay), // provide migration mode
schema.WithDialect(dialect.MySQL), // Ent dialect to use
}
// Generate migrations using Atlas support for TiDB (note the Ent dialect option passed above).
err = migrate.NamedDiff(ctx, "tidb://user:pass@localhost:3306/ent_dev", "my_migration", opts...)
if err != nil {
log.Fatalf("failed creating schema resources: %v", err)
log.Fatalf("failed generating migration file: %v", err)
}
}
```
You can then create a new set of migration files by simply calling `go run -mod=mod main.go`.
You can then generate a new set of migration files by simply calling `go run -mod=mod main.go`.
### From Graph
You can also generate new migration files without an instantiated Ent client. This can be useful if you want to make the
migration file creation part of a go generate workflow. Note, that in this case enabling the feature flag is optional.
```go
package main
import (
"context"
"log"
"ariga.io/atlas/sql/migrate"
"entgo.io/ent/dialect/sql"
"entgo.io/ent/dialect/sql/schema"
"entgo.io/ent/entc"
"entgo.io/ent/entc/gen"
)
func main() {
// Load the graph.
graph, err := entc.LoadGraph("./schema", &gen.Config{})
if err != nil {
log.Fatalln(err)
}
tbls, err := graph.Tables()
if err != nil {
log.Fatalln(err)
}
// Create a local migration directory.
d, err := migrate.NewLocalDir("migrations")
if err != nil {
log.Fatalln(err)
}
// Open connection to the database.
dlct, err := sql.Open("mysql", "root:pass@tcp(localhost:3306)/test")
if err != nil {
log.Fatalln(err)
}
// Inspect it and compare it with the graph.
m, err := schema.NewMigrate(dlct, schema.WithDir(d))
if err != nil {
log.Fatalln(err)
}
if err := m.Diff(context.Background(), tbls...); err != nil {
log.Fatalln(err)
}
// You can use the following method to give the migration files a name.
// if err := m.NamedDiff(context.Background(), "migration_name", tbls...); err != nil {
// log.Fatalln(err)
// }
}
```
:::info Note
If you want to inspect an existing database and compute the diff against your Ent schema, pass in the `ModeInspect`
migration mode.
:::
## Apply Migrations
@@ -165,18 +127,11 @@ versioned migration, you need to take some extra steps.
### Create an initial migration file reflecting the currently deployed state
To do this make sure your schema definition is in sync with your deployed version. Then spin up an empty database and
To do this make sure your schema definition is in sync with your deployed version(s). Then spin up an empty database and
run the diff command once as described above. This will create the statements needed to create the current state of
your schema graph.
If you happened to have [universal IDs](migrate.md#universal-ids) enabled before, the above command will create a
file called `.ent_types` containing a list of schema names similar to the following:
```text title=".ent_types"
atlas.sum ignore
users,groups
```
Once that has been created, one of the migration files will contain statements to create a table called
`ent_types`, as well as some inserts to it:
your schema graph. If you happened to have [universal IDs](migrate.md#universal-ids) enabled before, any deployment will
have a special database table named `ent_types`. The above command will create the necessary SQL statements to create
that table as well as its contents (similar to the following):
```sql
CREATE TABLE `users` (`id` integer NOT NULL PRIMARY KEY AUTOINCREMENT);
@@ -188,7 +143,7 @@ INSERT INTO `ent_types` (`type`) VALUES ('users'), ('groups');
```
In order to ensure to not break existing code, make sure the contents of that file are equal to the contents in the
table present in the database you created the diff from. For example, if you consider the `.ent_types` file from
table present in the database you created the diff from. For example, if you consider the migration file from
above (`users,groups`) but your deployed table looks like the one below (`groups,users`):
| id | type |
@@ -196,14 +151,14 @@ above (`users,groups`) but your deployed table looks like the one below (`groups
| 1 | groups |
| 2 | users |
You can see, that the order differs. In that case, you have to manually change both the entries in the
`.ent_types` file, as well in the generated migrations file. As a safety feature, Ent will warn you about type
drifts if you attempt to run a migration diff.
You can see, that the order differs. In that case, you have to manually change both the entries in the generated
migration file.
### Configure the tool you use to manage migrations to consider this file as applied
In case of `golang-migrate` this can be done by forcing your database version as
described [here](https://github.com/golang-migrate/migrate/blob/master/GETTING_STARTED.md#forcing-your-database-version).
described [here](https://github.com/golang-migrate/migrate/blob/master/GETTING_STARTED.md#forcing-your-database-version)
.
## Use a Custom Formatter
@@ -217,73 +172,62 @@ following four:
4. [Liquibase](https://www.liquibase.org/)
:::note
You need to have the latest master of Ent installed for this to be working.
You need to have the latest master of Ent installed for this to be working.
```shell
go get -u entgo.io/ent@master
```
:::
```go
package main
import (
"context"
"log"
"strings"
"text/template"
"time"
"context"
"log"
"ariga.io/atlas/sql/migrate"
"ariga.io/atlas/sql/sqltool"
"entgo.io/ent/dialect/sql"
"entgo.io/ent/dialect/sql/schema"
"entgo.io/ent/entc"
"entgo.io/ent/entc/gen"
"<project>/ent/migrate"
atlas "ariga.io/atlas/sql/migrate"
"ariga.io/atlas/sql/sqltool"
"entgo.io/ent/dialect"
"entgo.io/ent/dialect/sql/schema"
_ "github.com/go-sql-driver/mysql"
)
func main() {
// Load the graph.
graph, err := entc.LoadGraph("/.schema", &gen.Config{})
if err != nil {
log.Fatalln(err)
}
tbls, err := graph.Tables()
if err != nil {
log.Fatalln(err)
}
// Create a local migration directory.
d, err := migrate.NewLocalDir("migrations")
if err != nil {
log.Fatalln(err)
}
// Open connection to the database.
dlct, err := sql.Open("mysql", "root:pass@tcp(localhost:3306)/test")
if err != nil {
log.Fatalln(err)
}
// Inspect it and compare it with the graph.
m, err := schema.NewMigrate(dlct, schema.WithDir(d),
// highlight-start
// Chose one of the below.
schema.WithFormatter(sqltool.GolangMigrateFormatter),
schema.WithFormatter(sqltool.GooseFormatter),
schema.WithFormatter(sqltool.FlywayFormatter),
schema.WithFormatter(sqltool.LiquibaseFormatter),
// highlight-end
)
if err != nil {
log.Fatalln(err)
}
if err := m.Diff(context.Background(), tbls...); err != nil {
log.Fatalln(err)
}
ctx := context.Background()
// Create a local migration directory.
dir, err := atlas.NewLocalDir("migrations")
if err != nil {
log.Fatalf("failed creating atlas migration directory: %v", err)
}
// Write migration diff.
opts := []schema.MigrateOption{
schema.WithDir(dir), // provide migration directory
schema.WithMigrationMode(schema.ModeReplay), // provide migration mode
schema.WithDialect(dialect.MySQL), // Ent dialect to use
// highlight-start
// Choose one of the below.
schema.WithFormatter(sqltool.GolangMigrateFormatter),
schema.WithFormatter(sqltool.GooseFormatter),
schema.WithFormatter(sqltool.FlywayFormatter),
schema.WithFormatter(sqltool.LiquibaseFormatter),
// highlight-end
}
// Generate migrations using Atlas support for TiDB (note the Ent dialect option passed above).
err = migrate.NamedDiff(ctx, "tidb://user:pass@localhost:3306/ent_dev", "my_migration", opts...)
if err != nil {
log.Fatalf("failed generating migration file: %v", err)
}
}
```
### Note for using golang-migrate
If you use golang-migrate with MySQL, you need to add the `multiStatements` parameter to `true` as in the example below and then take the DSN we used in the documents with the param applied.
If you use golang-migrate with MySQL, you need to add the `multiStatements` parameter to `true` as in the example below
and then take the DSN we used in the documents with the param applied.
```
"user:password@tcp(host:port)/dbname?multiStatements=true"
@@ -360,58 +304,10 @@ Please note, that you need to have the Atlas CLI installed in your system for th
the [installation instructions](https://atlasgo.io/cli/getting-started/setting-up#install-the-cli) before proceeding.
:::
The first step is to tell the migration engine to create a sum file by using the `schema.WithSumFile()` option:
```go
package main
import (
"context"
"log"
"ariga.io/atlas/sql/migrate"
"ariga.io/atlas/sql/sqltool"
"entgo.io/ent/dialect/sql"
"entgo.io/ent/dialect/sql/schema"
"entgo.io/ent/entc"
"entgo.io/ent/entc/gen"
)
func main() {
// Load the graph.
graph, err := entc.LoadGraph("/.schema", &gen.Config{})
if err != nil {
log.Fatalln(err)
}
tbls, err := graph.Tables()
if err != nil {
log.Fatalln(err)
}
// Create a local migration directory.
d, err := migrate.NewLocalDir("migrations")
if err != nil {
log.Fatalln(err)
}
// Open connection to the database.
dlct, err := sql.Open("mysql", "root:pass@tcp(localhost:3306)/test")
if err != nil {
log.Fatalln(err)
}
// Inspect it and compare it with the graph.
m, err := schema.NewMigrate(dlct, schema.WithDir(d),
// highlight-start
// Enable the Atlas Migration Directory Integrity File.
schema.WithSumFile(),
// highlight-end
)
if err != nil {
log.Fatalln(err)
}
if err := m.Diff(context.Background(), tbls...); err != nil {
log.Fatalln(err)
}
}
```
In previous versions of Ent, the integrity file was opt-in. But we think this is a very important feature that provides
great value and safety to migrations. Therefore, generation of the sum file is now the default behavior and in the
future we might even remove the option to disable this feature. For now, if you really want to remove integrity file
generation, use the `schema.DisableChecksum()` option.
In addition to the usual `.sql` migration files the migration directory will contain the `atlas.sum` file. Every time
you let Ent generate a new migration file, this file is updated for you. However, every manual change made to the