golang/ent
1
0
Fork 0
mirror of https://github.com/ent/ent.git synced 2026-05-24 09:31:56 +03:00
Code Issues Packages Projects Releases Wiki Activity

doc/md: updating versioned migrations tutorial (#3334)

Browse Source 
* doc/md: updating versioned migrations tutorial

* Apply suggestions from code review

Co-authored-by: Hila Kashai <73284641+hilakashai@users.noreply.github.com>

---------

Co-authored-by: Hila Kashai <73284641+hilakashai@users.noreply.github.com>
This commit is contained in:
Rotem Tamir
2023-02-20 17:10:10 +02:00
committed by GitHub
parent 102f60ac12
commit eeccc754d3
8 changed files with 267 additions and 191 deletions
Download Patch File Download Diff File Expand all files Collapse all files

43
doc/md/versioned/02-auto-plan.mdx Normal file
View File

@@ -0,0 +1,43 @@
---
title: Automatic migration planning
id: auto-plan
---
import InstallationInstructions from '../components/_installation_instructions.mdx';
import AtlasMigrateDiff from '../components/_atlas_migrate_diff.mdx';
## Automatic migration planning
One of the convenient features of Automatic Migrations is that developers do not
need to write the SQL statements to create or modify the database schema. To
achieve similar benefits, we will now add a script to our project that will
automatically plan migration files for us based on the changes to our schema.
To do this, Ent uses [Atlas](https://atlasgo.io), an open-source tool for managing database
schemas, created by the same people behind Ent.
If you have been following our example repo, we have been using SQLite as our database
until this point. To demonstrate a more realistic use case, we will now switch to MySQL.
See this change in [PR #3](https://github.com/rotemtam/ent-versioned-migrations-demo/pull/3/files).
## Using the Atlas CLI to plan migrations
In this section, we will demonstrate how to use the Atlas CLI to automatically plan
schema migrations for us. In the past, users had to create a custom Go program to
do this (as described [here](07-programmatically.mdx)). With recent versions of Atlas,
this is no longer necessary: Atlas can natively load the desired databaes schema from an Ent schema.
<InstallationInstructions />
Then, run the following command to automatically generate migration files for your Ent schema:
<AtlasMigrateDiff/>
:::info The role of the [dev database](https://atlasgo.io/concepts/dev-database)
Atlas loads the **current state** by executing the SQL files stored in the migration directory onto the provided
[dev database](https://atlasgo.io/concepts/dev-database). It then compares this state against the **desired state**
defined by the `ent/schema` package and writes a migration plan for moving from the current state to the desired state.
:::
Next, let's see how to upgrade an existing production database to be managed with versioned migrations.

83
doc/md/versioned/02-enable-ff.mdx
View File

@@ -1,83 +0,0 @@
---
id: enable-ff
title: Enabling Versioned Migrations
---
import Tabs from '@theme/Tabs';
import TabItem from '@theme/TabItem';
## Supporting repository
The change described in this section can be found in PR [#2](https://github.com/rotemtam/ent-versioned-migrations-demo/pull/2/files)
in the supporting repository.
## Enable the versioned migration feature flag
The first step is to enable the versioned migration feature by passing in the `sql/versioned-migration` feature flag.
Depending on how you execute the Ent code generator, you have to use one of the two options:
<Tabs
defaultValue="ent"
values={[
{label: 'Using Ent CLI', value: 'ent'},
{label: 'Using the entc package', value: 'entc'},
]}>
<TabItem value="ent">
If you are using the default go generate configuration, simply add the `--feature sql/versioned-migration` to
the `ent/generate.go` file as follows:
```go
package ent
//go:generate go run -mod=mod entgo.io/ent/cmd/ent generate --feature sql/versioned-migration ./schema
```
</TabItem>
<TabItem value="entc">
If you are using the code generation package (e.g. if you are using an Ent extension like `entgql`),
add the feature flag as follows:
```go
//go:build ignore
package main
import (
"log"
"entgo.io/ent/entc"
"entgo.io/ent/entc/gen"
)
func main() {
err := entc.Generate("./schema", &gen.Config{
//highlight-next-line
Features: []gen.Feature{gen.FeatureVersionedMigration},
})
if err != nil {
log.Fatalf("running ent codegen: %v", err)
}
}
```
</TabItem>
</Tabs>
Next, re-run code-generation:
```shell
go generate ./...
```
After running the code-generation, you should see the following
[methods added](https://github.com/rotemtam/ent-versioned-migrations-demo/commit/e724fa32330d920fd405b9785fcfece2a46ea56c#diff-370235e5107bbdd35861063f3beff1507723ebdda6e29a39cdde1f1a944594d8)
to `ent/migrate/migrate.go`:
* `Diff`
* `NamedDiff`
These methods are used to compare the state read from a database connection or migration directory with the state defined
by the Ent schema.
In the next section, we will see how to automate the creation of versioned migrations.

71
doc/md/versioned/04-upgrade-prod.mdx → doc/md/versioned/03-upgrade-prod.mdx
View File

@@ -6,12 +6,14 @@ title: Upgrading Production
import Tabs from '@theme/Tabs';
import TabItem from '@theme/TabItem';
## Supporting repository
:::info Supporting repository
The change described in this section can be found in
[PR #5](https://github.com/rotemtam/ent-versioned-migrations-demo/pull/5/files)
in the supporting repository.
:::
## Upgrading our production database to use versioned migrations
If you have been following our tutorial to this point, you may be asking yourself how do we
@@ -19,72 +21,6 @@ upgrade the production instance of our database to be managed by the versioned m
With local development, we can just delete the database and start over, but that is not an option
for production for obvious reasons.
In this tutorial, we use the [Atlas CLI](https://atlasgo.io/docs/cli) to manage our production database.
If you haven't already, now is a good time to install it locally:
<Tabs
defaultValue="apple-intel-brew"
values={[
{label: 'macOS (Homebrew)', value: 'apple-intel-brew'},
{label: 'macOS (Plain)', value: 'apple-intel-plain'},
{label: 'Linux', value: 'linux'},
{label: 'Windows', value: 'windows'},
]}>
<TabItem value="apple-intel-brew">
Get the latest release with [Homebrew](https://brew.sh/):
```shell
brew install ariga/tap/atlas
```
</TabItem>
<TabItem value="apple-intel-plain">
Download latest release.
```shell
curl -LO https://release.ariga.io/atlas/atlas-darwin-amd64-latest
```
Make the Atlas binary executable.
```shell
chmod +x ./atlas-darwin-amd64-latest
```
Move the Atlas binary to a file location on your system PATH.
```shell
sudo mv ./atlas-darwin-amd64-latest /usr/local/bin/atlas
```
```shell
sudo chown root: /usr/local/bin/atlas
```
</TabItem>
<TabItem value="linux">
Download latest release.
```shell
curl -LO https://release.ariga.io/atlas/atlas-linux-amd64-latest
```
Move the Atlas binary to a file location on your system PATH.
```shell
sudo install -o root -g root -m 0755 ./atlas-linux-amd64-latest /usr/local/bin/atlas
```
</TabItem>
<TabItem value="windows">
Download the [latest release](https://release.ariga.io/atlas/atlas-windows-amd64-latest.exe) and
move the Atlas binary to a file location on your system PATH.
</TabItem>
</Tabs>
The binaries distributed in official releases are released under the [Apache 2 License](https://github.com/ariga/atlas/blob/master/LICENSE).
If you would like to build Atlas from source follow the instructions [here](https://atlasgo.io/cli-reference#building-from-source).
Like many other database schema management tools, [Atlas](https://atlasgo.io) uses a metadata table
on the target database to keep track of which migrations were already applied.
In the case where we start using Atlas on an existing database, we must somehow
@@ -135,7 +71,6 @@ Migration Status: OK
-- Next Version: Already at latest version
-- Executed Files: 1
-- Pending Files: 0
```
Great! We have successfully upgraded our project to use versioned migrations with Atlas.

65
doc/md/versioned/05-new-migration.mdx → doc/md/versioned/04-new-migration.mdx
View File

@@ -2,12 +2,19 @@
title: Planning a Migration
id: new-migration
---
## Supporting repository
import Tabs from '@theme/Tabs';
import TabItem from '@theme/TabItem';
:::info Supporting repository
The change described in this section can be found in
[PR #6](https://github.com/rotemtam/ent-versioned-migrations-demo/pull/6/files)
in the supporting repository.
:::
## Planning a migration
In this section, we will discuss how to plan a new schema migration when we
@@ -35,15 +42,61 @@ After adding the new field, we need to rerun code-gen for our project:
go generate ./...
```
Next, we need to create a new migration file for our change using the automatic
planning script we created earlier:
Next, we need to create a new migration file for our change using the Atlas CLI:
<Tabs
defaultValue="mysql"
values={[
{label: 'MySQL', value: 'mysql'},
{label: 'MariaDB', value: 'maria'},
{label: 'PostgreSQL', value: 'postgres'},
{label: 'SQLite', value: 'sqlite'},
]}>
<TabItem value="mysql">
```shell
go run ent/migrate/main.go add_user_title
atlas migrate diff add_user_title \
--dir "file://ent/migrate/migrations" \
--to "ent://ent/schema" \
--dev-url "docker://mysql/8/ent"
```
</TabItem>
<TabItem value="maria">
```shell
atlas migrate diff add_user_title \
--dir "file://ent/migrate/migrations" \
--to "ent://ent/schema" \
--dev-url "docker://mariadb/latest/test"
```
</TabItem>
<TabItem value="postgres">
```shell
atlas migrate diff add_user_title \
--dir "file://ent/migrate/migrations" \
--to "ent://ent/schema" \
--dev-url "docker://postgres/15/test?search_path=public"
```
</TabItem>
<TabItem value="sqlite">
```shell
atlas migrate diff add_user_title \
--dir "file://ent/migrate/migrations" \
--to "ent://ent/schema" \
--dev-url "sqlite://file?mode=memory&_fk=1"
```
</TabItem>
</Tabs>
Observe a new file named `20221115101649_add_user_title.sql` was created under
the `ent/migrate/migrations/ directory.` This file contains the SQL statements
the `ent/migrate/migrations/` directory. This file contains the SQL statements
to create the newly added `title` field in the `users` table:
```sql title=ent/migrate/migrations/20221115101649_add_user_title.sql
@@ -51,7 +104,7 @@ to create the newly added `title` field in the `users` table:
ALTER TABLE `users` ADD COLUMN `title` varchar(255) NULL;
```
Great! We've succesfully used our migration planning script to automatically
Great! We've successfully used the Atlas CLI to automatically
generate a new migration file for our change.
To apply the migration, we can run the following command:

4
doc/md/versioned/06-custom-migrations.md → doc/md/versioned/05-custom-migrations.md
View File

@@ -2,12 +2,14 @@
title: Custom migrations
id: custom-migrations
---
## Supporting repository
:::info Supporting repository
The change described in this section can be found in
[PR #7](https://github.com/rotemtam/ent-versioned-migrations-demo/pull/7/files)
in the supporting repository.
:::
## Custom migrations
In some cases, you may want to write custom migrations that are not automatically
generated by Atlas. This can be useful in cases where you want to perform changes

59
doc/md/versioned/07-verifying-safety.md → doc/md/versioned/06-verifying-safety.mdx
View File

@@ -2,12 +2,18 @@
title: Verifying migration safety
id: verifying-safety
---
## Supporting repository
import Tabs from '@theme/Tabs';
import TabItem from '@theme/TabItem';
:::info Supporting repository
The change described in this section can be found in
[PR #8](https://github.com/rotemtam/ent-versioned-migrations-demo/pull/8/files)
in the supporting repository.
:::
## Verifying migration safety
As the database is a critical component of our application, we want to make sure that when we
@@ -45,10 +51,57 @@ go generate ./...
Next, let's automatically generate a new migration:
<Tabs
defaultValue="mysql"
values={[
{label: 'MySQL', value: 'mysql'},
{label: 'MariaDB', value: 'maria'},
{label: 'PostgreSQL', value: 'postgres'},
{label: 'SQLite', value: 'sqlite'},
]}>
<TabItem value="mysql">
```shell
go run -mod=mod ent/migrate/main.go user_title_required```
atlas migrate diff user_title_required \
--dir "file://ent/migrate/migrations" \
--to "ent://ent/schema" \
--dev-url "docker://mysql/8/ent"
```
</TabItem>
<TabItem value="maria">
```shell
atlas migrate diff user_title_required \
--dir "file://ent/migrate/migrations" \
--to "ent://ent/schema" \
--dev-url "docker://mariadb/latest/test"
```
</TabItem>
<TabItem value="postgres">
```shell
atlas migrate diff user_title_required \
--dir "file://ent/migrate/migrations" \
--to "ent://ent/schema" \
--dev-url "docker://postgres/15/test?search_path=public"
```
</TabItem>
<TabItem value="sqlite">
```shell
atlas migrate diff user_title_required \
--dir "file://ent/migrate/migrations" \
--to "ent://ent/schema" \
--dev-url "sqlite://file?mode=memory&_fk=1"
```
</TabItem>
</Tabs>
A new migration file was created in the `ent/migrate/migrations` directory:
```sql title="ent/migrate/migrations/20221116051710_user_title_required.sql"
@@ -198,6 +251,8 @@ automatic migration to versioned migrations. To recap, we learned how to:
* Plan custom migrations for our project
* Verify migrations safely using `atlas migrate lint`
In the next steps
:::note For more Ent news and updates:
- Subscribe to our [Newsletter](https://entgo.substack.com/)

123
doc/md/versioned/03-auto-plan.md → doc/md/versioned/07-programmatically.mdx
View File

@@ -1,42 +1,114 @@
---
title: Automatic migration planning
id: auto-plan
id: programmatically
title: "Appendix: programmatic planning"
---
## Supporting repository
import Tabs from '@theme/Tabs';
import TabItem from '@theme/TabItem';
In the previous sections, we saw how to use the Atlas CLI to generate migration files. However, we can also
generate these files programmatically. In this section we will review how to write Go code that can be used for
automatically planning migration files.
## 1. Enable the versioned migration feature flag
:::info Supporting repository
The change described in this section can be found in PR [#2](https://github.com/rotemtam/ent-versioned-migrations-demo/pull/2/files)
in the supporting repository.
:::
The first step is to enable the versioned migration feature by passing in the `sql/versioned-migration` feature flag.
Depending on how you execute the Ent code generator, you have to use one of the two options:
<Tabs
defaultValue="ent"
values={[
{label: 'Using Ent CLI', value: 'ent'},
{label: 'Using the entc package', value: 'entc'},
]}>
<TabItem value="ent">
If you are using the default go generate configuration, simply add the `--feature sql/versioned-migration` to
the `ent/generate.go` file as follows:
```go
package ent
//go:generate go run -mod=mod entgo.io/ent/cmd/ent generate --feature sql/versioned-migration ./schema
```
</TabItem>
<TabItem value="entc">
If you are using the code generation package (e.g. if you are using an Ent extension like `entgql`),
add the feature flag as follows:
```go
//go:build ignore
package main
import (
"log"
"entgo.io/ent/entc"
"entgo.io/ent/entc/gen"
)
func main() {
err := entc.Generate("./schema", &gen.Config{
//highlight-next-line
Features: []gen.Feature{gen.FeatureVersionedMigration},
})
if err != nil {
log.Fatalf("running ent codegen: %v", err)
}
}
```
</TabItem>
</Tabs>
Next, re-run code-generation:
```shell
go generate ./...
```
After running the code-generation, you should see the following
[methods added](https://github.com/rotemtam/ent-versioned-migrations-demo/commit/e724fa32330d920fd405b9785fcfece2a46ea56c#diff-370235e5107bbdd35861063f3beff1507723ebdda6e29a39cdde1f1a944594d8)
to `ent/migrate/migrate.go`:
* `Diff`
* `NamedDiff`
These methods are used to compare the state read from a database connection or a migration directory with the state defined
by the Ent schema.
## 2. Automatic Migration planning script
:::info Supporting repository
The change described in this section can be found in PR [#4](https://github.com/rotemtam/ent-versioned-migrations-demo/pull/4/files)
in the supporting repository.
## Automatic migration planning
:::
One of the convenient features of Automatic Migrations is that developers do not
need to write the SQL statements to create or modify the database schema. To
achieve similar benefits, we will now add a script to our project that will
automatically plan migration files for us based on the changes to our schema.
To do this, Ent uses [Atlas](https://atlasgo.io), an open-source tool for managing database
schemas, created by the same people behind Ent.
If you have been following our example repo, we have been using SQLite as our database
until this point. To demonstrate a more realistic use case, we will now switch to MySQL.
See this change in [PR #3](https://github.com/rotemtam/ent-versioned-migrations-demo/pull/3/files).
## Dev database
### Dev database
To be able to plan accurate and consistent migration files, Atlas introduces the
concept of a [Dev database](https://atlasgo.io/concepts/dev-database), a temporary
database which is used to simulate the state of the database after different changes.
Therefore, to use Atlas to automatically plan migrations, we need to supply a connection
string to such a database to our migration planning script. Such a database is most commonly
string to such a database to our migration planning script. Such a database is most commonly
spun up using a local Docker container. Let's do this now by running the following command:
```shell
docker run --rm --name atlas-db-dev -d -p 3306:3306 -e MYSQL_DATABASE=dev -e MYSQL_ROOT_PASSWORD=pass mysql:8
```
## Migration planning script
Now that we have a Dev database, we can write a script that will use Atlas to plan
Using the Dev database we have just configured, we can write a script that will use Atlas to plan
migration files for us. Let's create a new file called `main.go` in the `ent/migrate` directory
of our project:
@@ -49,7 +121,7 @@ import (
"context"
"log"
"os"
// highlight-next-line
"<project>/ent/migrate"
@@ -95,7 +167,7 @@ func main() {
:::info
Notice that you need to make some modifications to the script above in the highlighted lines.
Edit the import path of the `migrate` package to match your project and to supply the connection
Edit the import path of the `migrate` package to match your project and to supply the connection
string to your Dev database.
:::
@@ -138,12 +210,11 @@ CREATE TABLE `blogs` (`id` bigint NOT NULL AUTO_INCREMENT, `title` varchar(255)
Atlas integrates very well with Ent, but it is not the only migration tool that can be used
to manage database schemas in Ent projects. The following is a list of other migration tools
that are supported:
* [Goose](https://github.com/pressly/goose)
* [Golang Migrate](https://github.com/golang-migrate/migrate)
* [Flyway](https://flywaydb.org)
* [Liquibase](https://www.liquibase.org)
* [dbmate](https://github.com/amacneil/dbmate)
To learn more about how to use these tools with Ent, see the [docs](https://entgo.io/docs/versioned-migrations#create-a-migration-files-generator) on this subject.
Next, let's see how to upgrade an existing production database to be managed with versioned migrations.
To learn more about how to use these tools with Ent, see the [docs](https://entgo.io/docs/versioned-migrations#create-a-migration-files-generator) on this subject.