diff --git a/doc/md/versioned/02-auto-plan.mdx b/doc/md/versioned/02-auto-plan.mdx new file mode 100644 index 000000000..7d3bd4dd4 --- /dev/null +++ b/doc/md/versioned/02-auto-plan.mdx @@ -0,0 +1,43 @@ +--- +title: Automatic migration planning +id: auto-plan +--- + +import InstallationInstructions from '../components/_installation_instructions.mdx'; +import AtlasMigrateDiff from '../components/_atlas_migrate_diff.mdx'; + +## Automatic migration planning + +One of the convenient features of Automatic Migrations is that developers do not +need to write the SQL statements to create or modify the database schema. To +achieve similar benefits, we will now add a script to our project that will +automatically plan migration files for us based on the changes to our schema. + +To do this, Ent uses [Atlas](https://atlasgo.io), an open-source tool for managing database +schemas, created by the same people behind Ent. + +If you have been following our example repo, we have been using SQLite as our database +until this point. To demonstrate a more realistic use case, we will now switch to MySQL. +See this change in [PR #3](https://github.com/rotemtam/ent-versioned-migrations-demo/pull/3/files). + +## Using the Atlas CLI to plan migrations + +In this section, we will demonstrate how to use the Atlas CLI to automatically plan +schema migrations for us. In the past, users had to create a custom Go program to +do this (as described [here](07-programmatically.mdx)). With recent versions of Atlas, +this is no longer necessary: Atlas can natively load the desired databaes schema from an Ent schema. + + + +Then, run the following command to automatically generate migration files for your Ent schema: + + + +:::info The role of the [dev database](https://atlasgo.io/concepts/dev-database) +Atlas loads the **current state** by executing the SQL files stored in the migration directory onto the provided +[dev database](https://atlasgo.io/concepts/dev-database). It then compares this state against the **desired state** +defined by the `ent/schema` package and writes a migration plan for moving from the current state to the desired state. +::: + + +Next, let's see how to upgrade an existing production database to be managed with versioned migrations. diff --git a/doc/md/versioned/02-enable-ff.mdx b/doc/md/versioned/02-enable-ff.mdx deleted file mode 100644 index 0f5f5c929..000000000 --- a/doc/md/versioned/02-enable-ff.mdx +++ /dev/null @@ -1,83 +0,0 @@ ---- -id: enable-ff -title: Enabling Versioned Migrations ---- - -import Tabs from '@theme/Tabs'; -import TabItem from '@theme/TabItem'; - -## Supporting repository - -The change described in this section can be found in PR [#2](https://github.com/rotemtam/ent-versioned-migrations-demo/pull/2/files) -in the supporting repository. - -## Enable the versioned migration feature flag - -The first step is to enable the versioned migration feature by passing in the `sql/versioned-migration` feature flag. -Depending on how you execute the Ent code generator, you have to use one of the two options: - - - - -If you are using the default go generate configuration, simply add the `--feature sql/versioned-migration` to -the `ent/generate.go` file as follows: - -```go -package ent - -//go:generate go run -mod=mod entgo.io/ent/cmd/ent generate --feature sql/versioned-migration ./schema -``` - - - - -If you are using the code generation package (e.g. if you are using an Ent extension like `entgql`), -add the feature flag as follows: - -```go -//go:build ignore - -package main - -import ( - "log" - - "entgo.io/ent/entc" - "entgo.io/ent/entc/gen" -) - -func main() { - err := entc.Generate("./schema", &gen.Config{ - //highlight-next-line - Features: []gen.Feature{gen.FeatureVersionedMigration}, - }) - if err != nil { - log.Fatalf("running ent codegen: %v", err) - } -} -``` - - - - -Next, re-run code-generation: - -```shell -go generate ./... -``` - -After running the code-generation, you should see the following -[methods added](https://github.com/rotemtam/ent-versioned-migrations-demo/commit/e724fa32330d920fd405b9785fcfece2a46ea56c#diff-370235e5107bbdd35861063f3beff1507723ebdda6e29a39cdde1f1a944594d8) -to `ent/migrate/migrate.go`: -* `Diff` -* `NamedDiff` - -These methods are used to compare the state read from a database connection or migration directory with the state defined -by the Ent schema. - -In the next section, we will see how to automate the creation of versioned migrations. \ No newline at end of file diff --git a/doc/md/versioned/04-upgrade-prod.mdx b/doc/md/versioned/03-upgrade-prod.mdx similarity index 62% rename from doc/md/versioned/04-upgrade-prod.mdx rename to doc/md/versioned/03-upgrade-prod.mdx index 24c1aac9b..ed2c26d67 100644 --- a/doc/md/versioned/04-upgrade-prod.mdx +++ b/doc/md/versioned/03-upgrade-prod.mdx @@ -6,12 +6,14 @@ title: Upgrading Production import Tabs from '@theme/Tabs'; import TabItem from '@theme/TabItem'; -## Supporting repository +:::info Supporting repository The change described in this section can be found in [PR #5](https://github.com/rotemtam/ent-versioned-migrations-demo/pull/5/files) in the supporting repository. +::: + ## Upgrading our production database to use versioned migrations If you have been following our tutorial to this point, you may be asking yourself how do we @@ -19,72 +21,6 @@ upgrade the production instance of our database to be managed by the versioned m With local development, we can just delete the database and start over, but that is not an option for production for obvious reasons. -In this tutorial, we use the [Atlas CLI](https://atlasgo.io/docs/cli) to manage our production database. -If you haven't already, now is a good time to install it locally: - - - - -Get the latest release with [Homebrew](https://brew.sh/): - -```shell -brew install ariga/tap/atlas -``` - - - - -Download latest release. -```shell -curl -LO https://release.ariga.io/atlas/atlas-darwin-amd64-latest -``` - -Make the Atlas binary executable. -```shell -chmod +x ./atlas-darwin-amd64-latest -``` - -Move the Atlas binary to a file location on your system PATH. -```shell -sudo mv ./atlas-darwin-amd64-latest /usr/local/bin/atlas -``` -```shell -sudo chown root: /usr/local/bin/atlas -``` - - - - -Download latest release. -```shell -curl -LO https://release.ariga.io/atlas/atlas-linux-amd64-latest -``` - -Move the Atlas binary to a file location on your system PATH. -```shell -sudo install -o root -g root -m 0755 ./atlas-linux-amd64-latest /usr/local/bin/atlas -``` - - - - -Download the [latest release](https://release.ariga.io/atlas/atlas-windows-amd64-latest.exe) and -move the Atlas binary to a file location on your system PATH. - - - - - -The binaries distributed in official releases are released under the [Apache 2 License](https://github.com/ariga/atlas/blob/master/LICENSE). -If you would like to build Atlas from source follow the instructions [here](https://atlasgo.io/cli-reference#building-from-source). - Like many other database schema management tools, [Atlas](https://atlasgo.io) uses a metadata table on the target database to keep track of which migrations were already applied. In the case where we start using Atlas on an existing database, we must somehow @@ -135,7 +71,6 @@ Migration Status: OK -- Next Version: Already at latest version -- Executed Files: 1 -- Pending Files: 0 - ``` Great! We have successfully upgraded our project to use versioned migrations with Atlas. diff --git a/doc/md/versioned/05-new-migration.mdx b/doc/md/versioned/04-new-migration.mdx similarity index 60% rename from doc/md/versioned/05-new-migration.mdx rename to doc/md/versioned/04-new-migration.mdx index 0ff979d5b..32f54fc49 100644 --- a/doc/md/versioned/05-new-migration.mdx +++ b/doc/md/versioned/04-new-migration.mdx @@ -2,12 +2,19 @@ title: Planning a Migration id: new-migration --- -## Supporting repository + +import Tabs from '@theme/Tabs'; +import TabItem from '@theme/TabItem'; + +:::info Supporting repository The change described in this section can be found in [PR #6](https://github.com/rotemtam/ent-versioned-migrations-demo/pull/6/files) in the supporting repository. +::: + + ## Planning a migration In this section, we will discuss how to plan a new schema migration when we @@ -35,15 +42,61 @@ After adding the new field, we need to rerun code-gen for our project: go generate ./... ``` -Next, we need to create a new migration file for our change using the automatic -planning script we created earlier: +Next, we need to create a new migration file for our change using the Atlas CLI: + + + + ```shell -go run ent/migrate/main.go add_user_title +atlas migrate diff add_user_title \ + --dir "file://ent/migrate/migrations" \ + --to "ent://ent/schema" \ + --dev-url "docker://mysql/8/ent" ``` + + + +```shell +atlas migrate diff add_user_title \ + --dir "file://ent/migrate/migrations" \ + --to "ent://ent/schema" \ + --dev-url "docker://mariadb/latest/test" +``` + + + + +```shell +atlas migrate diff add_user_title \ + --dir "file://ent/migrate/migrations" \ + --to "ent://ent/schema" \ + --dev-url "docker://postgres/15/test?search_path=public" +``` + + + + +```shell +atlas migrate diff add_user_title \ + --dir "file://ent/migrate/migrations" \ + --to "ent://ent/schema" \ + --dev-url "sqlite://file?mode=memory&_fk=1" +``` + + + + Observe a new file named `20221115101649_add_user_title.sql` was created under -the `ent/migrate/migrations/ directory.` This file contains the SQL statements +the `ent/migrate/migrations/` directory. This file contains the SQL statements to create the newly added `title` field in the `users` table: ```sql title=ent/migrate/migrations/20221115101649_add_user_title.sql @@ -51,7 +104,7 @@ to create the newly added `title` field in the `users` table: ALTER TABLE `users` ADD COLUMN `title` varchar(255) NULL; ``` -Great! We've succesfully used our migration planning script to automatically +Great! We've successfully used the Atlas CLI to automatically generate a new migration file for our change. To apply the migration, we can run the following command: diff --git a/doc/md/versioned/06-custom-migrations.md b/doc/md/versioned/05-custom-migrations.md similarity index 98% rename from doc/md/versioned/06-custom-migrations.md rename to doc/md/versioned/05-custom-migrations.md index 48b1f16c5..4910cd332 100644 --- a/doc/md/versioned/06-custom-migrations.md +++ b/doc/md/versioned/05-custom-migrations.md @@ -2,12 +2,14 @@ title: Custom migrations id: custom-migrations --- -## Supporting repository +:::info Supporting repository The change described in this section can be found in [PR #7](https://github.com/rotemtam/ent-versioned-migrations-demo/pull/7/files) in the supporting repository. +::: + ## Custom migrations In some cases, you may want to write custom migrations that are not automatically generated by Atlas. This can be useful in cases where you want to perform changes diff --git a/doc/md/versioned/07-verifying-safety.md b/doc/md/versioned/06-verifying-safety.mdx similarity index 86% rename from doc/md/versioned/07-verifying-safety.md rename to doc/md/versioned/06-verifying-safety.mdx index 2e407bda9..e831a680d 100644 --- a/doc/md/versioned/07-verifying-safety.md +++ b/doc/md/versioned/06-verifying-safety.mdx @@ -2,12 +2,18 @@ title: Verifying migration safety id: verifying-safety --- -## Supporting repository + +import Tabs from '@theme/Tabs'; +import TabItem from '@theme/TabItem'; + +:::info Supporting repository The change described in this section can be found in [PR #8](https://github.com/rotemtam/ent-versioned-migrations-demo/pull/8/files) in the supporting repository. +::: + ## Verifying migration safety As the database is a critical component of our application, we want to make sure that when we @@ -45,10 +51,57 @@ go generate ./... Next, let's automatically generate a new migration: + + + + ```shell -go run -mod=mod ent/migrate/main.go user_title_required``` +atlas migrate diff user_title_required \ + --dir "file://ent/migrate/migrations" \ + --to "ent://ent/schema" \ + --dev-url "docker://mysql/8/ent" ``` + + + +```shell +atlas migrate diff user_title_required \ + --dir "file://ent/migrate/migrations" \ + --to "ent://ent/schema" \ + --dev-url "docker://mariadb/latest/test" +``` + + + + +```shell +atlas migrate diff user_title_required \ + --dir "file://ent/migrate/migrations" \ + --to "ent://ent/schema" \ + --dev-url "docker://postgres/15/test?search_path=public" +``` + + + + +```shell +atlas migrate diff user_title_required \ + --dir "file://ent/migrate/migrations" \ + --to "ent://ent/schema" \ + --dev-url "sqlite://file?mode=memory&_fk=1" +``` + + + + A new migration file was created in the `ent/migrate/migrations` directory: ```sql title="ent/migrate/migrations/20221116051710_user_title_required.sql" @@ -198,6 +251,8 @@ automatic migration to versioned migrations. To recap, we learned how to: * Plan custom migrations for our project * Verify migrations safely using `atlas migrate lint` +In the next steps + :::note For more Ent news and updates: - Subscribe to our [Newsletter](https://entgo.substack.com/) diff --git a/doc/md/versioned/03-auto-plan.md b/doc/md/versioned/07-programmatically.mdx similarity index 64% rename from doc/md/versioned/03-auto-plan.md rename to doc/md/versioned/07-programmatically.mdx index 2d61f21c1..e8104abc2 100644 --- a/doc/md/versioned/03-auto-plan.md +++ b/doc/md/versioned/07-programmatically.mdx @@ -1,42 +1,114 @@ --- -title: Automatic migration planning -id: auto-plan +id: programmatically +title: "Appendix: programmatic planning" --- -## Supporting repository + +import Tabs from '@theme/Tabs'; +import TabItem from '@theme/TabItem'; + +In the previous sections, we saw how to use the Atlas CLI to generate migration files. However, we can also +generate these files programmatically. In this section we will review how to write Go code that can be used for +automatically planning migration files. + +## 1. Enable the versioned migration feature flag + +:::info Supporting repository + +The change described in this section can be found in PR [#2](https://github.com/rotemtam/ent-versioned-migrations-demo/pull/2/files) +in the supporting repository. + +::: + +The first step is to enable the versioned migration feature by passing in the `sql/versioned-migration` feature flag. +Depending on how you execute the Ent code generator, you have to use one of the two options: + + + + +If you are using the default go generate configuration, simply add the `--feature sql/versioned-migration` to +the `ent/generate.go` file as follows: + +```go +package ent + +//go:generate go run -mod=mod entgo.io/ent/cmd/ent generate --feature sql/versioned-migration ./schema +``` + + + + +If you are using the code generation package (e.g. if you are using an Ent extension like `entgql`), +add the feature flag as follows: + +```go +//go:build ignore + +package main + +import ( + "log" + + "entgo.io/ent/entc" + "entgo.io/ent/entc/gen" +) + +func main() { + err := entc.Generate("./schema", &gen.Config{ + //highlight-next-line + Features: []gen.Feature{gen.FeatureVersionedMigration}, + }) + if err != nil { + log.Fatalf("running ent codegen: %v", err) + } +} +``` + + + + +Next, re-run code-generation: + +```shell +go generate ./... +``` + +After running the code-generation, you should see the following +[methods added](https://github.com/rotemtam/ent-versioned-migrations-demo/commit/e724fa32330d920fd405b9785fcfece2a46ea56c#diff-370235e5107bbdd35861063f3beff1507723ebdda6e29a39cdde1f1a944594d8) +to `ent/migrate/migrate.go`: +* `Diff` +* `NamedDiff` + +These methods are used to compare the state read from a database connection or a migration directory with the state defined +by the Ent schema. + +## 2. Automatic Migration planning script + +:::info Supporting repository The change described in this section can be found in PR [#4](https://github.com/rotemtam/ent-versioned-migrations-demo/pull/4/files) in the supporting repository. -## Automatic migration planning +::: -One of the convenient features of Automatic Migrations is that developers do not -need to write the SQL statements to create or modify the database schema. To -achieve similar benefits, we will now add a script to our project that will -automatically plan migration files for us based on the changes to our schema. - -To do this, Ent uses [Atlas](https://atlasgo.io), an open-source tool for managing database -schemas, created by the same people behind Ent. - -If you have been following our example repo, we have been using SQLite as our database -until this point. To demonstrate a more realistic use case, we will now switch to MySQL. -See this change in [PR #3](https://github.com/rotemtam/ent-versioned-migrations-demo/pull/3/files). - -## Dev database +### Dev database To be able to plan accurate and consistent migration files, Atlas introduces the concept of a [Dev database](https://atlasgo.io/concepts/dev-database), a temporary database which is used to simulate the state of the database after different changes. Therefore, to use Atlas to automatically plan migrations, we need to supply a connection -string to such a database to our migration planning script. Such a database is most commonly +string to such a database to our migration planning script. Such a database is most commonly spun up using a local Docker container. Let's do this now by running the following command: ```shell docker run --rm --name atlas-db-dev -d -p 3306:3306 -e MYSQL_DATABASE=dev -e MYSQL_ROOT_PASSWORD=pass mysql:8 ``` -## Migration planning script - -Now that we have a Dev database, we can write a script that will use Atlas to plan +Using the Dev database we have just configured, we can write a script that will use Atlas to plan migration files for us. Let's create a new file called `main.go` in the `ent/migrate` directory of our project: @@ -49,7 +121,7 @@ import ( "context" "log" "os" - + // highlight-next-line "/ent/migrate" @@ -95,7 +167,7 @@ func main() { :::info Notice that you need to make some modifications to the script above in the highlighted lines. -Edit the import path of the `migrate` package to match your project and to supply the connection +Edit the import path of the `migrate` package to match your project and to supply the connection string to your Dev database. ::: @@ -138,12 +210,11 @@ CREATE TABLE `blogs` (`id` bigint NOT NULL AUTO_INCREMENT, `title` varchar(255) Atlas integrates very well with Ent, but it is not the only migration tool that can be used to manage database schemas in Ent projects. The following is a list of other migration tools that are supported: + * [Goose](https://github.com/pressly/goose) * [Golang Migrate](https://github.com/golang-migrate/migrate) * [Flyway](https://flywaydb.org) * [Liquibase](https://www.liquibase.org) * [dbmate](https://github.com/amacneil/dbmate) -To learn more about how to use these tools with Ent, see the [docs](https://entgo.io/docs/versioned-migrations#create-a-migration-files-generator) on this subject. - -Next, let's see how to upgrade an existing production database to be managed with versioned migrations. +To learn more about how to use these tools with Ent, see the [docs](https://entgo.io/docs/versioned-migrations#create-a-migration-files-generator) on this subject. \ No newline at end of file diff --git a/doc/website/sidebars.js b/doc/website/sidebars.js index 2366aa299..4c2c5b61d 100644 --- a/doc/website/sidebars.js +++ b/doc/website/sidebars.js @@ -121,10 +121,6 @@ module.exports = { type: 'doc', id: 'versioned/intro', }, - { - type: 'doc', - id: 'versioned/enable-ff', - }, { type: 'doc', id: 'versioned/auto-plan', @@ -144,7 +140,11 @@ module.exports = { { type: 'doc', id: 'versioned/verifying-safety', - } + }, + { + type: 'doc', + id: 'versioned/programmatically', + }, ] } ]