golang/ent
1
0
Fork 0
mirror of https://github.com/ent/ent.git synced 2026-03-05 19:35:23 +03:00
Code Issues Packages Projects Releases Wiki Activity

blog/upsert: minor changes to upsert blogpost

Browse Source
This commit is contained in:
Ariel Mashraki
2021-08-05 21:38:38 +03:00
committed by Ariel Mashraki
parent 1a4403c1b3
commit e0a85ab609
1 changed files with 77 additions and 52 deletions
Download Patch File Download Diff File Expand all files Collapse all files

129
doc/website/blog/2021-08-05-announcing-upsert-api.md
View File

@@ -7,65 +7,79 @@ authorTwitter: _rtam
---
It has been almost 4 months since our [last release](https://github.com/ent/ent/releases/tag/v0.8.0), and for a good reason. Version 0.9.0 (**ADD LINK**) which was released today is packed with some highly-anticipated features. Perhaps at the top of the list, is a feature that has been in discussion for [more than a year in a half](https://github.com/ent/ent/issues/139) and was one of the most commonly requested features in the [Ent User Survey](https://forms.gle/7VZSPVc7D1iu75GV9): the Upsert API!
It has been almost 4 months since our [last release](https://github.com/ent/ent/releases/tag/v0.8.0), and for a good reason.
Version [0.9.0](https://github.com/ent/ent/releases/tag/v0.9.0) which was released today is packed with some highly-anticipated
features. Perhaps at the top of the list, is a feature that has been in discussion for [more than a year in a half](https://github.com/ent/ent/issues/139)
and was one of the most commonly requested features in the [Ent User Survey](https://forms.gle/7VZSPVc7D1iu75GV9): the Upsert API!
Version 0.9.0 adds support for "Upsert" style statements using [a new feature flag](https://entgo.io/docs/feature-flags#upsert): `sql/upsert`. Ent has a [collection of feature flags](https://entgo.io/docs/feature-flags) that can be switched on to add more features to the code generated by Ent. This is used as both a mechanism to allow opt-in to some features that are not necessarily desired in every project and as a way to run experiments of features that may one day become part of Ent's core.
Version 0.9.0 adds support for "Upsert" style statements using [a new feature flag](https://entgo.io/docs/feature-flags#upsert): `sql/upsert`.
Ent has a [collection of feature flags](https://entgo.io/docs/feature-flags) that can be switched on to add more features
to the code generated by Ent. This is used as both a mechanism to allow opt-in to some features that are not necessarily
desired in every project and as a way to run experiments of features that may one day become part of Ent's core.
In this post, we will introduce the new feature, the places where it is useful, and demonstrate how to use it.
### Upserts
### Upsert
"Upsert" is a commonly-used term in data systems that is a [portmanteau](https://entgo.io/docs/feature-flags) of "update" and "insert" which usually refers to a statement that attempts to insert a record to a table, and if a uniqueness constraint is violated (usually: a record by that ID already exists) that record is updated instead. While none of the popular relational databases have a specific `UPSERT` statement, most of them support ways of achieving this type of behavior. For example, assume we have a table with this definition in a PostgreSQL database:
"Upsert" is a commonly-used term in data systems that is a portmanteau of "update" and "insert" which usually refers to
a statement that attempts to insert a record to a table, and if a uniqueness constraint is violated (e.g. a record by
that ID already exists) that record is updated instead. While none of the popular relational databases have a specific
`UPSERT` statement, most of them support ways of achieving this type of behavior.
For example, assume we have a table with this definition in an SQLite database:
```sql
CREATE TABLE users (
id SERIAL,
id integer PRIMARY KEY AUTOINCREMENT,
email varchar(255) UNIQUE,
full_name varchar(255),
PRIMARY KEY (id)
name varchar(255)
)
```
If we try to execute the same insert twice:
```sql
INSERT INTO users (email, full_name) VALUES ('rotem@entgo.io', 'Rotem Tamir');
INSERT INTO users (email, full_name) VALUES ('rotem@entgo.io', 'Rotem Tamir');
INSERT INTO users (email, name) VALUES ('rotem@entgo.io', 'Rotem Tamir');
INSERT INTO users (email, name) VALUES ('rotem@entgo.io', 'Rotem Tamir');
```
We get this error:
```
[2021-08-05 06:49:22] [23505] ERROR: duplicate key value violates unique constraint "users_email_key"
[2021-08-05 06:49:22] Detail: Key (email)=(rotem@entgo.io) already exists.
[2021-08-05 06:49:22] UNIQUE constraint failed: users.email
```
In many cases, it is useful to have write operations be [idempotent](https://en.wikipedia.org/wiki/Idempotence), meaning we can run them many times in a row while leaving the system in the same state. In other cases, it is not desirable to query if a record exists before trying to create it. For these kinds of situations, PostgreSQL supports the `ON CONFLICT` [Clause](https://www.postgresql.org/docs/9.5/sql-insert.html#SQL-ON-CONFLICT) in `INSERT` statements. To instruct Postgres to override an existing value with the new one we can execute:
In many cases, it is useful to have write operations be [idempotent](https://en.wikipedia.org/wiki/Idempotence),
meaning we can run them many times in a row while leaving the system in the same state.
In other cases, it is not desirable to query if a record exists before trying to create it. For these kinds of situations,
SQLite supports the [`ON CONFLICT` clause](https://www.sqlite.org/lang_upsert.html) in `INSERT`
statements. To instruct SQLite to override an existing value with the new one we can execute:
```sql
INSERT INTO users (email, full_name) values ('rotem@entgo.io', 'Tamir, Rotem')
ON CONFLICT (email) DO UPDATE
SET email=excluded.email, full_name=excluded.full_name;
INSERT INTO users (email, name) values ('rotem@entgo.io', 'Tamir, Rotem')
ON CONFLICT (email) DO UPDATE SET email=excluded.email, name=excluded.name;
```
If we prefer to keep the existing values, we can use the `DO NOTHING` conflict action:
```sql
INSERT INTO users (email, full_name) values ('rotem@entgo.io', 'Tamir, Rotem')
INSERT INTO users (email, name) values ('rotem@entgo.io', 'Tamir, Rotem')
ON CONFLICT DO NOTHING;
```
Sometimes we want to merge the two versions in some way, we can use the `DO UPDATE` action a little differently to achieve do something like:
Sometimes we want to merge the two versions in some way, we can use the `DO UPDATE` action a little differently to
achieve do something like:
```sql
INSERT INTO users (email, full_name) values ('rotem@entgo.io', 'Tamir, Rotem')
ON CONFLICT (email) DO UPDATE
SET full_name=excluded.full_name || ' (formerly: ' || users.full_name || ')'
ON CONFLICT (email) DO UPDATE SET name=excluded.name || ' (formerly: ' || users.name || ')'
```
In this case, after our second `INSERT` the value for the `full_name` column would be: `Tamir, Rotem (formerly: Rotem Tamir)`. Not very useful, but hopefully you can see that you can do cool things this way.
In this case, after our second `INSERT` the value for the `name` column would be: `Tamir, Rotem (formerly: Rotem Tamir)`.
Not very useful, but hopefully you can see that you can do cool things this way.
### Upserts with Ent
### Upsert with Ent
Assume we have an existing Ent project with an entity similar to the `users` table described above:
@@ -78,8 +92,10 @@ type User struct {
// Fields of the User.
func (User) Fields() []ent.Field {
return []ent.Field{
field.String("email_address").Unique(),
field.String("full_name"),
field.String("email").
Unique(),
field.String("name"),
}
}
```
@@ -119,7 +135,7 @@ Observe that a new method named `OnConflict` was added to the `ent/user_create.g
// // Override some of the fields with custom
// // update values.
// Update(func(u *ent.UserUpsert) {
// SetEmailAddress(v+v).
// SetEmailAddress(v+v)
// }).
// Exec(ctx)
//
@@ -131,7 +147,8 @@ func (uc *UserCreate) OnConflict(opts ...sql.ConflictOption) *UserUpsertOne {
}
```
This (along with more new generated code) will serve us in achieving upsert behavior for our `User` entity. To explore this, let's first start by writing a test to reproduce the uniqueness constraint error:
This (along with more new generated code) will serve us in achieving upsert behavior for our `User` entity.
To explore this, let's first start by writing a test to reproduce the uniqueness constraint error:
```go
func TestUniqueConstraintFails(t *testing.T) {
@@ -139,15 +156,17 @@ func TestUniqueConstraintFails(t *testing.T) {
ctx := context.TODO()
// Create the user for the first time.
client.User.Create().
SetEmailAddress("rotem@entgo.io").
SetFullName("Rotem Tamir").
client.User.
Create().
SetEmail("rotem@entgo.io").
SetName("Rotem Tamir").
SaveX(ctx)
// Try to create a user with the same email the second time.
_, err := client.User.Create().
SetEmailAddress("rotem@entgo.io").
SetFullName("Rotem Tamir").
_, err := client.User.
Create().
SetEmail("rotem@entgo.io").
SetName("Rotem Tamir").
Save(ctx)
if !ent.IsConstraintError(err) {
@@ -161,7 +180,7 @@ The test passes:
```bash
=== RUN TestUniqueConstraintFails
2021/08/05 07:12:11 second query failed with: ent: constraint failed: insert node to table "users": UNIQUE constraint failed: users.email_address
2021/08/05 07:12:11 second query failed with: ent: constraint failed: insert node to table "users": UNIQUE constraint failed: users.email
--- PASS: TestUniqueConstraintFails (0.00s)
```
@@ -173,31 +192,32 @@ func TestUpsertReplace(t *testing.T) {
ctx := context.TODO()
// Create the user for the first time.
orig := client.User.Create().
SetEmailAddress("rotem@entgo.io").
SetFullName("Rotem Tamir").
orig := client.User.
Create().
SetEmail("rotem@entgo.io").
SetName("Rotem Tamir").
SaveX(ctx)
// Try to create a user with the same email the second time.
// This time we set ON CONFLICT behavior, and use the `UpdateNewValues`
// modifier.
newId := client.User.Create().
SetEmailAddress("rotem@entgo.io").
SetFullName("Tamir, Rotem").
OnConflictColumns(user.FieldEmailAddress).
newID := client.User.Create().
SetEmail("rotem@entgo.io").
SetName("Tamir, Rotem").
OnConflict().
UpdateNewValues().
// we use the IDX method to receive the ID of the created/updated
// entity
// we use the IDX method to receive the ID
// of the created/updated entity
IDX(ctx)
// We expect the ID of the originally created user to be the same as
// the one that was just updated.
if orig.ID != newId {
if orig.ID != newID {
log.Fatalf("expected upsert to update an existing record")
}
current := client.User.GetX(ctx, orig.ID)
if current.FullName != "Tamir, Rotem" {
if current.Name != "Tamir, Rotem" {
log.Fatalf("expected upsert to replace with the new values")
}
}
@@ -210,7 +230,8 @@ Running our test:
--- PASS: TestUpsertReplace (0.00s)
```
Alternatively, we can use the `Ignore` modifier to instruct Ent to keep the old version when resolving the conflict. Let's write a test that shows this:
Alternatively, we can use the `Ignore` modifier to instruct Ent to keep the old version when resolving the conflict.
Let's write a test that shows this:
```go
func TestUpsertIgnore(t *testing.T) {
@@ -218,18 +239,20 @@ func TestUpsertIgnore(t *testing.T) {
ctx := context.TODO()
// Create the user for the first time.
orig := client.User.Create().
SetEmailAddress("rotem@entgo.io").
SetFullName("Rotem Tamir").
orig := client.User.
Create().
SetEmail("rotem@entgo.io").
SetName("Rotem Tamir").
SaveX(ctx)
// Try to create a user with the same email the second time.
// This time we set ON CONFLICT behavior, and use the `Ignore`
// modifier.
client.User.Create().
SetEmailAddress("rotem@entgo.io").
SetFullName("Tamir, Rotem").
OnConflictColumns(user.FieldEmailAddress).
client.User.
Create().
SetEmail("rotem@entgo.io").
SetName("Tamir, Rotem").
OnConflict().
Ignore().
ExecX(ctx)
@@ -244,7 +267,9 @@ You can read more about the feature in the [Feature Flag](https://entgo.io/docs/
### Wrapping Up
In this post, we presented the Upsert API, a long-anticipated capability, that is available by feature-flag in Ent v0.9.0. We discussed where upserts are commonly used in applications and the way they are implemented using common relational databases. Finally, we showed a simple example of how to get started with the Upsert API using Ent.
In this post, we presented the Upsert API, a long-anticipated capability, that is available by feature-flag in Ent v0.9.0.
We discussed where upserts are commonly used in applications and the way they are implemented using common relational databases.
Finally, we showed a simple example of how to get started with the Upsert API using Ent.
Have questions? Need help with getting started? Feel free to [join our Slack channel](https://entgo.io/docs/slack/).