User Merge

Merge the users that share an email address into one.

Django’s auth.User.email isn’t unique, so two users could be created with the same address, e.g. by the api, a manifest or a sign-up typed in a different case. Sign-in looks a user up by email address, so such a person couldn’t sign in with a password (issue #914). Migration account.0011_merge_duplicate_users merges them on every platform, and account.0012_unique_user_email then makes the address unique. manage.py merge_duplicate_users --dry-run reports what a merge would do, without changing anything.

Addresses are compared as the database compares them: without regard to case or trailing spaces. Blank addresses are not duplicates of each other, because a user who signed in with GitHub may have none.

Each group of users with one address keeps one user, the survivor: the one who signed in most recently, or, when none of them ever signed in, the oldest. Everything that the other users own is moved to the survivor, and then they are deleted:

• Their UserProfiles. When the survivor already has a profile in the same account, the resources of the duplicate’s profile (LLMClients, Secrets, Plugins, api keys, …) are moved to the survivor’s profile. A resource whose name the survivor’s profile already uses is renamed <name>_merged_<id>.

• Every other row that refers to the user: social sign-in links, knox tokens, the journal, onboarding hints, group and permission links, waffle flags, the admin log, … A row that the survivor already has an equivalent of, e.g. the same group, is deleted instead.

• The user’s sign-in history and privileges: the survivor keeps the latest last_login, the earliest date_joined, and is staff, superuser or active if any of them was. A survivor whose password is unusable (a social sign-in) takes a duplicate’s usable password.

The functions work on the models of a historical app registry, as a migration’s RunPython receives, never on the real models: those send signals, which deploy, undeploy or delete infrastructure. historical_apps() returns the registry for code outside a migration.

exception smarter.apps.account.user_merge.DryRunRollback

Bases: Exception

Raised inside a dry run’s transaction, so that everything it changed is rolled back.

smarter.apps.account.user_merge.MERGED_NAME_SUFFIX = '_merged_'

Inserted between the name and the id of a resource that is renamed because the survivor’s profile already uses its name.

smarter.apps.account.user_merge.UNIQUE_EMAIL_COLUMN = 'email_unique'

the invisible virtual column that the index is on, NULL for a blank address.

Type:

MariaDB and MySQL

smarter.apps.account.user_merge.UNIQUE_EMAIL_INDEX = 'auth_user_email_unique'

The unique index on auth_user’s email addresses.

class smarter.apps.account.user_merge.UserMergeReport(email, survivor_id, duplicate_ids=<factory>, moved=<factory>, renamed=<factory>, deleted=<factory>)

Bases: object

What merging one group of users with the same email address did, or would do in a dry run.

__init__(email, survivor_id, duplicate_ids=<factory>, moved=<factory>, renamed=<factory>, deleted=<factory>)

deleted: list[str]

The rows that were deleted, because the survivor already had an equivalent.

duplicate_ids: list[int]

The ids of the users who are merged into the survivor and deleted.

email: str

The masked email address that the users share.

moved: list[str]

The rows that were moved to the survivor or to one of their profiles, e.g. secret.Secret 12.

renamed: list[str]

The resources that were renamed, because the survivor’s profile already uses their name.

summary()

Describe the merge in one line.

Return type:

str

Returns:

e.g. a***@example.com: kept user 7, merged users [9]: 3 moved, 1 renamed, 0 deleted.

survivor_id: int

The id of the user who is kept.

smarter.apps.account.user_merge.clear_cache()

Clear the cache after a merge.

A merge sends no signals, so the receivers that invalidate a user’s cached objects don’t run. The cache would still hold the deleted users, and the survivor’s profiles without the ones that were moved to it. A cache that can’t be reached, e.g. while a new platform migrates, is logged and skipped: it holds nothing to clear.

Return type:

None

smarter.apps.account.user_merge.create_unique_email_index(schema_editor)

Make auth_user.email unique, ignoring blank addresses.

auth.User belongs to Django, so the index is created with SQL rather than a model constraint. A user who signed in with GitHub may have no address, so blank addresses are left out of it. MariaDB and MySQL have no partial indexes: the index is on an invisible virtual column that is NULL for a blank address, and a unique index allows any number of NULL values. The column’s collation ignores case and trailing spaces, as email_key() does. Other databases get a partial index on the lower-cased address.

Parameters:

schema_editor (BaseDatabaseSchemaEditor) – The schema editor of a migration, or connection.schema_editor().

Raises:

django.db.IntegrityError – when two users still have the same address.

Return type:

None

smarter.apps.account.user_merge.drop_unique_email_index(schema_editor)

Undo create_unique_email_index().

Parameters:

schema_editor (BaseDatabaseSchemaEditor) – The schema editor of a migration, or connection.schema_editor().

Return type:

None

smarter.apps.account.user_merge.email_key(email)

Return the form of an email address that the database’s unique index compares.

MariaDB’s utf8mb4_unicode_ci collation ignores case and trailing spaces.

Parameters:

email (str) – The email address.

Return type:

str

Returns:

The address, lower cased and without trailing spaces.

smarter.apps.account.user_merge.find_conflict(model, fk_field, row, new_target_id)

Return the unique fields that row would violate if its foreign key pointed at new_target_id.

Parameters:

• model (type[Model]) – The row’s model.

• fk_field (Field) – The foreign key that is moved.

• row (Model) – The row.

• new_target_id (int) – The id of the survivor, or of the survivor’s profile.

Return type:

tuple[str, ...] | None

Returns:

The violated set of fields, or None if the row can be moved as it is.

smarter.apps.account.user_merge.find_duplicate_users(apps)

Find the groups of users that share an email address.

Parameters:

apps (Apps) – The historical app registry.

Return type:

list[list[Model]]

Returns:

One list per address, with the survivor first. Empty if there are no duplicates.

smarter.apps.account.user_merge.historical_apps()

Return the app registry of the migrations that the database has applied.

Its models are the historical models that a migration works with. They have no custom methods and send no signals, so a merge can’t trigger a deploy or a DNS change. They match the tables as they are, so a dry run works on a database that a newer version hasn’t migrated yet.

Return type:

Apps

Returns:

The app registry.

smarter.apps.account.user_merge.merge_duplicate_users(apps, dry_run=False)

Merge every group of users that share an email address into one user.

Each group is merged in its own transaction. In a dry run, each transaction is rolled back, so that the reports say exactly what a merge would do.

Parameters:

• apps (Apps) – The historical app registry: a migration’s, or historical_apps().

• dry_run (bool) – Report the merges without keeping them.

Return type:

list[UserMergeReport]

Returns:

One report per group.

smarter.apps.account.user_merge.merge_user(apps, survivor, duplicate, report)

Move everything that one user owns to another, then delete it.

Parameters:

• apps (Apps) – The historical app registry.

• survivor (Model) – The user who is kept.

• duplicate (Model) – The user who is merged into it and deleted.

• report (UserMergeReport) – Collects what was moved, renamed and deleted.

Return type:

None

smarter.apps.account.user_merge.merge_user_fields(survivor, duplicate)

Give the survivor the duplicate’s sign-in history and privileges, where the survivor’s are weaker.

Parameters:

• survivor (Model) – The user who is kept. Saved by the caller.

• duplicate (Model) – The user who is merged into it.

Return type:

None

smarter.apps.account.user_merge.move_related_rows(target_model, old_id, new_id, report, skip_models=())

Point every row that refers to one user, or one profile, at another.

Every reverse relation of target_model is followed, including those of other apps and the automatic tables of many-to-many fields, so that a new model is merged without a code change. A row that would violate a unique set of fields that includes name is renamed. A row that would violate any other, e.g. the survivor’s link to the same group, is deleted.

Parameters:

• target_model (type[Model]) – auth.User or account.UserProfile.

• old_id (int) – The id of the duplicate, or of its profile.

• new_id (int) – The id of the survivor, or of its profile.

• report (UserMergeReport) – Collects what was moved, renamed and deleted.

• skip_models (tuple[type[Model], ...]) – Related models that the caller merges itself.

Return type:

None

smarter.apps.account.user_merge.survivor_sort_key(user)

Order users so that the one who should survive a merge comes first.

That is the one who signed in most recently. Users who never signed in come after, oldest first, then by id.

Parameters:

user (Model) – A user.

Return type:

tuple

Returns:

The sort key.

smarter.apps.account.user_merge.unique_field_sets(model, fk_field)

Return the model’s sets of unique fields that include a foreign key.

Parameters:

• model (type[Model]) – The model of the rows that refer to a user or profile.

• fk_field (Field) – The foreign key that refers to it.

Return type:

list[tuple[str, ...]]

Returns:

Each unique_together or unique constraint that includes fk_field.