---
title: Schema Migrations
description: >-
  The committed cocoon.lock, the interactive migrate command, the flags for
  scripts, and the on-device migration runner.
---
> ## Documentation Index
> Fetch the complete documentation index at: https://docs.page/redfieldchristabel/butterfly/llms.txt
> Use this file to discover all available pages before exploring further.

Cocoon never needs hand-written migrations. Field numbers are pinned by name
in a committed `cocoon.lock`, so most schema changes need nothing at
all — and a type change needs exactly one conversion line that the on-device
runner applies for you. The lock is YAML, formatted like `pubspec.lock`; a
project still carrying the legacy `cocoon.lock.json` is converted by the
next `migrate` run (which writes `cocoon.lock`, deletes the JSON and asks
you to commit both changes).

## What each change needs

| Change | What happens |
| --- | --- |
| Add a field | Fresh number. Old documents decode the default. Nothing to run — unless the field is non-nullable with no constructor default on an existing collection: `migrate` then asks what the old documents should read (see below). |
| Remove a field | Its number is reserved forever. Confirm the drop in `migrate`. |
| Reorder fields | Nothing — order never mattered. |
| Add/drop an index | Applied by diffing on open; no migration file. |
| Rename (same type) | Developer confirms; same number, data untouched. |
| Type change | New number, old reserved, migration file with a typed upgrade stub. |
| Rename + type change | Same as a type change. Numbers are never silently reused. |
| Move a collection to a different path | Refused — delete the documents and store them under a new collection instead. |

## The two tools

Two tools compare the models against the committed `cocoon.lock`:

- **build_runner** (local and CI) only generates codecs, typed fields and
  queries. It never writes the lock or migrations; it **fails** when models
  and lock disagree, and it fails while any migration file still contains an
  unimplemented conversion.
- **`dart run cocoon_generator:migrate`** runs on the developer's machine,
  interactively. It fixes the disagreement: writes the new lock and — only
  when stored data must change — a numbered migration file.

```sh
dart run cocoon_generator:migrate
```

## What the prompt looks like

The command asks only what a human must answer. For every removed field
with new fields alongside it, a numbered menu offers the rename candidates —
best first, `Enter` accepts the preselected one — and a final drop option:

```text
Txn: `currency` (String, #3) is gone and these are new:
  1) currencyCode (int)  rename, convert data
  2) drop currency (its number is reserved forever)
Choice [1]:
```

Fields without candidates ask a plain confirm (`y/N`, default `No` —
answering `No` aborts without writing anything):

```text
Txn: `memo` (String?, #6) is gone. Drop it (number 6 is reserved forever)? [y/N]
```

Removed collections and embedded classes confirm the same way:

```text
Collection Ledger has no model anymore. Drop it (existing documents become unreadable if the collection ever returns)? [y/N]
```

Everything else is decided without asking: added fields get fresh numbers,
reorders and index changes change nothing, and a type change always moves
the field to a fresh number (the old number is reserved forever) and writes
a conversion stub. A candidate already chosen for another field is not
offered again.

The command then writes the updated lock and, when needed, the migration
file:

```text
Wrote cocoon.lock (version 3).
Wrote lib/cocoon_migrations/m0003_txn_currency_to_currency_code.dart.
Next: implement the `upgrade...` functions in the new migration file, then run build_runner.
Wrote lib/cocoon_migrations/migrations.dart.
```

## Migration files

The generated file contains one typed conversion stub per type change —
implement them, and commit the lock, the migration file and the models
together so renames and conversions show up in review:

```dart
// lib/cocoon_migrations/m0003_txn_currency_to_currency_code.dart
// Txn.currency: String (#3) -> currencyCode: int (#10)
int upgradeTxnCurrency(String old) => switch (old) {
  'MYR' => 458,
  'SGD' => 702,
  _ => 0,
};
```

The build fails while any `upgrade...` function still throws
`UnimplementedError`. Stub types follow the stored representation:
`int` and `DateTime` fields convert as `int` (their microseconds), enums as
their `String` name, and only present values are converted — documents
without the old field are skipped, and a conversion returning `null` drops
the value.

## New required fields: the backfill question

One more question, for an added field of an **existing** collection that is
non-nullable and has no constructor default: such a field would silently
read its proto3 zero value (`''`, `0`, `false`, the epoch for `DateTime`,
the first enum value) in every document written before it existed — for
money and dates, a quiet data bug. The command shows the exact value the
old documents would read and asks:

```text
Txn: `currency` (String) is new and required. Existing documents would read ''.
  1) Write a backfill stub (a migration sets the value for existing documents) [default]
  2) Keep the zero value ('')
  3) Abort (add a default or make it nullable first)
```

The default writes a backfill stub into the new migration file — a function
that receives each decoded old document and returns the value to store, so
it can depend on the document's other fields:

```dart
// lib/cocoon_migrations/m0004_txn_currency_backfill.dart
String backfillTxnCurrency(Txn doc) => doc.memo == null ? 'MYR' : 'USD';
```

Implement the stubs — the build fails while any `UnimplementedError`
remains — and on the device the engine runs the backfill over every
existing document inside the open's one transaction: batched, one
change-log row per document, crash-safe, progress through
`onMigrationProgress`, a throwing stub rolls the whole open back. Choice 2
records the decision in the lock so the question is never asked again. A
brand-new collection asks nothing — it has no existing documents.

## Flags for scripts

Scripts and pre-commit hooks drive the command without a terminal:

| Flag | Meaning |
| --- | --- |
| `--rename Model.old=new` | Decide a rename without the menu (repeatable). |
| `--drop Model.field` | Drop a removed field, reserving its number (repeatable). |
| `--drop-collection Model` | Confirm a removed collection or embedded class (repeatable). |
| `--backfill Model.field` | Decide an added required field with a backfill stub, without the menu (repeatable). |
| `--keep-zero Model.field` | Record the keep-the-zero decision for an added required field (repeatable). |
| `--yes` | Accept the default answer of every rename menu; **never** covers drops; for an added required field it picks the default — a backfill stub, never a silent keep-zero. |
| `--check` | Exit 1 when the run would change anything; writes nothing, and reports pending required-field decisions and a pending `cocoon.lock.json` -> `cocoon.lock` conversion. |

```sh
# Scripted run on a developer machine (never in CI): fail when the lock is
# out of date, never write it
dart run cocoon_generator:migrate --check
```

When stdin is not a terminal and a decision is missing, the command exits 2
and lists the flags that would resolve it:

```text
Non-interactive stdin with 1 decision(s) missing:
  - Txn: removed field `currency` (String, #3): pass --rename Txn.currency=currencyCode (or --drop Txn.currency)
```

Exit codes: `0` up to date or written, `1` refused/aborted/`--check` would
change, `2` usage error or missing non-interactive decision. Output is
colored only when stdout is a terminal.

<Warning>
  **`migrate` is a development tool, not a CI step.** Run it on your machine
  after changing a model, answer its questions, then review and commit
  `cocoon.lock` and `lib/cocoon_migrations/` together. CI only builds:
  `dart run build_runner build` fails when the models and the committed lock
  disagree, so a model change that skipped `migrate` cannot pass CI — the fix
  is always "run migrate locally and commit", never "run migrate in CI".
  Generating migrations in a pipeline would skip the human decisions (rename
  vs drop, backfill vs zero) that keep stored data correct. The
  non-interactive flags exist for scripted runs on a developer machine (an AI
  agent working in a checkout, say), and `--check` is read-only but redundant
  in CI, because the build already fails.
</Warning>

## The on-device runner

Pass both artifacts to the engine when opening the database:

```dart
final db = await Cocoon.open(
  file,
  collections: [accounts, txns],
  schemaVersion: cocoonSchemaVersion, // from lib/cocoon_schema.g.dart
  migrations: cocoonMigrations, // from lib/cocoon_migrations/migrations.dart
  onMigrationProgress: (done, total) => print('$done/$total'),
);
```

On the device, opening compares the schema version stored in the internal
`_cocoon_meta` table with `cocoonSchemaVersion`:

- A **fresh database** just stores the version.
- An **older database** runs every pending migration once, in order, inside
  ONE transaction: documents are rewritten at the wire level in batches of
  500, one change-log row per rewritten document is appended, the migrated
  tables' columns and indexes are rebuilt from the rewritten bytes, and only
  then is the new version stored. Any failure — a throwing conversion
  included — rolls the whole open back.
- A database written by a **newer** app version refuses to open with a
  `SchemaTooNewException`; upgrade the app instead.

## Old files: storage-format upgrades

Files also carry a storage-format version in the internal `_cocoon_meta`
table, separate from your schema version. A file written by an older Cocoon
is upgraded on open — its collection tables are rebuilt to the current
layout (no stored `path` column; a document's identity is its path params
plus id) table by table, inside the open, crash-safe, with progress
reported through `onMigrationProgress` — and a file written by a newer
version refuses to open with a `StorageFormatTooNewException`.

Measured on a 1M file: the one-time upgrade takes ~5-9 s, keeps every
document and change-log row, and the file transiently grows to ~519 MiB —
the dropped tables' pages sit on the freelist. After the upgrade's
transaction commits, the open compacts the file once (`VACUUM`) when more
than a fifth of it is reclaimable: 390 → 305 MiB with the freelist back to
zero, ~1 s of `VACUUM` at 1M, ~34 ms on a 0.5 MiB file.

- `compactAfterUpgrade: false` keeps the old behavior — the next
  `backupTo`, which always compacts, reclaims the space instead (see
  [Backup & Restore](/libraries/cocoon/backup)).
- A compaction that cannot run — most often a full disk, since `VACUUM`
  needs about the file size in free space — is reported through
  `onCompactionFailed` and never fails the open.
- Normal opens never compact.

The wire-level mechanics of the rebuild are in
[Internals](/libraries/cocoon/internals).
