move-codex-session write-locks each destination database for a time that grows with the database, not the session #144
Loading…
Reference in a new issue
No description provided.
Delete branch "%!s()"
Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?
At
1a85c77, each copy into a destination database holds that database's write lock while the script hashes every unrelated row in it. A live Codex writing there waits, and a write that waits past Codex's 5 s busy timeout fails.copyState,copyHistoryandcopyThreadDatabaserunBEGIN IMMEDIATEon the destination, insert the moved rows, and then calltake()(lines 1184, 1217 and 1272) to confirm that no trigger changed other data.take()issnapshotUnmovedRows, which reads and hashes every row of every table except the moved ones. Its cost follows the size of the destination database, not the size of the session.snapshotBeforeLockalready runs the first such pass before the lock in WAL mode. Even so, the lock covers one full pass, and two in some cases:data_versionchanged.I measured this at
1a85c77with a synthetic destinationthread_history_1.sqlitein WAL mode, filled withthread_itemsrows of a thread that isn't moving, and moved one session with a child. A poller triedBEGIN IMMEDIATEon that database every 10 ms with a 1 ms timeout. The window is how long it could not get the lock, to within the 20 ms poll step. The right column adds one small commit from another process while the first pass runs. Three runs per size up to 642 MB, two at 1284 MB:A commit that landed before the first pass began left the window unchanged. The rate is about 2.4 ms per MB, so the quiet window passes 5 s near 2 GB and the doubled one near 1 GB. An earlier run on APFS clones of a real 1.66 GB
thread_history_1.sqlitegave 4.7 s quiet and 9.6 s with a writer committing every 1.5 s, which is within 20% of that rate.Codex opens its state pools with a 5 s busy timeout (
codex-rs/state/src/sqlite.rs:315inrust-v0.160.1). A connection using that timeout and committing a small write every 200 ms, standing in for the invoking Codex, gave:database is lockedafter 5.2 s, in both runs. The move still exited 0 with empty stderr.SKILL.md line 12 already says the destination is locked "for seconds on a large home" and that the invoking session can see
database is locked. What it doesn't say is that the time is unbounded and starts costing a write at about 1 GB. Nothing is lost: the move completes and the destination stays consistent. The cost is one failed write in the invoking Codex, and how that shows up depends on what the write was.A possible fix takes the whole-database pass out of the lock. A
node:sqlitesession (database.createSession()) opened before the inserts records the changes triggers make as well, so its changeset lists every row the inserts touched and can be compared with the moved rows in time proportional to the move. I checked on Node 26.10.0 that a trigger's update to a second table appears in the changeset. A table without a primary key doesn't, so such a table would keep the full pass or be refused. The alternative reads the triggers fromsqlite_masterand runs the pass only over tables a trigger on an inserted table can write to, which means parsing trigger bodies and is the weaker option. The rollback pathsdeleteThreadDatabaseRowsanddeleteDestinationDatabasesuse the same helpers and would benefit alike.Evidence: reproduced at
1a85c77on macOS with Node 26.10.0 on the synthetic databases above, with machine load varying between runs and the windows agreeing within about 7% per size. The 1.66 GB figures come from an earlier run on a real backup and were not repeated. The 5 s timeout comes from reading Codex 0.160.1.