Wait for host page-cache flush before Ctrl-D on Linux (#229)#494
Merged
makermelissa merged 9 commits intoMay 13, 2026
Merged
Conversation
) On Linux, writes through the File System Access API land in the kernel page cache and are flushed to a vfat-mounted CIRCUITPY drive on the kernel's writeback timer (~30s by default). Sending Ctrl-D before that flush completes makes CircuitPython try to import a half-written code.py and fail with OSError: [Errno 5] Input/output error. The File System Access API does not expose fsync, so we cannot force the flush from JS. Instead, gate every soft restart on the device's own view of the filesystem: poll os.stat(path)[6] via REPL until the size matches the bytes we just wrote, then send Ctrl-D. - FSAPI client now records {path, byteLength, at} on each writable.close() and exposes getLastWrite() / clearLastWrite(). - Workflow gains _waitForHostFlush() which is awaited before every softRestart() (run-current and reboot-button paths). It is a no-op on non-Linux, non-FSAPI, or when no write is pending. - The wait is wrapped in showBusy() so the loader is visible during the (potentially up-to-35s) wait. - Caps at 35s and falls through if the kernel never flushes; the existing 3-retry save logic recovers from a failed reboot. - isLinux() added to utilities.js (and exported), with the same ChromeOS/Android exclusions used elsewhere. Refs circuitpython#229
…uitpython#229) On Linux vfat, the kernel can update the FAT directory entry before flushing the actual file data sectors. os.stat() alone returns the correct size before the device can actually read the file, so a poll that only checks size is not a sufficient flush detector. Instead, each poll opens the file on the device, reads all bytes, and computes a small xor checksum. We compare it to a host-computed checksum recorded at write time. Only when size, readable length, and checksum all match do we proceed to softRestart. Tested on Raspberry Pi 5 (kernel 6.12) with a Feather RP2040 running CircuitPython 10.2.0. Polling typically resolves at ~33s (just inside the kernel's 30s dirty_expire window); bumped timeout from 35s to 40s for headroom.
…python#229) The host-flush wait was only wired into the editor's Run and Reboot button paths. A Ctrl-D typed directly in the terminal panel bypassed the wait and still raced the kernel page cache flush. Add serialTransmitWithFlushGuard() on the workflow base class. The terminal panel routes onData through it; when the user transmits a Ctrl-D (\x04) and there is a tracked pending FSAPI write, we run the same _waitForHostFlush() before passing the byte through. The fast path (no Ctrl-D or no pending write) has no extra overhead. Also add a Troubleshooting section to README documenting: - udev rule to mount CIRCUITPY with sync,flush - supervisor.runtime.autoreload = False in boot.py - vm.dirty_expire_centisecs sysctl tuning - ChromeOS limitation note Issue circuitpython#229.
The 'Choose a different workflow' back link (issue circuitpython#373) showed a focus rectangle after a mouse click. Use :focus / :focus-visible to suppress the outline on pointer activation while preserving a visible focus ring for keyboard users.
:focus-visible was still drawing the rectangle in some browsers. Match the pattern used by other anchors in the editor: no outline on any focus state. The hover underline is sufficient affordance.
README Option A now explicitly notes that mounting CIRCUITPY with sync,flush makes the flush-detector poll match on its first attempt, so save/run/reboot/Ctrl-D feel instant rather than waiting up to ~30s for the kernel page cache to flush.
The autoreload=False suggestion was a misdirect: CircuitPython's own filesystem-change reload is a separate path that the editor's flush-detector already handles correctly. Removing it leaves the two workarounds that actually address the root kernel-flush race.
Collaborator
|
This basically fixes the issue by waiting until the file has finished being written. |
dhalbert
approved these changes
May 13, 2026
Contributor
dhalbert
left a comment
dhalbert
left a comment
There was a problem hiding this comment.
This sounds good. I am surprised about the 30 seconds, but it's true that every editor I use does the equivalent of an fsync.
The user could be encouraged to type sync in a terminal window, which would speed up the waiting process.
The 40s timeout was calibrated against the default Linux dirty_expire_centisecs of 3000 (=30s), which covers a Pi 5 + SSD setup comfortably. On hosts running laptop-mode tools (which push the expire window to 60s+), on slow/contended USB buses, or when writing larger files, the 40s window could miss and fall through to the save-retry loop. Bump to 60s for headroom and call out the trade-off in the Troubleshooting section so users know to apply Options A-C if they hit the timeout regularly.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes #229
Problem
On Linux, saving
code.pyvia the FS Access API (USB workflow) and then sending a soft-reboot would frequently cause CircuitPython to read a partially-flushed file, raisingOSError: [Errno 5] Input/output error.Root cause: Linux mounts vfat (CIRCUITPY) without
syncby default. After the host writes a file, the kernel can hold the data in the page cache for up todirty_expire_centisecs(default 30s) before flushing the actual data sectors to disk. There is nofsyncavailable through the FS Access API, so the editor must wait until the device confirms it can see the full file before triggering a reload.Crucially,
os.stat()is not a sufficient flush detector: the kernel can update the FAT directory entry (giving the device the correct file size) before flushing the data sectors (so the device cannot yet read the file contents). Empirical testing showedos.stat()returns the correct size ~1s after a write, butopen()+read()still returns -1 (OSError) for another ~30s.Approach
This PR introduces a device-side flush detector that runs before every soft-reboot the editor sends to the device. The FSAPI client records the path, byte length, and xor checksum of the last write. Before
softRestart(), the workflow polls the device every 500ms with a small Python snippet that:os.stat()to read the FAT directory size.The wait completes when all three match the host-recorded values, confirming the data sectors are flushed. The poll is wrapped in
showBusy()so the user sees the existing Blinka loader instead of a frozen UI. A 40s timeout falls through to the existing 3-retry save loop if something goes wrong.The wait is gated on:
isLinux()excludes ChromeOS and Android, which include "Linux" in their UA strings)So macOS, Windows, ChromeOS, and the BLE/Web workflows are unaffected.
Soft-reboot paths covered
The wait fires before:
runCurrentCode()→softRestart()).restartDevice()→softRestart()).serialTransmitWithFlushGuard()interceptor on the terminal'sonDatahandler).User-side workarounds (also documented in README)
For users who want to eliminate the wait or the underlying race entirely:
sync,flush(recommended on Linux).supervisor.runtime.autoreload = Falseinboot.pyto suppress the device's own auto-reload-on-filesystem-change.vm.dirty_expire_centisecssysctl for host-wide flush tuning.boot.pyworkaround.What this still does NOT fix
boot.pyworkaround above addresses this.Test plan
Tested on Raspberry Pi 5 (kernel 6.12, vfat mounted async) with a Feather RP2040 running CircuitPython 10.2.0, and on macOS:
isLinux() === false)Files
js/common/utilities.js—isLinux()helper that excludes ChromeOS/Androidjs/common/fsapi-file-transfer.js— last-write tracker (path, byteLength, checksum) on the FSAPI clientjs/workflows/workflow.js—_waitForHostFlush()/_waitForHostFlushImpl()gated at bothsoftRestart()call sites, plusserialTransmitWithFlushGuard()for terminal-typed Ctrl-Djs/script.js— route terminalonDatathrough the flush guardREADME.md— Troubleshooting section with udev / boot.py / sysctl workarounds