Data Storage

Data Storage

Local-first data storage with IndexedDB, OPFS backup, integrity checks, and (optional) cloud sync

Overview

The Data Storage layer provides a local-first architecture. All user data is persisted in the browser's IndexedDB (with OPFS for full-state snapshots and large binary assets). It serves as the foundation for all CRUD operations, data export/import, schema migrations, and integrity validation. The app works fully offline; cloud sync is opt-in.

Requirements

Functional Requirements

Non-Functional Requirements

Data Requirements

• Storage engine: IndexedDB (primary), OPFS (large binary assets and full-state snapshots)
• Schema validation: JSON schema definitions in @data/schema/
• Normalizers: Per-entity normalizers in @data/normalizers/
• Migrations: Version-based migrations in @data/migrations/
• Encryption: AES-GCM encryption for sensitive fields via entityEncryption.js; salt stored in localStorage[gle_entity_encryption_salt_v1] and survives factory reset

Constraints & Limits

• IndexedDB storage quota varies by browser (typically 50–80 % of available disk)
• Export file size limited by available RAM (JSON serialization)
• The Storage Health "healthy" threshold is a single value (QUOTA_WARNING_THRESHOLD = 80 %); the progress bar's red/amber tints (90 % / 70 %) are cosmetic and do not affect any logic

Offline Behavior

• Fully offline — this IS the offline storage layer
• All CRUD, export, import, and migrations work without network
• Cloud sync is opt-in and provides additional cross-device backup when enabled

Dependencies

• Required by: All features use storageService for data persistence
• Required by: Sync Engine — sync reads/writes via storage
• Required by: Settings — export/import via Backup & Data tab

---

Accessing Data Storage

Data storage is automatic. Management lives in Settings → Backup & Data, which is split into eight sub-sections (Storage · Backup · Auto Backups · Cleanup · Data Integrity · Sync · Preferences Export · Factory Reset).

Managing Your Data

Export (Download Full Backup)

1. Open Settings → Backup & Data → Backup.
2. Click Download full backup.
3. A JSON file downloads containing all your projects, gear, contacts, settings, and time-tracking agreements.

Import (Merge a Backup)

1. Settings → Backup & Data → Backup.
2. Click Import full backup.
3. Select a previously exported JSON file.
4. Confirm. The imported data is merged with what's already there — duplicates are reconciled, not overwritten.

Restore from Device Auto-Backup

1. Settings → Backup & Data → Backup.
2. Click Restore from device backup to apply the latest on-device snapshot saved to OPFS.
3. For a per-project history, click Browse device backups instead — auto-saved snapshots are shown, and any one can be restored.

Cloud Backups (when sync is enabled)

If cloud sync is on, conflict-resolution snapshots are uploaded to the cloud. They live in the Cloud backups card (Settings → Backup & Data → Backup → Cloud backups). Click Load cloud backups to fetch the list, then Download or Restore any entry.

Check Data Integrity

1. Settings → Backup & Data → Data Integrity.
2. Verify runs five integrity checks (state structure, entity IDs, references, content, maintenance schedules) and shows a summary.
3. Repair runs the same checks and applies in-place fixes — bad foreign keys are cleared, missing IDs are generated, invalid statuses are reset to safe defaults. No entities are dropped by Repair. (For deletion, see Cleanup below.)
4. Export Diagnostic Data downloads a JSON bundle of your full state plus the verification result. Useful when filing a support request.

Cleanup (deletes data)

Settings → Backup & Data → Cleanup is the only path that removes content:

• Clear orphaned items — removes contacts not used by any project, owned gear, or company link, and removes custom devices not used by any project / favorite. Profile and production-company contacts are always kept.
• Clear old auto-backups — removes auto-backups older than your retention window. Set it in the Auto Backups sub-section: the field accepts 1–168 hours and starts at 24 h (one day). This page and Settings both called 168 h the default until 2026-09-07 — 168 is the field's maximum, not its starting value.
• Clear browser cache — clears the browser cache and resets the app shell. You may need to sign in again.

Factory Reset

1. Settings → Backup & Data → Factory Reset.
2. Click Run factory reset, then confirm twice: Continue, then type RESET and click Confirm.
3. The app downloads a full backup first. Cancelling that download cancels the reset — nothing is wiped unless the backup was saved. While the reset runs, Confirm reads Resetting… and the whole dialog is inert (Cancel, the confirm field, Escape and click-outside are all disabled) until the page reloads.
4. Then it wipes:
   • Every user-data IndexedDB (e.g. gear-list-sync, collab_baselines, gear-list-overrides, …).
   • The OPFS namespace.
   • localStorage, sessionStorage, cookies, browser caches, service workers.
5. Three things are deliberately preserved:
   • Free-tier counters (_gl_sys IndexedDB) — projects-created count, first-seen timestamp, calculator-usage count. Without this, clearing data would reset the free-tier limits.
   • Encryption salt (gle_entity_encryption_salt_v1 in localStorage) — keeps any cloud backups uploaded before the reset readable.
   • Terra Xplore's local store (terraxplore IndexedDB) — a deliberately standalone store for the public, login-less /terraxplore page. It shares no schema, no sync engine and no cloud path with the signed-in app, so resetting the app is not a reason to destroy it (owner decision on #409, 2026-08-09). This page listed only two preserved stores until 2026-09-07.
6. The page reloads with a cache-buster (?reset=<timestamp>).

The cloud session is revoked before the wipe, but that step is capped at six seconds: a slow or stuck revoke can no longer hold up the local wipe you confirmed. Nothing is lost by proceeding — the wipe deletes the stored session, so the device ends up signed out either way.

If you want a 100 % no-traces wipe (paywall counters and salt included), use the browser's site-data clear directly. Factory reset prioritizes "fresh app, same identity" over "scorched earth."

Storage Warnings

The Storage section shows four signals:

• IndexedDB — Available / Unavailable. Critical: if unavailable, the app falls back to in-memory state (data won't persist).
• OPFS Backup — Available / Not available. Nice-to-have: when missing, the secondary backup target is skipped silently.
• Storage Usage — MB used / MB quota and a percentage. Bar turns amber at 70 %+ and red at 90 %+.
• Storage is healthy / needs attention — true when IndexedDB is up and quota usage is below QUOTA_WARNING_THRESHOLD = 80 %.

If you see the warning state, export a full backup, then use Cleanup → Clear orphaned items / Clear old auto-backups to free space.

Private / Incognito Browsing

When the app detects private browsing (one of: tiny quota, localStorage error, IndexedDB blocked, FileSystem API restricted), a banner is shown. The app still works, but your data may not survive closing the tab. Recommended action: download a full backup before working on anything important.

Tips

1. Export regularly — JSON backups before major changes are cheap insurance.
2. Watch the storage signals — drop unused auto-backups before quota crosses 80 %.
3. Turn on cloud sync for automatic cross-device backup (requires an account; free tier limited).
4. Run Verify before Repair — Verify is read-only; you can read the report and decide whether Repair is appropriate before applying changes.
5. Cleanup ≠ Repair — Repair never deletes entities; only Cleanup does.

Related Documentation

• Settings
• Sync Engine
• Projects

---

Last Updated: 2026-09-07 Version: 0.790.2