Data Management

Data Management

What's new in v2 (0.169.0): the wizard now organises shoots as a 4-level hierarchy — Data Project → Shooting Day → Transfer Session → Offload. Drives live on the Data Project (reused across every shooting day). Cameras, sound mixer name, folder template, and reel counters carry forward day-to-day. Episodes auto-populate from the linked dashboard project. The wizard is 8 steps; new Project picker is Step 1, a Cards step was added at Step 4, and drives moved to Step 5.

Cine Power Planner's Data Management page automates the on-set offload routine: organise the shoot into Data Projects + Shooting Days, lay down a consistent folder structure on external drives (merging non-destructively across days), and copy the cards with verification — using the built-in engine or by handing the source/destination off to Offshoot Pro (Hedge), whichever you prefer.

Browser support: Data Management requires the File System Access API. That means Chromium-based browsers only — Chrome, Edge, Opera, Brave, Arc, Vivaldi. Safari and Firefox land on a friendly "not supported" panel with install instructions for one of the supported browsers.

---

What you need

Account and plan. Data Projects, shooting days, folder templates, and transfer history are stored per account and synced across your devices, so the page is available only to signed-in subscribers:

• Signed out — the page shows a sign-in card with a link to Settings → Account instead of the wizard.
• Signed in on the Free plan — the page shows an upgrade card describing what the subscription covers.
• Signed in with an active subscription — the full wizard, transfer history, and settings open normally.

Transfer history (/data-management/history) sits behind the same gate. The lock hides the UI only: existing data projects, folder templates, and transfer sessions stay stored and synced, and come back as soon as the plan covers the feature again.

The first time you visit /data-management the page shows a capability matrix — green check, amber warning, or red cross for each requirement. Anything red is a soft block; the rest of the page degrades gracefully.

---

The 8-step wizard at a glance

Carry-forward — what's seeded for a new shooting day

When you open the wizard for a fresh shoot day in an existing Data Project, these fields are pre-populated from your most recent prior shoot day:

• Cameras (letters + names)
• Reel counters (continue across days; scope is per-camera-per-Data-Project)
• Sound mixer override (if you set one yesterday)
• Folder template
• Target subfolder pattern

The DataProject's drives + project-level mixer name are NOT carry-forward — they're project-level constants the wizard reads each time.

Idempotent merge across shooting days

Every shoot day's folder creation runs through the SAME drive root. Existing folders from prior days are reused; the wizard never overwrites, renames, or deletes anything that's already on disk. The dry-run preview shows a yellow banner reminding you when this is happening.

This means the on-disk layout for a multi-day shoot looks like:

/Volumes/SSD-A/<DataProjectName>/
  Day_1_260424/  ← wrote on day 1 (untouched on subsequent days)
    01_Camera/A_CAM/A001_…/
    02_Sound/Reel_001/
  Day_2_260425/  ← created on day 2 (Day_1 not touched)
    01_Camera/A_CAM/A002_…/
    02_Sound/Reel_002/

(The exact template is whatever you pick — the merge behavior holds for any layout.)

---

1. Pick a drive

1. Click Pick drive, choose your USB-C / Thunderbolt / RAID volume, and grant permission.
2. Add as many drives as you mirror to — there is no fixed limit. They are stored on the Data Project, so every shoot day in it reuses the same set.
3. Drives persist across reloads. Use Probe to recheck whether the drive is online before each session — if you unplug a drive the indicator flips offline.
4. Rename any drive label (e.g. "Mirror A", "Mirror B") so the rest of the page reads naturally.

PWA tip: Install Cine Power Planner from the address bar (⊕) to keep drive permissions across browser restarts. Without the install, you may be re-prompted on next visit.

2. Pick a folder template

The app ships four built-in templates — Commercial shoot, Feature film, Documentary, Stills/photo. They cover the common offload structure (Camera/A_CAM, Camera/B_CAM, Sound, Proxies, …) and are read-only.

To create your own:

1. Click + New template in the picker.
2. The template editor opens with a tree view. Use + Add root folder for top-level entries, Enter to rename, Delete to remove, and Tab / Insert to nest a child.
3. Insert dynamic placeholders ({{date}}, {{shootDay}}, {{projectName}}, {{episodeName}}, {{cameraName}}, {{cameraLetter}}, {{reelNumber}}, {{location}}) to keep folder names per-shoot. {{location}} takes the Location from that day's Daily Shooting Report header, and resolves to nothing on a day where it was left blank.
4. Save. Custom templates sync to the cloud so the same structure appears on every signed-in device.

Use Duplicate on a built-in template to start from one of the presets without losing the original.

Closing an editor you have changed asks before discarding — the backdrop, Escape and Cancel all lead to the same question, and it compares the folder tree as well as the name, so nesting, renaming or deleting a folder counts. A template you opened without touching still closes on the first click.

3. Create the folder structure

1. Confirm a drive and template are selected.
2. Click Create structure. The app walks the tree depth-first, creating each folder via the FS Access API.
3. The progress strip shows current node count out of total. Each row reports created, existed (already there — left untouched), or failed (with the underlying error).
4. Cancel stops the walk cleanly; partially-written folders remain on disk.

Existing folders are never deleted or renamed by Data Management.

4. Transfer: two engines

Step 8 has two tabs, and the copy runs under whichever one you pick. The choice is remembered per Data Project.

The manifests the built-in engine writes are byte-compatible with the ones Offshoot, ShotPut and Silverstack produce — so any of those can verify a Cine Power Planner offload, and Cine Power Planner can verify any of theirs.

Built-in copy

The Built-in tab itself has two modes:

• Copy (the default) — copies the card and verifies it. Where it runs matters: in the Cine Power Planner Mac app the copy runs natively, with an fsync per file and an atomic rename, which is the most durable path available. In a Chromium browser it runs on the File System Access write surface. Safari and Firefox cannot copy at all — they have no write surface — and the panel says so rather than failing quietly.
• Verify — read-only verification of an offload that already exists, against its MHL manifest. This one only reads, so it works in more browsers, and it is equally happy checking a copy Offshoot made.

When the built-in engine finishes a card, the number of files it copied and verified is written onto that card — see Clips per card.

Offshoot Pro

1. Pick the source (camera card mount or staging folder) and one or more destinations (any drive you've granted permission to).
2. Click Trigger Offshoot. The app fires offshoot://actions?json=... and Offshoot opens with source + destination(s) pre-filled. Click Start in Offshoot — auto-start is not supported via the URL scheme.
3. If Offshoot is not installed, the page surfaces a "Not installed" banner after ~800 ms (the app detects whether the OS handed control to a scheme handler).
4. For long destination lists the URL builder splits the call into multiple offshoot:// invocations, spaced 1.5 s apart so Chromium's popup blocker doesn't suppress later calls.

The session UUID embedded in the source label is the handshake — it lets the app correlate the in-progress transfer with the events Offshoot fires below.

5. Real-time transfer status (optional)

To see file-by-file progress and verification issues inside Cine Power Planner:

1. Click Download Python hook on the page. You get a single cine-power-planner-notifier.py with your webhook URL and Bearer token interpolated. Do not commit this file — it contains a secret.
2. Move it to Offshoot's Scripts folder:
   • macOS: ~/Library/Application Support/Hedge/Offshoot/Scripts/
3. In Offshoot → Preferences → Scripts, register the file under both File Copy Completed and Verification Issue.
4. Back in Cine Power Planner, the Transfer Status panel updates in real time as Offshoot processes files: counts climb, verification issues bubble to the top, and the panel flips to Complete once Offshoot finishes.

The hook batches events (max 50 events or 1 second between flushes) so a fast-firing job doesn't hammer the webhook. Failed flushes leave the queue intact for the next event to retry.

6. Web Push notifications

Click Enable notifications on the page once. After approving the browser prompt, Cine Power Planner registers a Web Push (VAPID) subscription. When a transfer completes or surfaces a verification issue while you're in another tab, you get a system notification — clicking it deep-links back to the session.

7. Shooting Day Report PDF

After a transfer completes, the Save report button generates a Shooting Day Report PDF (powered by pdfMake) listing source paths, destinations, file counts, verification issues, timestamps, and the project link if any. The PDF is saved next to the drive root via showSaveFilePicker, defaulting to a Reports/ subfolder if you've configured one in the template.

8. Check the clips themselves

Two checks sit on top of the copy, both under Transfer → Built-in copy → Verify.

Compare the first and last clip

Point the panel at the card and at the folder it was copied to. It shows the first and last clip of each side next to each other and gives a verdict. This is aimed at the two ways a copy fails quietly: stopping before the last clip, and writing a last clip that is short. A gap in the middle is caught too, by the clip count.

The verdict is computed from clip names, byte sizes and the clip count — never from the picture. That is deliberate, and it is why the check works on ARRIRAW, RED and BRAW cards exactly as well as on XAVC ones. Where your browser has no decoder for a format, the preview cell says which format it cannot show and the verdict is unaffected.

What it will not claim:

• If either folder is too large to read completely, or one side has no clips, you get "Not enough to tell" — never a match.
• A match names what it compared and says plainly that it is not a full checksum of every file. For that, use the MHL verification directly above it.
• A card holding a single clip is reported as such rather than compared against itself.

Clips per card, and camera metadata

When the built-in copy engine finishes a card, the number of files it copied and verified is written onto that card and appears on the Shooting Day Report under Clips. A run you cancelled, or one that failed, writes nothing rather than a zero — an empty field means "unknown", and a zero would mean "the card was empty".

On the report, each camera has a Read from card button. It reads one clip's metadata — codec, resolution, sensor FPS, ISO/EI, colour space, gamma — and offers what it found. Nothing is written until you press Apply, and only the fields the clip actually carried are filled, so anything you typed by hand survives. Sony and Blackmagic sidecar files are read first (cheapest and most complete); QuickTime and MP4 clips are read from their header. ARRI, RED and Canon Raw cannot be read and say so rather than failing silently.

---

---

Standalone vs. project-linked sessions

Sessions can be:

• Project-linked — bound to a Cine Power Planner project. The report and Realtime updates show alongside the project's other artifacts.
• Standalone — for ad-hoc offloads (e.g. covering a friend's wedding pickups). Standalone sessions still produce a report PDF and push notifications, but aren't visible inside any project.

Pick the mode at the top of the page. Switching modes mid-session is not supported — start a new session.

---

Privacy & security

• No file contents leave your machine. The Python hook only POSTs metadata (file counts, paths, timestamps, verification status) to your own self-hosted Supabase instance.
• Bearer tokens in the hook are URL-safe random bytes (32 bytes, base64-encoded). They authenticate the hook to the webhook and use a constant-time compare on the server.
• Row-level security on offshoot_events ensures user A can never read user B's transfer history.
• The hook script is generated per-user and embeds your token. Treat it like a password — don't commit it to git, don't share it.
• Drive handles stored by the browser are user-granted and re-checked on every read or write. Removing a drive from the list also revokes its handle from IndexedDB.

---

Troubleshooting

---

Limitations

• Copying in a browser needs Chromium. Safari and Firefox expose no File System Access write surface, so the built-in engine cannot copy there — the read-only Verify mode still works. The Mac app has no such restriction and is the most durable copy path.
• No auto-start in Offshoot via URL scheme — addTransfers queues, the user clicks Start.
• Network drives mounted as local volumes work, but performance depends on the share. Folder creation is sequential by design (the FS Access API doesn't safely permit parallel creation under one parent).
• Maximum URL length for offshoot:// is ~8 KB — the URL builder splits long invocations automatically.

---

See also

• DATA_STORAGE.md — overall storage architecture (IndexedDB + OPFS + Supabase).
• Offshoot Automation API
• Offshoot Scripts

---

Last Updated: 2026-09-10 Version: 0.790.2