Data Hub

Troubleshooting

Fix the watcher by symptom: won't start, files not detected, files not uploading, auto-update never fires, connection errors, and where to find the logs.

ForLab operators

Find your symptom below. When you're stuck, the logs are the fastest way in.

Setup and registration

“Instrument is still pending activation”

The instrument was registered but an admin hasn't confirmed it yet. Ask your admin to open Instruments in the web app and click Confirm next to it. See Managing instruments.

“Instrument already has an active watcher”

Each instrument can have at most one active watcher. An earlier install (often on a different PC, or before a reimage) is still registered. The CLI prints the existing watcher's id; deregister it before re-running init:

  • Web app: go to Watchers, open the existing watcher, and click Deregister.
  • API: curl -X DELETE -H "Authorization: Bearer $DATA_HUB_API_KEY" https://<host>/api/v1/watchers/<existing_watcher_id>

Deregistration is a soft-delete: the old watcher's heartbeats, events, and runs stay visible under Watchers > Deregistered for auditing.

init rejects my API key

The wizard rejects values that don't start with dhub_, and strips zero-width and non-breaking-space characters that Windows clipboards inject. If a paste keeps failing, pass --show-key so you can see what the prompt actually received.

The watcher won't start

  • Run data-hub-watcher config show to confirm the config is present and points at the right environment.
  • Run data-hub-watcher watch --dry-run: it validates the config, checks the API is reachable, and confirms the instrument is active, printing the first failing check.
  • Confirm the instrument is active, not pending (see above).
  • On Windows, if the service won't start, see Windows service below.

“Connection error” or “Request timed out”

The watcher can't reach the Data Hub API. Check:

  • Your internet connection.
  • That the correct environment is set in the config (staging, production, or preview): data-hub-watcher config show.
  • That the API URL for your deployment is reachable. Each environment (staging, production, preview) has its own base URL; data-hub-watcher config show prints the one in use.

Files aren't being detected

  • Verify the watch directory is correct: data-hub-watcher config show.
  • Check your file patterns. The watcher uses glob matching: *.csv matches data.csv but not data.CSV on case-sensitive systems.
  • If your files live in subfolders, make sure run detection is recursive (see the Configuration reference).
  • Run data-hub-watcher watch --dry-run to see exactly what the watcher would pick up.
  • Look for pattern_mismatch events on the dashboard: they mean files were seen but didn't match run_detection.pattern.

Files are detected but not uploading

  • In manual mode, files aren't uploaded until approved through the upload queue. Check the dashboard / Managing watchers.
  • Check the log for upload_failed errors.
  • Verify your API token hasn't expired or been revoked: a revoked token surfaces as 401 Unauthorized. Re-run init with a new token (see Managing tokens).

Auto-update never fires

Check, in order:

  • The watcher is running as a service, not in a console window.
  • The environment isn't preview: auto-update is hard-disabled there.
  • The activity-window guard isn't holding it up. Run data-hub-watcher self-update for an immediate upgrade, or ask an admin to flag the release mandatory.
  • The dashboard's Last Heartbeat is recent. A stale watcher isn't ticking and won't auto-update.

More upgrade-specific failure modes (update_startedupdate_failed, editable installs, stale markers) are covered in Upgrading the watcher.

Windows service

The service exits immediately

If the service stops before writing to watcher.log:

  • Check C:\ProgramData\DataHubWatcher\service-bootstrap.log for a pre-dispatcher crash (missing pywin32, moved venv, corrupt install).
  • Query the Windows Application event log:
Get-WinEvent -FilterHashtable @{LogName='Application'; StartTime=(Get-Date).AddHours(-1)} |
  Where-Object { $_.ProviderName -match 'Python|DataHubWatcher' -or $_.Message -match 'DataHubWatcher' } |
  Format-List TimeCreated, ProviderName, Id, LevelDisplayName, Message
  • Run the service in the foreground to see the full traceback:
& "C:\path\to\venv\Scripts\python.exe" -m win32serviceutil debug DataHubWatcher

self-update says “Upgrade dispatched” but nothing happens

On Windows uv-tool installs, the upgrade runs out of process under SYSTEM and takes 30–60s. Track it:

Get-Service data-hub-watcher
Get-Content ~/.data-hub/upgrade-worker.log -Tail 20 -Wait

Turn on debug logging for the service

Add DATA_HUB_WATCHER_LOG_LEVEL=DEBUG to the env file the service loads (typically ~/.data-hub/.env.<environment>) and restart the service. No reinstall needed.

Where are the logs?

The watcher writes rotating logs (10 MB each, 5 backups). The CLI watch command and the Windows service write to the same file:

PlatformLog file
WindowsC:\ProgramData\DataHubWatcher\watcher.log
macOS / Linux~/.data-hub/watcher.log

Add --verbose for debug-level console output: data-hub-watcher --verbose watch. Upgrade-subprocess output on Windows uv-tool installs lands in ~/.data-hub/upgrade-worker.log.

MCP / API clients get 401 Unauthorized

The bearer token is missing, mistyped, revoked, or expired. Verify it at Settings > Access Tokens and re-issue if needed. See Managing tokens.

Getting more help

Use the Ask AI widget (top of any page) or the troubleshoot_instrument MCP prompt, which inspects an instrument's status, watcher heartbeats, and recent runs for you. See the API reference.

On this page