Troubleshoot a watcher

Find your symptom below. When you are stuck, the logs are the fastest way in. Admins can also inspect heartbeats and events under Watchers in the web app.

Setup and registration

“Instrument is still pending activation”

The instrument was registered but an admin has not confirmed it. Ask an admin to open Instruments and click Confirm. See Manage instruments.

“Instrument already has an active watcher”

Each instrument can have at most one active watcher. An earlier install (often on another PC, or before a reimage) is still registered. The CLI prints the existing watcher id; deregister it before re-running init:

• Web app: Watchers, open the existing watcher, click Deregister
• API: DELETE /api/v1/watchers/<existing_watcher_id> with a bearer token that has watchers:write

Deregistration is a soft-delete: history stays under Watchers → Deregistered.

init rejects my API key

The wizard rejects values that do not start with dhub_, and strips zero-width and non-breaking-space characters that Windows clipboards inject. If a paste keeps failing, pass --show-key so you can see what the prompt received.

The watcher will not start

• Run data-hub-watcher config show to confirm config and environment
• Run data-hub-watcher watch --dry-run: it prints the first failing check
• Confirm the instrument is active, not pending
• On Windows, if the service will not start, see Windows service

“Connection error” or “Request timed out”

The watcher cannot reach the Data Hub API. Check:

• Network connectivity from the instrument PC
• Environment in config (staging, production, or preview): data-hub-watcher config show
• That the API base URL for that environment is reachable

Files are not being detected

• Confirm the watch directory: data-hub-watcher config show
• Check file patterns. Globs are case-sensitive on some systems (*.csv does not match data.CSV)
• If files live in subfolders, set run detection to recursive; see Configure run detection and uploads
• Run data-hub-watcher watch --dry-run to preview matches
• Look for pattern_mismatch events on the dashboard

Files are detected but not uploading

• In manual mode, files wait for approval in the upload queue; see Manage the watcher fleet
• Check the log for upload_failed
• A revoked or expired token surfaces as 401 Unauthorized. Create a new token and re-run init; see Issue and revoke tokens

Auto-update never fires

Check, in order:

• Prefer a long-running service (or scheduled) install; a one-off console session may not stay up for the hourly tick
• Environment is not preview: auto-update is hard-disabled there
• Activity-window guard is not holding it up: run data-hub-watcher self-update, or ask an admin to flag the release mandatory
• Last Heartbeat on the dashboard is recent; a stale watcher is not ticking

Upgrade failure modes (update_started → update_failed, editable installs, missing Scheduled Task) are covered in Install and update the watcher and Roll out watcher releases.

Windows service

The service exits immediately

If it stops before writing to watcher.log:

• Check C:\ProgramData\DataHubWatcher\service-bootstrap.log for a pre-dispatcher crash (missing pywin32, moved venv, corrupt install)
• Query the Windows Application event log:

Get-WinEvent -FilterHashtable @{LogName='Application'; StartTime=(Get-Date).AddHours(-1)} |
  Where-Object { $_.ProviderName -match 'Python|DataHubWatcher' -or $_.Message -match 'DataHubWatcher' } |
  Format-List TimeCreated, ProviderName, Id, LevelDisplayName, Message

• Run the service in the foreground to see the traceback:

& "C:\path\to\venv\Scripts\python.exe" -m win32serviceutil debug DataHubWatcher

self-update says “Upgrade dispatched” but nothing happens

On Windows uv-tool installs, the upgrade runs out of process under SYSTEM and takes 30–60s. Track it:

Get-Service data-hub-watcher
Get-Content ~/.data-hub/upgrade-worker.log -Tail 20 -Wait

Turn on debug logging for the service

Add DATA_HUB_WATCHER_LOG_LEVEL=DEBUG to the env file the service loads (typically ~/.data-hub/.env.<environment>) and restart the service. No reinstall needed.

Where are the logs?

Rotating logs (10 MB each, 5 backups). CLI watch and the Windows service share one file:

Add --verbose for debug console output: data-hub-watcher --verbose watch. Windows uv-tool upgrade output lands in ~/.data-hub/upgrade-worker.log.

MCP / API clients get 401 Unauthorized

The bearer token is missing, mistyped, revoked, or expired. Check Settings → Access Tokens and re-issue if needed. See Issue and revoke tokens.

Getting more help

Use the Ask AI widget on any docs page, or the troubleshoot_instrument MCP prompt once MCP catalogs are published under MCP.