Data collection

Nothing arrives on a listener, Fluent Bit or syslog is silent, a pull source fails on permissions, Windows sources need a Windows node, events are unparsed or carry the wrong time, data stops at a collection policy or retention limit.

What you will do: follow the data from the source to Folders and Logs and find the step where it stops.

A listener receives nothing

  1. Reachability. From the source host, curl -k -X POST "https://<xplg-host>:30443/logeye/api/logger.jsp?token=<token>&hostName=test" -H "Content-Type: application/json" -d "{\"message\":\"hello\"}" must return quickly. A timeout is the network or a firewall; a 401 or an error body is the token.
  2. The right port and scheme. HTTP on 30303, HTTPS on 30443 (tls On in Fluent Bit); syslog UDP on 514, TCP on 1468. On X8 the listener role listens on 30305 behind Traefik.
  3. The listener account under PortX > Data > Listeners is enabled and bound to the node that receives the traffic (a dedicated listener node must be chosen on the account).
  4. Size limits. UDP syslog events above 1,024 bytes and TCP events above 8,192 bytes are truncated or dropped; move long lines to TCP or HTTP.
  5. System Health > Listeners shows received and dropped counts per listener.

Fluent Bit is running but no data shows

  • The output URI must carry the token and hostName; Format json with header_tag message is the current recommendation.
  • Check Fluent Bit's own log (journalctl -u fluent-bit, or the Windows service log) for http output retries; storage.type filesystem keeps data across restarts.
  • Keep the tail database on persistent storage, or every restart re-reads (duplicates) or skips (gaps).
  • Kubernetes: the DaemonSet needs read RBAC on pods and namespaces, and the cri parser on containerd nodes. See Fluent Bit.

A pull source fails

SymptomCauseFix
"Access denied" on a UNC path or shareThe Windows service runs as Local SystemSet a domain account on the XpoLogCenter service Log On tab and restart (Windows Server)
Windows Events not collectedThe reading node is LinuxWindows Events and shares are read by a Windows node only
SSH source times outKey or password, host key, or port 22 blockedTest from the XPLG host with the same user; the log/ssh log holds the exact error
S3, JDBC or a SaaS plugin failsOutbound HTTPS or the database port blocked, or credentials expiredAllow the outbound port (System requirements); for proxies add the JVM proxy flags
The wizard finds no logsThe path, include / exclude patterns or the subdirectory limitScan a narrower path; the scanner log lists what was examined

Every pull source needs read permission and runs on its Collection Policy schedule; data from a yesterday-only policy appears tomorrow.

Events arrive but look wrong

  • Unparsed or one giant field: set the listener's Message JSON Key, Split by JSON Columns or JSON Parsing Level, or let the pattern parser learn the format and then assign the pattern to the log.
  • Wrong time: the source's time zone differs from the log's; System Health > Data flags parsing and time-zone mismatches under Data Configuration. Fix the time zone on the log, not on the host.
  • Duplicates: two listeners or two Fluent Bit inputs read the same file; or the tail database was lost.
  • Masked values where you expected data: PortX Protect masking or a forwarder flow is applied to that field by policy; see Secure.

Data stops after a while

  • System Health > Data raises "Abnormal data behavior" when a source stops sending or exceeds its volume rule; the rule names the source.
  • The license daily volume was reached ("Allowed data volume exceeded") — see License and updates.
  • Retention: the Collection Policy's hot retention expired and the data moved to the archive; search the archive or raise the retention.
  • Disk: System Health > Disk Space below its threshold stops indexing; free space or move the data path.

Next: Clusters and performance

Source: XPLG wiki, pages 1595637847, 1594131765, 1919418369, 1594132938, 1594132942, 1594132976, 1594132180, 3161489464 (https://xpolog.atlassian.net/wiki/spaces/XPOL/pages/1595637847).