Indexing and retention
What XpoLog indexes, where the index and the hot data live, how to turn indexing off per log, the retention and archive settings that govern them, and the cluster roles involved.
What you will do: understand which data is searchable, for how long, on which node, and change those settings for one log or one policy.
XpoLog stores parsed events with time-range navigation and a lifecycle aligned to your retention policy. Two settings govern the lifecycle: whether a log is indexed (searchable) and which Collection Policy (storage path, retention, archive) it uses.
Indexing
- By default every log is indexed; indexing is what makes search fast and the wiki recommends keeping it on.
- Disable it per log under PortX > Search > General. A log that is not indexed is still collected and readable in the Log Viewer, but it is not available in the Search console (and so to anything that runs a search).
- Listeners index received data at their Indexing Interval, on their Indexing Node in a cluster.
- The Support Portal (administrators only, PortX > Settings > About) shows the index status of every log under Data Information > Indexing, and can delete or rebuild an index; Actions shows the last indexing time and runs indexing by hand.
- On X8 the processor role (and the optional processor-1) carries indexing and the listener role receives; workloads redistribute automatically when a node fails. On V7 the same engine runs inside the single process or the cluster's processing nodes, where one node is the active processor and a UI node can take over; see Sizing and capacity planning.
- The index share of the PortX volume is also a licensing dimension (the XpoLog tier, 50 % by default); see How licensing works.
Retention
| Data | Setting | Where |
|---|---|---|
| Hot (indexed, searchable) | Delete files older than on the Storage tab | Collection Policy |
| Cold (archive, compressed) | Delete archive files older than on the Archive Location tab | Collection Policy |
| Audit and system logs | kept as long as needed | Support Portal |
| Configuration backups | daily, 30 days | EXTERNAL_CONFIGURATION_DIRECTORY/temp/backups/ |
Hot data is a segmented-binary format readable only through XPLG; the raw copy is never altered by parsing or enrichment and can be exported. The archive is compressed flat files, about 90 % smaller, with an optional MD5, SHA1 or SHA-256 checksum; restored data is re-added as a local log and becomes searchable again. The full policy is on Collection policies and archive.
Steps: a retention class
- Create a Collection Policy named for the class (
app-30d,security-13m). - Set the hot path on fast storage with the retention your dashboards and monitors need.
- Enable the archive on slower or object storage with the retention the regulation requires and a checksum.
- Bind the logs; set the policy on every listener and wizard that feeds the class.
- For logs you only ever read raw, disable indexing and leave them to the Log Viewer.
Checks
* | datastream | group by ext.log | order by data volume desc | display data volume in volume formatshows what each log contributes to the indexed volume.- Searches that return nothing for a known log: confirm the log is indexed and inside its hot retention.
- Disk alerts for the storage path appear in System Health.
About the "indexless" claim
xplg.com describes the platform as indexless. The wiki documents an index on every log by default, a per-log switch to turn it off and the node that performs it. This page follows the wiki; the wording of the claim is pending confirmation from XPLG engineering.
Next: LogX
Source: XPLG wiki, pages 1594133714, 1594132976, 1594132940, 1598685520, 1595637849, 1594131316, 3150774276, 1602584512, 1595637833, 1602584807, 3161587738, 1594132401, 1594132942, 1594132938 (https://xpolog.atlassian.net/wiki/spaces/XPOL/pages/1594133714).
