Collection policies and archive
A Collection Policy binds every log to a schedule, a hot storage path with retention, and a cold archive (local or S3) with its own retention and checksum; how to define one and restore from it.
What you will do: create a Collection Policy for a group of logs, place its hot and cold tiers on the right disks, and set how long each keeps data.
Every log added to XPLG uses a Collection Policy — the default one or a custom one. The policy decides how the log is collected, where its indexed (hot) data lives and for how long, and whether a compressed (cold) copy is archived, where, for how long and with which checksum. Listeners and the Add Data wizards ask for it; AppTags can group policies.
The six tabs
| Tab | Settings |
|---|---|
| General | name, description |
| Logs (Members) | the folders and logs bound to the policy |
| Collection Schedule | Never, Daily, Weekly or Monthly, with the parameters of the chosen frequency |
| Storage | Storage Repository path (default the internal data directory ${xpolog.root.path}data; use fast disk, this is where most reads and writes happen) and Delete files older than |
| Archive Location | Enable Archiving, Mirror Folders and Logs structure on file system (recommended), Local (recommended for large volumes) or AWS S3 Bucket account, Archive Path, Archive all data, Delete archive files older than, Checksum algorithm None / MD5 / SHA1 / SHA-256 |
| AppTags | the AppTags the policy belongs to |
The console (PortX > Data > Collection Policies) offers Add, Edit, Duplicate and Delete, plus Schedule Archive Task (the archive runs every hour by default; reschedule it here when the load must move off-peak) and Schedule Security Task (the checksum, best at off-peak hours).
Hot tier
Hot data is stored in a segmented-binary, non-readable format that only an authenticated XPLG user can read; it cannot be changed after it entered the processing engine, parsing and enrichment never touch the raw copy, and the raw data can be exported from the system. Tampering raises an alert. Retention is the policy's Delete files older than; the X8 rule that /node/data never sits on NFS applies to this path. Indexing is per log and on by default; see Indexing and retention.
Cold tier (archive)
- Archived data is compressed flat files (zip, about 90 % smaller), one directory per folder and one zip per log, mirroring the Folders and Logs tree when the mirror option is on, so moving a log in the console moves it on disk.
- It is not searchable. To read it again, restore it and add it as a local log (a manual step), then search it like any other log.
- Archive all data makes the first run archive everything already stored — slow on a large repository; off, archiving starts from the first execution forward.
- The checksum signature is stored (by default in the internal repository, configurable) and compared on each run; a mismatch alerts. Flux verifies synced files with its own per-file checksum, a different mechanism; see Security and integrity.
- The 2026 security specifications describe the same tier as policy-based retention by project, environment, regulation or period, to customer-managed storage, reloadable and re-forwardable on demand.
Steps
- Open PortX > Data > Collection Policies and click New Collection Policy.
- Name it after the retention class it represents (for example
security-13-months). - In Logs, tick the folders and logs; new listeners and logs can be bound later.
- In Storage, set the path on fast storage and the hot retention.
- In Archive Location, enable archiving, keep the mirror option, choose Local or an S3 account (created under Data > Accounts), set the archive retention and a checksum.
- Save, then schedule the archive and security tasks outside business hours.
Checks
- Hot retention is at least the window your dashboards and monitors query.
- Archive retention matches the regulation that applies to the data; a checksum mismatch raises a system alert, and server-side task runs are logged in the
system auditfamily. - Disk alerts in System Health cover both paths.
- Replay to a destination needs the data in the hot tier; see Reduce.
Next: Use cases
Source: XPLG wiki, pages 1594132976, 1594132980, 1594132982, 1628143750, 1594132992, 1594131319, 3161587738, 3161489464, 1602846618, 1594133053, 3150544899, 3150807046, 3151953922, 1594132938 (https://xpolog.atlassian.net/wiki/spaces/XPOL/pages/1594132976).
