Appearance
Store & Forward and Data Infrastructure
Last updated
License feature: Store & ForwardStore & forward keeps data that could not be written — because InfluxDB or a remote database was unavailable — on the appliance, and sends it when the target is back. The Data Infrastructure page shows those buffers and also holds the SQL historian settings, transaction groups and outbound REST calls.
Where and who
Visualize → Data Infrastructure (/data-infra). The sidebar entry appears when any of store_forward, transaction_groups or rest_api_outbound is licensed and available to you. The page has up to five tabs, each shown only when its feature is licensed:
| Tab | Needs | Covers |
|---|---|---|
| Historian | store_forward, and the admin role | Where tag history is stored (InfluxDB, SQL, or both) |
| InfluxDB S&F | store_forward | Buffered InfluxDB writes and the quarantine |
| Remote DB S&F | store_forward + remote_databases | Buffered remote database writes |
| Transaction Groups | transaction_groups | See Transaction Groups |
| Outbound API | rest_api_outbound | See Outbound REST |
Anyone with the feature can view the buffers. Retry, Purge, quarantine actions and historian settings need admin.
The Historian tab is shown to admins only, because its settings are admin-only.
What is buffered
| Write | Buffered? | Where |
|---|---|---|
| Tag history and other points written to InfluxDB | Yes | store_forward_buffer table in the configuration database |
| Smart Assist sample batches to InfluxDB | No — dropped while InfluxDB is down | |
| Data Tables editor and script writes to included remote tables | Yes | remote_db_forward_buffer table |
| Transaction group rows to a remote database, when run by their trigger | Yes | remote_db_forward_buffer table |
| Inserts into included SQL Server tables | No — the caller gets the error | |
| Named-query writes to a remote database | No | |
| SQL historian rows | In memory only (below) | |
| Sparkplug, outbound REST, workflow HTTP calls | No |
Without the store_forward license nothing is buffered: InfluxDB points written during an outage are lost, and remote-table writes are lost while still reporting success.
InfluxDB store & forward
When a write to InfluxDB fails (InfluxDB not answering its health check, or an error response), the point is stored in SQLite instead.
- Replay: every 15 seconds, up to 500 entries at a time by default (see Forwarding settings below), oldest first, as one request with a 10-second timeout. A pass still running when the next is due is not started twice.
- InfluxDB still down: entries stay Pending and are retried indefinitely; the attempt count does not grow.
- InfluxDB answers 5xx: each attempt counts; after 10 attempts the entry becomes Failed.
- InfluxDB rejects the data (4xx) — a field type conflict, an unparseable line: the batch is split until the bad entries are found; good entries are sent and the bad ones go straight to Failed with InfluxDB's reason.
- Size limit. The buffer holds at most Store & forward buffer limit rows (Settings → Data Retention, 1,000,000 by default — roughly 250 MB). When it is full, rows are removed in this order: sent entries first, then quarantined (Failed) entries, and the oldest pending entries last, so the newest readings are kept. Every time unsent entries are dropped, the appliance logs a warning, writes an entry to the audit log, and the card shows how many were dropped and when. Set the limit to 0 for no limit.
- Sent entries are deleted automatically once they are older than Delivered store & forward rows (7 days by default). Purge Sent (7d+) does the same on demand.
InfluxDB Buffer Status card:
| Item | Meaning |
|---|---|
| Pending | Waiting to be sent |
| Sent | Already delivered (kept for reference) |
| Failed | Gave up — these are the same entries listed under Quarantine |
| Total | All entries in the buffer |
| Buffer limit line | The row limit from Settings → Data Retention, and a warning if unsent entries were ever dropped for it |
| Oldest pending entry | Timestamp of the oldest unsent point — how far behind history is |
| Retry Failed | Put every failed entry back to pending |
| Purge Sent (7d+) | Delete sent entries older than 7 days |
Forwarding settings
Each buffer tab has a Forwarding box at the top of its status card:
| Field | What it does | Default (InfluxDB / Remote DB) | Limits |
|---|---|---|---|
| Every (seconds) | Time between forwarding passes | 15 / 20 | 1–600 |
| Rows per pass | Most entries sent in one pass (per connection, for remote databases) | 500 / 100 | 1–5000 |
Click Save forwarding; the new values take effect immediately, without a restart. Only an admin can change them; everyone else sees them read-only. Larger batches and shorter intervals catch up faster after a long outage but put more load on the destination and on the appliance. Below the fields the box shows when entries were last forwarded and how many ("Last forwarded n row(s) at …"), and "Last attempt failed: …" with the reason when the most recent pass failed. The change is written to the audit log.
Quarantine lists failed writes with the measurement, time, attempts and InfluxDB's reason, 25 per page. Select entries and Retry after fixing the cause, or Discard them (Empty quarantine discards all). Discarding is permanent data loss and is written to the audit log.
Tags opens InfluxDB - Tag Notifications, where you can pick three tags that Scada.Ink sets to true/false: Buffering (connection lost), Forwarding In Progress and Forwarding Complete. Use local tags, and use them in alarms, workflows or a dashboard indicator. They are set in Scada.Ink only, never written to a PLC, and they can be wrong just after a restart. Click Save Tag Config.
Remote database store & forward
- Replay: every 20 seconds, up to 100 operations per connection by default (see Forwarding settings above), in the order they were made. The first is tried as a probe; replay for that connection stops at the first failure.
- Retries: after 10 failed attempts an operation becomes Failed and the ones behind it can proceed.
- The Remote DB S&F tab shows Pending, Sent, Failed, Oldest pending entry, Pending by Connection, Retry Failed and Purge Sent (7d+), plus the same three notification tags (for all connections together).
Warning: An operation is buffered whatever the reason it failed, including SQL errors that will never succeed. Such an operation holds up every later write to that connection for ten replay attempts (about three minutes) before it is marked failed. Operations for a deleted connection stay in the buffer.
SQL historian (Historian tab)
The Historian tab chooses where tag history is written. Details of history itself are in Historian Storage.
| Option | Behavior |
|---|---|
| InfluxDB | The built-in time-series database. The default |
| Both (migrating) | Write to both, read from SQL. Use while you verify the SQL copy |
| SQL only | Write to and read from a remote database connection |
| Field | What it does | Default | Limits |
|---|---|---|---|
| SQL connection | Which remote database (see Remote Databases) | none | Required for SQL and Both |
| Table name | Table to write | tag_history | Letters, digits, underscore; up to 63 characters |
| Batch size | Rows per INSERT | 500 | 1–5000 |
| Flush interval (ms) | How often a partial batch is written | 2000 | 250–60000 |
| Retention (days) | Delete older rows every 6 hours | 0 (keep forever) | 0–3650 |
Buttons: Save; then Create / verify table (creates device_id, tag_path, data_type, num_value, str_value, ts with an index), Flush now, and Prune now when retention is set. Changing the backend affects new history only.
- The SQL write buffer is in memory, up to 50,000 rows. If the database is unreachable long enough to fill it, the oldest rows are dropped and counted under dropped in Backend status. Rows still in memory are lost if the API restarts.
- On SQL Server, each flush is automatically capped at 333 rows regardless of Batch size — each row uses 6 parameters and SQL Server allows 2100 per statement. A Batch size above that just splits across more flushes rather than failing; the setting works as configured on MySQL, PostgreSQL, ODBC and Oracle.
- Oracle versions before 23ai do not accept the multi-row insert the historian uses.
- A raw query over a range with more than 1000 points returns about 1000 per-window averages spanning the whole range, and an aggregation window too small to fit the range in 2000 points is widened. ODBC connections cannot aggregate, so a long range there returns the newest 1000 raw points.
Backend status (refreshes every 10 seconds) shows which backend serves reads, whether InfluxDB is reachable, the SQL table, rows pending and dropped, the last write and the last error.
History rollups, below it, turns the hourly and daily rollups on or off, shows how many historized tags are up to date and how many rollup rows are kept, and has Run now. See Historian Storage.
API
| Method | Path | Role | Purpose |
|---|---|---|---|
| GET | /api/data-infra/store-forward/stats | viewer | InfluxDB buffer counts, forwarding interval and batch size, last pass |
| GET | /api/data-infra/store-forward/settings | viewer | { influx: { flushIntervalMs, batchSize }, remoteDb: { … } } |
| PUT | /api/data-infra/store-forward/settings | admin | Change either or both; out-of-range values answer 400 |
| POST | /api/data-infra/store-forward/retry-failed | admin | Retry every failed entry |
| POST | /api/data-infra/store-forward/purge | admin | { olderThanDays } (default 7) — delete sent entries |
| GET | /api/data-infra/store-forward/quarantine | viewer | Failed entries; limit ≤ 200, offset |
| POST | /api/data-infra/store-forward/quarantine/retry | admin | { ids } |
| POST | /api/data-infra/store-forward/quarantine/discard | admin | { ids } or { all: true } |
| GET | /api/data-infra/remote-db-forward/stats | viewer | Remote DB buffer counts |
| GET | /api/data-infra/remote-db-forward/entries | viewer | Buffered operations; status, limit |
| POST | /api/data-infra/remote-db-forward/retry-failed | admin | Retry failed operations |
| POST | /api/data-infra/remote-db-forward/purge | admin | Delete old sent operations |
| GET | /api/data-infra/sf-tags/influx | viewer | Notification tags (InfluxDB) |
| PUT | /api/data-infra/sf-tags/remotedb/:connectionId | admin | Notification tags for one connection, or global |
| GET | /api/historian/settings | admin | SQL historian settings |
| PUT | /api/historian/settings | admin | Save them |
Troubleshooting
| Symptom | Cause | Fix |
|---|---|---|
| Pending keeps growing | InfluxDB or the database is still down | Fix the target; replay starts on its own within one forwarding interval (15–20 s by default) |
| Failed entries with a field type conflict | A tag changed data type, so InfluxDB refuses the new type | Fix or discard; InfluxDB keeps the first type per field |
ids must be a non-empty array of entry ids | Retry/Discard called with nothing selected | Select entries |
| The buffer shows dropped entries | An outage lasted long enough to fill the buffer | Fix the target; raise the limit under Settings → Data Retention if storage allows |
Select a SQL connection before enabling the SQL backend | SQL or Both chosen without a connection | Pick SQL connection |
The selected remote connection is disabled | Connection disabled | Enable it |
| dropped count rising | SQL database unreachable and the 50,000-row buffer is full | Restore the database; the dropped rows are gone |