memory full due to influxdb connection issue

[jordanbauduin]

Hi there , first of all thx for the great work , while deploying it to test env running on LWS VPS , I have faced repetitive memory leaks visible in the below graphs

It looks like the tool wasn't able to write the acquired data inside the influxDB , stored it in local buffer 'till the memory got exhausted and process crashed

Setup is such

Siemens HMI located in BEL <-OPC-UA -> VPS running OPCUA-logger in PARIS LWS <-> influxDB running on the same VPS , GRAFANA running on the same VPS and consuming the data inside influxDB.

Haven't had time yet to dig on it , but will in the coming days , it looks more a pb on the influx DB , but will check that

See below generated logs

[2019-10-30T18:15:23.486] [ERROR] buffer - Failed to write points from membuffer. { error: 'connect ECONNREFUSED 127.0.0.1:8086' }
[2019-10-30T18:15:23.487] [ERROR] buffer - Will try again in 5s.
[2019-10-30T18:15:28.490] [ERROR] influx - connect ECONNREFUSED 127.0.0.1:8086
[2019-10-30T18:15:28.491] [ERROR] buffer - Failed to write points from membuffer. { error: 'connect ECONNREFUSED 127.0.0.1:8086' }
[2019-10-30T18:15:28.491] [ERROR] buffer - Will try again in 5s.
[2019-10-30T18:15:33.495] [ERROR] influx - connect ECONNREFUSED 127.0.0.1:8086
[2019-10-30T18:15:33.495] [ERROR] buffer - Failed to write points from membuffer. { error: 'connect ECONNREFUSED 127.0.0.1:8086' }
[2019-10-30T18:15:33.496] [ERROR] buffer - Will try again in 5s.
[2019-10-30T18:15:38.498] [ERROR] influx - connect ECONNREFUSED 127.0.0.1:8086
[2019-10-30T18:15:38.498] [ERROR] buffer - Failed to write points from membuffer. { error: 'connect ECONNREFUSED 127.0.0.1:8086' }
[2019-10-30T18:15:38.498] [ERROR] buffer - Will try again in 5s.
[2019-10-30T18:15:43.501] [ERROR] influx - connect ECONNREFUSED 127.0.0.1:8086
[2019-10-30T18:15:43.501] [ERROR] buffer - Failed to write points from membuffer. { error: 'connect ECONNREFUSED 127.0.0.1:8086' }
[2019-10-30T18:15:43.501] [ERROR] buffer - Will try again in 5s.
[2019-10-30T18:15:48.504] [ERROR] influx - connect ECONNREFUSED 127.0.0.1:8086
[2019-10-30T18:15:48.505] [ERROR] buffer - Failed to write points from membuffer. { error: 'connect ECONNREFUSED 127.0.0.1:8086' }
[2019-10-30T18:15:48.505] [ERROR] buffer - Will try again in 5s.
[2019-10-30T18:15:53.589] [ERROR] influx - No host available
[2019-10-30T18:15:53.589] [ERROR] buffer - Failed to write points from membuffer. { error: 'No host available' }
[2019-10-30T18:15:53.589] [ERROR] buffer - Will try again in 5s.
[2019-10-30T18:15:58.593] [ERROR] influx - connect ECONNREFUSED 127.0.0.1:8086
[2019-10-30T18:15:58.593] [ERROR] buffer - Failed to write points from membuffer. { error: 'connect ECONNREFUSED 127.0.0.1:8086' }
[2019-10-30T18:15:58.593] [ERROR] buffer - Will try again in 5s.
[2019-10-30T18:16:03.596] [ERROR] influx - No host available
[2019-10-30T18:16:03.596] [ERROR] buffer - Failed to write points from membuffer. { error: 'No host available' }
[2019-10-30T18:16:03.597] [ERROR] buffer - Will try again in 5s.
[2019-10-30T18:16:08.599] [ERROR] influx - connect ECONNREFUSED 127.0.0.1:8086
[2019-10-30T18:16:08.599] [ERROR] buffer - Failed to write points from membuffer. { error: 'connect ECONNREFUSED 127.0.0.1:8086' }
[2019-10-30T18:16:08.600] [ERROR] buffer - Will try again in 20s.
[2019-10-30T18:31:37.510] [WARN] opcuaclient - Time travel detected: drift: 510. Skipping tick.
[2019-10-30T18:31:38.002] [WARN] opcuaclient - Tick discrepancy: 38 vs 36
Timeout .... waiting for response for OpenSecureChannelRequest { /RequestHeader/
authenticationToken /* NodeId /: ns=0;i=0
timestamp / DateTime /: 2019-10-31T02:39:50.298Z
requestHandle / UInt32 /: 241864 0x3b0c8
returnDiagnostics / UInt32 /: 0 0x0
auditEntryId / UAString /: null
timeoutHint / UInt32 /: 0 0x0
additionalHeader / ExtensionObject */: null
};
[2019-10-31T03:56:32.780] [INFO] opcuaclient - Socket was closed!
[2019-10-31T03:56:32.781] [FATAL] main - connection_break
[2019-10-31T03:56:32.781] [INFO] buffer - Persisting membuffer.
[root@vps51703 ~]#

[coussej]

Hi, actually, it seems the OPCUA connection was interrupted, and the process shut down gracefully, persisting any points that were left in the buffer. It should be handled by the system daemon (windows service, systemd on linux, ...) to restart the process. Are you sure the memory increase is due to the logger process? Normally, the membuffer should contain no more than 2 * the write batch size, which is 1000 by default. 2000 points will not take up so much memory to have a memory overflow.