Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Crucial memory leak Investigation: requesting author's expertise for help in resolving the issue #28

Open
calebeaires opened this issue Jul 24, 2023 · 0 comments

Comments

@calebeaires
Copy link

Using this library in an application where a single connection is created to perform a Clickhouse SELECT query to retrieve a response, followed by another SELECT query and subsequently repeating the entire process multiple times, the Axios HTTP library is not releasing memory.

I have provided a simple and practical example below to simulate what I have been doing. The number of example calls is for illustration purposes only. The objective is to demonstrate that a heapsnapshot reveals that memory in multiple requests leads to a memory leak. After running for 2 minutes, the memory usage increases from 70MB to 90MB and continues to rise progressively.

I have created this issue to discuss how to resolve a potential memory leak when making multiple requests. I have considered disabling KeepAlive or implementing better socket control.

Do you have any suggestions?

import { ClickHouseClient } from '@depyronick/clickhouse-client';

let a = 1
const client = new ClickHouseClient({
  host: 'localhost',
  password: 'mysecret',
  settings: {
    wait_end_of_query: 1
  }
});

const init = async () => {
  await client.queryPromise('SELECT * from some_database.some_table limit 1')
  a = a +1
  console.log(a);
}

const loopIterations = 1000000;

const runLoop = async () => {
  for (let i = 0; i < loopIterations; i++) {
    await init();
  }
}

runLoop();

We know that the issue does not lie with ClickHouse, but rather in the inability of HTTP/Axios to release memory in subsequent requests. This is the key aspect where we can attempt to find a solution.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant