Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Store prefetch results in LFC cache once as soon as they are received #10442

Open
wants to merge 1 commit into
base: main
Choose a base branch
from

Conversation

knizhnik
Copy link
Contributor

Problem

Prefetch is performed locally, so different backers can request the same pages form PS.
Such duplicated request increase load of page server and network traffic.
Making prefetch global seems to be very difficult and undesirable, because different queries can access chunks on different speed. Storing prefetch chunks in LFC will not completely eliminate duplicates, but can minimise such requests.

The problem with storing prefetch result in LFC is that in this case page is not protected by share buffer lock.
So we will have to perform extra synchronisation at LFC side.

See:
https://neondb.slack.com/archives/C0875PUD0LC/p1736772890602029?thread_ts=1736762541.116949&cid=C0875PUD0LC

@MMeent implementation of prewarm:
See #10312

Summary of changes

Use conditional variables to sycnhronize access to LFC entry.

Copy link

github-actions bot commented Jan 19, 2025

7385 tests run: 7005 passed, 0 failed, 380 skipped (full report)


Flaky tests (3)

Postgres 17

Postgres 16

Code coverage* (full report)

  • functions: 33.6% (8445 of 25104 functions)
  • lines: 49.2% (70717 of 143866 lines)

* collected from Rust tests only


The comment gets automatically updated with the latest test results
071142f at 2025-01-20T20:34:33.613Z :recycle:

Copy link
Contributor

@erikgrinaker erikgrinaker left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks! I've only done a high-level review, but the approach makes sense to me. I expect the synchronization overhead would be negligible since this is on an IO path, but would be interested to see how benchmarks are affected.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants