Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

ci: add delay after link creation for test add remove static arp #2968

Merged
merged 1 commit into from
Sep 16, 2024

Conversation

QxBytes
Copy link
Contributor

@QxBytes QxBytes commented Aug 28, 2024

Reason for Change:

Linux AddRemoveStaticArp is flaky. It creates the static arp with no errors, but when we try to delete it, we get a response that the file is not found. This logs as an error but the pipeline retries 3 times, so this typically goes unnoticed. If the purpose of the deletion is to ensure the arp entry is deleted, this message likely indicates the arp entry is no longer present.

Observed behavior is that the arp entry is added to the arp table, but then is automatically and immediately deleted a fraction of a second later. When using an existing link, or a link that has been up for some time (> 1 second), the arp entry sticks around.

Theory as to why this happens:

  1. We create a link
  2. We add an arp entry to the link immediately after the link is created
  3. The link likely changes in some way some time afterwards as part of the creation process, deleting the arp entry
  4. We try to delete the arp entry but it is no longer there, causing an error

The solution is adding a delay after creating the link.

Issue Fixed:

Requirements:

Notes:
Relevant pipeline examples:
Running the test 100 times, no error detected with fix: https://msazure.visualstudio.com/One/_build/results?buildId=101831502&view=logs&j=792982d5-3bb1-5d82-222e-228148b73448&t=fe31bb4d-d8c6-5505-3ab3-3c9a2560c42b&l=959
Running the test 10 times, no error detected with fix: https://msazure.visualstudio.com/One/_build/results?buildId=101952522&view=logs&j=792982d5-3bb1-5d82-222e-228148b73448&t=fe31bb4d-d8c6-5505-3ab3-3c9a2560c42b&l=962
Example error: https://msazure.visualstudio.com/One/_build/results?buildId=101778154&view=logs&j=792982d5-3bb1-5d82-222e-228148b73448&t=fe31bb4d-d8c6-5505-3ab3-3c9a2560c42b&l=1091

@QxBytes
Copy link
Contributor Author

QxBytes commented Aug 28, 2024

/azp run Azure Container Networking PR

Copy link

Azure Pipelines successfully started running 1 pipeline(s).

@QxBytes QxBytes self-assigned this Aug 28, 2024
@QxBytes QxBytes added cni Related to CNI. fix Fixes something. ci Infra or tooling. labels Aug 28, 2024
@QxBytes QxBytes marked this pull request as ready for review August 28, 2024 20:22
@QxBytes QxBytes requested a review from a team as a code owner August 28, 2024 20:22
@QxBytes QxBytes requested a review from thatmattlong August 28, 2024 20:22
@QxBytes QxBytes force-pushed the alew/remove-arp-ut branch 3 times, most recently from f214ec7 to 117facf Compare August 28, 2024 22:49
@QxBytes QxBytes force-pushed the alew/remove-arp-ut branch from ea58db9 to e2f6bf2 Compare August 29, 2024 18:52
@QxBytes
Copy link
Contributor Author

QxBytes commented Aug 29, 2024

/azp run Azure Container Networking PR

Copy link

Azure Pipelines successfully started running 1 pipeline(s).

@QxBytes QxBytes changed the title ci: ignore no such file error for test add remove static arp ci: add delay after link creation for test add remove static arp Aug 29, 2024
@QxBytes QxBytes enabled auto-merge August 30, 2024 00:57
Copy link

This pull request is stale because it has been open for 2 weeks with no activity. Remove stale label or comment or this will be closed in 7 days

@github-actions github-actions bot added the stale Stale due to inactivity. label Sep 13, 2024
@QxBytes QxBytes removed the stale Stale due to inactivity. label Sep 13, 2024
@QxBytes QxBytes added this pull request to the merge queue Sep 13, 2024
@github-merge-queue github-merge-queue bot removed this pull request from the merge queue due to failed status checks Sep 14, 2024
@paulyufan2 paulyufan2 added this pull request to the merge queue Sep 14, 2024
@github-merge-queue github-merge-queue bot removed this pull request from the merge queue due to failed status checks Sep 14, 2024
@paulyufan2 paulyufan2 added this pull request to the merge queue Sep 15, 2024
Merged via the queue into master with commit cc1ba09 Sep 16, 2024
14 checks passed
@paulyufan2 paulyufan2 deleted the alew/remove-arp-ut branch September 16, 2024 03:20
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
ci Infra or tooling. cni Related to CNI. fix Fixes something.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants