Integrate e2e-test-cycloud job with Buildmaster #260
Conversation
Issue: FEI-6125
|
|
||
| dir('webapp/services/static') { | ||
| // We build the secrets first, so that we can create test users in our | ||
| // tests. | ||
| sh("yarn cypress:secrets"); | ||
| exec(runE2ETestsArgs); | ||
| } |
There was a problem hiding this comment.
The idea is stashing of individual worker results stash includes: "${resultsDir}/junit-${workerId}.xml", name: "worker-${workerId}-results";
There was a problem hiding this comment.
Are you planning on doing that in this PR, or a follow-up PR?
There was a problem hiding this comment.
Sorry for the confusion, but I think yes, we will work on the junit implementation after this PR.
| "run-lambdatest": { runLambdaTest(); }, | ||
| "test-cycloud": { runCypressCloud(); }, | ||
| ]); | ||
| // runLambdaTest(); | ||
| } | ||
|
|
||
| stage("Analyzing results") { |
There was a problem hiding this comment.
// Collect results from the triggered job
stage("Collect Cypress Results") {
def resultsDir = "results";
def mergedResults = "${resultsDir}/junit-merged.xml";
sh("mkdir -p ${resultsDir}");
// Unstash results from all workers
for (int i = 0; i < params.NUM_WORKER_MACHINES.toInteger(); i++) {
unstash name: "worker-${i}-results";
}
// Merge results into a single JUnit XML file
sh("""
find ${resultsDir} -name '*.xml' -print0 | xargs -0 cat > ${mergedResults}
""");
echo("Merged results written to ${mergedResults}");
}
There was a problem hiding this comment.
Is this how results get merged for unit tests too? If not, what's different and why?
There was a problem hiding this comment.
I think it’s a slightly different alternative way of how we might collect junit results - maybe more straightforward.
|
|
||
| dir('webapp/services/static') { | ||
| // We build the secrets first, so that we can create test users in our | ||
| // tests. | ||
| sh("yarn cypress:secrets"); |
There was a problem hiding this comment.
Cc @jeresig -- is this something we need to be doing some other way, now?
There was a problem hiding this comment.
Yes - this will need to use the new wrapper script I added here: https://github.com/Khan/webapp/pull/28829 good catch! I'm going to be landing the script to master today.
There was a problem hiding this comment.
We don't need to call yarn cypress:secrets at all. Secrets are handled internally to the CyCloud runner and passed as environment variables option, rather than a JSON file.
There was a problem hiding this comment.
Yeah, makes sense we can remove this yarn secrets call.
| @@ -166,15 +166,25 @@ def runAllTestClients() { | |||
| def runE2ETests(workerId) { | |||
There was a problem hiding this comment.
What directory are we in when this function is run?
There was a problem hiding this comment.
If I’m not mistaken, the script executes the runE2ETests function within the webapp/services/static directory. This is defined in the dir('webapp/services/static') block in the parent function (runAllTestClients).
| def resultsDir = "results"; // Directory to store results | ||
| def junitFile = "${resultsDir}/junit-${workerId}.xml"; | ||
|
|
||
| // Ensure the results directory exists |
There was a problem hiding this comment.
Don't you also need to clear out any old xml files that might be there?
There was a problem hiding this comment.
Oh, thanks! Yeah, we need to clean up the old xml files first to avoid conflicts.
|
|
||
| dir('webapp/services/static') { | ||
| // We build the secrets first, so that we can create test users in our | ||
| // tests. | ||
| sh("yarn cypress:secrets"); | ||
| exec(runE2ETestsArgs); | ||
| } |
There was a problem hiding this comment.
Are you planning on doing that in this PR, or a follow-up PR?
| "./dev/cypress/e2e/tools/start-cypress-cloud-run.ts", | ||
| "--cyConfig baseUrl=\"${E2E_URL}\"", | ||
| "--reporter mocha-junit-reporter", | ||
| "--reporter-options mochaFile=${junitFile}" |
There was a problem hiding this comment.
note: This invocation will need to change. These args aren't supported by the runner and I'm not sure we need to provide them like this. I think most of this can be calculated internally to the code.
There was a problem hiding this comment.
In addition, we're not going to do junit right away. That's a stretch goal once we have things working.
Issue: FEI-6125
Summary:
This draft PR introduces a Cypress Cloud job that runs in parallel with the existing Lambda e2e-test job. It also includes a Mocha implementation for each worker. (Note: The Mocha implementation requires updates with
yarn, which can be addressed in a subsequent PR.)The primary goal is to utilize Jenkins stash functionality to capture and merge all junit reports into a single file, enabling streamlined processing.