-
Notifications
You must be signed in to change notification settings - Fork 4
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Fixup tests and views for the 2025 rollover #715
Fixup tests and views for the 2025 rollover #715
Conversation
4b7b839
to
3a1302c
Compare
3a1302c
to
83826a1
Compare
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Amazing work here @jeancochrane. These are definitely some tricky issues. Glad we'll have tests to catch them going forward.
|
||
{% macro pre_mailed_filters(tablename) %} | ||
{{- pre_stage_filters(tablename, "CCAOVALUE") -}} | ||
and cardinality(stages.procnames) = 0 |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
suggestion: I would maybe add a brief comment here explaining what cardinality()
is doing. I had to look it up and then take a moment to grok how it relates to procnames
. Copying and pasting your PR comment below about the logic is fine IMO.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Good call, done in 28b78f8.
28b78f8
to
276c1ee
Compare
Now that we have 2025 data in Athena, some of our data integrity tests are failing due to views that aren't properly equipped to handle data following a rollover. This PR updates those views and bumps the
data_test_iasworld_year_start
variable to 2025 so that all of our tests are looking at the latest year of data.Data integrity test output: https://github.com/ccao-data/data-architecture/actions/runs/13000380643/job/36257605843
iasWorld data test output: https://github.com/ccao-data/data-architecture/actions/runs/13000382805/job/36257611283