You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
This repository has been archived by the owner on Sep 23, 2024. It is now read-only.
Describe the bug
Field level metadata is removed from the catalog when discovery is performed
To Reproduce
Steps to reproduce the behavior:
Provide field-level metadata in the catalog, for example:
{
'breadcrumb': ['properties', 'char_name'],'metadata': {'arbitrary_field_metadata': 'should be preserved'
}
}
Run the tap
This affects at least Meltano users who use the meltano select feature to populate the selected metadatum to indicate whether fields should be copied or not. In that case, the selected field is being stripped and therefore ignored and all fields get copied regardless.
Expected behavior
Field-level metadata should be preserved through the stream_utils.refresh_streams_schema() with only discoverable metadata being updated/added.
Your environment
Version of tap: master
Version of python: 3.8
Additional context
I'm using the tap with meltano and setting a field metadatum: selected: True, however this is stripped from the metadata and in turn not honored by the tap.
The text was updated successfully, but these errors were encountered:
Describe the bug
Field level metadata is removed from the catalog when discovery is performed
To Reproduce
Steps to reproduce the behavior:
This affects at least Meltano users who use the meltano select feature to populate the selected metadatum to indicate whether fields should be copied or not. In that case, the selected field is being stripped and therefore ignored and all fields get copied regardless.
Expected behavior
Field-level metadata should be preserved through the
stream_utils.refresh_streams_schema()
with only discoverable metadata being updated/added.Your environment
Additional context
I'm using the tap with meltano and setting a field metadatum:
selected: True
, however this is stripped from the metadata and in turn not honored by the tap.The text was updated successfully, but these errors were encountered: