Skip to content

ability to overwrite asset instead of merge #160

@bsushmith

Description

@bsushmith

Is your feature request related to a problem? Please describe.
As part of UpdatePatchAsset, we patch and merge the asset before writing into DB.

For the asset field data which contains the crux of asset information, it is impossible to overwrite in the current code. Only if the existing asset data is nil, it is possible to overwrite it. Otherwise, the existing asset.data is merged with incoming changes before writing to DB.

Would like to have a feature flag, where the user can decide if they would like to always take it in the latest changes or merge it with existing values.

Describe the solution you'd like
A feature flag to decide whether to merge or take it latest changes without merging.

Describe alternatives you've considered
I wanted to remove preview rows for bigquery tables in a project, and configured meteor recipe in such a way by having max_preview_rows value set as 0. But since compass does only merge rather than overwrite, it didn't happen. I've tried to delete the asset and then insert the asset again. but since the DeleteAsset takes in asset id, it became cumbersome to delete all the assets using the id.

Additional context
Am not sure about the process of finding asset diff here - https://github.com/odpf/compass/blob/main/internal/store/postgres/asset_repository.go#L301. By the time, the code flow reaches here, the asset is already merged with incoming changes. This diff is used to track asset versions in asset_versions table.
Please correct me If I am wrong here, but shouldn't the diff be calculated with incoming changes rather than merged changes?

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions