Skip to content

avoid conflicting data type in properties attributes field #32

@mabdh

Description

@mabdh

Describe the bug
Elasticsearch is silently dropping ingested metadata when there is a data type conflict in properties.attributes field. Columbus returns 200 but the metadata is not ingested.

To Reproduce
Steps to reproduce the behavior:

  1. Create a new type/index called dashboard
  2. Ingest a new data to the dashboard index with properties.attributes.id field is in integer e.g. 1234
  3. Check and verify elasticsearch, the data supposed to be ingested properly
  4. Ingest another new data to the dashboard index with properties.attributes.id field is in string e.g. "df431-54abf42-xxxx"
  5. Check and verify elasticsearch, the other new data is not ingested

Expected behavior
Ingesting different kind of metadata that has the same type should always be succeed.

Additional context
This is happened when we tried to ingest metabase and tableau dashboard data. We ingest metabase metadata first, then tableau data. tableau data is not ingested to elasticsearch because it has id in string and metabase has id in long.

Metadata

Metadata

Assignees

Labels

bugSomething isn't working

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions