Implement clear parquet metadata cache command by arthurpassos · Pull Request #713 · Altinity/ClickHouse

arthurpassos · 2025-04-04T12:45:22Z

Changelog category (leave one):

New Feature

Changelog entry (a user-readable short description of the changes that goes to CHANGELOG.md):

SYSTEM DROP PARQUET METADATA CACHE command

Documentation entry for user-facing changes

altinity-robot · 2025-04-04T13:35:39Z

This is an automated comment for commit 6971379 with description of existing statuses. It's updated for the latest CI running

❌ Click here to open a full report in a separate page

Check name	Description	Status
Integration tests	The integration tests report. In parenthesis the package type is given, and in square brackets are the optional part/total tests	❌ error
Regression aarch64 Tiered Storage s3amazon	There's no description for the check yet, please add it to tests/ci/ci_config.py:CHECK_DESCRIPTIONS	❌ failure
Regression aarch64 Tiered Storage s3gcs	There's no description for the check yet, please add it to tests/ci/ci_config.py:CHECK_DESCRIPTIONS	❌ failure
Sign aarch64	There's no description for the check yet, please add it to tests/ci/ci_config.py:CHECK_DESCRIPTIONS	❌ error
Sign release	There's no description for the check yet, please add it to tests/ci/ci_config.py:CHECK_DESCRIPTIONS	❌ error
Stateless tests	Runs stateless functional tests for ClickHouse binaries built in various configurations -- release, debug, with sanitizers, etc	❌ failure
Stress test	Runs stateless functional tests concurrently from several clients to detect concurrency-related errors	❌ failure

Successful checks

Check name	Description	Status
Builds	There's no description for the check yet, please add it to tests/ci/ci_config.py:CHECK_DESCRIPTIONS	✅ success
Compatibility check	Checks that clickhouse binary runs on distributions with old libc versions. If it fails, ask a maintainer for help	✅ success
Docker keeper image	The check to build and optionally push the mentioned image to docker hub	✅ success
Docker server image	The check to build and optionally push the mentioned image to docker hub	✅ success
Install packages	Checks that the built packages are installable in a clear environment	✅ success
Ready for release	There's no description for the check yet, please add it to tests/ci/ci_config.py:CHECK_DESCRIPTIONS	✅ success
Regression aarch64 Alter move partition	There's no description for the check yet, please add it to tests/ci/ci_config.py:CHECK_DESCRIPTIONS	✅ success
Regression aarch64 Benchmark aws_s3	There's no description for the check yet, please add it to tests/ci/ci_config.py:CHECK_DESCRIPTIONS	✅ success
Regression aarch64 Benchmark gcs	There's no description for the check yet, please add it to tests/ci/ci_config.py:CHECK_DESCRIPTIONS	✅ success
Regression aarch64 Benchmark minio	There's no description for the check yet, please add it to tests/ci/ci_config.py:CHECK_DESCRIPTIONS	✅ success
Regression aarch64 LDAP authentication	There's no description for the check yet, please add it to tests/ci/ci_config.py:CHECK_DESCRIPTIONS	✅ success
Regression aarch64 LDAP external_user_directory	There's no description for the check yet, please add it to tests/ci/ci_config.py:CHECK_DESCRIPTIONS	✅ success
Regression aarch64 LDAP role_mapping	There's no description for the check yet, please add it to tests/ci/ci_config.py:CHECK_DESCRIPTIONS	✅ success
Regression aarch64 Parquet aws_s3	There's no description for the check yet, please add it to tests/ci/ci_config.py:CHECK_DESCRIPTIONS	✅ success
Regression aarch64 Parquet minio	There's no description for the check yet, please add it to tests/ci/ci_config.py:CHECK_DESCRIPTIONS	✅ success
Regression aarch64 Parquet	There's no description for the check yet, please add it to tests/ci/ci_config.py:CHECK_DESCRIPTIONS	✅ success
Regression aarch64 aes_encryption	There's no description for the check yet, please add it to tests/ci/ci_config.py:CHECK_DESCRIPTIONS	✅ success
Regression aarch64 atomic_insert	There's no description for the check yet, please add it to tests/ci/ci_config.py:CHECK_DESCRIPTIONS	✅ success
Regression aarch64 base_58	There's no description for the check yet, please add it to tests/ci/ci_config.py:CHECK_DESCRIPTIONS	✅ success
Regression aarch64 data_types	There's no description for the check yet, please add it to tests/ci/ci_config.py:CHECK_DESCRIPTIONS	✅ success
Regression aarch64 datetime64_extended_range	There's no description for the check yet, please add it to tests/ci/ci_config.py:CHECK_DESCRIPTIONS	✅ success
Regression aarch64 dns	There's no description for the check yet, please add it to tests/ci/ci_config.py:CHECK_DESCRIPTIONS	✅ success
Regression aarch64 engines	There's no description for the check yet, please add it to tests/ci/ci_config.py:CHECK_DESCRIPTIONS	✅ success
Regression aarch64 example	There's no description for the check yet, please add it to tests/ci/ci_config.py:CHECK_DESCRIPTIONS	✅ success
Regression aarch64 extended_precision_data_types	There's no description for the check yet, please add it to tests/ci/ci_config.py:CHECK_DESCRIPTIONS	✅ success
Regression aarch64 kafka	There's no description for the check yet, please add it to tests/ci/ci_config.py:CHECK_DESCRIPTIONS	✅ success
Regression aarch64 kerberos	There's no description for the check yet, please add it to tests/ci/ci_config.py:CHECK_DESCRIPTIONS	✅ success
Regression aarch64 key_value	There's no description for the check yet, please add it to tests/ci/ci_config.py:CHECK_DESCRIPTIONS	✅ success
Regression aarch64 memory	There's no description for the check yet, please add it to tests/ci/ci_config.py:CHECK_DESCRIPTIONS	✅ success
Regression aarch64 part_moves_between_shards	There's no description for the check yet, please add it to tests/ci/ci_config.py:CHECK_DESCRIPTIONS	✅ success
Regression aarch64 selects	There's no description for the check yet, please add it to tests/ci/ci_config.py:CHECK_DESCRIPTIONS	✅ success
Regression aarch64 session_timezone	There's no description for the check yet, please add it to tests/ci/ci_config.py:CHECK_DESCRIPTIONS	✅ success
Regression aarch64 window_functions	There's no description for the check yet, please add it to tests/ci/ci_config.py:CHECK_DESCRIPTIONS	✅ success
Regression release Alter move partition	There's no description for the check yet, please add it to tests/ci/ci_config.py:CHECK_DESCRIPTIONS	✅ success
Regression release Benchmark aws_s3	There's no description for the check yet, please add it to tests/ci/ci_config.py:CHECK_DESCRIPTIONS	✅ success
Regression release Benchmark gcs	There's no description for the check yet, please add it to tests/ci/ci_config.py:CHECK_DESCRIPTIONS	✅ success
Regression release Benchmark minio	There's no description for the check yet, please add it to tests/ci/ci_config.py:CHECK_DESCRIPTIONS	✅ success
Stateful tests	Runs stateful functional tests for ClickHouse binaries built in various configurations -- release, debug, with sanitizers, etc	✅ success

arthurpassos · 2025-04-06T15:36:17Z

I have added some tests, but haven't executed them locally yet. When I get home, with better internet connection to download dependencies, I'll run it

ianton-ru · 2025-04-07T14:55:11Z

src/Interpreters/InterpreterSystemQuery.cpp

+            getContext()->checkAccess(AccessType::SYSTEM_DROP_PARQUET_METADATA_CACHE);
+            ParquetFileMetaDataCache::instance()->clear();
+#endif
+            break;


NIT: May be throw exception when USE_PARQUET not defined, as for USE_EMBEDDED_COMPILER below?
(I don't think than sometimes someone assemble clickhouse without parquet support)

Enmk

LGTM

Implement clear parquet metadata cache command

it was introduced in #713

arthurpassos · 2026-02-09T14:13:16Z

Marking it as obsolete in favor of #1039

implement clear parquet metadata cache command, no tests for cluster yet

44faec6

arthurpassos changed the title ~~implement clear parquet metadata cache command, no tests for cluster yet~~ Implement clear parquet metadata cache command Apr 4, 2025

ianton-ru previously approved these changes Apr 5, 2025

View reviewed changes

add cluster tests

6971379

arthurpassos dismissed ianton-ru’s stale review via 6971379 April 6, 2025 15:35

arthurpassos added 6 commits April 7, 2025 09:32

Merge branch 'antalya' into clear_parquet_metadata_cache

e53b670

Update test.py

58f3cb1

Update test.py

0baf53d

move config dir

7f60fc2

fix minioendpoint and withminio

f612bd5

fix integ tests

32f5d58

ianton-ru previously approved these changes Apr 7, 2025

View reviewed changes

arthurpassos dismissed ianton-ru’s stale review via f0e46ab April 7, 2025 14:59

throw in case support is disabled and command is used

97ca59e

arthurpassos force-pushed the clear_parquet_metadata_cache branch from f0e46ab to 97ca59e Compare April 7, 2025 15:02

Enmk approved these changes Apr 7, 2025

View reviewed changes

Enmk merged commit 6b8543b into antalya Apr 7, 2025
169 of 264 checks passed

MyroTk added the antalya-25.2.2 Planned for 25.2.2 release label Apr 7, 2025

vzakaznikov added antalya antalya-25.2 labels May 22, 2025

arthurpassos pushed a commit that referenced this pull request May 22, 2025

Merge pull request #713 from Altinity/clear_parquet_metadata_cache

3445959

Implement clear parquet metadata cache command

Enmk mentioned this pull request May 27, 2025

25.3 Antalya port - Parquet metadata caching #795

Merged

13 tasks

Enmk added a commit that referenced this pull request Aug 7, 2025

Removed test_parquet_drop_metadata_cache

9105bb0

it was introduced in #713

svb-alt added the port-antalya PRs to be ported to all new Antalya releases label Feb 6, 2026

arthurpassos added antalya-obsolete PR exists in higher Altinity release (doesn’t need forward porting) and removed port-antalya PRs to be ported to all new Antalya releases labels Feb 9, 2026

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Implement clear parquet metadata cache command#713

Implement clear parquet metadata cache command#713
Enmk merged 9 commits intoantalyafrom
clear_parquet_metadata_cache

arthurpassos commented Apr 4, 2025

Uh oh!

altinity-robot commented Apr 4, 2025 •

edited

Loading

Uh oh!

arthurpassos commented Apr 6, 2025

Uh oh!

ianton-ru Apr 7, 2025

Uh oh!

Enmk left a comment

Uh oh!

Uh oh!

arthurpassos commented Feb 9, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

7 participants

Conversation

arthurpassos commented Apr 4, 2025

Changelog category (leave one):

Changelog entry (a user-readable short description of the changes that goes to CHANGELOG.md):

Documentation entry for user-facing changes

Uh oh!

altinity-robot commented Apr 4, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

arthurpassos commented Apr 6, 2025

Uh oh!

ianton-ru Apr 7, 2025

Choose a reason for hiding this comment

Uh oh!

Enmk left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

arthurpassos commented Feb 9, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

7 participants

altinity-robot commented Apr 4, 2025 •

edited

Loading