@filimonov I believe you can close this issue

Currently, the only way is to do this manually:

1. Count the number of rows in MySQL, either by doing ANALYZE TABLE ... and querying INFORMATION_SCHEMA.TABLES, or by SELECT COUNT(*) in all the tables. If you are using the materialized_mysql_tables_list setting, adjust accordingly.
2. While MaterializedMySQL is in full sync, you can monitor the progress by querying system.parts in ClickHouse: SELECT sum(rows) FROM system.parts WHERE database = 'yourdb' AND active.

@antonio2368 I believe this PR is ready to be merged. The two failing (flaky) tests look unrelated?

Support optimize_group_by_function_keys on top of QueryTree

Optimize keys of GROUPING SETS

Add tests

Support scalar subqueries cache  Implementation:

• Added a map with hash of the node (without alias) and the evaluated value to Context. Testing:
• Added a test-case with new analyser in 02174_cte_scalar_cache.sql

Fixed style check issues - Support scalar subqueries cache

save

Something working

Some tests + setting

Add GROUP BY w/o aggrecation functions analyze + test

Fix type

Fix + CROSS JOIN tests

Merge remote-tracking branch 'origin/master' into igor/remove_redundant_order_by

GROUP BY with check if aggr func depends on order

• tests

Respect stateful functions in the optimization

Merge remote-tracking branch 'origin/master' into igor/remove_redundant_order_by

Polishing

Merge remote-tracking branch 'origin/master' into igor/remove_redundant_order_by

Fix: remove ORDER BY before "mergeExpressions" optimization

When removing ORDER BY we remove corresponding expression, so to remove only ORDER BY expression, do it before "mergeExpressions" optimization is applied

Merge remote-tracking branch 'origin/master' into igor/remove_redundant_order_by

Merge branch 'master' into igor/remove_redundant_order_by

@stigsb you can actually add SETTINGS clause inside the engine config of each system table. It does require from you to manually write engine expression instead of just using predefined config keys but I still think it's better than adding a specific config field for every needed setting including storage_policy. WDYT?

I see you covered that case, but I'm still conflicted because it doesn't seem like a scalable solution. Maybe add settings field inside system table definition where keys will specify setting and the value, value of that setting?

I see your point, and I considered that. I ended up going for a separate setting because I believe that would be more appropriate to avoid exposing too many details. To illustrate my point, the log tables recently changed schema (dropped the microsecond column). Isolating this setting protects the user from that type of concern.

IMHO a bit of code complexity is acceptable if the user experience benefits.

Allow configuring storage policy for system tables. This can be useful for keeping system tables on volatile local volumes, but the main databases on EBS or similar.

Changelog category (leave one):

• Improvement

Changelog entry (a user-readable short description of the changes that goes to CHANGELOG.md):

Add <storage_policy> config parameter for system logs

Documentation entry for user-facing changes

• [X] Documentation is written (mandatory for new features)

Information about CI checks: https://clickhouse.com/docs/en/development/continuous-integration/

Add <storage_policy> config parameter for system logs