ClickHouse-Kafka引擎，如果遇到Kafka消息源增加列，怎么办
ClickHouse-Kafka引擎，如果遇到Kafka消息源增加列，怎么办。ClickHouse-Kafka引擎，消息存储表新增列，更改消息源的结构。
基于：
一、删除Kafka消息表的使用
|
1 |
DROP TABLE cppla.kafka_readings_queue; |
二、消费存储表增加新列
|
1 |
ALTER TABLE cppla.kafka_readings ADD COLUMN name String; |
三、创建新的新增列的Kafka消费表
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 |
CREATE TABLE cppla.kafka_readings_queue ( dd_by_cppla String, msg String, platform String, data String, dt DateTime, name String ) ENGINE = Kafka SETTINGS kafka_broker_list = '172.21.0.5:49153,172.21.0.5:49154,172.21.0.5:49155', kafka_topic_list = 'test_2', kafka_group_name = 'consumer_group2', kafka_format = 'JSONEachRow', kafka_skip_broken_messages = 20000, kafka_num_consumers = 1; |
四、删除旧视图表并重新构建新视图表
|
1 2 3 4 5 6 7 |
# 删除旧物化视图表 DROP TABLE kafka_readings_view; # 新增新的物化视图表 CREATE MATERIALIZED VIEW cppla.kafka_readings_view TO cppla.kafka_readings AS SELECT dd_by_cppla, msg, platform, data, dt, name FROM cppla.kafka_readings_queue; |
五、插入新消息测试 并 查看
|
1 2 3 4 5 6 7 8 |
# 插入Kafka消息 {"dd_by_cppla":"2021-08-19 15:49:07","msg":"DD","platform":"DDS","data":"mmm-mmm-mmm","dt":"2021-08-19 15:48:05"} {"dd_by_cppla":"2021-08-19 15:49:07","msg":"DD","platform":"DDS","data":"nnn-nnnn-nnn","dt":"2021-08-19 15:48:07"} {"dd_by_cppla":"2021-08-19 15:49:07","msg":"DD","platform":"DDS","data":"mmm-mmm-mmm","dt":"2021-08-19 15:48:06","name":"cpp.la"} {"dd_by_cppla":"2021-08-19 15:49:07","msg":"DD","platform":"DDS","data":"nnn-nnnn-nnn","dt":"2021-08-19 15:48:16","name":"cpp.la"} # 查看新数据 select * from cppla.kafka_readings; |
《ClickHouse-Kafka引擎，如果遇到Kafka消息源增加列，怎么办》，有参阅ClickHouse和相关文献，by: cpp.la