Need help with EARLIEST_BY_OFFSET and GROUP BY
Confluent Community » KSQL DB
by nagababu9652
2w ago
My objective is to create an alert stream for a condition when speed is non zero and Geo-spacial data was not updating. I want to do this by comparing the two events continuously. I understood that we can’t do that directly. So, i wanted to create an stream from the main stream containing that both current record and last record or ( last two records) data for further analysis. CREATE STREAM STRGEOALERT_01 WITH (VALUE_FORMAT = ‘JSON’, RETENTION_MS = 3600000) AS SELECT deviceid, LATEST_BY_OFFSET(speed, 2) AS speed_, LATEST_BY_OFFSET(uniqueid, 2) AS uniqueid_, LATEST_BY_OFFSET(devicedatetime,2 ..read more
Visit website
Eliminating all but last CDC record for a topic within 2 sec window
Confluent Community » KSQL DB
by tim.jones.001
2w ago
We have a legacy source SQL Server DB that produces multiple CDC records for each table update due to post-insert/update triggers. We are using debezium to capture the SQL Server transactions and can see how we could filter all but the first within a couple of seconds in a SMT, but not the last. Rather than processing each record, we would like to generate a filtered topic with only the last record of the series, on which to do further processing. We can see how we can do this manually by reading the topic with its duplicates, and caching records for a second and only producing a filtered reco ..read more
Visit website
Need help with Tumbling Window behaviour
Confluent Community » KSQL DB
by nagababu9652
3w ago
I have a stream with 50 partitions. I Created an Tumbling window Table upon it. Suppose window size is 1 hour and at present running from 1 am to 2 am. A) What happens when the incorrect data with future timestamp (like 5am) comes and B) same for past data (yesterday) comes when window was created before and finished window not created for past data. 1 post - 1 participant Read full topic ..read more
Visit website
جلب الحبيب 254 39 270 002010
Confluent Community » KSQL DB
by sdggvhh
3w ago
Sfgjggjj dhjjv fgjkvhh 1 post - 1 participant Read full topic ..read more
Visit website
KsqlDB Table-Table join on STRUCT Type Primary Key
Confluent Community » KSQL DB
by ghosh.sayak
3w ago
We have 2 compact topics (C_USERS [key - AID] and A_USERS [key - account_name]) both having AVRO Key & Value Schema with “record” type, and we want to do left join, take some attributes from A_USERS topic and put the data to target compacted topic (USERS_ENRICHED [key - AID]) And I am following this thread as reference. Please Note: The “A_USERS” topic has been produced by a Transactional Producer. Please find the steps we performed: A) For Topic: C_USERS CREATE OR REPLACE STREAM DEV1_KSTREAM_C_USERS WITH (KAFKA_TOPIC='dev-1-C_USERS', FORMAT='AVRO', VALUE_SCHEMA_ID=xxx, KEY_SCHEMA_ID=xx ..read more
Visit website
Data Discrepancy in Tumbling Windowed Table
Confluent Community » KSQL DB
by nagababu9652
3w ago
Describe the bug We are currently facing a significant discrepancy in the results when creating a windowed table from a stream of IoT data in our ksqlDB setup. The observed output significantly differs from the expected results. The stream includes a devicedatetime field, which we have designated as the timestamp field for creating windows. CREATE STREAM STRTRACKERDATA_TRIM WITH (KAFKA_TOPIC=‘strtrackerdata_trim’, PARTITIONS=50, REPLICAS=1, RETENTION_MS=3600000, TIMESTAMP=‘devicedatetime’, VALUE_FORMAT=‘JSON’) AS SELECT DEVICEID, REASON, TENANTGROUPUID, RMSEVENTTYPE, DEVICEDATETIME, LATITUDE ..read more
Visit website
KSQL-Kafka Stream/tables
Confluent Community » KSQL DB
by Dinesh
3w ago
I have Kafka streams as follows: topic1: event example in Avro: { “ID”: {“string”:“S1”}, “text”:{“string”:“department”} } { “ID”: {“string”:“S2”}, “text”:{“string”:“minimum”} } { “ID”: {“string”:“S3”}, “text”:{“string”:“maximum”} } { “ID”: {“string”:“S4”}, “text”:{“string”:“telephone”} } stream1 which is a reference has details on any abbreviations built on topic1: stream1 definition: full_name string example values: —±---- |ID|text| —±---- |S1|department |S2|cellphone |S3|fax |S4|telephone topic2: event example in Avro: { “ID”: {“string”:“S1”}, “value”:{“string”:“department”} } { “ID”: {“stri ..read more
Visit website
Ksqldb not starting
Confluent Community » KSQL DB
by iomari
1M ago
Greetings, I have a working kafka installation on my laptop. I installed confluent-ksqldb-0.29.0 into /opt in Ubuntu Linux. When I execute: “sudo /opt/confluent/bin/ksql-server-start /opt/confluent/etc/ksqldb/ksql-server.properties” I get the following error: [2023-09-25 10:00:14,305] INFO KsqlConfig values: ksql.access.validator.enable = auto ksql.assert.schema.default.timeout.ms = 1000 ksql.assert.topic.default.timeout.ms = 1000 ksql.authorization.cache.expiry.time.secs = 30 ksql.authorization.cache.max.entries = 10000 ksql.cast.strings.prese ..read more
Visit website
Ksqldb how to join with multiple condition
Confluent Community » KSQL DB
by patharapon
1M ago
Hi, how to join multiple Condition on ksqlDB Example data Create stream C as select a.id1, a.id2, a.username, a.name, a.birthdate, b.id1, b.id2, b.username, b.lastname From a Join b **on** a.id1 = b.id1 **AND** a.username = b.username **OR** a.id2 = b.id2; 1 post - 1 participant Read full topic ..read more
Visit website
KSQLDB UDF configuration in CFK
Confluent Community » KSQL DB
by shubhamshirur1
1M ago
Hello, I have created an UDF for ksqlDB. Dev environment is deployed in EKS which is a CFK. I want to configure this UDF in the same cluster. Can you please help me documentation explaining how to configure UDF is CFK for ksqldb? Thanks, Shubham 1 post - 1 participant Read full topic ..read more
Visit website

Follow Confluent Community » KSQL DB on FeedSpot

Continue with Google
Continue with Apple
OR