Overview : About Kafka


-          What is Kafka - Kafka is a cluster.
-          What is Cluster - It’s a group of system
-          What all are the component inside Kafka - Kafka Broker and Zookeeper
-          What is Zookeeper - Zookeeper is used to balance the load. Ex : If primary node server is down and it will automatically move it into secondary node… So, the application will not get impact.
-          What is Kafka Broker - It will help to pass the message (data) into consumer.

Overall flow is:

Multiple producer will feed the data into Kafka cluster and multiple consumer will get the data from Kafka and will move it into data center.

The data will be moved based on below identity.

1.       Producer + Topic + Offset

Example : Producer is Shopname + Topic is Sales + Offset (sequence data )

Shop Name = 101
Sales = 201
Offset = 301

The consumer will receive the data like 101201301 and will feed it into datacenter.

Note: Kafka cluster can able to candle more than one producer and consumer, the consumer will get the data based upon kafka broker partition.

Comments

Post a Comment

Popular posts from this blog

General information about Netezza : Default Error details

General information about Netezza : Default Error details     There are around 140 default error details available in Netezza. For your reference please find below details. ERRORCODE ERRORTEXT 1 Operation failed 2 Interaction aborted 3 Statement aborted 4 Could not serialize - transaction aborted 5 Transaction no longer active 6 Out of memory 7 Incorrect data type 8 Illegal value 9 Internal: SPUs crosstalking 10 Internal pipe I/O failed 11 Cannot determine hostname 12 Rogue SPU trying to register 13 No such table exists 14 No such field exists in table 15 Table already exists 16 Plan already exists in plan dictionary 17 F...
Read more

Minus in Teradata?...

Minus : It will provide the result which one is not matching from both tables... Note: It will provide the not matching result from first select table only. Select Emp_No from Emp Minus Select Emp_no from Emp_Dep;
Read more