Skip to content

Repository files navigation

Standalone kafka worker, to be consumed by gpdb.

Architecture

gpssfmt = UDF(kafka_in)

client(psql) ---> Master: SELECT * FROM kafka_in('...');
  /|\               |
   | grpc           |
   | Add            |
   | Start          |
   | Offset         |
   |               \|/
  \|/     grpc  / Segment (kafka_in)
worker <--------- Segment (kafka_in)
        Consume \ Segment (kafka_in)
  1. client->worker, add kafka job
  2. client->worker, start kafka job
  3. client->master, SELECT * FROM kafka_in('');
  4. segment->worker, consume kafka data

About

No description, website, or topics provided.

Resources

Stars

0 stars

Watchers

0 watching

Forks

Releases

Packages

Contributors

Languages