Ref: https://medium.com/@ahmet9417/golang-thread-pool-and-scheduler-434dd094715a

Implement the worker pool in Golang.

Task description:

Imagine you have a factory that produces a product. In this factory, you have a manager and workers also you have a production line that holds raw material and this line has capacity. If line capacity full you can not put additional raw material.

What we want to achieve:

• Manager assign jobs to workers (can specify which worker)
• Workers can say its ready and ask for jobs

• Worker send its job queue to pool, so that pool can put job directly into their queue

• Send the waitGroup from pool to worker. In worker, all worker will do Add(1) if they are running, and do Done() (count - 1) when they stopped. So that in pool we can track if workers all finished before we exit

  • In Pool, we can do .Wait() waiting for all worker (or pool itself) exit when we run Stop()

• Use channel for sending signal (e.g. quit)

  • quit: let pool/worker know I want to quit

• In the goroutine, we can use following structure, So that even if there's no job in jobQueue (goroutine keep waiting), we can still capture the quit message

for {
    select {
    case job := <- jobQueue: {...}
    case <- quit: {...}
    }
}

• Pool struct

  • property
    • jobQueue
      • a set of jobs to be done
    • readyWorkerPool
      • Pool of worker that's ready for new jobs

• Worker

  • property
    • id
    • job chan -> get from pool
    • readyWorkerPoolchan -> put itself into the pool saying that it is ready

We need to wait for workers somewhere. Use *sync.WaitGroup showing that this worker is done.