Are worker clusters supported?

[nodesocket]

Can I create multiple workers on separate servers and have the clients round robin tasks to the workers? Extra credit, clients should detect if a worker is down and retry the task on another worker automatically.

[iamduo]

@nodesocket Yes, multiple workers working on the same group of jobs/tasks is the natural workflow for workq. The jobs are distributed on a first come first serve basis. The workers can be on as many servers as needed, but there can only be a single workq server as it is a standalone "single node" service.

Retry is also supported per-job as needed and at a somewhat granular level. Retry is only supported on background jobs, foreground jobs are up to caller (since they control the retry attempt). Retry can be configured on the job with the following flags:

(Extracted from protocol doc)

• max-attempts - Max number of allowed attempts, default is "unlimited" up to the job's TTL. If a value is set, max is 255. An attempt is counted anytime a job is leased regardless of the outcome. This includes a job timing out due to TTR.
• max-fails - Max number of explicit job failures. Explicit job failures are not from timeouts, but from explicit workers reporting results by the fail command. Use this when you want a job to be retried on "true" worker failures.

The key concept to note though is that the client configures a job with how they want the job retried and the server will hand out the job to the workers as needed. As a concrete example, if you want a job run successfully executed within 5 seconds, but want it retried at least once in case the initial worker timed out, the add call in Go would look like this:

job := &workq.BgJob{
	ID: "6ba7b810-9dad-11d1-80b4-00c04fd430c4",
	Name: "ping",
	TTL: 60000,      // Expire the job after 60 seconds
	TTR: 5000,       // 5 second time-to-run limit for a single execution.
	Payload: []byte("ping"),
	Priority: 10,    // @OPTIONAL Numeric priority, default 0.
	MaxAttempts: 1,  // @OPTIONAL Absolute max num of attempts.
}
err := client.Add(job)
if err != nil {
	// ...
}

In the above case, if a worker picks up the job, but fails to complete, another worker will pick it up and attempt it once more. If you don't provide a MaxAttempts value, it will retry forever until the job expires via the TTL.

Hope the above example provides some clarity. Let me know if you have any more questions.

[nodesocket]

@iamduo really appreciate the awesome and detailed reply. Is there a client in Node.js by chance?

[iamduo]

@nodesocket I am not aware of a node js client at the moment. The protocol is a plain text based protocol so if you chose to, it would be on the easier side to implement. You can refer to: https://github.com/iamduo/workq/blob/master/doc/protocol.md for reference and use the official Go client as a working example. It is also not required to implement the full protocol, just the commands you need if you want to reduce the effort (e.g., "add", "lease", "complete", "fail", "result")

[iamduo]

@nodesocket If you do end up creating a client, let me know and I will list it under the README#clients section. Thanks.