Streaming throwing queue.get() error

[rtalaricw]

Description

Dockerfile: faster_transformer(v1.2)
Model: GPT-J

Reproduced Steps

The streaming example in issue_requests.py throws the following error when passing in a request:

Traceback (most recent call last):
  File "/usr/lib/python3.8/multiprocessing/process.py", line 315, in _bootstrap
    self.run()
  File "/usr/lib/python3.8/multiprocessing/process.py", line 108, in run
    self._target(*self._args, **self._kwargs)
  File "/home/rtalari/coreweave/kubernetes-cloud/online-inference/fastertransformer/client/example_http.py", line 94, in stream_consumer
    result = queue.get()
  File "/usr/lib/python3.8/multiprocessing/queues.py", line 116, in get
    return _ForkingPickler.loads(res)
TypeError: __init__() missing 1 required positional argument: 'msg'

A simple call to a sample request is passed which works with HTTP with output for "She is a good girl":

Ouput: She is a good girl. She's a good girl. She's a good girl...'

'But you were in bed when we came in,' she says, and I

[byshiue]

I don't understand your question clearly, but the problem may be because issue_request.py needs to receive a request like requests/sample_request.json.

If this is not your problem, please provide more information and details. Please provide your end-to-end workflow step by step to help us understand your questions, thank you.

[shanekong]

i meet the same promblem, by running python3 tools/issue_request.py tools/requests/sample_request_stream.json

Process Process-1:
Traceback (most recent call last):
File "/data/kongxiangxing/env/miniconda3/envs/python37/lib/python3.7/multiprocessing/process.py", line 297, in _bootstrap
self.run()
File "/data/kongxiangxing/env/miniconda3/envs/python37/lib/python3.7/multiprocessing/process.py", line 99, in run
self._target(*self._args, **self._kwargs)
File "tools/issue_request.py", line 110, in stream_consumer
result = queue.get()
File "/data/kongxiangxing/env/miniconda3/envs/python37/lib/python3.7/multiprocessing/queues.py", line 113, in get
return _ForkingPickler.loads(res)
TypeError: init() missing 1 required positional argument: 'msg'

can anyone give any suggestions?