Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

RuntimeError: Event loop is closed when running dashboard on docker #737

Open
rahulkrprajapati opened this issue Aug 19, 2024 · 0 comments
Labels
bug Something isn't working

Comments

@rahulkrprajapati
Copy link

rahulkrprajapati commented Aug 19, 2024

I can't seem to run even one test successfully with uptrain running on docker. This is the error I'm getting

my JSONL has data in the following format:

{"question": "What restrictions are placed on solar energy devices to ensure they do not threaten public health or safety?", "context": "context 1:\nIf the fence is not a solid fence which blocks view of the Device, the ACC may require the Device be placed in a location behind a structure or otherwise require visual screening. The ACC may consider installation of Devices on properties without a fenced yard if there is adequate screening from public view from any street or common area.\n7.\ncontext 2:\nThe guidelines are effective upon recordation in the County, and supersede any guidelines for solar energy devices which may have previously been in effect.\ncontext 3:\nThe term includes a mechanical or chemical device that has the ability to store solar-generated energy for use in heating or cooling or in the production of power.\n2.\nSuch Devices may be installed with advance approval of the Architectural Control Committee (\u201cACC\u201d) subject to these guidelines.\n3.", "response": "Restrictions on solar energy devices include the requirement for visual screening if the device is not placed behind a solid fence that blocks the view. The Architectural Control Committee (ACC) may mandate that devices be installed in locations that are adequately screened from public view, particularly from streets or common areas. Additionally, installation of such devices requires advance approval from the ACC, ensuring compliance with established guidelines.", "retreiver_method": "auto-merging"}

The final error is something like:

2024-08-19 14:09:25 server-1     | 2024-08-19 08:39:25.877 | ERROR    | app:create_project:766 - Error running the eval: found multiple elements in the same group, please specify an aggregation function
2024-08-19 14:09:25 server-1     | Traceback (most recent call last):
2024-08-19 14:09:25 server-1     | 
2024-08-19 14:09:25 server-1     |   File "<string>", line 1, in <module>
2024-08-19 14:09:25 server-1     |   File "/usr/local/lib/python3.10/multiprocessing/spawn.py", line 116, in spawn_main
2024-08-19 14:09:25 server-1     |     exitcode = _main(fd, parent_sentinel)
2024-08-19 14:09:25 server-1     |                │     │   └ 3
2024-08-19 14:09:25 server-1     |                │     └ 14
2024-08-19 14:09:25 server-1     |<function _main at 0xffffb52c65f0>
2024-08-19 14:09:25 server-1     |   File "/usr/local/lib/python3.10/multiprocessing/spawn.py", line 129, in _main
2024-08-19 14:09:25 server-1     |     return self._bootstrap(parent_sentinel)
2024-08-19 14:09:25 server-1     |            │    │          └ 3
2024-08-19 14:09:25 server-1     |            │    └ <function BaseProcess._bootstrap at 0xffffb555d870>
2024-08-19 14:09:25 server-1     |<SpawnProcess name='SpawnProcess-2' parent=1 started>
2024-08-19 14:09:25 server-1     |   File "/usr/local/lib/python3.10/multiprocessing/process.py", line 314, in _bootstrap
2024-08-19 14:09:25 server-1     |     self.run()
2024-08-19 14:09:25 server-1     |     │    └ <function BaseProcess.run at 0xffffb555cee0>
2024-08-19 14:09:25 server-1     |<SpawnProcess name='SpawnProcess-2' parent=1 started>
2024-08-19 14:09:25 server-1     |   File "/usr/local/lib/python3.10/multiprocessing/process.py", line 108, in run
2024-08-19 14:09:25 server-1     |     self._target(*self._args, **self._kwargs)
2024-08-19 14:09:25 server-1     |     │    │        │    │        │    └ {'config': <uvicorn.config.Config object at 0xffffb5318070>, 'target': <bound method Process.target of <uvicorn.supervisors.m...
2024-08-19 14:09:25 server-1     |     │    │        │    │        └ <SpawnProcess name='SpawnProcess-2' parent=1 started>
2024-08-19 14:09:25 server-1     |     │    │        │    └ ()
2024-08-19 14:09:25 server-1     |     │    │        └ <SpawnProcess name='SpawnProcess-2' parent=1 started>
2024-08-19 14:09:25 server-1     |     │    └ <function subprocess_started at 0xffffb44a0b80>
2024-08-19 14:09:25 server-1     |<SpawnProcess name='SpawnProcess-2' parent=1 started>
2024-08-19 14:09:25 server-1     |   File "/usr/local/lib/python3.10/site-packages/uvicorn/_subprocess.py", line 80, in subprocess_started
2024-08-19 14:09:25 server-1     |     target(sockets=sockets)
2024-08-19 14:09:25 server-1     |     │              └ [<socket.socket fd=4, family=AddressFamily.AF_INET, type=SocketKind.SOCK_STREAM, proto=0, laddr=('0.0.0.0', 4300)>]
2024-08-19 14:09:25 server-1     |<bound method Process.target of <uvicorn.supervisors.multiprocess.Process object at 0xffffb53180a0>>
2024-08-19 14:09:25 server-1     |   File "/usr/local/lib/python3.10/site-packages/uvicorn/supervisors/multiprocess.py", line 63, in target
2024-08-19 14:09:25 server-1     |     return self.real_target(sockets)
2024-08-19 14:09:25 server-1     |            │    │           └ [<socket.socket fd=4, family=AddressFamily.AF_INET, type=SocketKind.SOCK_STREAM, proto=0, laddr=('0.0.0.0', 4300)>]
2024-08-19 14:09:25 server-1     |            │    └ <bound method Server.run of <uvicorn.server.Server object at 0xffffb53180d0>>
2024-08-19 14:09:25 server-1     |<uvicorn.supervisors.multiprocess.Process object at 0xffffb53180a0>
2024-08-19 14:09:25 server-1     |   File "/usr/local/lib/python3.10/site-packages/uvicorn/server.py", line 65, in run
2024-08-19 14:09:25 server-1     |     return asyncio.run(self.serve(sockets=sockets))
2024-08-19 14:09:25 server-1     |            │       │   │    │             └ [<socket.socket fd=4, family=AddressFamily.AF_INET, type=SocketKind.SOCK_STREAM, proto=0, laddr=('0.0.0.0', 4300)>]
2024-08-19 14:09:25 server-1     |            │       │   │    └ <function Server.serve at 0xffffb448fc70>
2024-08-19 14:09:25 server-1     |            │       │   └ <uvicorn.server.Server object at 0xffffb53180d0>
2024-08-19 14:09:25 server-1     |            │       └ <function run at 0xffffb50248b0>
2024-08-19 14:09:25 server-1     |<module 'asyncio' from '/usr/local/lib/python3.10/asyncio/__init__.py'>
2024-08-19 14:09:25 server-1     |   File "/usr/local/lib/python3.10/asyncio/runners.py", line 44, in run
2024-08-19 14:09:25 server-1     |     return loop.run_until_complete(main)
2024-08-19 14:09:25 server-1     |            │    │                  └ <coroutine object Server.serve at 0xffff8814d2a0>
2024-08-19 14:09:25 server-1     |            │    └ <function BaseEventLoop.run_until_complete at 0xffffb4989f30>
2024-08-19 14:09:25 server-1     |<_UnixSelectorEventLoop running=True closed=False debug=False>
2024-08-19 14:09:25 server-1     |   File "/usr/local/lib/python3.10/asyncio/base_events.py", line 636, in run_until_complete
2024-08-19 14:09:25 server-1     |     self.run_forever()
2024-08-19 14:09:25 server-1     |     │    └ <function BaseEventLoop.run_forever at 0xffffb4989ea0>
2024-08-19 14:09:25 server-1     |<_UnixSelectorEventLoop running=True closed=False debug=False>
2024-08-19 14:09:25 server-1     |   File "/usr/local/lib/python3.10/asyncio/base_events.py", line 603, in run_forever
2024-08-19 14:09:25 server-1     |     self._run_once()
2024-08-19 14:09:25 server-1     |     │    └ <function BaseEventLoop._run_once at 0xffffb498b9a0>
2024-08-19 14:09:25 server-1     |<_UnixSelectorEventLoop running=True closed=False debug=False>
2024-08-19 14:09:25 server-1     |   File "/usr/local/lib/python3.10/asyncio/base_events.py", line 1909, in _run_once
2024-08-19 14:09:25 server-1     |     handle._run()
2024-08-19 14:09:25 server-1     |     │      └ <function Handle._run at 0xffffb4927370>
2024-08-19 14:09:25 server-1     |<Handle Task.task_wakeup(<Future finis...fff81d0c430>)>)>
2024-08-19 14:09:25 server-1     |   File "/usr/local/lib/python3.10/asyncio/events.py", line 80, in _run
2024-08-19 14:09:25 server-1     |     self._context.run(self._callback, *self._args)
2024-08-19 14:09:25 server-1     |     │    │            │    │           │    └ <member '_args' of 'Handle' objects>
2024-08-19 14:09:25 server-1     |     │    │            │    │           └ <Handle Task.task_wakeup(<Future finis...fff81d0c430>)>)>
2024-08-19 14:09:25 server-1     |     │    │            │    └ <member '_callback' of 'Handle' objects>
2024-08-19 14:09:25 server-1     |     │    │            └ <Handle Task.task_wakeup(<Future finis...fff81d0c430>)>)>
2024-08-19 14:09:25 server-1     |     │    └ <member '_context' of 'Handle' objects>
2024-08-19 14:09:25 server-1     |<Handle Task.task_wakeup(<Future finis...fff81d0c430>)>)>
2024-08-19 14:09:25 server-1     |   File "/usr/local/lib/python3.10/site-packages/uvicorn/protocols/http/h11_impl.py", line 406, in run_asgi
2024-08-19 14:09:25 server-1     |     result = await app(  # type: ignore[func-returns-value]
2024-08-19 14:09:25 server-1     |<uvicorn.middleware.proxy_headers.ProxyHeadersMiddleware object at 0xffff830bcee0>
2024-08-19 14:09:25 server-1     |   File "/usr/local/lib/python3.10/site-packages/uvicorn/middleware/proxy_headers.py", line 70, in __call__
2024-08-19 14:09:25 server-1     |     return await self.app(scope, receive, send)
2024-08-19 14:09:25 server-1     |                  │    │   │      │        └ <bound method RequestResponseCycle.send of <uvicorn.protocols.http.h11_impl.RequestResponseCycle object at 0xffff827551b0>>
2024-08-19 14:09:25 server-1     |                  │    │   │      └ <bound method RequestResponseCycle.receive of <uvicorn.protocols.http.h11_impl.RequestResponseCycle object at 0xffff827551b0>>
2024-08-19 14:09:25 server-1     |                  │    │   └ {'type': 'http', 'asgi': {'version': '3.0', 'spec_version': '2.4'}, 'http_version': '1.1', 'server': ('172.20.0.3', 4300), 'c...
2024-08-19 14:09:25 server-1     |                  │    └ <fastapi.applications.FastAPI object at 0xffff830bf880>
2024-08-19 14:09:25 server-1     |<uvicorn.middleware.proxy_headers.ProxyHeadersMiddleware object at 0xffff830bcee0>
2024-08-19 14:09:25 server-1     |   File "/usr/local/lib/python3.10/site-packages/fastapi/applications.py", line 1054, in __call__
2024-08-19 14:09:25 server-1     |     await super().__call__(scope, receive, send)
2024-08-19 14:09:25 server-1     |                            │      │        └ <bound method RequestResponseCycle.send of <uvicorn.protocols.http.h11_impl.RequestResponseCycle object at 0xffff827551b0>>
2024-08-19 14:09:25 server-1     |                            │      └ <bound method RequestResponseCycle.receive of <uvicorn.protocols.http.h11_impl.RequestResponseCycle object at 0xffff827551b0>>
2024-08-19 14:09:25 server-1     |                            └ {'type': 'http', 'asgi': {'version': '3.0', 'spec_version': '2.4'}, 'http_version': '1.1', 'server': ('172.20.0.3', 4300), 'c...
2024-08-19 14:09:25 server-1     |   File "/usr/local/lib/python3.10/site-packages/starlette/applications.py", line 123, in __call__
2024-08-19 14:09:25 server-1     |     await self.middleware_stack(scope, receive, send)
2024-08-19 14:09:25 server-1     |           │    │                │      │        └ <bound method RequestResponseCycle.send of <uvicorn.protocols.http.h11_impl.RequestResponseCycle object at 0xffff827551b0>>
2024-08-19 14:09:25 server-1     |           │    │                │      └ <bound method RequestResponseCycle.receive of <uvicorn.protocols.http.h11_impl.RequestResponseCycle object at 0xffff827551b0>>
2024-08-19 14:09:25 server-1     |           │    │                └ {'type': 'http', 'asgi': {'version': '3.0', 'spec_version': '2.4'}, 'http_version': '1.1', 'server': ('172.20.0.3', 4300), 'c...
2024-08-19 14:09:25 server-1     |           │    └ <starlette.middleware.errors.ServerErrorMiddleware object at 0xffff82732c50>
2024-08-19 14:09:25 server-1     |<fastapi.applications.FastAPI object at 0xffff830bf880>
2024-08-19 14:09:25 server-1     |   File "/usr/local/lib/python3.10/site-packages/starlette/middleware/errors.py", line 164, in __call__
2024-08-19 14:09:25 server-1     |     await self.app(scope, receive, _send)
2024-08-19 14:09:25 server-1     |           │    │   │      │        └ <function ServerErrorMiddleware.__call__.<locals>._send at 0xffff827d2290>
2024-08-19 14:09:25 server-1     |           │    │   │      └ <bound method RequestResponseCycle.receive of <uvicorn.protocols.http.h11_impl.RequestResponseCycle object at 0xffff827551b0>>
2024-08-19 14:09:25 server-1     |           │    │   └ {'type': 'http', 'asgi': {'version': '3.0', 'spec_version': '2.4'}, 'http_version': '1.1', 'server': ('172.20.0.3', 4300), 'c...
2024-08-19 14:09:25 server-1     |           │    └ <starlette.middleware.cors.CORSMiddleware object at 0xffff827333a0>
2024-08-19 14:09:25 server-1     |<starlette.middleware.errors.ServerErrorMiddleware object at 0xffff82732c50>
2024-08-19 14:09:25 server-1     |   File "/usr/local/lib/python3.10/site-packages/starlette/middleware/cors.py", line 93, in __call__
2024-08-19 14:09:25 server-1     |     await self.simple_response(scope, receive, send, request_headers=headers)
2024-08-19 14:09:25 server-1     |           │    │               │      │        │                     └ Headers({'host': 'localhost:4300', 'connection': 'keep-alive', 'content-length': '329299', 'sec-ch-ua': '"Chromium";v="127", ...
2024-08-19 14:09:25 server-1     |           │    │               │      │        └ <function ServerErrorMiddleware.__call__.<locals>._send at 0xffff827d2290>
2024-08-19 14:09:25 server-1     |           │    │               │      └ <bound method RequestResponseCycle.receive of <uvicorn.protocols.http.h11_impl.RequestResponseCycle object at 0xffff827551b0>>
2024-08-19 14:09:25 server-1     |           │    │               └ {'type': 'http', 'asgi': {'version': '3.0', 'spec_version': '2.4'}, 'http_version': '1.1', 'server': ('172.20.0.3', 4300), 'c...
2024-08-19 14:09:25 server-1     |           │    └ <function CORSMiddleware.simple_response at 0xffff83ece440>
2024-08-19 14:09:25 server-1     |<starlette.middleware.cors.CORSMiddleware object at 0xffff827333a0>
2024-08-19 14:09:25 server-1     |   File "/usr/local/lib/python3.10/site-packages/starlette/middleware/cors.py", line 148, in simple_response
2024-08-19 14:09:25 server-1     |     await self.app(scope, receive, send)
2024-08-19 14:09:25 server-1     |           │    │   │      │        └ functools.partial(<bound method CORSMiddleware.send of <starlette.middleware.cors.CORSMiddleware object at 0xffff827333a0>>, ...
2024-08-19 14:09:25 server-1     |           │    │   │      └ <bound method RequestResponseCycle.receive of <uvicorn.protocols.http.h11_impl.RequestResponseCycle object at 0xffff827551b0>>
2024-08-19 14:09:25 server-1     |           │    │   └ {'type': 'http', 'asgi': {'version': '3.0', 'spec_version': '2.4'}, 'http_version': '1.1', 'server': ('172.20.0.3', 4300), 'c...
2024-08-19 14:09:25 server-1     |           │    └ <starlette.middleware.exceptions.ExceptionMiddleware object at 0xffff82732ec0>
2024-08-19 14:09:25 server-1     |<starlette.middleware.cors.CORSMiddleware object at 0xffff827333a0>
2024-08-19 14:09:25 server-1     |   File "/usr/local/lib/python3.10/site-packages/starlette/middleware/exceptions.py", line 65, in __call__
2024-08-19 14:09:25 server-1     |     await wrap_app_handling_exceptions(self.app, conn)(scope, receive, send)
2024-08-19 14:09:25 server-1     |           │                            │    │    │     │      │        └ functools.partial(<bound method CORSMiddleware.send of <starlette.middleware.cors.CORSMiddleware object at 0xffff827333a0>>, ...
2024-08-19 14:09:25 server-1     |           │                            │    │    │     │      └ <bound method RequestResponseCycle.receive of <uvicorn.protocols.http.h11_impl.RequestResponseCycle object at 0xffff827551b0>>
2024-08-19 14:09:25 server-1     |           │                            │    │    │     └ {'type': 'http', 'asgi': {'version': '3.0', 'spec_version': '2.4'}, 'http_version': '1.1', 'server': ('172.20.0.3', 4300), 'c...
2024-08-19 14:09:25 server-1     |           │                            │    │    └ <starlette.requests.Request object at 0xffff6c095570>
2024-08-19 14:09:25 server-1     |           │                            │    └ <fastapi.routing.APIRouter object at 0xffff83142b90>
2024-08-19 14:09:25 server-1     |           │                            └ <starlette.middleware.exceptions.ExceptionMiddleware object at 0xffff82732ec0>
2024-08-19 14:09:25 server-1     |<function wrap_app_handling_exceptions at 0xffff83e7e830>
2024-08-19 14:09:25 server-1     |   File "/usr/local/lib/python3.10/site-packages/starlette/_exception_handler.py", line 53, in wrapped_app
2024-08-19 14:09:25 server-1     |     await app(scope, receive, sender)
2024-08-19 14:09:25 server-1     |           │   │      │        └ <function wrap_app_handling_exceptions.<locals>.wrapped_app.<locals>.sender at 0xffff827d2320>
2024-08-19 14:09:25 server-1     |           │   │      └ <bound method RequestResponseCycle.receive of <uvicorn.protocols.http.h11_impl.RequestResponseCycle object at 0xffff827551b0>>
2024-08-19 14:09:25 server-1     |           │   └ {'type': 'http', 'asgi': {'version': '3.0', 'spec_version': '2.4'}, 'http_version': '1.1', 'server': ('172.20.0.3', 4300), 'c...
2024-08-19 14:09:25 server-1     |<fastapi.routing.APIRouter object at 0xffff83142b90>
2024-08-19 14:09:25 server-1     |   File "/usr/local/lib/python3.10/site-packages/starlette/routing.py", line 754, in __call__
2024-08-19 14:09:25 server-1     |     await self.middleware_stack(scope, receive, send)
2024-08-19 14:09:25 server-1     |           │    │                │      │        └ <function wrap_app_handling_exceptions.<locals>.wrapped_app.<locals>.sender at 0xffff827d2320>
2024-08-19 14:09:25 server-1     |           │    │                │      └ <bound method RequestResponseCycle.receive of <uvicorn.protocols.http.h11_impl.RequestResponseCycle object at 0xffff827551b0>>
2024-08-19 14:09:25 server-1     |           │    │                └ {'type': 'http', 'asgi': {'version': '3.0', 'spec_version': '2.4'}, 'http_version': '1.1', 'server': ('172.20.0.3', 4300), 'c...
2024-08-19 14:09:25 server-1     |           │    └ <bound method Router.app of <fastapi.routing.APIRouter object at 0xffff83142b90>>
2024-08-19 14:09:25 server-1     |<fastapi.routing.APIRouter object at 0xffff83142b90>
2024-08-19 14:09:25 server-1     |   File "/usr/local/lib/python3.10/site-packages/starlette/routing.py", line 774, in app
2024-08-19 14:09:25 server-1     |     await route.handle(scope, receive, send)
2024-08-19 14:09:25 server-1     |           │     │      │      │        └ <function wrap_app_handling_exceptions.<locals>.wrapped_app.<locals>.sender at 0xffff827d2320>
2024-08-19 14:09:25 server-1     |           │     │      │      └ <bound method RequestResponseCycle.receive of <uvicorn.protocols.http.h11_impl.RequestResponseCycle object at 0xffff827551b0>>
2024-08-19 14:09:25 server-1     |           │     │      └ {'type': 'http', 'asgi': {'version': '3.0', 'spec_version': '2.4'}, 'http_version': '1.1', 'server': ('172.20.0.3', 4300), 'c...
2024-08-19 14:09:25 server-1     |           │     └ <function Route.handle at 0xffff83e7fd90>
2024-08-19 14:09:25 server-1     |APIRoute(path='/api/public/create_project', name='create_project', methods=['POST'])
2024-08-19 14:09:25 server-1     |   File "/usr/local/lib/python3.10/site-packages/starlette/routing.py", line 295, in handle
2024-08-19 14:09:25 server-1     |     await self.app(scope, receive, send)
2024-08-19 14:09:25 server-1     |           │    │   │      │        └ <function wrap_app_handling_exceptions.<locals>.wrapped_app.<locals>.sender at 0xffff827d2320>
2024-08-19 14:09:25 server-1     |           │    │   │      └ <bound method RequestResponseCycle.receive of <uvicorn.protocols.http.h11_impl.RequestResponseCycle object at 0xffff827551b0>>
2024-08-19 14:09:25 server-1     |           │    │   └ {'type': 'http', 'asgi': {'version': '3.0', 'spec_version': '2.4'}, 'http_version': '1.1', 'server': ('172.20.0.3', 4300), 'c...
2024-08-19 14:09:25 server-1     |           │    └ <function request_response.<locals>.app at 0xffff830fec20>
2024-08-19 14:09:25 server-1     |APIRoute(path='/api/public/create_project', name='create_project', methods=['POST'])
2024-08-19 14:09:25 server-1     |   File "/usr/local/lib/python3.10/site-packages/starlette/routing.py", line 77, in app
2024-08-19 14:09:25 server-1     |     await wrap_app_handling_exceptions(app, request)(scope, receive, send)
2024-08-19 14:09:25 server-1     |           │                            │    │        │      │        └ <function wrap_app_handling_exceptions.<locals>.wrapped_app.<locals>.sender at 0xffff827d2320>
2024-08-19 14:09:25 server-1     |           │                            │    │        │      └ <bound method RequestResponseCycle.receive of <uvicorn.protocols.http.h11_impl.RequestResponseCycle object at 0xffff827551b0>>
2024-08-19 14:09:25 server-1     |           │                            │    │        └ {'type': 'http', 'asgi': {'version': '3.0', 'spec_version': '2.4'}, 'http_version': '1.1', 'server': ('172.20.0.3', 4300), 'c...
2024-08-19 14:09:25 server-1     |           │                            │    └ <starlette.requests.Request object at 0xffff827b8700>
2024-08-19 14:09:25 server-1     |           │                            └ <function request_response.<locals>.app.<locals>.app at 0xffff827d2170>
2024-08-19 14:09:25 server-1     |<function wrap_app_handling_exceptions at 0xffff83e7e830>
2024-08-19 14:09:25 server-1     |   File "/usr/local/lib/python3.10/site-packages/starlette/_exception_handler.py", line 53, in wrapped_app
2024-08-19 14:09:25 server-1     |     await app(scope, receive, sender)
2024-08-19 14:09:25 server-1     |           │   │      │        └ <function wrap_app_handling_exceptions.<locals>.wrapped_app.<locals>.sender at 0xffff827d2200>
2024-08-19 14:09:25 server-1     |           │   │      └ <bound method RequestResponseCycle.receive of <uvicorn.protocols.http.h11_impl.RequestResponseCycle object at 0xffff827551b0>>
2024-08-19 14:09:25 server-1     |           │   └ {'type': 'http', 'asgi': {'version': '3.0', 'spec_version': '2.4'}, 'http_version': '1.1', 'server': ('172.20.0.3', 4300), 'c...
2024-08-19 14:09:25 server-1     |<function request_response.<locals>.app.<locals>.app at 0xffff827d2170>
2024-08-19 14:09:25 server-1     |   File "/usr/local/lib/python3.10/site-packages/starlette/routing.py", line 74, in app
2024-08-19 14:09:25 server-1     |     response = await f(request)
2024-08-19 14:09:25 server-1     |                      │ └ <starlette.requests.Request object at 0xffff827b8700>
2024-08-19 14:09:25 server-1     |<function get_request_handler.<locals>.app at 0xffff830fe710>
2024-08-19 14:09:25 server-1     |   File "/usr/local/lib/python3.10/site-packages/fastapi/routing.py", line 278, in app
2024-08-19 14:09:25 server-1     |     raw_response = await run_endpoint_function(
2024-08-19 14:09:25 server-1     |<function run_endpoint_function at 0xffff83eacca0>
2024-08-19 14:09:25 server-1     |   File "/usr/local/lib/python3.10/site-packages/fastapi/routing.py", line 191, in run_endpoint_function
2024-08-19 14:09:25 server-1     |     return await dependant.call(**values)
2024-08-19 14:09:25 server-1     |                  │         │      └ {'user_id': 'a2bbbad751ce4491989a43c79edb3b49', 'db': <sqlalchemy.orm.session.Session object at 0xffff827bbfa0>, 'fsspec_fs':...
2024-08-19 14:09:25 server-1     |                  │         └ <function create_project at 0xffff830fdf30>
2024-08-19 14:09:25 server-1     |<fastapi.dependencies.models.Dependant object at 0xffff83106050>
2024-08-19 14:09:25 server-1     | 
2024-08-19 14:09:25 server-1     | > File "/app/app.py", line 761, in create_project
2024-08-19 14:09:25 server-1     |     results = user_client.evaluate_experiments(
2024-08-19 14:09:25 server-1     |               │           └ <function EvalLLM.evaluate_experiments at 0xffff88444a60>
2024-08-19 14:09:25 server-1     |<uptrain.framework.evalllm.EvalLLM object at 0xffff827b9de0>
2024-08-19 14:09:25 server-1     | 
2024-08-19 14:09:25 server-1     |   File "/usr/local/lib/python3.10/site-packages/uptrain/framework/evalllm.py", line 468, in evaluate_experiments
2024-08-19 14:09:25 server-1     |     exp_results = results.pivot(
2024-08-19 14:09:25 server-1     |                   │       └ <function DataFrame.pivot at 0xffffaefaf1c0>
2024-08-19 14:09:25 server-1     |shape: (197, 18)
2024-08-19 14:09:25 server-1     |                     ┌───────────┬───────────┬───────────┬───────────┬───┬───────────┬───────────┬───────────┬──────────┐
2024-08-19 14:09:25 server-1     |quest...
2024-08-19 14:09:25 server-1     |   File "/usr/local/lib/python3.10/site-packages/polars/_utils/deprecation.py", line 91, in wrapper
2024-08-19 14:09:25 server-1     |     return function(*args, **kwargs)
2024-08-19 14:09:25 server-1     |            │         │       └ {'values': ['score_response_conciseness', 'explanation_response_completeness', 'score_response_relevance', 'explanation_conte...
2024-08-19 14:09:25 server-1     |            │         └ (shape: (197, 18)
2024-08-19 14:09:25 server-1     |            │           ┌───────────┬───────────┬───────────┬───────────┬───┬───────────┬───────────┬───────────┬──────────┐
2024-08-19 14:09:25 server-1     |            │           │ ques...
2024-08-19 14:09:25 server-1     |<function DataFrame.pivot at 0xffffaefaf130>
2024-08-19 14:09:25 server-1     |   File "/usr/local/lib/python3.10/site-packages/polars/dataframe/frame.py", line 8032, in pivot
2024-08-19 14:09:25 server-1     |     self._df.pivot_expr(
2024-08-19 14:09:25 server-1     |     │    │   └ <method 'pivot_expr' of 'builtins.PyDataFrame' objects>
2024-08-19 14:09:25 server-1     |     │    └ <builtins.PyDataFrame object at 0xffff6d9f8690>
2024-08-19 14:09:25 server-1     |shape: (197, 18)
2024-08-19 14:09:25 server-1     |       ┌───────────┬───────────┬───────────┬───────────┬───┬───────────┬───────────┬───────────┬──────────┐
2024-08-19 14:09:25 server-1     |quest...
2024-08-19 14:09:25 server-1     | 
2024-08-19 14:09:25 server-1     | polars.exceptions.ComputeError: found multiple elements in the same group, please specify an aggregation function
2024-08-19 14:09:25 server-1     | INFO:     192.168.65.1:59106 - "POST /api/public/create_project HTTP/1.1" 500 Internal Server Error

and a whole lot of RuntimeError in between calls to openai

100%|██████████| 197/197 [01:47<00:00,  1.83it/s]
2024-08-19 13:59:28 server-1     | 2024-08-19 08:29:28.070 | WARNING  | uptrain.operators.language.llm:fetch_responses:268 - Detected a running event loop, scheduling requests in a separate thread.
2024-08-19 13:59:28 server-1     | 
  0%|          | 0/197 [00:00<?, ?it/s]Task exception was never retrieved
2024-08-19 13:59:28 server-1     | future: <Task finished name='Task-4493' coro=<AsyncClient.aclose() done, defined at /usr/local/lib/python3.10/site-packages/httpx/_client.py:2011> exception=RuntimeError('Event loop is closed')>
2024-08-19 13:59:28 server-1     | Traceback (most recent call last):
2024-08-19 13:59:28 server-1     |   File "/usr/local/lib/python3.10/site-packages/httpx/_client.py", line 2018, in aclose
2024-08-19 13:59:28 server-1     |     await self._transport.aclose()
2024-08-19 13:59:28 server-1     |   File "/usr/local/lib/python3.10/site-packages/httpx/_transports/default.py", line 385, in aclose
2024-08-19 13:59:28 server-1     |     await self._pool.aclose()
2024-08-19 13:59:28 server-1     |   File "/usr/local/lib/python3.10/site-packages/httpcore/_async/connection_pool.py", line 313, in aclose
2024-08-19 13:59:28 server-1     |     await self._close_connections(closing_connections)
2024-08-19 13:59:28 server-1     |   File "/usr/local/lib/python3.10/site-packages/httpcore/_async/connection_pool.py", line 305, in _close_connections
2024-08-19 13:59:28 server-1     |     await connection.aclose()
2024-08-19 13:59:28 server-1     |   File "/usr/local/lib/python3.10/site-packages/httpcore/_async/connection.py", line 171, in aclose
2024-08-19 13:59:28 server-1     |     await self._connection.aclose()
2024-08-19 13:59:28 server-1     |   File "/usr/local/lib/python3.10/site-packages/httpcore/_async/http11.py", line 265, in aclose
2024-08-19 13:59:28 server-1     |     await self._network_stream.aclose()
2024-08-19 13:59:28 server-1     |   File "/usr/local/lib/python3.10/site-packages/httpcore/_backends/anyio.py", line 55, in aclose
2024-08-19 13:59:28 server-1     |     await self._stream.aclose()
2024-08-19 13:59:28 server-1     |   File "/usr/local/lib/python3.10/site-packages/anyio/streams/tls.py", line 202, in aclose
2024-08-19 13:59:28 server-1     |     await self.transport_stream.aclose()
2024-08-19 13:59:28 server-1     |   File "/usr/local/lib/python3.10/site-packages/anyio/_backends/_asyncio.py", line 1202, in aclose
2024-08-19 13:59:28 server-1     |     self._transport.close()
2024-08-19 13:59:28 server-1     |   File "/usr/local/lib/python3.10/asyncio/selector_events.py", line 706, in close
2024-08-19 13:59:28 server-1     |     self._loop.call_soon(self._call_connection_lost, None)
2024-08-19 13:59:28 server-1     |   File "/usr/local/lib/python3.10/asyncio/base_events.py", line 753, in call_soon
2024-08-19 13:59:28 server-1     |     self._check_closed()
2024-08-19 13:59:28 server-1     |   File "/usr/local/lib/python3.10/asyncio/base_events.py", line 515, in _check_closed
2024-08-19 13:59:28 server-1     |     raise RuntimeError('Event loop is closed')
2024-08-19 13:59:28 server-1     | RuntimeError: Event loop is closed
2024-08-19 13:59:28 server-1     | Task exception was never retrieved
2024-08-19 13:59:28 server-1     | future: <Task finished name='Task-4494' coro=<AsyncClient.aclose() done, defined at /usr/local/lib/python3.10/site-packages/httpx/_client.py:2011> exception=RuntimeError('Event loop is closed')>
2024-08-19 13:59:28 server-1     | Traceback (most recent call last):
2024-08-19 13:59:28 server-1     |   File "/usr/local/lib/python3.10/site-packages/httpx/_client.py", line 2018, in aclose
2024-08-19 13:59:28 server-1     |     await self._transport.aclose()
2024-08-19 13:59:28 server-1     |   File "/usr/local/lib/python3.10/site-packages/httpx/_transports/default.py", line 385, in aclose
2024-08-19 13:59:28 server-1     |     await self._pool.aclose()
2024-08-19 13:59:28 server-1     |   File "/usr/local/lib/python3.10/site-packages/httpcore/_async/connection_pool.py", line 313, in aclose
2024-08-19 13:59:28 server-1     |     await self._close_connections(closing_connections)
2024-08-19 13:59:28 server-1     |   File "/usr/local/lib/python3.10/site-packages/httpcore/_async/connection_pool.py", line 305, in _close_connections
2024-08-19 13:59:28 server-1     |     await connection.aclose()
2024-08-19 13:59:28 server-1     |   File "/usr/local/lib/python3.10/site-packages/httpcore/_async/connection.py", line 171, in aclose
2024-08-19 13:59:28 server-1     |     await self._connection.aclose()
2024-08-19 13:59:28 server-1     |   File "/usr/local/lib/python3.10/site-packages/httpcore/_async/http11.py", line 265, in aclose
2024-08-19 13:59:28 server-1     |     await self._network_stream.aclose()
2024-08-19 13:59:28 server-1     |   File "/usr/local/lib/python3.10/site-packages/httpcore/_backends/anyio.py", line 55, in aclose
2024-08-19 13:59:28 server-1     |     await self._stream.aclose()
2024-08-19 13:59:28 server-1     |   File "/usr/local/lib/python3.10/site-packages/anyio/streams/tls.py", line 202, in aclose
2024-08-19 13:59:28 server-1     |     await self.transport_stream.aclose()
2024-08-19 13:59:28 server-1     |   File "/usr/local/lib/python3.10/site-packages/anyio/_backends/_asyncio.py", line 1202, in aclose
2024-08-19 13:59:28 server-1     |     self._transport.close()
2024-08-19 13:59:28 server-1     |   File "/usr/local/lib/python3.10/asyncio/selector_events.py", line 706, in close
2024-08-19 13:59:28 server-1     |     self._loop.call_soon(self._call_connection_lost, None)
2024-08-19 13:59:28 server-1     |   File "/usr/local/lib/python3.10/asyncio/base_events.py", line 753, in call_soon
2024-08-19 13:59:28 server-1     |     self._check_closed()
2024-08-19 13:59:28 server-1     |   File "/usr/local/lib/python3.10/asyncio/base_events.py", line 515, in _check_closed
2024-08-19 13:59:28 server-1     |     raise RuntimeError('Event loop is closed')
2024-08-19 13:59:28 server-1     | RuntimeError: Event loop is closed
2024-08-19 13:59:28 server-1     | Task exception was never retrieved
2024-08-19 13:59:28 server-1     | future: <Task finished name='Task-4495' coro=<AsyncClient.aclose() done, defined at /usr/local/lib/python3.10/site-packages/httpx/_client.py:2011> exception=RuntimeError('Event loop is closed')>
2024-08-19 13:59:28 server-1     | Traceback (most recent call last):
2024-08-19 13:59:28 server-1     |   File "/usr/local/lib/python3.10/site-packages/httpx/_client.py", line 2018, in aclose
2024-08-19 13:59:28 server-1     |     await self._transport.aclose()
2024-08-19 13:59:28 server-1     |   File "/usr/local/lib/python3.10/site-packages/httpx/_transports/default.py", line 385, in aclose
2024-08-19 13:59:28 server-1     |     await self._pool.aclose()
2024-08-19 13:59:28 server-1     |   File "/usr/local/lib/python3.10/site-packages/httpcore/_async/connection_pool.py", line 313, in aclose
2024-08-19 13:59:28 server-1     |     await self._close_connections(closing_connections)
2024-08-19 13:59:28 server-1     |   File "/usr/local/lib/python3.10/site-packages/httpcore/_async/connection_pool.py", line 305, in _close_connections
2024-08-19 13:59:28 server-1     |     await connection.aclose()
2024-08-19 13:59:28 server-1     |   File "/usr/local/lib/python3.10/site-packages/httpcore/_async/connection.py", line 171, in aclose
2024-08-19 13:59:28 server-1     |     await self._connection.aclose()
2024-08-19 13:59:28 server-1     |   File "/usr/local/lib/python3.10/site-packages/httpcore/_async/http11.py", line 265, in aclose
2024-08-19 13:59:28 server-1     |     await self._network_stream.aclose()
2024-08-19 13:59:28 server-1     |   File "/usr/local/lib/python3.10/site-packages/httpcore/_backends/anyio.py", line 55, in aclose
2024-08-19 13:59:28 server-1     |     await self._stream.aclose()
2024-08-19 13:59:28 server-1     |   File "/usr/local/lib/python3.10/site-packages/anyio/streams/tls.py", line 202, in aclose
2024-08-19 13:59:28 server-1     |     await self.transport_stream.aclose()
2024-08-19 13:59:28 server-1     |   File "/usr/local/lib/python3.10/site-packages/anyio/_backends/_asyncio.py", line 1202, in aclose
2024-08-19 13:59:28 server-1     |     self._transport.close()
2024-08-19 13:59:28 server-1     |   File "/usr/local/lib/python3.10/asyncio/selector_events.py", line 706, in close
2024-08-19 13:59:28 server-1     |     self._loop.call_soon(self._call_connection_lost, None)
2024-08-19 13:59:28 server-1     |   File "/usr/local/lib/python3.10/asyncio/base_events.py", line 753, in call_soon
2024-08-19 13:59:28 server-1     |     self._check_closed()
2024-08-19 13:59:28 server-1     |   File "/usr/local/lib/python3.10/asyncio/base_events.py", line 515, in _check_closed
2024-08-19 13:59:28 server-1     |     raise RuntimeError('Event loop is closed')
2024-08-19 13:59:28 server-1     | RuntimeError: Event loop is closed
@rahulkrprajapati rahulkrprajapati added the bug Something isn't working label Aug 19, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

No branches or pull requests

1 participant