(APIServer pid=814) INFO 09-19 01:23:06 [api_utils.py:286] non-default args: {'model_tag': '/workspace/prisma/modelos/Qwen3.8-27B-FP8', 'enable_auto_tool_choice': True, 'tool_call_parser': 'qwen3_xml', 'enable_log_outputs': True, 'host': '127.0.0.1', 'model': '/workspace/prisma/modelos/Qwen3.8-27B-FP8', 'max_model_len': 65536, 'served_model_name': ['qwen3.8-27b'], 'reasoning_parser': 'qwen3', 'ena
(EngineCore pid=1550) INFO 09-19 01:26:38 [default_loader.py:430] Loading weights took 90.77 seconds
(EngineCore pid=1550) INFO 09-19 01:26:39 [model_runner.py:404] Model loading took 28.51 GiB memory and 97.318091 seconds
(EngineCore pid=1550) INFO 09-19 01:27:56 [monitor.py:53] torch.compile took 50.24 s in total
(EngineCore pid=1550) INFO 09-19 01:29:35 [gpu_worker.py:625] Available KV cache memory: 54.86 GiB
(EngineCore pid=1550) INFO 09-19 01:29:35 [kv_cache_utils.py:2032] GPU KV cache size: 834,491 tokens, Maximum concurrency for 65,536 tokens per request: 12.73x
(APIServer pid=814) INFO 09-19 01:30:41 [entry.py:139] Starting vLLM server on http://127.0.0.1:8000
(APIServer pid=814) INFO:     Application startup complete.
(APIServer pid=814) INFO:     127.0.0.1:58552 - "POST /v1/chat/completions HTTP/1.1" 200 OK
(APIServer pid=814) INFO:     127.0.0.1:58552 - "POST /v1/chat/completions HTTP/1.1" 200 OK
(APIServer pid=814) INFO 09-19 01:32:12 [loggers.py:310] Engine 000: Avg prompt throughput: 972.2 tokens/s, Avg generation throughput: 5.4 tokens/s, Running: 0 reqs, Waiting: 0 reqs, GPU KV cache usage: 0.0%, Prefix cache hit rate: 28.7%
(APIServer pid=814) INFO 09-19 01:32:22 [loggers.py:310] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Waiting: 0 reqs, GPU KV cache usage: 0.0%, Prefix cache hit rate: 28.7%
(APIServer pid=814) INFO:     127.0.0.1:47634 - "POST /v1/chat/completions HTTP/1.1" 200 OK
(APIServer pid=814) INFO:     127.0.0.1:47634 - "POST /v1/chat/completions HTTP/1.1" 200 OK
(APIServer pid=814) INFO 09-19 01:32:32 [loggers.py:310] Engine 000: Avg prompt throughput: 578.9 tokens/s, Avg generation throughput: 36.1 tokens/s, Running: 1 reqs, Waiting: 0 reqs, GPU KV cache usage: 1.7%, Prefix cache hit rate: 43.1%
(APIServer pid=814) INFO 09-19 01:32:42 [loggers.py:310] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 45.1 tokens/s, Running: 1 reqs, Waiting: 0 reqs, GPU KV cache usage: 1.7%, Prefix cache hit rate: 43.1%
(APIServer pid=814) INFO 09-19 01:32:52 [loggers.py:310] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 45.2 tokens/s, Running: 1 reqs, Waiting: 0 reqs, GPU KV cache usage: 1.7%, Prefix cache hit rate: 43.1%
(APIServer pid=814) INFO:     127.0.0.1:47634 - "POST /v1/chat/completions HTTP/1.1" 200 OK
(APIServer pid=814) INFO:     127.0.0.1:47634 - "POST /v1/chat/completions HTTP/1.1" 200 OK
(APIServer pid=814) INFO 09-19 01:33:02 [loggers.py:310] Engine 000: Avg prompt throughput: 1198.7 tokens/s, Avg generation throughput: 30.1 tokens/s, Running: 1 reqs, Waiting: 0 reqs, GPU KV cache usage: 2.6%, Prefix cache hit rate: 43.5%
(APIServer pid=814) INFO 09-19 01:33:12 [loggers.py:310] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 44.9 tokens/s, Running: 1 reqs, Waiting: 0 reqs, GPU KV cache usage: 2.6%, Prefix cache hit rate: 43.5%
(APIServer pid=814) INFO:     127.0.0.1:47634 - "POST /v1/chat/completions HTTP/1.1" 200 OK
(APIServer pid=814) INFO 09-19 01:33:22 [loggers.py:310] Engine 000: Avg prompt throughput: 84.4 tokens/s, Avg generation throughput: 44.2 tokens/s, Running: 1 reqs, Waiting: 0 reqs, GPU KV cache usage: 1.0%, Prefix cache hit rate: 48.5%
(APIServer pid=814) INFO:     127.0.0.1:47634 - "POST /v1/chat/completions HTTP/1.1" 200 OK
(APIServer pid=814) INFO:     127.0.0.1:47634 - "POST /v1/chat/completions HTTP/1.1" 200 OK
(APIServer pid=814) INFO 09-19 01:33:32 [loggers.py:310] Engine 000: Avg prompt throughput: 583.9 tokens/s, Avg generation throughput: 40.8 tokens/s, Running: 1 reqs, Waiting: 0 reqs, GPU KV cache usage: 1.8%, Prefix cache hit rate: 53.4%
(APIServer pid=814) INFO:     127.0.0.1:47634 - "POST /v1/chat/completions HTTP/1.1" 200 OK
(APIServer pid=814) INFO 09-19 01:33:42 [loggers.py:310] Engine 000: Avg prompt throughput: 90.5 tokens/s, Avg generation throughput: 44.3 tokens/s, Running: 1 reqs, Waiting: 0 reqs, GPU KV cache usage: 1.1%, Prefix cache hit rate: 56.4%
(APIServer pid=814) INFO:     127.0.0.1:47634 - "POST /v1/chat/completions HTTP/1.1" 200 OK
(APIServer pid=814) INFO 09-19 01:33:52 [loggers.py:310] Engine 000: Avg prompt throughput: 116.1 tokens/s, Avg generation throughput: 38.4 tokens/s, Running: 0 reqs, Waiting: 0 reqs, GPU KV cache usage: 0.0%, Prefix cache hit rate: 59.2%
(APIServer pid=814) INFO 09-19 01:34:02 [loggers.py:310] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Waiting: 0 reqs, GPU KV cache usage: 0.0%, Prefix cache hit rate: 59.2%
(APIServer pid=814) INFO:     127.0.0.1:56298 - "POST /v1/chat/completions HTTP/1.1" 200 OK
(APIServer pid=814) INFO:     127.0.0.1:56298 - "POST /v1/chat/completions HTTP/1.1" 200 OK
(APIServer pid=814) INFO 09-19 01:35:02 [loggers.py:310] Engine 000: Avg prompt throughput: 555.7 tokens/s, Avg generation throughput: 22.3 tokens/s, Running: 1 reqs, Waiting: 0 reqs, GPU KV cache usage: 1.6%, Prefix cache hit rate: 59.1%
(APIServer pid=814) INFO:     127.0.0.1:56298 - "POST /v1/chat/completions HTTP/1.1" 200 OK
(APIServer pid=814) INFO:     127.0.0.1:56298 - "POST /v1/chat/completions HTTP/1.1" 200 OK
(APIServer pid=814) INFO 09-19 01:35:12 [loggers.py:310] Engine 000: Avg prompt throughput: 556.1 tokens/s, Avg generation throughput: 41.2 tokens/s, Running: 1 reqs, Waiting: 0 reqs, GPU KV cache usage: 1.6%, Prefix cache hit rate: 59.0%
(APIServer pid=814) INFO:     127.0.0.1:56298 - "POST /v1/chat/completions HTTP/1.1" 200 OK
(APIServer pid=814) INFO:     127.0.0.1:56298 - "POST /v1/chat/completions HTTP/1.1" 200 OK
(APIServer pid=814) INFO 09-19 01:35:22 [loggers.py:310] Engine 000: Avg prompt throughput: 375.3 tokens/s, Avg generation throughput: 42.1 tokens/s, Running: 1 reqs, Waiting: 0 reqs, GPU KV cache usage: 1.4%, Prefix cache hit rate: 59.8%
(APIServer pid=814) INFO:     127.0.0.1:56298 - "POST /v1/chat/completions HTTP/1.1" 200 OK
(APIServer pid=814) INFO:     127.0.0.1:56298 - "POST /v1/chat/completions HTTP/1.1" 200 OK
(APIServer pid=814) INFO 09-19 01:35:32 [loggers.py:310] Engine 000: Avg prompt throughput: 375.7 tokens/s, Avg generation throughput: 28.7 tokens/s, Running: 1 reqs, Waiting: 0 reqs, GPU KV cache usage: 1.4%, Prefix cache hit rate: 60.5%
(APIServer pid=814) INFO:     127.0.0.1:56298 - "POST /v1/chat/completions HTTP/1.1" 200 OK
(APIServer pid=814) INFO:     127.0.0.1:56298 - "POST /v1/chat/completions HTTP/1.1" 200 OK
(APIServer pid=814) INFO 09-19 01:35:42 [loggers.py:310] Engine 000: Avg prompt throughput: 41.9 tokens/s, Avg generation throughput: 28.9 tokens/s, Running: 0 reqs, Waiting: 0 reqs, GPU KV cache usage: 0.0%, Prefix cache hit rate: 61.4%
(APIServer pid=814) INFO:     127.0.0.1:56298 - "POST /v1/chat/completions HTTP/1.1" 200 OK
(APIServer pid=814) INFO 09-19 01:35:52 [loggers.py:310] Engine 000: Avg prompt throughput: 442.6 tokens/s, Avg generation throughput: 38.6 tokens/s, Running: 1 reqs, Waiting: 0 reqs, GPU KV cache usage: 0.8%, Prefix cache hit rate: 61.6%
(APIServer pid=814) INFO:     127.0.0.1:56298 - "POST /v1/chat/completions HTTP/1.1" 200 OK
(APIServer pid=814) INFO 09-19 01:36:02 [loggers.py:310] Engine 000: Avg prompt throughput: 400.7 tokens/s, Avg generation throughput: 25.0 tokens/s, Running: 0 reqs, Waiting: 0 reqs, GPU KV cache usage: 0.0%, Prefix cache hit rate: 61.0%
(APIServer pid=814) INFO:     127.0.0.1:56298 - "POST /v1/chat/completions HTTP/1.1" 200 OK
(APIServer pid=814) INFO:     127.0.0.1:56298 - "POST /v1/chat/completions HTTP/1.1" 200 OK
(APIServer pid=814) INFO:     127.0.0.1:56298 - "POST /v1/chat/completions HTTP/1.1" 200 OK
(APIServer pid=814) INFO 09-19 01:36:12 [loggers.py:310] Engine 000: Avg prompt throughput: 246.8 tokens/s, Avg generation throughput: 31.1 tokens/s, Running: 0 reqs, Waiting: 0 reqs, GPU KV cache usage: 0.0%, Prefix cache hit rate: 62.9%
(APIServer pid=814) INFO:     127.0.0.1:56298 - "POST /v1/chat/completions HTTP/1.1" 200 OK
(APIServer pid=814) INFO:     127.0.0.1:56298 - "POST /v1/chat/completions HTTP/1.1" 200 OK
(APIServer pid=814) INFO:     127.0.0.1:56298 - "POST /v1/chat/completions HTTP/1.1" 200 OK
(APIServer pid=814) INFO 09-19 01:36:22 [loggers.py:310] Engine 000: Avg prompt throughput: 242.0 tokens/s, Avg generation throughput: 33.6 tokens/s, Running: 0 reqs, Waiting: 0 reqs, GPU KV cache usage: 0.0%, Prefix cache hit rate: 64.5%
(APIServer pid=814) INFO:     127.0.0.1:56298 - "POST /v1/chat/completions HTTP/1.1" 200 OK
(APIServer pid=814) INFO:     127.0.0.1:56298 - "POST /v1/chat/completions HTTP/1.1" 200 OK
(APIServer pid=814) INFO 09-19 01:36:32 [loggers.py:310] Engine 000: Avg prompt throughput: 1023.4 tokens/s, Avg generation throughput: 27.4 tokens/s, Running: 1 reqs, Waiting: 0 reqs, GPU KV cache usage: 2.4%, Prefix cache hit rate: 62.7%
(APIServer pid=814) INFO:     127.0.0.1:56298 - "POST /v1/chat/completions HTTP/1.1" 200 OK
(APIServer pid=814) INFO:     127.0.0.1:56298 - "POST /v1/chat/completions HTTP/1.1" 200 OK
(APIServer pid=814) INFO 09-19 01:36:42 [loggers.py:310] Engine 000: Avg prompt throughput: 1023.8 tokens/s, Avg generation throughput: 30.3 tokens/s, Running: 1 reqs, Waiting: 0 reqs, GPU KV cache usage: 2.4%, Prefix cache hit rate: 61.2%
(APIServer pid=814) INFO:     127.0.0.1:56298 - "POST /v1/chat/completions HTTP/1.1" 200 OK
(APIServer pid=814) INFO:     127.0.0.1:56298 - "POST /v1/chat/completions HTTP/1.1" 200 OK
(APIServer pid=814) INFO 09-19 01:36:52 [loggers.py:310] Engine 000: Avg prompt throughput: 536.4 tokens/s, Avg generation throughput: 30.0 tokens/s, Running: 1 reqs, Waiting: 0 reqs, GPU KV cache usage: 1.6%, Prefix cache hit rate: 61.1%
(APIServer pid=814) INFO:     127.0.0.1:56298 - "POST /v1/chat/completions HTTP/1.1" 200 OK
(APIServer pid=814) INFO:     127.0.0.1:56298 - "POST /v1/chat/completions HTTP/1.1" 200 OK
(APIServer pid=814) INFO 09-19 01:37:02 [loggers.py:310] Engine 000: Avg prompt throughput: 783.8 tokens/s, Avg generation throughput: 19.7 tokens/s, Running: 1 reqs, Waiting: 0 reqs, GPU KV cache usage: 1.8%, Prefix cache hit rate: 60.4%
(APIServer pid=814) INFO:     127.0.0.1:56298 - "POST /v1/chat/completions HTTP/1.1" 200 OK
(APIServer pid=814) INFO:     127.0.0.1:56298 - "POST /v1/chat/completions HTTP/1.1" 200 OK
(APIServer pid=814) INFO 09-19 01:37:12 [loggers.py:310] Engine 000: Avg prompt throughput: 443.8 tokens/s, Avg generation throughput: 30.9 tokens/s, Running: 1 reqs, Waiting: 0 reqs, GPU KV cache usage: 1.5%, Prefix cache hit rate: 60.6%
(APIServer pid=814) INFO:     127.0.0.1:56298 - "POST /v1/chat/completions HTTP/1.1" 200 OK
(APIServer pid=814) INFO:     127.0.0.1:56298 - "POST /v1/chat/completions HTTP/1.1" 200 OK
(APIServer pid=814) INFO:     127.0.0.1:56298 - "POST /v1/chat/completions HTTP/1.1" 200 OK
(APIServer pid=814) INFO 09-19 01:37:22 [loggers.py:310] Engine 000: Avg prompt throughput: 153.6 tokens/s, Avg generation throughput: 31.5 tokens/s, Running: 0 reqs, Waiting: 0 reqs, GPU KV cache usage: 0.0%, Prefix cache hit rate: 61.3%
(APIServer pid=814) INFO:     127.0.0.1:56298 - "POST /v1/chat/completions HTTP/1.1" 200 OK
(APIServer pid=814) INFO:     127.0.0.1:56298 - "POST /v1/chat/completions HTTP/1.1" 200 OK
(APIServer pid=814) INFO 09-19 01:37:32 [loggers.py:310] Engine 000: Avg prompt throughput: 188.6 tokens/s, Avg generation throughput: 40.1 tokens/s, Running: 0 reqs, Waiting: 0 reqs, GPU KV cache usage: 0.0%, Prefix cache hit rate: 62.1%
(APIServer pid=814) INFO:     127.0.0.1:56298 - "POST /v1/chat/completions HTTP/1.1" 200 OK
(APIServer pid=814) INFO:     127.0.0.1:56298 - "POST /v1/chat/completions HTTP/1.1" 200 OK
(APIServer pid=814) INFO:     127.0.0.1:56298 - "POST /v1/chat/completions HTTP/1.1" 200 OK
(APIServer pid=814) INFO 09-19 01:37:42 [loggers.py:310] Engine 000: Avg prompt throughput: 1878.5 tokens/s, Avg generation throughput: 32.4 tokens/s, Running: 1 reqs, Waiting: 0 reqs, GPU KV cache usage: 2.7%, Prefix cache hit rate: 64.4%
(APIServer pid=814) INFO:     127.0.0.1:56298 - "POST /v1/chat/completions HTTP/1.1" 200 OK
(APIServer pid=814) INFO 09-19 01:37:52 [loggers.py:310] Engine 000: Avg prompt throughput: 84.1 tokens/s, Avg generation throughput: 42.8 tokens/s, Running: 1 reqs, Waiting: 0 reqs, GPU KV cache usage: 2.8%, Prefix cache hit rate: 66.2%
(APIServer pid=814) INFO 09-19 01:38:02 [loggers.py:310] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 44.3 tokens/s, Running: 1 reqs, Waiting: 0 reqs, GPU KV cache usage: 2.9%, Prefix cache hit rate: 66.2%
(APIServer pid=814) INFO:     127.0.0.1:56298 - "POST /v1/chat/completions HTTP/1.1" 200 OK
(APIServer pid=814) INFO:     127.0.0.1:56298 - "POST /v1/chat/completions HTTP/1.1" 200 OK
(APIServer pid=814) INFO 09-19 01:38:12 [loggers.py:310] Engine 000: Avg prompt throughput: 462.0 tokens/s, Avg generation throughput: 41.2 tokens/s, Running: 1 reqs, Waiting: 0 reqs, GPU KV cache usage: 1.5%, Prefix cache hit rate: 66.1%
(APIServer pid=814) INFO:     127.0.0.1:56298 - "POST /v1/chat/completions HTTP/1.1" 200 OK
(APIServer pid=814) INFO:     127.0.0.1:56298 - "POST /v1/chat/completions HTTP/1.1" 200 OK
(APIServer pid=814) INFO 09-19 01:38:22 [loggers.py:310] Engine 000: Avg prompt throughput: 292.6 tokens/s, Avg generation throughput: 37.6 tokens/s, Running: 1 reqs, Waiting: 0 reqs, GPU KV cache usage: 1.3%, Prefix cache hit rate: 66.3%
(APIServer pid=814) INFO 09-19 01:38:32 [loggers.py:310] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 14.3 tokens/s, Running: 0 reqs, Waiting: 0 reqs, GPU KV cache usage: 0.0%, Prefix cache hit rate: 66.3%
(APIServer pid=814) INFO 09-19 01:38:42 [loggers.py:310] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Waiting: 0 reqs, GPU KV cache usage: 0.0%, Prefix cache hit rate: 66.3%
(APIServer pid=814) INFO:     127.0.0.1:39220 - "POST /v1/chat/completions HTTP/1.1" 200 OK
(APIServer pid=814) INFO 09-19 01:38:52 [loggers.py:310] Engine 000: Avg prompt throughput: 170.8 tokens/s, Avg generation throughput: 3.2 tokens/s, Running: 0 reqs, Waiting: 0 reqs, GPU KV cache usage: 0.0%, Prefix cache hit rate: 66.0%, MM cache hit rate: 0.0%
(APIServer pid=814) INFO 09-19 01:39:02 [loggers.py:310] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Waiting: 0 reqs, GPU KV cache usage: 0.0%, Prefix cache hit rate: 66.0%, MM cache hit rate: 0.0%
(APIServer pid=814) INFO:     127.0.0.1:39352 - "POST /v1/chat/completions HTTP/1.1" 200 OK
(APIServer pid=814) INFO:     127.0.0.1:39352 - "POST /v1/chat/completions HTTP/1.1" 200 OK
(APIServer pid=814) INFO 09-19 01:39:12 [loggers.py:310] Engine 000: Avg prompt throughput: 108.6 tokens/s, Avg generation throughput: 19.7 tokens/s, Running: 1 reqs, Waiting: 0 reqs, GPU KV cache usage: 1.4%, Prefix cache hit rate: 66.8%, MM cache hit rate: 0.0%
(APIServer pid=814) INFO 09-19 01:39:22 [loggers.py:310] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 45.2 tokens/s, Running: 1 reqs, Waiting: 0 reqs, GPU KV cache usage: 1.4%, Prefix cache hit rate: 66.8%, MM cache hit rate: 0.0%
(APIServer pid=814) INFO 09-19 01:39:32 [loggers.py:310] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 45.2 tokens/s, Running: 1 reqs, Waiting: 0 reqs, GPU KV cache usage: 1.5%, Prefix cache hit rate: 66.8%, MM cache hit rate: 0.0%
(APIServer pid=814) INFO:     127.0.0.1:39352 - "POST /v1/chat/completions HTTP/1.1" 200 OK
(APIServer pid=814) INFO:     127.0.0.1:39352 - "POST /v1/chat/completions HTTP/1.1" 200 OK
(APIServer pid=814) INFO 09-19 01:39:42 [loggers.py:310] Engine 000: Avg prompt throughput: 161.4 tokens/s, Avg generation throughput: 19.7 tokens/s, Running: 0 reqs, Waiting: 0 reqs, GPU KV cache usage: 0.0%, Prefix cache hit rate: 66.9%, MM cache hit rate: 0.0%
(APIServer pid=814) INFO 09-19 01:39:52 [loggers.py:310] Engine 000: Avg prompt throughput: 1455.8 tokens/s, Avg generation throughput: 42.4 tokens/s, Running: 1 reqs, Waiting: 0 reqs, GPU KV cache usage: 3.1%, Prefix cache hit rate: 65.1%, MM cache hit rate: 0.0%
(APIServer pid=814) INFO:     127.0.0.1:39352 - "POST /v1/chat/completions HTTP/1.1" 200 OK
(APIServer pid=814) INFO:     127.0.0.1:39352 - "POST /v1/chat/completions HTTP/1.1" 200 OK
(APIServer pid=814) INFO:     127.0.0.1:39352 - "POST /v1/chat/completions HTTP/1.1" 200 OK
(APIServer pid=814) INFO 09-19 01:40:02 [loggers.py:310] Engine 000: Avg prompt throughput: 231.1 tokens/s, Avg generation throughput: 24.2 tokens/s, Running: 0 reqs, Waiting: 0 reqs, GPU KV cache usage: 0.0%, Prefix cache hit rate: 65.6%, MM cache hit rate: 0.0%
(APIServer pid=814) INFO 09-19 01:40:12 [loggers.py:310] Engine 000: Avg prompt throughput: 751.2 tokens/s, Avg generation throughput: 42.6 tokens/s, Running: 1 reqs, Waiting: 0 reqs, GPU KV cache usage: 2.3%, Prefix cache hit rate: 65.1%, MM cache hit rate: 0.0%
(APIServer pid=814) INFO:     127.0.0.1:39352 - "POST /v1/chat/completions HTTP/1.1" 200 OK
(APIServer pid=814) INFO 09-19 01:40:22 [loggers.py:310] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 28.2 tokens/s, Running: 1 reqs, Waiting: 0 reqs, GPU KV cache usage: 1.6%, Prefix cache hit rate: 65.3%, MM cache hit rate: 0.0%
(APIServer pid=814) INFO:     127.0.0.1:39352 - "POST /v1/chat/completions HTTP/1.1" 200 OK
(APIServer pid=814) INFO:     127.0.0.1:39352 - "POST /v1/chat/completions HTTP/1.1" 200 OK
(APIServer pid=814) INFO 09-19 01:40:32 [loggers.py:310] Engine 000: Avg prompt throughput: 1239.9 tokens/s, Avg generation throughput: 37.4 tokens/s, Running: 1 reqs, Waiting: 0 reqs, GPU KV cache usage: 2.4%, Prefix cache hit rate: 65.1%, MM cache hit rate: 0.0%
(APIServer pid=814) INFO 09-19 01:40:42 [loggers.py:310] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 44.7 tokens/s, Running: 1 reqs, Waiting: 0 reqs, GPU KV cache usage: 2.5%, Prefix cache hit rate: 65.1%, MM cache hit rate: 0.0%
(APIServer pid=814) INFO 09-19 01:40:52 [loggers.py:310] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 44.7 tokens/s, Running: 1 reqs, Waiting: 0 reqs, GPU KV cache usage: 2.6%, Prefix cache hit rate: 65.1%, MM cache hit rate: 0.0%
(APIServer pid=814) INFO 09-19 01:41:02 [loggers.py:310] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 31.6 tokens/s, Running: 0 reqs, Waiting: 0 reqs, GPU KV cache usage: 0.0%, Prefix cache hit rate: 65.1%, MM cache hit rate: 0.0%
(APIServer pid=814) INFO:     127.0.0.1:39352 - "POST /v1/chat/completions HTTP/1.1" 200 OK
(APIServer pid=814) INFO 09-19 01:41:12 [loggers.py:310] Engine 000: Avg prompt throughput: 200.3 tokens/s, Avg generation throughput: 40.2 tokens/s, Running: 1 reqs, Waiting: 0 reqs, GPU KV cache usage: 1.6%, Prefix cache hit rate: 65.2%, MM cache hit rate: 0.0%
(APIServer pid=814) INFO 09-19 01:41:22 [loggers.py:310] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 9.0 tokens/s, Running: 0 reqs, Waiting: 0 reqs, GPU KV cache usage: 0.0%, Prefix cache hit rate: 65.2%, MM cache hit rate: 0.0%
(APIServer pid=814) INFO 09-19 01:41:32 [loggers.py:310] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Waiting: 0 reqs, GPU KV cache usage: 0.0%, Prefix cache hit rate: 65.2%, MM cache hit rate: 0.0%