Here's a minimal reproducible example of my FastAPI app. I have a strange behavior and I'm not sure I understand the reason.
I'm using ApacheBench (ab) to send multiple requests as follows:
ab -n 1000 -c 50 -H 'accept: application/json' -H 'x-data-origin: source''http://localhost:8001/test/async'FastAPI app
import timeimport asyncioimport enumfrom typing import Anyfrom fastapi import FastAPI, Path, Bodyfrom starlette.concurrency import run_in_threadpoolapp = FastAPI()loop = asyncio.get_running_loop()def sync_func() -> None: time.sleep(3) print("sync func")async def sync_async_with_fastapi_thread() -> None: await run_in_threadpool( time.sleep, 3) print("sync async with fastapi thread")async def sync_async_func() -> None: await loop.run_in_executor(None, time.sleep, 3)async def async_func() -> Any: await asyncio.sleep(3) print("async func")@app.get("/test/sync")def test_sync() -> None: sync_func() print("sync")@app.get("/test/async")async def test_async() -> None: await async_func() print("async")@app.get("/test/sync_async")async def test_sync_async() -> None: await sync_async_func() print("sync async")@app.get("/test/sync_async_fastapi")async def test_sync_async_with_fastapi_thread() -> None: await sync_async_with_fastapi_thread() print("sync async with fastapi thread")Here's the ApacheBench results:
async with (asyncio.sleep) :*Concurrency Level: 50
- Time taken for tests: 63.528 seconds
- Complete requests: 1000
- Failed requests: 0
- Total transferred: 128000 bytes
- HTML transferred: 4000 bytes
- Requests per second: 15.74 [#/sec] (mean)
- Time per request: 3176.407 [ms] (mean)
- Time per request: 63.528 [ms] (mean, across all concurrent requests)Transfer rate: 1.97 [Kbytes/sec] received*
sync (with time.sleep):Concurrency Level: 50
- *Time taken for tests: 78.615 seconds
- Complete requests: 1000
- Failed requests: 0
- Total transferred: 128000 bytes
- HTML transferred: 4000 bytes
- Requests per second: 12.72 [#/sec] (mean)
- Time per request: 3930.751 [ms] (mean)
- Time per request: 78.615 [ms] (mean, across all concurrent requests)Transfer rate: 1.59 [Kbytes/sec] received*
sync_async (time sleep with run_in_executor) : *Concurrency Level: 50
- Time taken for tests: 256.201 seconds
- Complete requests: 1000
- Failed requests: 0
- Total transferred: 128000 bytes
- HTML transferred: 4000 bytes
- Requests per second: 3.90 [#/sec] (mean)
- Time per request: 12810.038 [ms] (mean)
- Time per request: 256.201 [ms] (mean, across all concurrent requests)Transfer rate: 0.49 [Kbytes/sec] received*
sync_async_fastapi (time sleep with run_in threadpool):*Concurrency Level: 50
- Time taken for tests: 78.877 seconds
- Complete requests: 1000
- Failed requests: 0
- Total transferred: 128000 bytes
- HTML transferred: 4000 bytes
- Requests per second: 12.68 [#/sec] (mean)
- Time per request: 3943.841 [ms] (mean)
- Time per request: 78.877 [ms] (mean, across all concurrent requests)Transfer rate: 1.58 [Kbytes/sec] received*
In conclusion, I'm experiencing a surprising disparity in results, especially when using run_in_executor, where I'm encountering significantly higher average times (12 seconds). I don't understand this outcome.