I need to send a large number (approximately 50) of HTTP POST requests, each with a size of around 5 MB. I use the asyncio.gather function to wait for the responses from these requests. However, the JSON serialization process takes approximately 30-40 milliseconds and blocks the EventLoop.
This causes all requests to be sent to the server simultaneously. For instance, if there are 50 requests and the first request is available at time T, it is only sent at (T + (50 * 30) = T + 1500ms). Additionally, at T+1500 ms, all 50 requests are sent to the server. I want Aiohttp to send requests as soon as they become available.
Can I bypass the JSON serialization step to achieve this?
Also, this line is where aiohttp is waiting for the eventloop. Our APIs are latency sensitive. Is there any way to speed up this? or any other alternatives?
