I have a Python API interfacing with a C# application. The C# component transmits an image to the API, which undergoes byte conversion followed by base64 encoding to facilitate transmission within a JSON request.
Subsequently, the API (python) decodes the base64 data to reconstruct the image for processing within a Convolutional Neural Network (CNN). The CNN outputs a binary response, indicating the presence or absence of a detected object.
However, the current end-to-end processing time is suboptimal, averaging approximately 1.5 seconds. I seek to achieve a significantly faster response time, ideally targeting a latency of around 200 milliseconds.
Are there alternative approaches to include the image in the request without prior conversion, such as transmitting a grayscale matrix or the object image directly?
Concerning the CNN model, I possess the H5 file. Are there alternative model formats conducive to faster inference? Attempts to mitigate latency by image resizing and reducing CNN hidden layers have proven inadequate.
My codes:
Python
@app.post('/overlap/')async def overlap(request: Request): global cnn body = await request.json() image_np = np.frombuffer(base64.b64decode(body["image"]), dtype=np.uint8) image = cv2.imdecode(image_np, cv2.IMREAD_GRAYSCALE) image_resized = cv2.resize(image, (360, 300)) gamma_corrected = tf.pow(image_resized / 255.0, 0.6) * 255.0 image_batch = np.expand_dims(gamma_corrected, axis=0) result = cnn.predict(image_batch) if result[0][0] > result[0][1]: overlap_result = False else: overlap_result = True return [overlap_result, str(result[0][0]), str(result[0][1])]C#
using System;using System.Diagnostics;using System.IO;using System.Net;using System.Text;using System.Threading;namespace comunica_api{ class Program { static void Main(string[] args) { string imagePath = @"C:\my_photo.jpg"; try { Stopwatch stopwatch1 = new Stopwatch(); stopwatch1.Start(); byte[] imageBytes = File.ReadAllBytes(imagePath); string base64String = Convert.ToBase64String(imageBytes); Console.WriteLine("Image read"); string url = "http://localhost:8000/overlap/"; string json = "{\"image\": \"" + base64String +"\"}"; var request = (HttpWebRequest)WebRequest.Create(url); request.Method = "POST"; request.ContentType = "application/json"; request.ContentLength = json.Length; byte[] jsonBytes = Encoding.UTF8.GetBytes(json); using (Stream requestStream = request.GetRequestStream()) using (StreamWriter writer = new StreamWriter(requestStream)) { writer.Write(json); } stopwatch1.Stop(); Stopwatch stopwatch2 = new Stopwatch(); stopwatch2.Start(); using (var response = (HttpWebResponse)request.GetResponse()) { using (var streamReader = new StreamReader(response.GetResponseStream())) { string responseText = streamReader.ReadToEnd(); stopwatch2.Stop(); Console.WriteLine("API Response:"); Console.WriteLine(responseText); long elapsedTimeMs1 = stopwatch1.ElapsedMilliseconds; long elapsedTimeMs2 = stopwatch2.ElapsedMilliseconds; Console.WriteLine($"Image conversion time: {elapsedTimeMs1} ms"); Console.WriteLine($"API response time: {elapsedTimeMs2} ms"); Thread.Sleep(3000); } } } catch (FileNotFoundException) { Console.WriteLine("The specified image could not be found."); Thread.Sleep(3000); } catch (WebException ex) { Console.WriteLine("API communication error:"); Console.WriteLine(ex.Message); Thread.Sleep(3000); } Console.ReadLine(); } }}Any idea for this problem?