Python Socket Send

I'm attempting to transmit CV frames and additional data via a socket in Python. My approach includes a "header" that specifies the data type, its size, and the timestamp of its creation. This method works seamlessly for smaller images (640x480x3), delivering data with minimal latency and ensuring accurate data capture.

Combine data with header

    def encapsulate(self, data, data_type) -> bytes:        # Data should be frame obtained from cap.read(), no need to do anything        if data_type == 'IMAGE':            buffer = cv.imencode('.jpg', data)[1].tobytes()            self.frame_counter += 1        elif data_type == 'SERVO':            ## Tuple of two floads            buffer = struct.pack('ff', *data)        else:            logging.critical("Encapsulate: Data type not supported")            return None        data_type_encode = data_type.encode('utf-8').ljust(8, b'\0')  # Ensure 8-byte length, left justified        header = struct.pack(self.HEADER_FORMAT, data_type_encode,                              len(buffer),                              self.frame_counter if data_type == 'IMAGE' else -1,                              time.time()) # convert to fixed sized bytes        # Split data into chunks        all_data = header + buffer        total_length = len(all_data)        num_chunks = total_length // self.BUFFER_SIZE + (1 if total_length % self.BUFFER_SIZE else 0)        logging.debug(f"Data Length: {len(buffer)}, num_chunks: {num_chunks}")        for i in range(num_chunks):            chunk = all_data[i*self.BUFFER_SIZE : (i+1)*self.BUFFER_SIZE]            yield total_length, chunk # return a generator

send chunk out

    def send_all(self, soc, chunk, length):        while length:            try:                sent = soc.sendall(chunk)            except ConnectionError as e:                logging.error(f"Send: Connection error {e} ")                logging.error(f"Send: Remote host closing the connection, network issues leading to a dropped connection, or attempts to interact with a socket that has already been closed.")                self.remove_socket(soc)                return False            except BlockingIOError as e:                logging.error(f"Send: BlockingIOError {e}")                time.sleep(0.01)                continue            except OSError as e:                logging.error(f"Send: Other OSError {e}")                self.remove_socket(soc)                return False            if sent == 0:                logging.error(f"Send: Sent 0 bytes")            break        return True

Separate header from data

    def decapsulate(self, header) -> tuple:        assert len(header) == self.HEADER_SIZE, f"Invalid header size: {len(header)}"        data_type, total_byte, frame_index, sent_time = struct.unpack(self.HEADER_FORMAT, header)        data_type = data_type.decode('utf-8').rstrip('\0')        assert data_type in ['IMAGE', 'SERVO', 'TEXT'], f'Invalid data type: {data_type}'        if data_type == 'IMAGE':            logging.info(f"Frame index: {frame_index}, time different: {time.time() - sent_time}")        # Returned data_type is a string, total_byte is an integer        return data_type, total_byte

and receiver function similar in socket HOWTO

  def recv_all(self, sock, count):        buf = bytearray()        while count:            try:                newbuf = sock.recv(count)            except OSError as e:                logging.warning(f"Client close with Error: {e}")                newbuf = None            if not newbuf:                 logging.info("Connection closed")                self.remove_socket(sock)                return None            buf.extend(newbuf)            count -= len(newbuf)        return buf

However, when I increase the image size to 1280x720x3, I encounter an error on the receiver's end, indicating a decoding issue. Further investigation revealed that this is due to a buffer full, specifically a BlockingIOError. Im using non-blocking socket.

As a result of this error, the receiver fails to obtain the correct data, leading to decoding problems. I attempted to resolve this by replacing sendall with send and dividing the data into smaller segments, mirroring the approach on the receiver side. Unfortunately, the socket.send() function tends to return 0 after sending a few chunks.

It appears that the buffer on the sender side is unable to transmit data effectively, as indicated by the receiver not receiving any data.

I would appreciate any advice on resolving this issue.

Thank you in advance!