I'm trying to count the number of objects (larvae in this case), from a video.
This being the overall goal, I first tried density map on all the frames from the video.On looking at density plots more closely, I notice the algorithm is counting objects which are not even larvae, so I thought let me narrow down the question to just one frame at the moment.In the frames, I applied the k-means cluster on the colours present in the image. After some manipulation, I understand, what is the colour of the larvae and mask the images for the colour.Post these modification I apply density map on masked image for the colour of interest.
It now happens that the count is of the order 1e3, but in reality this is wrong as actual count ~50.When looking closely at the density maps, it is predicting/ counting the density of point objects and this is the reason for wrong calculations.
Now the question is How do I change the density map such that it doesn't count point objects, but give me a closer answer for my actual count.
The code for density map is:
import numpy as npimport tensorflow as tffrom tensorflow.keras.layers import Conv2D, Inputfrom tensorflow.keras.models import Modelfrom tensorflow.keras.losses import MeanSquaredErrorimport cv2# Create the density map estimation modeldef create_density_map_model(input_shape): inputs = Input(shape=input_shape) x = Conv2D(64, (3, 3), activation='relu', padding='same')(inputs) x = Conv2D(64, (3, 3), activation='relu', padding='same')(x) density_map = Conv2D(1, (1, 1), activation='linear', padding='same')(x) model = Model(inputs=inputs, outputs=density_map) return model# Load the imageimage_path = 'color_17.png'image = cv2.imread(image_path, cv2.IMREAD_UNCHANGED)# Extract the alpha channel from the imagealpha_channel = image[:, :, 3]# Threshold the alpha channel to get a binary mask where non-transparent pixels are white (255) and transparent pixels are black (0)_, binary_mask = cv2.threshold(alpha_channel, 0, 255, cv2.THRESH_BINARY)# Get the height and width of the imageimage_height, image_width = image.shape[:2]# Create and compile the modelinput_shape = (image_height, image_width, 1)model = create_density_map_model(input_shape)model.compile(optimizer='adam', loss=MeanSquaredError())# Preprocess the imageimage_gray = cv2.cvtColor(image, cv2.COLOR_BGR2GRAY)image_normalized = image_gray / 255.0image_input = np.expand_dims(image_normalized, axis=-1)image_input = np.expand_dims(image_input, axis=0)# Predict the density map for the imagepredicted_density_map = model.predict(image_input)# Reshape the predicted density map to match the shape of the imagepredicted_density_map = np.reshape(predicted_density_map, (image_height, image_width, 1))# Create a size factor map (assuming all pixels have the same size factor for simplicity)size_factor_map = np.ones_like(predicted_density_map)# Apply the size factor map to the predicted density mappredicted_density_map_filtered = predicted_density_map * size_factor_map# Expand the dimensions of the binary mask to match the number of channels in the predicted density mapbinary_mask_reshaped = np.expand_dims(binary_mask, axis=-1)# Apply the binary mask to the filtered density map to only keep values where the image isn't transparentpredicted_density_map_filtered = predicted_density_map_filtered * (binary_mask_reshaped / 255.0)## Apply the binary mask to the filtered density map to only keep values where the image isn't transparent#predicted_density_map_filtered = predicted_density_map_filtered * (binary_mask / 255.0)# Save the filtered density map as an imagecv2.imwrite('density_plot_filtered.jpg', (predicted_density_map_filtered * 255.0).astype(np.uint8))# Calculate the number of larvae predicted from the filtered density mapnum_larvae_predicted = int(np.sum(predicted_density_map_filtered))# Print the number of larvae predictedprint(f"Number of larvae predicted: {num_larvae_predicted}")The video that I used for the first code, i.e., the kmeans for identifying the top 20 expressed colours can be found here:https://iitk-my.sharepoint.com/:v:/g/personal/abint21_iitk_ac_in/EexjUqun0pZFmzRTxTBHGFoB8ML2hX5iZ6luH9QVWpLjKA?e=mLQLfb
The first frame without any filtering can be found here:
To add, the colour of interest when viewed with rgba palette looks like this:
The density plot for this image comes out as something like this:
I hope this information was helpful for making the details clear. Please let me know in case more details are needed.