I have a pandas dataframe with 4 columns as shown below. The text_vector column contains vectors that are 1500 dimensions:
When I write this out using: df.to_csv("./data/train_clean_vec.csv", index=False, encoding='utf-8'), the values in the text_vector column get truncated as seen when looking at the csv in a text editor as shown below:
Is there a way to write this dataframe such that all the vector values are preserved? If not, any suggestions as how to write this out in some other format where I can easily read the data back in? I'm "data format agnostic" in other words, I don't care what the format is (csv, json, HDF5, whatever...) as my goal is to preserve these vectors so I read them back in instead of having to regenerate them each time I need to work with this data.