I am fetching a constant stream of data into a dataframe, which I now need to save into multiple CSVs.
For example,I am fetching OHLCV data from Binance and I get this dataframe:
sym o h l \2024-02-09 11:15:59.594 ETHUSDT 2419.56000000 2479.88000000 2419.16000000 c v barcomplete 2024-02-09 11:15:59.594 2471.79000000 170696.13700000 False sym o h l \2024-02-09 11:15:59.622 IDUSDT 0.54534000 0.64866000 0.53587000 c v barcomplete 2024-02-09 11:15:59.622 0.60634000 93122120.00000000 False sym o h l \2024-02-09 11:15:59.658 ICPUSDT 12.18600000 12.81000000 12.16500000 c v barcomplete 2024-02-09 11:15:59.658 12.62300000 1065607.61000000 False sym o h l \2024-02-09 11:15:59.594 ETHUSDT 2400.56000000 2422.88000000 2399.16000000 c v barcomplete 2024-02-08 11:15:59.594 2419.79000000 160696.13700000 False
Index is the timestamp. I get multiple data rows for the same sym (ETHUSDT price today, yesterday, day before and so on). I want to save coins/ETHUSDT rows into their own separate CSV (ETHUSDT.csv, IDUSDT.csv, etc), with new data rows being appended to those CSVs as they get fetched.
I am using this but its slow:
for coin in df.sym: filename = r"{}.csv".format(coin) print(filename) df['sym'].to_csv(filename, mode='a', header=False)
But I just can't get this working properly. Please advise. (I just started learning pandas so please be kind) :)