I have a dataset saved in a local drive, and i can load and convert this dataset to json or csv format and saved it to my local drive. I want to know , if there is a way i can stream this data and iterate through item. how can i iterate through ds? I can save it as a csv and process data row by row. or can i stream the data and iterate through each item?
from datasets import load_from_diskimport jsonds = load_from_disk(f"local_path"...)ds.to_json(mydata.json)