Quantcast
Channel: Active questions tagged python - Stack Overflow
Viewing all articles
Browse latest Browse all 16773

Error in Loading Serialized Map Data with InMemMap.deserialize

$
0
0

I am attempting to load map data from a pickle file using map_connn = InMemMap.deserialize(map_conn). After exporting the map data from memory with map_con.dump() as a dictionary, an error occurs:

RTreeError: Error in "Index_CreateWithStream": Spatial Index Error: IllegalArgumentException: SpatialIndex::DiskStorageManager: Index/Data file cannot be created."1.

map_con.dump()
import pickleimport osmap_data_file = 'E:\jupyternotebook\共享单车路径匹配\map_con\myosm.pkl'if os.path.exists(map_data_file):    with open(map_data_file, 'rb') as f:        map_conn = pickle.load(f)
map_connn = InMemMap.deserialize(map_conn)---------------------------------------------------------------------------RTreeError                                Traceback (most recent call last)Cell In[5], line 1----> 1 map_connn = InMemMap.deserialize(map_conn)File D:\miniconda3\envs\Map_Matching\lib\site-packages\leuvenmapmatching\map\inmem.py:144, in InMemMap.deserialize(cls, data)    141 @classmethod    142 def deserialize(cls, data):    143     """Create a new instance from a dictionary."""--> 144     nmap = cls(data["name"], dir=data.get("dir", None),    145                use_latlon=data["use_latlon"], use_rtree=data["use_rtree"],    146                index_edges=data["index_edges"],    147                crs_lonlat=data.get("crs_lonlat", None), crs_xy=data.get("crs_xy", None),    148                graph=data.get("graph", None), linked_edges=data.get("linked_edges", None),    149                deserializing=True)    150     return nmapFile D:\miniconda3\envs\Map_Matching\lib\site-packages\leuvenmapmatching\map\inmem.py:81, in InMemMap.__init__(self, name, use_latlon, use_rtree, index_edges, crs_lonlat, crs_xy, graph, linked_edges, dir, deserializing)     79 self.use_rtree = use_rtree     80 if self.use_rtree:---> 81     self.setup_index(deserializing=deserializing)     83 self.crs_lonlat = 'EPSG:4326' if crs_lonlat is None else crs_lonlat  # GPS     84 self.crs_xy = 'EPSG:3395' if crs_xy is None else crs_xy  # Mercator projectionFile D:\miniconda3\envs\Map_Matching\lib\site-packages\leuvenmapmatching\map\inmem.py:384, in InMemMap.setup_index(self, force, deserializing)    382 else:    383     logger.debug(f"Creating new in-memory rtree index (args={args}) ...")--> 384 self.rtree = rtree.index.Index(*args)    385 t_delta = time.time() - t_start    386 logger.debug(f"... done: rtree size = {self.rtree_size()}, time = {t_delta} sec")File D:\miniconda3\envs\Map_Matching\lib\site-packages\rtree\index.py:273, in Index.__init__(self, *args, **kwargs)    271 if stream and self.properties.type == RT_RTree:    272     self._exception = None--> 273     self.handle = self._create_idx_from_stream(stream)    274     if self._exception:    275         raise self._exceptionFile D:\miniconda3\envs\Map_Matching\lib\site-packages\rtree\index.py:1263, in Index._create_idx_from_stream(self, stream)   1260     return 0   1262 stream = core.NEXTFUNC(py_next_item)-> 1263 return IndexStreamHandle(self.properties.handle, stream)File D:\miniconda3\envs\Map_Matching\lib\site-packages\rtree\index.py:1396, in Handle.__init__(self, *args, **kwargs)   1395 def __init__(self, *args: Any, **kwargs: Any) -> None:-> 1396     self._ptr = self._create(*args, **kwargs)File D:\miniconda3\envs\Map_Matching\lib\site-packages\rtree\core.py:25, in check_void(result, func, cargs)     23     msg = f'Error in "{func.__name__}": {s}'     24     rt.Error_Reset()---> 25     raise RTreeError(msg)     26 return resultRTreeError: Error in "Index_CreateWithStream": Spatial Index Error: IllegalArgumentException: SpatialIndex::DiskStorageManager: Index/Data file cannot be created.

Whenever I create a map in-memory object, it requires a significant amount of computation, including importing points, importing edges, and performing deduplication operations. This results in a long processing time each time I generate an in-memory map object. To overcome this, I attempted to export my pre-computed map in-memory object and load it directly from a file for future use.

Initially, I tried using Python's built-in pickle module. However, when I read the exported .pkl file, I found that it couldn't be used for path matching, as the matched paths were empty. I suspected that certain attributes, such as R-tree, might have been lost during the export of the map in-memory object.

To address this, I consulted the official documentation of LeuvenMap and discovered the dump() and deserialize() methods provided by the package. I attempted to use these recommended backup and loading methods. However, during the process, I encountered the aforementioned error.

I would greatly appreciate assistance in resolving this issue.


Viewing all articles
Browse latest Browse all 16773

Trending Articles



<script src="https://jsc.adskeeper.com/r/s/rssing.com.1596347.js" async> </script>