Quantcast
Viewing all articles
Browse latest Browse all 14069

Selenium: "element not interactable error" when accessing the CAISO website but only for downloadNetDemandCSV, but not for downloadDemandCSV

I am trying to fix this code to extract data from it. The code is publicly available here: https://github.com/grgmiller/CAISO_operations/blob/master/CAISO_operations.py

This code used to worked in 2020, but currently does not due to slight changes in the website, and its use of now-depreciated methods.

The first thing I did was to downgrade selenium to version 4.0.0 to factor out the errors created by the depreciated methods. Next, I added one line of code to the downloadCurtialment function after

curtail_read = pd.read_csv(curtailments / 'curtailment_data.csv', dtype=ct_dtypes) #load the csv into a dataframe

:

curtail_read = curtail_read.drop(columns=['Reason']) # Reason is not needed

The 'Reason' column was not originally in the dataset used in the code, so I dropped it from the dataframe.

The problem I am currently having is with the second to last line of code in the downloadDemand function:

browser.find_element_by_id('downloadNetDemandCSV').click() #download CSV file

I get this error output:

File "/home/luis/Research/CAISO_operations/CAISO_operations.py", line 101, in maindownloadDemand(browser, dataDate)File "/home/luis/Research/CAISO_operations/CAISO_operations.py", line 253, in downloadDemandbrowser.find_element_by_id('downloadNetDemandCSV').click() #download CSV fileFile "/home/luis/.local/lib/python3.10/site-packages/selenium/webdriver/remote/webelement.py", line 80, in clickself._execute(Command.CLICK_ELEMENT)File "/home/luis/.local/lib/python3.10/site-packages/selenium/webdriver/remote/webelement.py", line 693, in _executereturn self._parent.execute(command, params)File "/home/luis/.local/lib/python3.10/site-packages/selenium/webdriver/remote/webdriver.py", line 418, in executeself.error_handler.check_response(response)File "/home/luis/.local/lib/python3.10/site-packages/selenium/webdriver/remote/errorhandler.py", line 243, in check_responseraise exception_class(message, screen, stacktrace)selenium.common.exceptions.ElementNotInteractableException: Message: element not interactable(Session info: chrome=123.0.6312.58)

I am a little bit lost about this error since it can do

browser.find_element_by_id('downloadDemandCSV').click() #download CSV file

just fine with downloadDemand.csv, and I checked the browser console, and both 'downloadDemandCSV'and 'downloadNetDemandCSV' are there in the dropdown medium. I cannot figure out what can be there error. Here is the website the code references. http://www.caiso.com/TodaysOutlook/Pages/default.aspx#section-net-demand-trend

Thank you all for the help, and apologize if it is something trivial I am new to selenium and web scraping in general.


Viewing all articles
Browse latest Browse all 14069

Trending Articles



<script src="https://jsc.adskeeper.com/r/s/rssing.com.1596347.js" async> </script>