Quantcast
Channel: Active questions tagged python - Stack Overflow
Viewing all articles
Browse latest Browse all 13951

Trying to scrape Canon Webpage using selenium

$
0
0

One of the categories has Load More button which load cameras. I need to write function that scrolls to Load more button and clicks iturl: https://www.usa.canon.com/shop/cameras/mirrorless-cameras

def find_load_more(driver, xpath):    try:        return WebDriverWait(driver, 10).until(            EC.element_to_be_clickable((By.XPATH, xpath))        )    except TimeoutException:        print("Timeout waiting for the load more button.")        return Nonedef scroll_to_load_more(driver, xpath):    print(xpath)    try:        load_more_button = find_load_more(driver, xpath)        if load_more_button:            driver.execute_script("arguments[0].scrollIntoView();", load_more_button)            load_more_button.click()        else:            print("Load more button not found.")    except NoSuchElementException:        print("The 'Load More' button does not exist.")def scrape_canon_preview(category, driver):    url = f"https://www.usa.canon.com/shop/cameras/{category}"    driver.get(url)    wait_for_page_load(driver)    # Click Load more to load all the cameras    scroll_to_load_more(driver, "//button[@class='primary amscroll-load-button']")

But It returns Load more button not found and Timeout waiting for the load more button.


Viewing all articles
Browse latest Browse all 13951

Trending Articles



<script src="https://jsc.adskeeper.com/r/s/rssing.com.1596347.js" async> </script>