Quantcast
Channel: Active questions tagged python - Stack Overflow
Viewing all articles
Browse latest Browse all 23160

Auto-parse on python

$
0
0

I'm trying to write a script that will automatically parse product cards. I managed to cope with one page. How can I make the script automatically go to another page?

I've seen several answers where selenium helped. But I couldn't figure it out.

Here's the code:

import randomimport stringimport csv import requestsfrom bs4 import BeautifulSoupurl = "https://game29.ru/products?category=926"response = requests.get(url)html = response.textmulti_class = {'class': ['row'], 'style': 'border: 2px solid #898989;border-radius: 7px;padding: 2px;margin-top: -2px;'}soup = BeautifulSoup(html, "html.parser")products = soup.find_all("div", {"class":"row"})identifaer = "".join([random.choice(string.ascii_letters + string.digits) for n in range(32)])ad_status = "Free"category = "Игры, приставкиипрограммы"goods_type = "Игрыдляприставок"ad_type = "Продаюсвоё"adress = ""discription = ""condition = "Новое"data_begin = "2024-04-03"data_end = "2024-05-03" allow_email = "Нет"contact_phone = ""contact_method = "Потелефонуивсообщениях"all_products = []for product in products:    if product.attrs == multi_class:        identifaer        image ="https://www.game29.ru" + product.find("img")["src"]        if image != "https://game29.ru/zaglushka.png":            title = product.find("div", {"class":"cart-item-name"}).text            price = product.find("div", {"class": "cart-item-price"}).text.strip().replace("руб.", "")            all_products.append([identifaer, ad_status, category, goods_type, ad_type, adress, title, discription, condition, price, data_begin, data_end, allow_email, contact_phone, image, contact_method])# names = ["Id", "AdStatus", "Category", "GoodsType", "Adtype", "Adress", "Title", "Discription", "Condition", "Price", "DataBegin", "DataEnd", "AllowEmail", "ContactPhone","ImageUrls", "ContactMethod"]with open("data.csv", "a", newline='') as csv.file:    writer = csv.writer(csv.file, delimiter=',')    # writer.writerow(names)    for product in all_products:        writer.writerow(product)

I really thought selenium would help me. And I think that the answer is there, but unfortunately, I don't understand it yet, and I don’t have much time. I would be glad if you could help me.


Viewing all articles
Browse latest Browse all 23160

Trending Articles



<script src="https://jsc.adskeeper.com/r/s/rssing.com.1596347.js" async> </script>