iT邦幫忙

0

webdriver問題,爬不出照片

  • 分享至 

  • xImage

各位大大您好:

以下是我根據影片做的ig爬蟲

from selenium import webdriver
from selenium.webdriver.common.by import By
from selenium.webdriver.support.ui import WebDriverWait
from selenium.common.exceptions import TimeoutException
from selenium.webdriver.support import expected_conditions as EC
from selenium.webdriver.common.keys import Keys
from selenium.webdriver.chrome.service import Service
import time
import os
import wget

try:
s=Service("/Users/xxxxxxx/Downloads/chromedriver")
driver= webdriver.Chrome(service=s)
driver.get("https://www.instagram.com/")

username = WebDriverWait(driver, 10).until(
    EC.presence_of_element_located((By.NAME, "username"))
)
password = WebDriverWait(driver, 10).until(
    EC.presence_of_element_located((By.NAME, "password"))
)
login = driver.find_element("xpath",'//*[@id="loginForm"]/div/div[3]/button')

username.clear()
password.clear()
username.send_keys('帳號')
password.send_keys('密碼')
login.click()

search = WebDriverWait(driver, 5).until(
    EC.presence_of_element_located((By.XPATH, '//*[@id="react-root"]/section/nav/div[2]/div/div/div[2]/input'))
)
keyword = "欲爬取之帳號"
search.send_keys(keyword)
time.sleep(1)
search.send_keys(Keys.RETURN)
time.sleep(1)
search.send_keys(Keys.RETURN)

WebDriverWait(driver, 5).until(
    EC.presence_of_element_located((By.CLASS_NAME, "FFVAD"))
)

for i in range(5):
    driver.execute_script("window.scrollTo(0, document.body.scrollHeight);")
    time.sleep(5)

imgs = driver.find_elements_by_class_name("FFVAD")

path = os.path.join(keyword)
os.mkdir(path)

count = 0
for img in imgs:
    save_as = os.path.join(path, keyword + str(count) + '.jpg')
    wget.download(img.get_attribute("src"), save_as)
    count += 1

except TimeoutException as ex:
isrunning = 0
print("Exception has been thrown. " + str(ex))
driver.close()**

但最後得出來的結果一直是這個:
Exception has been thrown. Message:
Stacktrace:
0 chromedriver 0x0000000100c91a90 chromedriver + 3889808
1 chromedriver 0x0000000100c20b54 chromedriver + 3427156
2 chromedriver 0x0000000100912238 chromedriver + 221752
3 chromedriver 0x00000001009434c8 chromedriver + 423112
4 chromedriver 0x000000010096c944 chromedriver + 592196
5 chromedriver 0x000000010093854c chromedriver + 378188
6 chromedriver 0x0000000100c652f8 chromedriver + 3707640
7 chromedriver 0x0000000100c68ea8 chromedriver + 3722920
8 chromedriver 0x0000000100c6f3d8 chromedriver + 3748824
9 chromedriver 0x0000000100c699a8 chromedriver + 3725736
10 chromedriver 0x0000000100c44de8 chromedriver + 3575272
11 chromedriver 0x0000000100c83d1c chromedriver + 3833116
12 chromedriver 0x0000000100c83e84 chromedriver + 3833476
13 chromedriver 0x0000000100c98210 chromedriver + 3916304
14 libsystem_pthread.dylib 0x000000018329826c _pthread_start + 148
15 libsystem_pthread.dylib 0x000000018329308c thread_start + 8

想請問有人知道為什麼嗎TAT非常感謝。

Time iT邦新手 4 級 ‧ 2022-09-20 15:47:32 檢舉
我通靈了一下,我猜是 wait 的東西沒出來直接跳 timeout,你可以用 debug mode 慢慢檢查
圖片
  直播研討會
圖片
{{ item.channelVendor }} {{ item.webinarstarted }} |
{{ formatDate(item.duration) }}
直播中

1 個回答

1
hello02923
iT邦新手 5 級 ‧ 2022-09-21 13:37:16

要debug 最好不要try except寫在外層,要去一行一行確認程式碼,請可參考webdriver問題,看起來error跟你相似。

我要發表回答

立即登入回答