如題,原本可以正常執行的程式碼,今天想要換主題測試,發現會出錯!
都沒有調整過範例的語法,懇請對於爬蟲了解的前輩能解答或協助除錯。
#----------------------------------------#
import requests
from bs4 import BeautifulSoup
import json
import pandas as pd
import time
from tqdm import tqdm
import time
import re
import random
#----------------------------------------#
keyword = '運動內衣'
page = 1
#----------------------------------------#
#封包標頭檔
my_headers = {'authority' : 'shopee.tw',
'method': 'GET',
'path': '/api/v1/item_detail/?item_id=1147052312&shop_id=17400098',
'scheme': 'https',
'accept': '/',
'accept-encoding': 'gzip, deflate, br',
'accept-language': 'zh-TW,zh;q=0.9,en-US;q=0.8,en;q=0.7,ja;q=0.6',
'cookie': '_ga=GA1.2.1087113924.1519696808; SPC_IA=-1; SPC_F=SDsFai6wYMRFvHCNzyBRCvFIp92UnuU3; REC_T_ID=f2be85da-1b61-11e8-a60b-d09466041854; __BWfp=c1519696822183x3c2b15d09; __utmz=88845529.1521362936.1.1.utmcsr=(direct)|utmccn=(direct)|utmcmd=(none); _atrk_siteuid=HEgUlHUKcEXQZWpB; SPC_EC=-; SPC_U=-; SPC_T_ID="vBBUETICFqj4EWefxIdZzfzutfKhrgytH2wyevGxiObL3hFEfy0dpQSOM/yFzaGYQLUANrPe7QZ4hqLZotPs72MhLd8aK0qhIwD5fqDrlRs="; SPC_T_IV="IpxA2sGrOUQhMH4IaolDSA=="; cto_lwid=2fc9d64c-3cfd-4cf9-9de7-a1516b03ed79; csrftoken=EDL9jQV76T97qmB7PaTPorKtfMlU7eUO; bannerShown=true; _gac_UA-61915057-6=1.1529645767.EAIaIQobChMIwvrkw8bm2wIVkBiPCh2bZAZgEAAYASAAEgIglPD_BwE; _gid=GA1.2.1275115921.1529896103; SPC_SI=2flgu0yh38oo0v2xyzns9a2sk6rz9ou8; __utma=88845529.1087113924.1519696808.1528465088.1529902919.7; __utmc=88845529; appier_utmz=%7B%22csr%22%3A%22(direct)%22%2C%22timestamp%22%3A1529902919%7D; _atrk_sync_cookie=true; _gat=1',
'if-none-match': "55b03-9ff4fb127aff56426f5ec9022baec594",
'referer': 'https://shopee.tw/6-9-%F0%9F%87%B0%F0%9F%87%B7%E9%9F%93%E5%9C%8B%E9%80%A3%E7%B7%9A-omg!%E6%96%B0%E8%89%B2%E7%99%BB%E5%A0%B4%F0%9F%94%A5%E4%BA%A4%E5%8F%89%E7%BE%8E%E8%83%8CBra%E5%BD%88%E5%8A%9B%E8%83%8C%E5%BF%83-i.17400098.1147052312',
'user-agent': 'Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/66.0.3359.139 Safari/537.36',
'x-api-source': 'pc',
'x-requested-with': 'XMLHttpRequest'
}
#----------------------------------------#
#專門用來請求蝦皮的相關商品基本資料
def request_shopee(keyword, page):
url = 'https://shopee.tw/api/v2/search_items/?by=relevancy&keyword=' + keyword + '&limit=100&newest=' + str(page*100) + '&order=desc&page_type=search&version=2'
#開始請求
list_req = requests.get(url,headers = my_headers)
soup = BeautifulSoup(list_req.content, "html.parser")
#將扒下來的文字轉成Json
getjson=json.loads(soup.text)
return getjson['items']
#----------------------------------------#
items_json = request_shopee(keyword = '運動內衣', page = 0)
#----------------------------------------#
原本都正常執行,今天使用發現會出現:
看起來就是 getjson 拿到的東西沒有 'key' 這個鍵
建議先 print 出 soup.txt 檢查回傳值
如果原本正常,這次突然發生,需要思考的方向就是: