0
176
Viewssplash:mouse_click couldn't open pop up window
I'm trying to scrape part-time job information from townwork(A Japanese job information site). To get the address, I want to open a popup window. Then I started to use Splash. Although I could open other links and popups using splash:mouse_click, I couldn't open the desired pop-up window.
Any answer would be greatly appreciated.
The link that I wanted to click.
Desired result.
Target page URL https://townwork.net/detail/clc_2578209001/joid_Y00BKDF4/
Code
function main(splash, args)
assert(splash:go(args.url))
local map = splash:select('#jsi-inflow-target > div > dl:nth-child(3) > dd > p > span > a')
local bounds = map:bounds()
local x_dif = bounds.right - bounds.left
assert(map:mouse_click({x=bounds.width/3, y=bounds.height/3}))
splash:wait(1)
return {
html = splash:html(),
png = splash:png(),
bounds = bounds
}
end
·
Juan Pablo Isaza
1 answers
Answer question0
I think that the speed at which this popup opens is a cause.
I found another solution other than the open popup page for the sake of the address: getting latitude and longitude from the first page.
def parse(self, response):
dt_list = response.css('dt')
for dt in dt_list:
if dt.css('::text').get == "勤務地":
a_tag = BeautifulSoup(dt.xpath('./following-sibling::dd').get()).find('a')
if a_tag is None:
continue
item['lat'] = a_tag['data-lat']
item['lon'] = a_tag['data-lon']
·
Juan Pablo Isaza
Report
Answer question