I'm trying to build a simple javascript web scrapping project which extracts headlines from some news website.
the page I'm trying to scrape:
https://www.chinatimes.com/?chdtv
It has over 30 script tags
The variable I wanted to access is data on line 1610 in view-source mode(as shown below), which contains all the headline titles and URLs(unfortunately, they are available only here)
<script type="c04ba886ffa04b8e214f17e6-text/javascript">
//速報
$(document).ready(function () {
$(".breaking-news").verticalMarquee({
"onContentHeightChanged": function (baseDiv, contentHeight) {
var $self = $(baseDiv),
contentPadding = 12;
$self.height(contentHeight + contentPadding);
}
});
});
//焦點大圖
$(document).ready(function () {
var photoUrl;
var data = [...];
...
</script>
I tried targeting the script tag with puppeteer using $eval + css attribute selector, but the value of type for each scripts are dynamic(constantly changing)
Any solutions, opinions, keywords to google for will be deeply appreciated