What I'm looking to do is to get content inside a div on another website (which I do not control) and display the content on my own website. For example, we will use these two sites:
On othersite.com, there is some text content inside a div that I want to fetch with javascript and update periodically (suppose, check every hour). For example, <div class="some-div">9384</div>.
Now, on mysite.com, I want to output this result into my own div using javascript. I suppose this would effectively be web scraping.
To do this, I would like to save a variable which updates occasionally like every hour.
Using jquery, I could define a variable and pull the content. If it was on the same site, which it is not, I could do this:
/* HTML with content to fetch */
<div class="some-div">9384</div>
/* get the content with jquery */
var getDiv = document.getElementsByClassName('some-div');
var getContent = getDiv.innerHTML;
If it was on the same site, then it would get the content inside the div and store it to the variable getContent. Here, getContent will equal 9384.
Then I could use it anywhere I want, like replacing the content of a blank div on mysite.com with the content fetched from the somediv, like this:
/* HTML where the content will be output */
<div class="output-div"></div>
/* replace the content with jquery form a stored variable */
var getMyDiv = document.getElementsByClassName('output-div');
getMyDiv.innerHTML = getContent.innerHTML;
The result would be:
<div class="output-div">9384</div>
But, the content for the some-div div is not on the same URL (and I don't control the other site).
How do I do the above, but instead of fetching some-div from the current page or site, to instead fetch some-div content from an external site (that I do not control)?
If you want to use jQuery:
$.get('http://othersite.com', function(data) {
var $othersite = $(data);
var getDiv = $othersite.find('.some-div');
var getContent = getDiv.html();
var getMyDiv = $('.output-div');
getMyDiv.html(getContent);
});
If you want to use JavaScript without jQuery:
var xhr = new XMLHttpRequest();
xhr.open('GET', 'http://othersite.com', true);
xhr.responseType = 'document';
xhr.send();
xhr.onload = function(e) {
var othersite = e.target.responseXML;
var getDiv = othersite.getElementsByClassName('some-div');
var getContent = getDiv.innerHTML;
var getMyDiv = document.getElementsByClassName('output-div');
getMyDiv.innerHTML = getContent;
}
In other not to worry about CORS, you can try these steps:
Example - Step 1 and Step 3
$.get('mysite.com/some-php-file.php', function(data) {
let $othersiteHtml = $('<div />').html(data);
let getSomeText = $othersiteHtml.find('.some-div').text();
$('.output-div').html(getSomeText);
});
Step 2 - You can put below code in a PHP file on your server.
<?php
$ch = curl_init();
curl_setopt($ch, CURLOPT_URL, "othersite.com");
curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);
$output = curl_exec($ch);
curl_close($ch);
echo $output;
?>