Business
Jobs
  • About Us
  • Solutions
    • Job Postings
      Post your job and receive qualified candidates in 48h.
    • Candidate Assessments
      500+ technical and psychological tests, plus anti-fraud.
    • Headhunting
      Tailor-made executive search from start to finish.
    • Payroll + EOR
      Payroll dispersal and EOR across 15+ LATAM countries.
  • Pricing
  • Jobs

0

198
Views
PHP and MySql Web Crawler

I need some help with my web crawler exercise.I should write a crawler for saving the first pages of some websites and all of their content in a MySql database. I am using xampp MySql database. Here is my code:

<?php
 $main_url="webpage";
 $str = file_get_contents($main_url);

 // Gets Webpage Title
 if(strlen($str)>0)
 {
  $str = trim(preg_replace('/\s+/', ' ', $str)); // supports line breaks inside <title>
  preg_match("/\<title\>(.*)\<\/title\>/i",$str,$title); // ignore case
  $title=$title[1];
 }

 // Gets Webpage Description
 $b =$main_url;
 @$url = parse_url( $b );
 @$tags = get_meta_tags($url['scheme'].'://'.$url['host'] );
 $description=$tags['description'];

 // Gets Webpage Internal Links
 $doc = new DOMDocument; 
 @$doc->loadHTML($str); 

 $items = $doc->getElementsByTagName('a'); 
 foreach($items as $value) 
 { 
  $attrs = $value->attributes; 
  $sec_url[]=$attrs->getNamedItem('href')->nodeValue;
 }
 $all_links=implode(",",$sec_url);

 //Gets Webpage images
 require_once('C:\xampp\htdocs\simple_html_dom.php');
 require_once('C:\xampp\htdocs\url_to_absolute.php');
 $url='webpage'
 $html=file_get_html('webpage');
foreach($html->find('img') as $element) {
    echo url_to_absolute($url, $element->src), "\n";
}
$images=

 // Store Data In Database
 $host="localhost";
 $username="root";
 $password="";
 $databasename="db";
 $connect=mysql_connect($host,$username,$password);
 $db=mysql_select_db($databasename);

 mysql_query("insert into webpage_details values('$main_url','$title','$description','$all_links','$images')");

?>

I should save everything from a home page, i have a problem saving the images. Any ideas? Are there any other items I need to save?

Thanks!

over 4 years ago · Santiago Trujillo
Answer question
Find remote jobs

Discover the new way to find a job!

Top jobs
Top job categories
Business
Post vacancy Pricing Sales
Legal
Terms and conditions Privacy policy
© 2026 PeakU Inc. All Rights Reserved.
Andres GPT
Show me some job opportunities
There's an error!