Posted By

level09 on 05/13/11

Tagged

curl
php

Versions (?)

Last Edited at 05/13/11 01:13am

Statistics

Viewed 1057 times

Favorited by 0 user(s)

Related snippets

byPAss curl / file_get_contents blocking for scrapping

/ Published in: PHP

Expand | Embed | Plain Text

Copy this code and paste it in your HTML

$userAgent = 'Googlebot/2.1 (http://www.googlebot.com/bot.html)';
        $curl = curl_init($url);
        curl_setopt($curl, CURLOPT_USERAGENT, $userAgent);
        curl_setopt($curl, CURLOPT_AUTOREFERER, true);
        curl_setopt($curl, CURLOPT_FOLLOWLOCATION, true);
        curl_setopt($curl, CURLOPT_RETURNTRANSFER, 1 );
        curl_setopt($curl, CURLOPT_TIMEOUT, 2 );                
        $html = curl_exec( $curl );
	    curl_close( $curl);