欧美bbbwbbbw肥妇,免费乱码人妻系列日韩,一级黄片

PHPCrawl爬蟲(chóng)庫(kù)實(shí)現(xiàn)抓取酷狗歌單的方法示例

 更新時(shí)間:2017年12月21日 10:30:18   作者:飛蓬058  
這篇文章主要介紹了PHPCrawl爬蟲(chóng)庫(kù)實(shí)現(xiàn)抓取酷狗歌單的方法,涉及PHPCrawl爬蟲(chóng)庫(kù)的使用及正則匹配相關(guān)操作技巧,需要的朋友可以參考下

本文實(shí)例講述了PHPCrawl爬蟲(chóng)庫(kù)實(shí)現(xiàn)抓取酷狗歌單的方法。分享給大家供大家參考,具體如下:

本人看了網(wǎng)絡(luò)爬蟲(chóng)相關(guān)的視頻后,手癢癢,想爬點(diǎn)什么。最近Facebook上表情包大戰(zhàn)很激烈,就想著把所有表情包都爬下來(lái),卻一時(shí)沒(méi)有找到合適的VPN,因此把酷狗最近一月精選歌曲和簡(jiǎn)單介紹抓取到本地。代碼寫(xiě)得有點(diǎn)亂,自己不是很滿意,并不想放上來(lái)丟人現(xiàn)眼。不過(guò)轉(zhuǎn)念一想,這好歹是自己第一次爬蟲(chóng),于是...就有了如下不堪入目的代碼~~~(由于抓取的數(shù)據(jù)量較小,所以沒(méi)有考慮多進(jìn)程什么的,不過(guò)我看了一下PHPCrawl的文檔,發(fā)現(xiàn)PHPCrawl庫(kù)已經(jīng)把我能想到的功能都封裝好了,實(shí)現(xiàn)起來(lái)很方便)

<?php
header("Content-type:text/html;charset=utf-8");
// It may take a whils to crawl a site ...
set_time_limit(10000);
include("libs/PHPCrawler.class.php");
class MyCrawler extends PHPCrawler {
  function handleDocumentInfo($DocInfo) {
    // Just detect linebreak for output ("\n" in CLI-mode, otherwise "<br>").
    if (PHP_SAPI == "cli") $lb = "\n";
    else $lb = "<br />";
    $url = $DocInfo->url;
    $pat = "/http:\/\/www\.kugou\.com\/yy\/special\/single\/\d+\.html/";
    if(preg_match($pat,$url) > 0){
    $this->parseSonglist($DocInfo);
    }
    flush();
  }
  public function parseSonglist($DocInfo){
    $content = $DocInfo->content;
    $songlistArr = array();
    $songlistArr['raw_url'] = $DocInfo->url;
    //解析歌曲介紹
    $matches = array();
    $pat = "/<span>名稱:<\/span>([^(<br)]+)<br/";
    $ret = preg_match($pat,$content,$matches);
    if($ret>0){
      $songlistArr['title'] = $matches[1];
    }else{
      $songlistArr['title'] = '';
    }
    //解析歌曲
    $pat = "/<a title=\"([^\"]+)\" hidefocus=\"/";
    $matches = array();
    preg_match_all($pat,$content,$matches);
    $songlistArr['songs'] = array();
    for($i = 0;$i < count($matches[0]);$i++){
      $song_title = $matches[1][$i];
      array_push($songlistArr['songs'],array('title'=>$song_title));
    }
    echo "<pre>";
    print_r($songlistArr);
    echo "</pre>";
    }
  }
$crawler = new MyCrawler();
// URL to crawl
$start_url="http://www.kugou.com/yy/special/index/1-0-2.html";
$crawler->setURL($start_url);
// Only receive content of files with content-type "text/html"
$crawler->addContentTypeReceiveRule("#text/html#");
//鏈接擴(kuò)展
$crawler->addURLFollowRule("#http://www\.kugou\.com/yy/special/single/\d+\.html$# i");
$crawler->addURLFollowRule("#http://www.kugou\.com/yy/special/index/\d+-\d+-2\.html$# i");
// Store and send cookie-data like a browser does
$crawler->enableCookieHandling(true);
// Set the traffic-limit to 1 MB(1000 * 1024) (in bytes,
// for testing we dont want to "suck" the whole site)
//爬取大小無(wú)限制
$crawler->setTrafficLimit(0);
// Thats enough, now here we go
$crawler->go();
// At the end, after the process is finished, we print a short
// report (see method getProcessReport() for more information)
$report = $crawler->getProcessReport();
if (PHP_SAPI == "cli") $lb = "\n";
else $lb = "<br />";
echo "Summary:".$lb;
echo "Links followed: ".$report->links_followed.$lb;
echo "Documents received: ".$report->files_received.$lb;
echo "Bytes received: ".$report->bytes_received." bytes".$lb;
echo "Process runtime: ".$report->process_runtime." sec".$lb; 
?>

PS:這里再為大家提供2款非常方便的正則表達(dá)式工具供大家參考使用:

JavaScript正則表達(dá)式在線測(cè)試工具:
http://tools.jb51.net/regex/javascript

正則表達(dá)式在線生成工具:
http://tools.jb51.net/regex/create_reg

更多關(guān)于PHP相關(guān)內(nèi)容感興趣的讀者可查看本站專題:《php正則表達(dá)式用法總結(jié)》、《PHP數(shù)組(Array)操作技巧大全》、《PHP基本語(yǔ)法入門(mén)教程》、《PHP運(yùn)算與運(yùn)算符用法總結(jié)》、《php面向?qū)ο蟪绦蛟O(shè)計(jì)入門(mén)教程》、《PHP網(wǎng)絡(luò)編程技巧總結(jié)》、《php字符串(string)用法總結(jié)》、《php+mysql數(shù)據(jù)庫(kù)操作入門(mén)教程》及《php常見(jiàn)數(shù)據(jù)庫(kù)操作技巧匯總

希望本文所述對(duì)大家PHP程序設(shè)計(jì)有所幫助。

相關(guān)文章

最新評(píng)論