<blockquote>使用 Laravel5.5 开发一个自动交割的项目,把使用到的开源扩展包及特性整理起来,以供后续使用。</blockquote> <h2>一、安装IDE提示工具</h2> <p>Laravel IDE Helper 是一个极其好用的代码提示及补全工具,可以给编写代码带来极大的便利。</p> <h3>1、安装</h3>
# 如果只想在开发环境安装请加上 --dev
composer require barryvdh/laravel-ide-helper
<p>安装 doctrine/dbal 「请装上它,在为模型注释字段的时候必须用到它」</p>
# 如果只想在开发环境安装请加上 --dev
composer require "doctrine/dbal: ~2.3"
<p>详细安装方法,请参考这篇博文: <a href="https://laravel-china.org/articles/10172/laravel-super-good-code-prompt-tool-laravel-ide-helper" rel="nofollow noreferrer">Laravel 超好用代码提示工具 Laravel IDE Helper</a></p> <p>三个常用命令</p> <blockquote><ul> <li>php artisan ide-helper:generate - 为 Facades 生成注释</li> <li>php artisan ide-helper:models - 为数据模型生成注释</li> <li>php artisan ide-helper:meta - 生成 PhpStorm Meta file</li> </ul></blockquote> <h2>二、Monolog日志包</h2> <p>日志的重要程度不言而喻, 不管是在开发过程中, 还是部署到生产环境后, 都是经常使用的.<br>随着 <code>psr-3</code> 的出现, 终于统一了 <code>php</code> 中日志的风格.但是, 好用的记录日志系统, 也很重要.<br><code>monolog</code> 是我遇到的最好的日志系统.而且, laravel 中也是用的 <code>monolog</code>。</p> <h3>安装</h3>
composer require monolog/monolog
<h3>用法</h3> <p>Github地址:<a href="https://github.com/Seldaek/monolog" rel="nofollow noreferrer">monolog/monolog</a></p>
<?php
use Monolog\Logger;
use Monolog\Handler\StreamHandler;
// create a log channel
$log = new Logger('name');
$log->pushHandler(new StreamHandler('path/to/your.log', Logger::WARNING));
// $logger->pushHandler(new StreamHandler(storage_path() . '/logs/spider.log'));
// add records to the log
$log->warning('Foo');
$log->error('Bar');
<h2>三、抓包工具</h2> <p><a href="https://github.com/guzzle/guzzle" rel="nofollow noreferrer">Guzzle</a> 是一个十分强大的php的模拟HTTP client的第三方库,可以通过composer安装</p> <p><a href="https://github.com/FriendsOfPHP/Goutte" rel="nofollow noreferrer">Goutte</a> 是一个用来解析HTML文档的第三方库,可以通过composer安装</p> <h3>安装</h3>
composer require fabpot/goutte
composer require guzzlehttp/guzzle
<h3>创建命令</h3>
php artisan make:command Spider
<h3>命令参数</h3>
// concurrency为并发数 keyWords为查询关键词
protected $signature = 'command:spider {concurrency} {keyWords*}';
<h3>实战</h3>
<?php
namespace App\Console\Commands;
use Goutte\Client as GoutteClient;
use GuzzleHttp\Client as GuzzleClient;
use GuzzleHttp\Pool;
use Illuminate\Console\Command;
use Monolog\Logger;
use Monolog\Handler\StreamHandler;
class Spider extends Command
{
private $totalPageCount;
private $counter = 1;
private $concurrency = 7; // 同时并发抓取
private $logger = null;
private $urls = [
'https://www.feixiaohao.com/currencies/bitcoin/', // BTC
'https://www.feixiaohao.com/currencies/decred/', // DCR
];
/**
* The name and signature of the console command.
*
* @var string
*/
protected $signature = 'test:spider-request'; //concurrency为并发数 keyWords为查询关键词
/**
* The console command description.
*
* @var string
*/
protected $description = 'php spider';
/**
* Create a new command instance.
*
* @return void
*/
public function __construct()
{
parent::__construct();
}
/**
* Execute the console command.
*
* @return mixed
*/
public function handle()
{
// 实例化一个日志实例, 参数是 channel name
$logger = new Logger('spider');
$logger->pushHandler(new StreamHandler(storage_path() . '/logs/spider.log'));
$this->totalPageCount = count($this->urls);
$guzzleClent = new GuzzleClient();
$client = new GoutteClient();
$client->setClient($guzzleClent);
$request = function ($total) use ($client){
foreach ($this->urls as $url){
yield function () use($client, $url){
return $client->request('GET',$url);
};
}
};
// @DOC http://docs.guzzlephp.org/en/stable/quickstart.html?highlight=pool
// /Users/kaiyiwang/Code/digcoin/vendor/symfony/dom-crawler/Crawler.php
$pool = new Pool($guzzleClent,$request($this->totalPageCount), [
'concurrency' => $this->concurrency,
'fulfilled' => function ($response, $index) use ($logger){
$res = $response->html();
// print_r($res);
$logger->info($res);
$this->info("请求第 $index 个请求,连接 " . $this->urls[$index]);
$this->countedAndCheckEnded();
},
'rejected' => function ($reason, $index){
$this->error("rejected" );
$this->error("rejected reason: " . $reason );
$this->countedAndCheckEnded();
},
]);
// 开始发送请求
$promise = $pool->promise();
$promise->wait();
}
public function countedAndCheckEnded()
{
if ($this->counter < $this->totalPageCount){
$this->counter++;
return;
}
$this->info("请求结束!");
}
// 运行命令:php artisan test:spider-request
}
<h3>2.项目中添加定时命令</h3> <p>在 <code>App\Console\Kernel</code> 类的 <code>schedule</code> 方法中定义预定的命令:</p>
protected function schedule(Schedule $schedule)
{
// $schedule->command('inspire')
// ->hourly();
// php artisan test:spider-request, 每十分钟调用一次
$schedule->command('test:spider-request')
->everyFifteenMinutes()->withoutOverlapping();
}
<p>添加好了之后,我们可以直接使用这个命令测试定时任务是否可以执行:</p>
> php /home/vagrant/Code/digcoin/artisan test:spider-request
<p>OK,只需要简单的两步便可实现laravel的定时任务添加。</p> <p>更多关于Laravel的任务调度,请看考该文:<a href="https://laravel-china.org/docs/laravel/5.5/scheduling/1325" rel="nofollow noreferrer">Laravel 的任务调度(计划任务)功能 Task Scheduling</a></p>
原文地址:https://segmentfault.com/a/1190000015968429
来源:oschina
链接:https://my.oschina.net/u/4408081/blog/3748353