快捷導(dǎo)航

Pyinstaller打包Scrapy項(xiàng)目的實(shí)現(xiàn)步驟

更新時(shí)間：2020年09月22日 11:45:02 作者：vample

這篇文章主要介紹了Pyinstaller打包Scrapy項(xiàng)目的實(shí)現(xiàn)步驟，文中通過(guò)示例代碼介紹的非常詳細(xì)，對(duì)大家的學(xué)習(xí)或者工作具有一定的參考學(xué)習(xí)價(jià)值，需要的朋友們下面隨著小編來(lái)一起學(xué)習(xí)學(xué)習(xí)吧

1.安裝pyinstaller http://www.dbjr.com.cn/article/177160.htm

2.安裝pywin32 http://www.dbjr.com.cn/article/187388.htm

3.安裝其他模塊

注意點(diǎn):

scrapy用pyinstaller打包不能用

cmdline.execute('scrapy crawl douban -o test.csv --nolog'.split())

我用的是CrawlerProcess方式來(lái)輸出

舉個(gè)栗子：

1、在scrapy項(xiàng)目根目錄下建一個(gè)crawl.py(你可以自己定義)如下圖

cralw.py代碼如下

# -*- coding: utf-8 -*-
from scrapy.crawler import CrawlerProcess
from scrapy.utils.project import get_project_settings
from douban.spiders.douban_spider import Douban_spider

#打包需要的import
import urllib.robotparser
import scrapy.spiderloader
import scrapy.statscollectors
import scrapy.logformatter
import scrapy.dupefilters
import scrapy.squeues
import scrapy.extensions.spiderstate
import scrapy.extensions.corestats
import scrapy.extensions.telnet
import scrapy.extensions.logstats
import scrapy.extensions.memusage
import scrapy.extensions.memdebug
import scrapy.extensions.feedexport
import scrapy.extensions.closespider
import scrapy.extensions.debug
import scrapy.extensions.httpcache
import scrapy.extensions.statsmailer
import scrapy.extensions.throttle
import scrapy.core.scheduler
import scrapy.core.engine
import scrapy.core.scraper
import scrapy.core.spidermw
import scrapy.core.downloader
import scrapy.downloadermiddlewares.stats
import scrapy.downloadermiddlewares.httpcache
import scrapy.downloadermiddlewares.cookies
import scrapy.downloadermiddlewares.useragent
import scrapy.downloadermiddlewares.httpproxy
import scrapy.downloadermiddlewares.ajaxcrawl
import scrapy.downloadermiddlewares.chunked
import scrapy.downloadermiddlewares.decompression
import scrapy.downloadermiddlewares.defaultheaders
import scrapy.downloadermiddlewares.downloadtimeout
import scrapy.downloadermiddlewares.httpauth
import scrapy.downloadermiddlewares.httpcompression
import scrapy.downloadermiddlewares.redirect
import scrapy.downloadermiddlewares.retry
import scrapy.downloadermiddlewares.robotstxt
import scrapy.spidermiddlewares.depth
import scrapy.spidermiddlewares.httperror
import scrapy.spidermiddlewares.offsite
import scrapy.spidermiddlewares.referer
import scrapy.spidermiddlewares.urllength
import scrapy.pipelines
import scrapy.core.downloader.handlers.http
import scrapy.core.downloader.contextfactory

from douban.pipelines import DoubanPipeline
from douban.items import DoubanItem
import douban.settings

if __name__ == '__main__':
  setting = get_project_settings()
  process = CrawlerProcess(settings=setting)
  process.crawl(Douban_spider)
  process.start()

2、在crawl.py目錄下pyinstaller crawl.py 生成dist，build（可刪）和crawl.spec（可刪）。

3、在crawl.exe目錄下創(chuàng)建文件夾scrapy，然后到自己安裝的scrapy文件夾中把VERSION和mime.types兩個(gè)文件復(fù)制到剛才創(chuàng)建的scrapy文件夾中。

4、發(fā)布程序包括douban/dist 和douban/scrapy.cfg

如果沒有scrapy.cfg無(wú)法讀取settings.py和pipelines.py的配置

5、在另外一臺(tái)機(jī)器上測(cè)試成功

6、對(duì)于自定義的pipelines和settings，貌似用pyinstaller打包后的 exe無(wú)法讀取到settings和pipelines，哪位高手看看能解決這個(gè)問題？？？

到此這篇關(guān)于Pyinstaller打包Scrapy項(xiàng)目的實(shí)現(xiàn)步驟的文章就介紹到這了,更多相關(guān)Pyinstaller打包Scrapy內(nèi)容請(qǐng)搜索腳本之家以前的文章或繼續(xù)瀏覽下面的相關(guān)文章希望大家以后多多支持腳本之家！

您可能感興趣的文章: