利用Python腳本生成sitemap.xml的實(shí)現(xiàn)方法
安裝lxml
首先需要pip install lxml
安裝lxml庫。
如果你在ubuntu上遇到了以下錯(cuò)誤:
#include "libxml/xmlversion.h" compilation terminated. error: command 'x86_64-linux-gnu-gcc' failed with exit status 1 ---------------------------------------- Cleaning up... Removing temporary dir /tmp/pip_build_root... Command /usr/bin/python -c "import setuptools, tokenize;__file__='/tmp/pip_build_root/lxml/setup.py';exec(compile(getattr(tokenize, 'open', open)(__file__).read().replace('\r\n', '\n'), __file__, 'exec'))" install --record /tmp/pip-O4cIn6-record/install-record.txt --single-version-externally-managed --compile failed with error code 1 in /tmp/pip_build_root/lxml Exception information: Traceback (most recent call last): File "/usr/lib/python2.7/dist-packages/pip/basecommand.py", line 122, in main status = self.run(options, args) File "/usr/lib/python2.7/dist-packages/pip/commands/install.py", line 283, in run requirement_set.install(install_options, global_options, root=options.root_path) File "/usr/lib/python2.7/dist-packages/pip/req.py", line 1435, in install requirement.install(install_options, global_options, *args, **kwargs) File "/usr/lib/python2.7/dist-packages/pip/req.py", line 706, in install cwd=self.source_dir, filter_stdout=self._filter_install, show_stdout=False) File "/usr/lib/python2.7/dist-packages/pip/util.py", line 697, in call_subprocess % (command_desc, proc.returncode, cwd)) InstallationError: Command /usr/bin/python -c "import setuptools, tokenize;__file__='/tmp/pip_build_root/lxml/setup.py';exec(compile(getattr(tokenize, 'open', open)(__file__).read().replace('\r\n', '\n'), __file__, 'exec'))" install --record /tmp/pip-O4cIn6-record/install-record.txt --single-version-externally-managed --compile failed with error code 1 in /tmp/pip_build_root/lxml
請(qǐng)安裝以下依賴:
sudo apt-get install libxml2-dev libxslt1-dev
Python代碼
下面是生成sitemap和sitemapindex索引的代碼,可以按照需求傳入需要的參數(shù),或者增加字段:
#!/usr/bin/env python # -*- coding:utf-8 -*- import io import re from lxml import etree def generate_xml(filename, url_list): """Generate a new xml file use url_list""" root = etree.Element('urlset', xmlns="http://www.sitemaps.org/schemas/sitemap/0.9") for each in url_list: url = etree.Element('url') loc = etree.Element('loc') loc.text = each url.append(loc) root.append(url) header = u'<?xml version="1.0" encoding="UTF-8"?>\n' s = etree.tostring(root, encoding='utf-8', pretty_print=True) with io.open(filename, 'w', encoding='utf-8') as f: f.write(unicode(header+s)) def update_xml(filename, url_list): """Add new url_list to origin xml file.""" f = open(filename, 'r') lines = [i.strip() for i in f.readlines()] f.close() old_url_list = [] for each_line in lines: d = re.findall('<loc>(http:\/\/.+)<\/loc>', each_line) old_url_list += d url_list += old_url_list generate_xml(filename, url_list) def generatr_xml_index(filename, sitemap_list, lastmod_list): """Generate sitemap index xml file.""" root = etree.Element('sitemapindex', xmlns="http://www.sitemaps.org/schemas/sitemap/0.9") for each_sitemap, each_lastmod in zip(sitemap_list, lastmod_list): sitemap = etree.Element('sitemap') loc = etree.Element('loc') loc.text = each_sitemap lastmod = etree.Element('lastmod') lastmod.text = each_lastmod sitemap.append(loc) sitemap.append(lastmod) root.append(sitemap) header = u'<?xml version="1.0" encoding="UTF-8"?>\n' s = etree.tostring(root, encoding='utf-8', pretty_print=True) with io.open(filename, 'w', encoding='utf-8') as f: f.write(unicode(header+s)) if __name__ == '__main__': urls = ['http://www.baidu.com'] * 10 mods = ['2004-10-01T18:23:17+00:00'] * 10 generatr_xml_index('index.xml', urls, mods)
效果
生成的效果應(yīng)該是這種格式:
sitemap格式:
<?xml version="1.0" encoding="UTF-8"?> <urlset xmlns="http://www.sitemaps.org/schemas/sitemap/0.9"> <url> <loc>http://www.example.com/foo.html</loc> </url> </urlset>
sitemapindex格式:
<?xml version="1.0" encoding="UTF-8"?> <sitemapindex xmlns="http://www.sitemaps.org/schemas/sitemap/0.9"> <sitemap> <loc>http://www.example.com/sitemap1.xml.gz</loc> <lastmod>2004-10-01T18:23:17+00:00</lastmod> </sitemap> <sitemap> <loc>http://www.example.com/sitemap2.xml.gz</loc> <lastmod>2005-01-01</lastmod> </sitemap> </sitemapindex>
lastmod時(shí)間格式的問題
格式是用ISO 8601的標(biāo)準(zhǔn),如果是linux/unix系統(tǒng),可以使用以下函數(shù)獲取
def get_lastmod_time(filename): time_stamp = os.path.getmtime(filename) t = time.localtime(time_stamp) # return time.strftime('%Y-%m-%dT%H:%M:%S+08:00', t) return time.strftime('%Y-%m-%dT%H:%M:%SZ', t)
優(yōu)化
一般來說,用lxml效率低并且內(nèi)存占用比較大,可以直接用文件的write方法創(chuàng)建。
def generate_xml(filename, url_list): with gzip.open(filename,"w") as f: f.write("""<?xml version="1.0" encoding="utf-8"?> <urlset xmlns="http://www.sitemaps.org/schemas/sitemap/0.9">\n""") for i in url_list: f.write("""<url><loc>%s</loc></url>\n"""%i) f.write("""</urlset>""") def append_xml(filename, url_list): with gzip.open(filename, 'r') as f: for each_line in f: d = re.findall('<loc>(http:\/\/.+)<\/loc>', each_line) url_list.extend(d) generate_xml(filename, set(url_list)) def modify_time(filename): time_stamp = os.path.getmtime(filename) t = time.localtime(time_stamp) return time.strftime('%Y-%m-%dT%H:%M:%S:%SZ', t) def new_xml(filename, url_list): generate_xml(filename, url_list) root = dirname(filename) with open(join(dirname(root), "sitemap.xml"),"w") as f: f.write('<?xml version="1.0" encoding="utf-8"?>\n<sitemapindex xmlns="http://www.sitemaps.org/schemas/sitemap/0.9">\n') for i in glob.glob(join(root,"*.xml.gz")): lastmod = modify_time(i) i = i[len(CONFIG.SITEMAP_PATH):] f.write("<sitemap>\n<loc>http:/%s</loc>\n"%i) f.write("<lastmod>%s</lastmod>\n</sitemap>\n"%lastmod) f.write('</sitemapindex>')
總結(jié)
以上就是這篇文章的全部?jī)?nèi)容了,希望本文的內(nèi)容對(duì)大家學(xué)習(xí)或者使用python能帶來一定的幫助,如果有疑問大家可以留言交流。謝謝大家對(duì)腳本之家的支持。
- python生成xml時(shí)規(guī)定dtd實(shí)例方法
- Python根據(jù)指定文件生成XML的方法
- Python如何生成xml文件
- 利用 Python ElementTree 生成 xml的實(shí)例
- python 批量修改 labelImg 生成的xml文件的方法
- 對(duì)python 生成拼接xml報(bào)文的示例詳解
- 使用Python生成XML的方法實(shí)例
- Python中使用dom模塊生成XML文件示例
- python網(wǎng)絡(luò)編程學(xué)習(xí)筆記(八):XML生成與解析(DOM、ElementTree)
- python將xml xsl文件生成html文件存儲(chǔ)示例講解
- python 生成xml文件,以及美化的實(shí)例代碼
相關(guān)文章
Python數(shù)據(jù)分析numpy的Nan和Inf使用注意點(diǎn)詳解
這篇文章主要為大家介紹了Python數(shù)據(jù)分析numpy的Nan和Inf使用注意點(diǎn),有需要的朋友可以借鑒參考下,希望能夠有所幫助,祝大家多多進(jìn)步,早日升職加薪2023-08-08Python?NumPy教程之?dāng)?shù)據(jù)類型對(duì)象詳解
每個(gè)?ndarray?都有一個(gè)關(guān)聯(lián)的數(shù)據(jù)類型?(dtype)?對(duì)象。這個(gè)數(shù)據(jù)類型對(duì)象(dtype)告訴我們數(shù)組的布局。本文將通過示例詳細(xì)講講NumPy的數(shù)據(jù)類型對(duì)象,需要的可以參考一下2022-08-08OpenCV實(shí)現(xiàn)機(jī)器人對(duì)物體進(jìn)行移動(dòng)跟隨的方法實(shí)例
這篇文章主要給大家介紹了關(guān)于OpenCV實(shí)現(xiàn)機(jī)器人對(duì)物體進(jìn)行移動(dòng)跟隨的相關(guān)資料,文中通過示例代碼介紹的非常詳細(xì),對(duì)大家的學(xué)習(xí)或者工作具有一定的參考學(xué)習(xí)價(jià)值,需要的朋友們下面隨著小編來一起學(xué)習(xí)學(xué)習(xí)吧2020-11-11Python更新所有安裝的包的實(shí)現(xiàn)方式
這篇文章主要介紹了Python更新所有安裝的包的實(shí)現(xiàn)方式,具有很好的參考價(jià)值,希望對(duì)大家有所幫助,如有錯(cuò)誤或未考慮完全的地方,望不吝賜教2024-03-03Python中threading模塊join函數(shù)用法實(shí)例分析
這篇文章主要介紹了Python中threading模塊join函數(shù)用法,以實(shí)例形式較為詳細(xì)的分析了join函數(shù)的功能與使用方法,需要的朋友可以參考下2015-06-06