Python爬蟲爬取王者榮耀英雄信息并保存到圖數(shù)據(jù)庫的操作方法
更新時間:2024年09月28日 13:58:24 作者:叁拾舞
本文介紹了如何使用Python爬蟲技術從王者榮耀官方獲取英雄信息,并將數(shù)據(jù)保存到圖數(shù)據(jù)庫中,文章詳細說明了爬取英雄名稱、類型及皮膚名稱的過程,并展示了創(chuàng)建英雄類型節(jié)點和英雄信息節(jié)點的方法
爬取信息說明
- 英雄名稱
- 英雄類型
- 英雄包含的所有皮膚名稱
創(chuàng)建英雄類型節(jié)點
王者榮耀官方給出的英雄類型是以下幾種:
直接準備好英雄類型詞典
hero_type_dict = [ '戰(zhàn)士', '法師', '坦克', '刺客', '射手', '輔助' ]
添加到圖數(shù)據(jù)庫中
def create_hero_type_node(): for hero_type in hero_type_dict: cypher = "MERGE (n:HeroType{label: '" + hero_type + "'})" graph.run(cypher).data() print('創(chuàng)建英雄類型節(jié)點成功')
創(chuàng)建英雄信息節(jié)點
獲取英雄信息
def get_hero_info_list(): # 英雄的全部信息的url hero_info = 'https://pvp.qq.com/web201605/js/herolist.json' # 獲取英雄的全部信息 response = requests.get(hero_info) # 轉為字典格式 hero_info_dict = json.loads(response.text) return hero_info_dict
打印的內容如下:
這里需要注意的是,部分英雄包含兩個英雄類別。
保存英雄信息
def create_hero_node(): hero_info_dict = get_hero_info_list() # 1戰(zhàn)士 2法師 3坦克 4刺客 5射手 6輔助 for hero in hero_info_dict: # print(hero) # print(str(hero.get('cname')) + '===' + str(hero_type[hero.get('hero_type')-1]) + '===' + str(hero.get('skin_name'))) hero_type_list = [str(hero_type_dict[hero.get('hero_type') - 1])] if '|' in str(hero.get('skin_name')): skin_name_list = hero.get('skin_name').split('|') else: skin_name_list = [hero.get('skin_name')] if 'hero_type2' in str(hero): hero_type_list.append(str(hero_type_dict[hero.get('hero_type2') - 1])) # 創(chuàng)建英雄信息節(jié)點 hero_cypher = "MERGE (n:Hero{label: '" + str(hero.get('cname')) + "'})" graph.run(hero_cypher).data() # 創(chuàng)建英雄->類型關系 for hero_type in hero_type_list: cypher_rel = "MATCH(h:Hero{label:'" + str( hero.get('cname')) + "'}),(t:HeroType{label:'" + hero_type + "'}) MERGE (h)-[r:類型]->(t) RETURN h,r,t" graph.run(cypher_rel).data() for skin_name in skin_name_list: # 創(chuàng)建英雄皮膚節(jié)點 cypher = "MERGE (n:Skin{label:'" + skin_name + "'})" graph.run(cypher).data() # 創(chuàng)建英雄->皮膚關系 cypher_rel = "MATCH(h:Hero{label:'" + str( hero.get('cname')) + "'}),(s:Skin{label:'" + skin_name + "'}) MERGE (h)-[r:皮膚]->(s) RETURN h,r,s" graph.run(cypher_rel).data() print(str(hero.get('cname')) + '===' + str(hero_type_list) + '===' + str(skin_name_list))
完整代碼
import json import requests from bs4 import BeautifulSoup from py2neo import Graph, RelationshipMatcher, NodeMatcher from dict import hero_type_dict url = "bolt://localhost:7687" username = "neo4j" password = 'Suns3535' graph = Graph(url, auth=(username, password), name="wzry") node_matcher = NodeMatcher(graph=graph) relationship_matcher = RelationshipMatcher(graph=graph) def get_hero_info_list(): # 英雄的全部信息的url hero_info = 'https://pvp.qq.com/web201605/js/herolist.json' # 獲取英雄的全部信息 response = requests.get(hero_info) # 轉為字典格式 hero_info_dict = json.loads(response.text) return hero_info_dict def create_hero_type_node(): for hero_type in hero_type_dict: cypher = "MERGE (n:HeroType{label: '" + hero_type + "'})" graph.run(cypher).data() print('創(chuàng)建英雄類型節(jié)點成功') def create_hero_node(): hero_info_dict = get_hero_info_list() # 1戰(zhàn)士 2法師 3坦克 4刺客 5射手 6輔助 for hero in hero_info_dict: # print(hero) # print(str(hero.get('cname')) + '===' + str(hero_type[hero.get('hero_type')-1]) + '===' + str(hero.get('skin_name'))) hero_type_list = [str(hero_type_dict[hero.get('hero_type') - 1])] if '|' in str(hero.get('skin_name')): skin_name_list = hero.get('skin_name').split('|') else: skin_name_list = [hero.get('skin_name')] if 'hero_type2' in str(hero): hero_type_list.append(str(hero_type_dict[hero.get('hero_type2') - 1])) # 創(chuàng)建英雄信息節(jié)點 hero_cypher = "MERGE (n:Hero{label: '" + str(hero.get('cname')) + "'})" graph.run(hero_cypher).data() # 創(chuàng)建英雄->類型關系 for hero_type in hero_type_list: cypher_rel = "MATCH(h:Hero{label:'" + str( hero.get('cname')) + "'}),(t:HeroType{label:'" + hero_type + "'}) MERGE (h)-[r:類型]->(t) RETURN h,r,t" graph.run(cypher_rel).data() for skin_name in skin_name_list: # 創(chuàng)建英雄皮膚節(jié)點 cypher = "MERGE (n:Skin{label:'" + skin_name + "'})" graph.run(cypher).data() # 創(chuàng)建英雄->皮膚關系 cypher_rel = "MATCH(h:Hero{label:'" + str( hero.get('cname')) + "'}),(s:Skin{label:'" + skin_name + "'}) MERGE (h)-[r:皮膚]->(s) RETURN h,r,s" graph.run(cypher_rel).data() print(str(hero.get('cname')) + '===' + str(hero_type_list) + '===' + str(skin_name_list)) # 創(chuàng)建英雄類型節(jié)點 create_hero_type_node() # 創(chuàng)建英雄信息 create_hero_node()
實現(xiàn)效果
到此這篇關于Python爬蟲爬取王者榮耀英雄信息并保存到圖數(shù)據(jù)庫的文章就介紹到這了,更多相關Python爬取王者榮耀英雄信息內容請搜索腳本之家以前的文章或繼續(xù)瀏覽下面的相關文章希望大家以后多多支持腳本之家!
相關文章
Python Opencv實現(xiàn)單目標檢測的示例代碼
這篇文章主要介紹了Python Opencv實現(xiàn)單目標檢測的示例代碼,文中通過示例代碼介紹的非常詳細,對大家的學習或者工作具有一定的參考學習價值,需要的朋友們下面隨著小編來一起學習學習吧2020-09-09