python中的多cpu并行編程
多cpu并行編程
- python多線程只能算并發(fā),因?yàn)樗悄苁褂靡粋€(gè)cpu內(nèi)核
- python下pp包支持多cpu并行計(jì)算
安裝
pip install pp
使用
#-*- coding: UTF-8 -*- import math, sys, time import pp def IsPrime(n): """返回n是否是素?cái)?shù)""" if not isinstance(n, int): raise TypeError("argument passed to is_prime is not of 'int' type") if n < 2: return False if n == 2: return True max = int(math.ceil(math.sqrt(n))) i = 2 while i <= max: if n % i == 0: return False i += 1 return True def SumPrimes(n): for i in xrange(15): sum([x for x in xrange(2,n) if IsPrime(x)]) """計(jì)算從2-n之間的所有素?cái)?shù)之和""" return sum([x for x in xrange(2,n) if IsPrime(x)]) inputs = (100000, 100100, 100200, 100300, 100400, 100500, 100600, 100700) # start_time = time.time() # for input in inputs: # print SumPrimes(input) # print '單線程執(zhí)行,總耗時(shí)', time.time() - start_time, 's' # # tuple of all parallel python servers to connect with ppservers = () #ppservers = ("10.0.0.1",) if len(sys.argv) > 1: ncpus = int(sys.argv[1]) # Creates jobserver with ncpus workers job_server = pp.Server(ncpus, ppservers=ppservers) else: # Creates jobserver with automatically detected number of workers job_server = pp.Server(ppservers=ppservers) print "pp 可以用的工作核心線程數(shù)", job_server.get_ncpus(), "workers" start_time = time.time() jobs = [(input, job_server.submit(SumPrimes,(input,), (IsPrime,), ("math",))) for input in inputs]#submit提交任務(wù) for input, job in jobs: print "Sum of primes below", input, "is", job()# job()獲取方法執(zhí)行結(jié)果 print "多線程下執(zhí)行耗時(shí): ", time.time() - start_time, "s" job_server.print_stats()#輸出執(zhí)行信息
執(zhí)行結(jié)果
pp 可以用的工作核心線程數(shù) 4 workers
Sum of primes below 100000 is 454396537
Sum of primes below 100100 is 454996777
Sum of primes below 100200 is 455898156
Sum of primes below 100300 is 456700218
Sum of primes below 100400 is 457603451
Sum of primes below 100500 is 458407033
Sum of primes below 100600 is 459412387
Sum of primes below 100700 is 460217613
多線程下執(zhí)行耗時(shí): 15.4971420765 s
Job execution statistics:
job count | % of all jobs | job time sum | time per job | job server
8 | 100.00 | 60.9828 | 7.622844 | local
Time elapsed since server creation 15.4972219467
0 active tasks, 4 cores
submit 函數(shù)定義
def submit(self, func, args=(), depfuncs=(), modules=(), callback=None, callbackargs=(), group='default', globals=None): """Submits function to the execution queue func - function to be executed 執(zhí)行的方法 args - tuple with arguments of the 'func' 方法傳入的參數(shù),使用元組 depfuncs - tuple with functions which might be called from 'func' 傳入自己寫的方法 ,元組 modules - tuple with module names to import 傳入 模塊 callback - callback function which will be called with argument list equal to callbackargs+(result,) as soon as calculation is done callbackargs - additional arguments for callback function group - job group, is used when wait(group) is called to wait for jobs in a given group to finish globals - dictionary from which all modules, functions and classes will be imported, for instance: globals=globals() """
多核cpu并行計(jì)算
- 多進(jìn)程實(shí)現(xiàn)并行計(jì)算的簡單示例
- 這里我們開兩個(gè)進(jìn)程,計(jì)算任務(wù)也簡潔明了
# 多進(jìn)程 import multiprocessing as mp def job(q, a, b): print('aaa') q.put(a**1000+b*1000) # 把計(jì)算結(jié)果放到隊(duì)列 # 多進(jìn)程 if __name__ == '__main__': q = mp.Queue() # 一個(gè)隊(duì)列 p1 = mp.Process(target=job, args=(q, 100, 200)) p2 = mp.Process(target=job, args=(q, 100, 200)) p1.start() p1.join() # print(p1.ident) p2.start() p2.join() res = q.get() + q.get() # 讀取隊(duì)列,這里面保存了兩次計(jì)算得到的結(jié)果 print('result:', res)
以上為個(gè)人經(jīng)驗(yàn),希望能給大家一個(gè)參考,也希望大家多多支持腳本之家。
相關(guān)文章
Python 實(shí)現(xiàn)微信自動回復(fù)的方法
這篇文章主要介紹了Python 實(shí)現(xiàn)微信自動回復(fù)的方法,幫助大家更好的理解和使用python,感興趣的朋友可以了解下2020-09-09Python?queue雙端隊(duì)列模塊及用法小結(jié)
雙端隊(duì)列是一種具有隊(duì)列和棧性質(zhì)的線性數(shù)據(jù)結(jié)構(gòu),本文主要介紹了Python?queue雙端隊(duì)列模塊及用法小結(jié),文中通過示例代碼介紹的非常詳細(xì),需要的朋友們下面隨著小編來一起學(xué)習(xí)學(xué)習(xí)吧2024-02-02Flask項(xiàng)目中實(shí)現(xiàn)短信驗(yàn)證碼和郵箱驗(yàn)證碼功能
這篇文章主要介紹了Flask項(xiàng)目中實(shí)現(xiàn)短信驗(yàn)證碼和郵箱驗(yàn)證碼功能,需本文通過截圖實(shí)例代碼的形式給大家介紹的非常詳細(xì),需要的朋友可以參考下2019-12-12深入講解Python函數(shù)中參數(shù)的使用及默認(rèn)參數(shù)的陷阱
這篇文章主要介紹了Python函數(shù)中參數(shù)的使用及默認(rèn)參數(shù)的陷阱,文中將函數(shù)的參數(shù)分為必選參數(shù)、默認(rèn)參數(shù)、可變參數(shù)和關(guān)鍵字參數(shù)來講,要的朋友可以參考下2016-03-03Python XML RPC服務(wù)器端和客戶端實(shí)例
這篇文章主要介紹了Python XML RPC服務(wù)器端和客戶端實(shí)例,本文給出了實(shí)現(xiàn)代碼以及運(yùn)行效果,需要的朋友可以參考下2014-11-11