python中的多cpu并行編程
多cpu并行編程
- python多線程只能算并發(fā),因為它智能使用一個cpu內(nèi)核
- python下pp包支持多cpu并行計算
安裝
pip install pp
使用
#-*- coding: UTF-8 -*- import math, sys, time import pp def IsPrime(n): """返回n是否是素數(shù)""" if not isinstance(n, int): raise TypeError("argument passed to is_prime is not of 'int' type") if n < 2: return False if n == 2: return True max = int(math.ceil(math.sqrt(n))) i = 2 while i <= max: if n % i == 0: return False i += 1 return True def SumPrimes(n): for i in xrange(15): sum([x for x in xrange(2,n) if IsPrime(x)]) """計算從2-n之間的所有素數(shù)之和""" return sum([x for x in xrange(2,n) if IsPrime(x)]) inputs = (100000, 100100, 100200, 100300, 100400, 100500, 100600, 100700) # start_time = time.time() # for input in inputs: # print SumPrimes(input) # print '單線程執(zhí)行,總耗時', time.time() - start_time, 's' # # tuple of all parallel python servers to connect with ppservers = () #ppservers = ("10.0.0.1",) if len(sys.argv) > 1: ncpus = int(sys.argv[1]) # Creates jobserver with ncpus workers job_server = pp.Server(ncpus, ppservers=ppservers) else: # Creates jobserver with automatically detected number of workers job_server = pp.Server(ppservers=ppservers) print "pp 可以用的工作核心線程數(shù)", job_server.get_ncpus(), "workers" start_time = time.time() jobs = [(input, job_server.submit(SumPrimes,(input,), (IsPrime,), ("math",))) for input in inputs]#submit提交任務(wù) for input, job in jobs: print "Sum of primes below", input, "is", job()# job()獲取方法執(zhí)行結(jié)果 print "多線程下執(zhí)行耗時: ", time.time() - start_time, "s" job_server.print_stats()#輸出執(zhí)行信息
執(zhí)行結(jié)果
pp 可以用的工作核心線程數(shù) 4 workers
Sum of primes below 100000 is 454396537
Sum of primes below 100100 is 454996777
Sum of primes below 100200 is 455898156
Sum of primes below 100300 is 456700218
Sum of primes below 100400 is 457603451
Sum of primes below 100500 is 458407033
Sum of primes below 100600 is 459412387
Sum of primes below 100700 is 460217613
多線程下執(zhí)行耗時: 15.4971420765 s
Job execution statistics:
job count | % of all jobs | job time sum | time per job | job server
8 | 100.00 | 60.9828 | 7.622844 | local
Time elapsed since server creation 15.4972219467
0 active tasks, 4 cores
submit 函數(shù)定義
def submit(self, func, args=(), depfuncs=(), modules=(), callback=None, callbackargs=(), group='default', globals=None): """Submits function to the execution queue func - function to be executed 執(zhí)行的方法 args - tuple with arguments of the 'func' 方法傳入的參數(shù),使用元組 depfuncs - tuple with functions which might be called from 'func' 傳入自己寫的方法 ,元組 modules - tuple with module names to import 傳入 模塊 callback - callback function which will be called with argument list equal to callbackargs+(result,) as soon as calculation is done callbackargs - additional arguments for callback function group - job group, is used when wait(group) is called to wait for jobs in a given group to finish globals - dictionary from which all modules, functions and classes will be imported, for instance: globals=globals() """
多核cpu并行計算
- 多進(jìn)程實現(xiàn)并行計算的簡單示例
- 這里我們開兩個進(jìn)程,計算任務(wù)也簡潔明了
# 多進(jìn)程 import multiprocessing as mp def job(q, a, b): print('aaa') q.put(a**1000+b*1000) # 把計算結(jié)果放到隊列 # 多進(jìn)程 if __name__ == '__main__': q = mp.Queue() # 一個隊列 p1 = mp.Process(target=job, args=(q, 100, 200)) p2 = mp.Process(target=job, args=(q, 100, 200)) p1.start() p1.join() # print(p1.ident) p2.start() p2.join() res = q.get() + q.get() # 讀取隊列,這里面保存了兩次計算得到的結(jié)果 print('result:', res)
以上為個人經(jīng)驗,希望能給大家一個參考,也希望大家多多支持腳本之家。
相關(guān)文章
深入講解Python函數(shù)中參數(shù)的使用及默認(rèn)參數(shù)的陷阱
這篇文章主要介紹了Python函數(shù)中參數(shù)的使用及默認(rèn)參數(shù)的陷阱,文中將函數(shù)的參數(shù)分為必選參數(shù)、默認(rèn)參數(shù)、可變參數(shù)和關(guān)鍵字參數(shù)來講,要的朋友可以參考下2016-03-03