これは日々の作業を通して学んだことや毎日の生活で気づいたことをを記録しておく備忘録である。
HTML ファイル生成日時: 2024/12/21 11:44:57.596 (台灣標準時)
Python の multiprocessing を使って並列計算を行う方法を学んでみたでござる。
まず、十萬までの素数を探すプログラムを書いてみるでござる。
#!/usr/pkg/bin/python3.9 # # Time-stamp: <2023/07/06 22:55:42 (CST) daisuke> # # output file name file_output = 'prime_numbers_single.data' # start and end numbers n_start = 2 n_end = 10**5 # a list containing prime numbers results = [] # function to check whether or not the number is a prime number def is_prime_number (x): # resetting parameter "pn" pn = 1 # examining if the number is divisible by numbers between 2 and (x-1) for k in range (2, x): # if the number is divisible by k if (x % k == 0): # then subtract 1 from "pn" pn -= 1 # leaving from the loop break # returning number and a flag "pn" return (x, pn) # numbers to be checked for i in range (n_start, n_end + 1): # examining whether the number is a prime number result = is_prime_number (i) # appending result to the list "results" results.append (result) # opening file for writing with open (file_output, 'w') as fh_out: # for each number in the list "list_pn" for x, pn in results: # writing number to the file if (pn): fh_out.write (f'{x}\n')
上のプログラムを実行してみると以下のようになるでござる。
% time ./find_prime_numbers_single.py 17.033u 0.012s 0:17.04 100.0% 0+0k 0+112io 0pf+0w
17 秒ほどの時間がかかったでござる。
% ls -lF prime_numbers_single.data -rw-r--r-- 1 daisuke taiwan 56126 Jul 6 23:06 prime_numbers_single.data % wc prime_numbers_single.data 9592 9592 56126 prime_numbers_single.data % head prime_numbers_single.data 2 3 5 7 11 13 17 19 23 29 % tail prime_numbers_single.data 99877 99881 99901 99907 99923 99929 99961 99971 99989 99991
次に、 multiprocessing を使って、素数を並列に探すプログラムを書いてみ たでござる。 multiprocessing.Pool () を使って pool of workers を作り、 作った pool に対して .map () メソッドを使えばよいようでござる。
#!/usr/pkg/bin/python3.9 # # Time-stamp: <2023/07/06 22:56:08 (CST) daisuke> # # importing multiprocessing module import multiprocessing # number of CPUs to be used n_cpu = 5 # output file name file_output = 'prime_numbers_multiprocess.data' # start and end numbers n_start = 2 n_end = 10**5 # function to check whether or not the number is a prime number def is_prime_number (x): # resetting parameter "pn" pn = 1 # examining if the number is divisible by numbers between 2 and (x-1) for k in range (2, x): # if the number is divisible by k if (x % k == 0): # then subtract 1 from "pn" pn -= 1 # leaving from the loop break # returning number and a flag "pn" return (x, pn) # numbers to be checked numbers = range (n_start, n_end + 1) # making a pool of workers pool = multiprocessing.Pool (n_cpu) # executing function for different numbers in parallel results = pool.map (is_prime_number, numbers) # opening file for writing with open (file_output, 'w') as fh_out: # for each number in the list "list_pn" for x, pn in results: # writing number to the file if (pn): fh_out.write (f'{x}\n')
CPU core を五個使って、素数を探してみるでござる。
% time ./find_prime_numbers_multiprocess.py 17.245u 0.041s 0:04.17 414.3% 0+0k 0+112io 0pf+0w
今度は、四秒ほどで終了したでござる。
% ls -lF prime_numbers_* -rw-r--r-- 1 daisuke taiwan 56126 Jul 6 23:10 prime_numbers_multiprocess.data -rw-r--r-- 1 daisuke taiwan 56126 Jul 6 23:06 prime_numbers_single.data % wc prime_numbers_* 9592 9592 56126 prime_numbers_multiprocess.data 9592 9592 56126 prime_numbers_single.data 19184 19184 112252 total % diff prime_numbers_*