In the Python multiprocessing
library, is there a variant of pool.map
which supports multiple arguments?
import multiprocessing
text = "test"
def harvester(text, case):
X = case[0]
text + str(X)
if __name__ == '__main__':
pool = multiprocessing.Pool(processes=6)
case = RAW_DATASET
pool.map(harvester(text, case), case, 1)
pool.close()
pool.join()
Best Answer
Python 3.3 includes
pool.starmap()
method:For older versions:
Output
Notice how
itertools.izip()
anditertools.repeat()
are used here.Due to the bug mentioned by @unutbu you can't use
functools.partial()
or similar capabilities on Python 2.6, so the simple wrapper functionfunc_star()
should be defined explicitly. See also the workaround suggested byuptimebox
.