parallelize

Functions to parallelize of processing of a list of inputs. Adapted from https://docs.python.org/3/library/multiprocessing.html

pdm_utils.functions.parallelize.count_processors(inputs, num_processors)

Programmatically determines whether the specified num_processors is appropriate. There’s no need to use more processors than there are inputs, and it’s impossible to use fewer than 1 processor or more than exist on the machine running the code. :param inputs: list of inputs :param num_processors: specified number of processors :return: num_processors (optimized)

pdm_utils.functions.parallelize.parallelize(inputs, num_processors, task, verbose=True)

Parallelizes some task on an input list across the specified number of processors :param inputs: list of inputs :param num_processors: number of processor cores to use :param task: name of the function to run :param verbose: updating progress bar output? :return: results

pdm_utils.functions.parallelize.start_processes(inputs, num_processors, verbose)

Creates input and output queues, and runs the jobs :param inputs: jobs to run :param num_processors: optimized number of processors :param verbose: updating progress bar output? :return: results

pdm_utils.functions.parallelize.worker(input_queue, output_queue)