Asynchronous functions allow you to give different tasks to different members of the process pool. However, giving functions one by one is not very efficient. It would be good to be able to combine mapping with asynchronous functions, i.e. be able to give different mapping tasks simultanously to the pool of workers.

Create a new python script called `asyncmap.py`

and copy into it

asyncmap.py

```
import os
import time
from concurrent.futures import ProcessPoolExecutor
from functools import reduce
def add(x, y):
"""Return the sum of the arguments"""
print(f"Worker {os.getpid()} is processing add({x}, {y})")
time.sleep(1)
return x + y
def product(x, y):
"""Return the product of the arguments"""
print(f"Worker {os.getpid()} is processing product({x}, {y})")
time.sleep(1)
return x * y
if __name__ == "__main__":
a = [1, 2, 3, 4, 5, 6, 7, 8, 9, 10]
b = [11, 12, 13, 14, 15, 16, 17, 18, 19, 20]
# Now create a Pool of workers
with ProcessPoolExecutor() as pool:
sum_results = pool.map(add, a, b)
product_results = pool.map(product, a, b)
print("All the jobs are submitted, now we wait for results...")
total_sum = reduce(lambda x, y: x + y, sum_results)
total_product = reduce(lambda x, y: x + y, product_results)
print(f"Sum of sums of 'a' and 'b' is {total_sum}")
print(f"Sum of products of 'a' and 'b' is {total_product}")
```

Running this script using should give output similar to:

$

```
python asyncmap.py
```

This script provides two functions, `add`

and `product`

, which are mapped using the `ProcessPoolExecutor.map`

function. We didn't mention it before but the map is performed asynchronously, just like the `submit`

function was. The `map`

function does not return an explicit `Future`

object though, instead the results will be automatically waited-for as you loop through the result (in this case they are looped-through implicitly by the `reduce`

).

By default, the `ProcessPoolExecutor.map`

function divides the work over the pool of workers by assigning pieces of work one-by-one. In the example above, the work to be performed was:

```
add(1, 11)
add(2, 12)
add(3, 13)
...
add(10,20)
product(1, 11)
product(2, 12)
product(3, 13)
...
product(10, 20)
```

The work was assigned one by one to the four workers on my computer, i.e. the first worker process was given `add(1, 11)`

, the second `add(2, 12)`

, the third `add(3, 13)`

the then the fourth `add(4, 14)`

. The first worker to finish was then given `add(5, 15)`

, then the next given `add(6, 16)`

etc. etc.

Giving work one by one can be very inefficient for quick tasks, as the time needed by a worker process to stop and get new work can be longer than it takes to actually complete the task. To solve this problem, you can control how many work items are handed out to each worker process at a time. This is known as chunking, and the number of work items is known as the *chunk* of work to perform.

You can control the number of work items to perform per worker (the *chunk size*) by setting the `chunksize`

argument, e.g.

```
sum_results = pool.map(add, a, b, chunksize=5)
```

would suggest to `pool`

that each worker be given a chunk of five pieces of work. Note that this is just a suggestion, and pool may decide to use a slightly smaller or larger chunk size depending on the amount of work and the number of workers available.

Modify your `asyncmap.py`

script and set the `chunksize`

to `5`

for both of the asynchronous maps for `add`

and `product`

. Re-run your script. You should see something like;

```
All the jobs are submitted, now we wait for results...
Worker 10531 is processing add(1, 11)
Worker 10532 is processing add(6, 16)
Worker 10533 is processing product(6, 16)
Worker 10534 is processing product(1, 11)
Worker 10531 is processing add(2, 12)
Worker 10532 is processing add(7, 17)
Worker 10533 is processing product(7, 17)
Worker 10534 is processing product(2, 12)
Worker 10531 is processing add(3, 13)
Worker 10533 is processing product(8, 18)
Worker 10532 is processing add(8, 18)
Worker 10534 is processing product(3, 13)
Worker 10531 is processing add(4, 14)
Worker 10533 is processing product(9, 19)
Worker 10532 is processing add(9, 19)
Worker 10534 is processing product(4, 14)
Worker 10531 is processing add(5, 15)
Worker 10533 is processing product(10, 20)
Worker 10532 is processing add(10, 20)
Worker 10534 is processing product(5, 15)
Sum of sums of 'a' and 'b' is 210
Sum of products of 'a' and 'b' is 935
```

My laptop has four workers. The first worker is assigned the first five items of work, i.e. `add(1, 11)`

to `add(5, 15)`

, and it starts by running `add(1, 11)`

, hence why `add(1, 11)`

is printed first.

The next worker is given the next five items of work, i.e. `add(6, 16)`

to `add(10,20)`

, and starts by running `add(6, 16)`

, hence why `add(6, 16)`

is printed second.

The next worker is given the next five items of work, i.e. `product(1, 11)`

to `product(5, 15)`

, and it starts by running `product(1, 11)`

, hence why this is printed third.

The last worker is given the next five items of work, i.e. `product(6, 16)`

to `product(10, 20)`

, and it starts by running `product(6, 16)`

, hence why this is printed fourth.

Once each worker has finished its first item of work, it moves onto its second. This is why `add(2, 12)`

, `add(7, 17)`

, `product(2, 12)`

and `product(7, 17)`

are printed next. Then, each worker moves onto its third piece of work etc. etc.

If you donâ€™t specify the chunksize then it is equal to `1`

. When writing a new script you should experiment with different values of chunksize to find the value that gives best performance.

Edit your script written in answer to exercise 2 of Parallel Map/Reduce, in which you count all of the words used in all Shakespeare plays in `coundwords.py`

(e.g. an example answer is here).

Edit the script so that you set the chunk size of the `map`

.

If you get stuck or want inspiration, a possible answer is given here.