Quantcast
Channel: Active questions tagged python - Stack Overflow
Viewing all articles
Browse latest Browse all 23131

How to wait for all tasks from ProcessPoolExecutor to be completed before checking on with as_completed?

$
0
0
def main():    start_time = time()    cpu_cores = os.cpu_count() - LIMIT    part_size = len(CHARACTERS)//cpu_cores    for length in range(MAX_LENGTH):        with Manager() as manager:            event = manager.Event()            with ProcessPoolExecutor() as executor:                attacks = {}                for part in range(cpu_cores):                    if part == cpu_cores - 1:                        first_bit = CHARACTERS[part_size * part :]                    else:                        first_bit = CHARACTERS[part_size * part : part_size * (part+1)]                    future = executor.submit(brute_force, SECRET_STRING, first_bit, event, *([CHARACTERS] * length)).result()                    attacks[future] =  f"Brute Force {length} - {first_bit}"                for attack in as_completed(attacks):                    password = attack.result()                    if password:                        attack_type = attacks[attack]                        print(f"Password found by {attack_type} attack: {password}")                        event.set()                        break        if not password:            print(f"Password not found on length of {length}")        elif password:            break    end_time = time()    duration = end_time - start_time    print(f"Total execution time: {duration:.5f} seconds\n")
Traceback (most recent call last):  File "/home/almus/Documents/BruteForce/crack_4.0.py", line 72, in <module>    main()  File "/home/almus/Documents/BruteForce/crack_4.0.py", line 53, in main    for attack in as_completed(attacks):  File "/usr/lib/python3.10/concurrent/futures/_base.py", line 224, in as_completed    with _AcquireFutures(fs):  File "/usr/lib/python3.10/concurrent/futures/_base.py", line 151, in __enter__    future._condition.acquire()AttributeError: 'NoneType' object has no attribute '_condition'

The problem seems came from as_completed checking the map when the executor have not submitting the result.

When i use ProcessPoolExecutor and as_comppleted on something fast like multiplying every number in a list, it works well. This time i use ProcessPoolExecutor and as_completed on tasks that takes time. This is the problem, is there a way to wait for all the tasks to complete before the main code goes to as_completed?


Viewing all articles
Browse latest Browse all 23131

Trending Articles