Optimizing Automated Testing with Concurrent Bash Scripts

Hello Seekers,

Automated testing plays a vital role in software development, ensuring the reliability and quality of our code. However, managing multiple processes concurrently while adhering to threshold limits can be challenging. In this article, I will share my experience and provide you with a practical solution using bash scripting.

The Challenge

During a recent project, I faced the task of executing ten processes simultaneously, with a constraint of launching only three at a time. The conventional approach of using the wait command and waiting for all processes to finish proved to be time-consuming, especially when certain scripts completed quickly.

The Solution

To address this challenge, I devised a mechanism in bash that allowed the initiation of a new process as soon as one finished. Here’s the modified code snippet with my solution incorporated:

#!/bin/bash
# This script will be our main script, which launches multiple concurrent scripts

# List all scripts you want to run
allscripts=$(cat list.txt)
totalscripts=$(cat list.txt | wc -l)

# How many scripts you want to run in parallel, let's say 3
concurrent_run=3
current_run=0
start_time=$(date +%s)

launch() {
    local index=$1
    local scriptR="$2"
    echo "Launched script $index"
    # Here, the script will be launched in the background
    bash $scriptR "$index" &
}

index=1
for script in $allscripts; do
    launch "$index" "$script"
    ((current_run++))
    ((index++))

    # This will maintain the concurrency of background processes
    while [ "$current_run" -ge "$concurrent_run" ]; do
        # This will give all the current jobs managed by this script
        job=$(jobs -p)

        # Jobs here is referred to as the background scripts
        # Modify the current_run stats
        current_run=$(echo "$job" | wc -l)
    done
done

# As we have 10 processes in the list, our loop will maintain all the concurrency.
# We can add the while condition here for waiting for the last jobs, but it will be better
# to wait for the last jobs instead of processing in while.

wait
end_time=$(date +%s)
spantime=$((end_time - start_time))
echo "Total time : $spantime"
Explanation

In the code snippet, the allscripts variable contains the names of the scripts to be executed. The concurrent_run variable sets the maximum number of processes that can run concurrently. We iterate through each script in the allscripts array and launch it in the background using the bash command. The current_run counter keeps track of the number of currently running processes.

Once the number of running processes reaches the concurrent_run limit, we use the jobs -p command to find all the currently running jobs. We modify the current_run variable based on the list of jobs available. As soon as current_run is less than concurrent_run, it breaks the while loop. This allows us to launch a new process immediately, ensuring a continuous flow of execution.

Finally, we use wait to wait for the remaining processes to finish before the script exits. By implementing this mechanism, you can maximize productivity, reduce idle waiting time, and optimize the overall completion time of your automated testing suite.

Conclusion

Incorporating bash scripting techniques for automated testing can significantly enhance efficiency and streamline your software development projects. By carefully managing the execution of multiple processes, you can minimize waiting periods and achieve faster results.

I hope this practical solution inspires you to explore the power of bash scripting in your own projects. Embrace the potential of automation and efficiency, and unlock new levels of productivity in your software development endeavors.

Thank you for joining me on this exploration of the "Optimizing Automated Testing with Concurrent Bash Scripts". Stay tuned for more insights on tech trends, career advice, and creative stories. Feel free to share your thoughts and experiences in the comments below. I'd love to hear from you!

Until next time, 

Satyam