Python-based social science applications’ profiling and optimization on HPC systems using task and data parallelism
Downloads
Published
DOI:
https://doi.org/10.58414/SCIENTIFICTEMPER.2023.14.3.48Keywords:
Python-based social science applications, High-performance computing systems, task and data parallelism, Optimization methodology, Machine learning model evaluationDimensions Badge
Issue
Section
License
Copyright (c) 2023 The Scientific Temper
![Creative Commons License](http://i.creativecommons.org/l/by-nc-sa/4.0/88x31.png)
This work is licensed under a Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International License.
This research addresses the pressing need to optimize Python-based social science applications for high-performance computing (HPC)Abstract
systems, emphasizing the combined use of task and data parallelism techniques. The paper delves into a substantial body of research,
recognizing Python’s interpreted nature as a challenge for efficient social science data processing. The paper introduces a Python
program that exemplifies the proposed methodology. This program uses task parallelism with multi-processing and data parallelism
with dask to optimize data processing workflows. It showcases how researchers can effectively manage large datasets and intricate
computations on HPC systems. The research offers a comprehensive framework for optimizing Python-based social science applications
on HPC systems. It addresses the challenges of Python’s performance limitations, data-intensive processing, and memory efficiency.
Incorporating insights from a rich literature survey, it equips researchers with valuable tools and strategies for enhancing the efficiency
of their social science applications in HPC environments.
How to Cite
Downloads