HighPerformanceMathematics
/
Calcolo_Parallelo_Cluster_Steffe
mirror of https://git.phc.dm.unipi.it/3dY_0/Calcolo_Parallelo_Cluster_Steffe
2
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity
You cannot select more than 25 topics Topics must start with a letter or number, can include dashes ('-') and can be up to 35 characters long.
main
Branches Tags
${ item.name }
Create tag ${ searchTerm }
Create branch ${ searchTerm }
from '3a26062708'
${ noResults }
Calcolo_Parallelo_Cluster_S.../SortingAlg
History
edoardocoli 3a26062708 README 9 months ago
..
dataset SortingAlgorithm 9 months ago
Distributed_Sorting.2405.out README 9 months ago
Makefile various fix 9 months ago
README.md README 9 months ago
checkSort.exe MPI sorter with buffers, missing SnowPlow technique 12 months ago
doTest.sh SortingAlgorithm 9 months ago
eDistributed_Sorting.2405.err README 9 months ago
launcher.sh README 9 months ago
main.cpp SortingAlgorithm 9 months ago
main.o README 9 months ago
report.pdf SortingAlgorithm 9 months ago
sort_big_file README 9 months ago

README.md

MPI C++ Sorting Program

This folder contains a C++ program for distributed sorting using MPI (Message Passing Interface). The program is designed to hopefully efficiently sort large files (more than available composed RAM) across multiple nodes, leveraging the parallel processing capabilities of MPI.

Features

  • Parallel Sorting: MPI is utilized to distribute the sorting task across multiple nodes, enabling efficient sorting of large datasets.
  • Slurm Integration: The included Makefile facilitates job submission using Slurm, allowing users to specify job parameters such as number of nodes, partition, time, memory, etc.
  • Optimized Compilation: Compiler flags (-O3) are configured for optimization, ensuring high-performance execution.

Usage

  1. Clone the repository:

    bash
git clone https://git.phc.dm.unipi.it/3dY_0/Calcolo_Parallelo_Cluster_Steffe.git
cd Calcolo_Parallelo_Cluster_Steffe/SortingAlg

  2. Edit the Makefile to customize compiler options, file names, and other parameters based on your requirements.

  3. Compile the program:

    bash
make all

  4. Run the program with MPI and Slurm:

    bash
make run NODES="1-5"

    Make sure to replace "1-5" with the desired node range.

    It returns a file with extension '.sort' in the /mnt/raid/tmp/ directory. Make sure to have space before. Use arguments in the make as ARGS="stuff".

    make run NODES="1-5" ARGS="/path/to/file.bin"

  5. Monitor job status with squeue and cancel a job with scancel jobid if needed.

Cleaning

  • Remove object files and temporary files:

    make clean

  • Additionally, remove the executable and generated scripts:

    make fclean

  • Remove all generated files, including Slurm output files:

    make cleanall

Additional Information

  • View compiler and linker flags used by mpicxx:

    make detail

  • For a full clean, rebuild the executable using:

    make re

Feel free to customize the program, explore the code, and adapt it to suit your specific sorting requirements. If you encounter any issues or have suggestions for improvement, please open an issue or submit a pull request. Happy sorting!