README

SortingAlg/Distributed_Sorting.2405.out

@@ -0,0 +1,3 @@
+Usage: sort_big_file <file_to_parse>
+It returns a file with extension '.sort' in the /mnt/raid/tmp/ directory. Make sure to have space before.
+Use arguments in the make as ARGS="stuff". Example 'make run ARGS="/path/to/file"'.

SortingAlg/README.md

@@ -0,0 +1,71 @@
+# MPI C++ Sorting Program
+
+This folder contains a C++ program for distributed sorting using MPI (Message Passing Interface). The program is designed to hopefully efficiently 
+sort large files (more than available composed RAM) across multiple nodes, leveraging the parallel processing capabilities of MPI.
+
+## Features
+- **Parallel Sorting:** MPI is utilized to distribute the sorting task across multiple nodes, enabling efficient sorting of large datasets.
+- **Slurm Integration:** The included `Makefile` facilitates job submission using Slurm, allowing users to specify job parameters such as number of nodes, partition, time, memory, etc.
+- **Optimized Compilation:** Compiler flags (`-O3`) are configured for optimization, ensuring high-performance execution.
+
+## Usage
+1. Clone the repository:
+   ```
+   bash
+   git clone https://git.phc.dm.unipi.it/3dY_0/Calcolo_Parallelo_Cluster_Steffe.git
+   cd Calcolo_Parallelo_Cluster_Steffe/SortingAlg
+   ```
+
+2. Edit the `Makefile` to customize compiler options, file names, and other parameters based on your requirements.
+
+3. Compile the program:
+   ```
+   bash
+   make all
+   ```
+
+4. Run the program with MPI and Slurm:
+   ```
+   bash
+   make run NODES="1-5"
+   ```
+
+   Make sure to replace `"1-5"` with the desired node range.  
+
+   It returns a file with extension '.sort' in the /mnt/raid/tmp/ directory. Make sure to have space before.
+   Use arguments in the make as ARGS="stuff".
+   ```
+   make run NODES="1-5" ARGS="/path/to/file.bin"
+   ```
+
+
+5. Monitor job status with `squeue` and cancel a job with `scancel jobid` if needed.
+
+## Cleaning
+- Remove object files and temporary files:
+  ```bash
+  make clean
+  ```
+
+- Additionally, remove the executable and generated scripts:
+  ```bash
+  make fclean
+  ```
+
+- Remove all generated files, including Slurm output files:
+  ```bash
+  make cleanall
+  ```
+
+## Additional Information
+- View compiler and linker flags used by `mpicxx`:
+  ```bash
+  make detail
+  ```
+
+- For a full clean, rebuild the executable using:
+  ```bash
+  make re
+  ```
+
+Feel free to customize the program, explore the code, and adapt it to suit your specific sorting requirements. If you encounter any issues or have suggestions for improvement, please open an issue or submit a pull request. Happy sorting!

SortingAlg/launcher.sh

@@ -0,0 +1,22 @@
+#!/bin/bash
+## sbatch is the command line interpreter for Slurm
+
+## specify the name of the job in the queueing system
+#SBATCH --job-name=Distributed_Sorting
+## specify the partition for the resource allocation. if not specified, slurm is allowed to take the default(the one with a star *)
+#SBATCH --partition=production
+## format for time is days-hours:minutes:seconds, is used as time limit for the execution duration
+#SBATCH --time=12:00:00
+## specify the real memory required per node. suffix can be K-M-G-T but if not present is MegaBytes by default
+#SBATCH --mem=3G
+## format for hosts as a range(steffe[1-4,10-15,20]), to specify hosts needed to satisfy resource requirements
+#SBATCH --nodelist=steffe[1]
+## to specify the number of processors per task, default is one
+#SBATCH --cpus-per-task=1
+## to specify the number of tasks to be invoked on each node
+#SBATCH --ntasks-per-node=1
+## to specify the file of utput and error
+#SBATCH --output ./%x.%j.out
+#SBATCH --error ./e%x.%j.err
+
+mpirun sort_big_file