First Steps Guide

After the server has booted the newly installed Qlustar OS, log in as root and start the post-install configuration process by running the command

This will first thoroughly check your network connectivity and then complete the installation by executing the remaining configuration steps as detailed below. During the package update process, you might be asked whether to keep locally modified configuration files. In this case always choose the option

Remaining configuration steps run-through

If your chosen hostname can’t be resolved via DNS, you will see a non-fatal error message reminding you that the hostname should be registered in some (external) name service (typically DNS).

Reboot again once all the previous steps are complete by pressing Enter. After the head-node is up and running again, test its network connectivity by pinging its public IP address (hostname). Do the same for the virtual front-end node, if you have chosen to configure one. It should have booted as well, after the head-node is up and running. You can try to login to it using ssh.

A test mail should have been sent to the e-mail address(es) you specified during the installation. If you didn’t receive one, review your settings in /etc/aliases and/or /etc/postfix/main.cf. In case some of them are wrong, you can execute

to modify them.

If you have chosen to configure a virtual demo-cluster, you can start it by executing the command:

and to stop it

These commands use the configuration file /etc/qlustar/vm-configs/demo-system.conf. If you find that the (automatically calculated) amount of RAM per VM is not right, you can change the variable CN_MEM to some other value in that file. The consoles of the virtual nodes (and also of the virtual front-end node if you chose to set one up) are accessible in a screen session. Type

to attach to the console session of the virtual FE node and

to attach to the console sessions of the virtual demo cluster nodes. Note that the tmux command character is Ctrl-t. To detach from the tmux session, type Ctrl-t d, to switch to the next/previous window type Ctrl-t n / Ctrl-t p. More details on the usage of tmux is available in the corresponding man pages. To check whether all nodes are up and running, type

dsh or pdsh can be used to execute arbitrary commands on groups of nodes. Check their man pages and the corresponding section in the QluMan guide for further information.

At this stage, the following services are configured and running on your head-node:

Authenticating users in the cluster can be done in many ways, hence the creation of users depends on what method is used. The most basic method is to use NIS. If there is no requirement of keeping user authentification in sync with some external service like e.g. LDAP, this is sufficient. A NIS database is setup during the initial installation process and cluster users can be authenticated against it. Creating accounts that should use other authentification mechanisms is more complex and beyond the scope of this guide. Some options are explained in the admin manual. Add a test user by executing a command like this:

The behavior of the adduser.sh script can be customized in its configuration file /etc/qlustar/common/adduser.cf. It also contains the definition of the initial default user password.

MPI (Message Passing Interface) is the de facto standard for distributed parallel programming on Linux clusters. The default _MPI variant in Qlustar is OpenMPI and is automatically installed in the standard chroot during installation under Ubuntu and also under CentOS if OpenHPC has also been selected.

You can test the correct installation of MPI with two small hello world test programs (one in C ,the other one in FORTRAN90) as the test user you created earlier. The procedure is slightly different depending on whether you’re on Ubuntu or CentOS. Login on the front-end node as this test user and if on Ubuntu execute

On CentOS, you need to activate the correct compiler and openmpi stack from OpenHPC first via the modules infrastructure

After this you should have created two executables. Check it with

Now we’re prepared to test the queuing system with these two programs.

Before running a multi-node MPI job via slurm on the demo nodes, the slurm config most likely needs to be updated with the exact RAM values available to these nodes for slurm jobs. This is due to the fact that Qlustar nodes report their available RAM during the boot process to the qlumand on the head-node, and these values can be different to the ones originally assumed by the installer. To accomplish this, write the slurm config as explained here.

It is also possible, that the demo nodes will still be in the slurm state DRAINED after this. To check this and possibly change the state, use the Slurm Node State Management dialog of the QluMan GUI. If the nodes are indeed in state DRAINED, first restart slurmd on the nodes and then undrain them using this dialog.

Once all this is done and the nodes are in state IDLE, still being logged in as the test user and assuming at least two demo nodes are started, we can submit the two "hello world" programs created previously to slurm as follows:

This will run the job interactively on 2 nodes with 2 processes each (total of 4 processes). You should obtain an output like this:

Similarly, the F90 version can be submitted as a batch job using the script hello-world-f90-slurm.sh (to check the output, execute cat slurm-<job#>.out after the job has finished):

The Linpack benchmark is used to classify supercomputers in the Top 500 list. That’s why on most clusters, it’s probably run as one of the first parallel programs to check functionality, stability and performance. Qlustar comes with an optimized pre-compiled version of Linpack (using a current version of the OpenBlas library), and a script to auto-generate the necessary input file given the number of nodes, processes per node and total amount of RAM for the run.

The test user has some pre-defined shell aliases to simplify the submission of Linpack jobs. Type alias to see what’s available. They are defined in $HOME/.bash/alias. Example submission (assuming you have 4 running demo nodes):

Check that the job is started (output should be similar):

Now ssh to one of the nodes in the NODELIST and check with top that Linpack is running at full steam, like:

You can check the output of each Linpack run in the files: $HOME/bench/hpl/run/job-<jobid>-/openblas/job-<jobid>--<run#>.out where <<jobid>> is the slurm JOBID (see the squeue command above) and <<run#>> is an integer starting from 1. The way the script is designed, it will run indefinitely, restarting Linpack in an infinite loop. So to stop it, you need to cancel the job like