During the installation of Qlustar, the basic configuration parameters for the cluster network had to be entered. Often, additional networks need to be defined later. This can be accomplished within the networks dialog selectable via Manage Cluster  Networks from the main windows menu.

The Networks window displays all defined networks each in a separate tab. Each tab shows all the configurable parameter that define the corresponding network. The base settings of a network are its IP address and netmask as well as an optional gateway address. QluMan distinguishes two types of networks: Primary Networks and Slave Networks.

A primary network is one that determines the IP address of a host using one of the options listed in the Configure via section of the dialog. The available options are:

A slave network on the other hand, is tied to a primary network. The idea is, that the IP of a host in the slave network is determined by mapping the host part of the IP in the primary network into the network range of the slave network. In most cases, this means that the host IP of the primary and slave networks end in the same number. This is a convenient feature, mostly used for Infiniband and IPMI networks. It saves you from registering/managing additional MAC/IP addresses and makes IPs easily recognizable as belonging to the same host.

Configuring the Boot network

Qlustar uses network-booting via PXE to start nearly all hosts in the cluster. The only system(s) that can not be network-booted is the head-node(s) itself. Every cluster should therefore have a network that is a Primary Network and configured via DHCP (boot). Such a network is created by the Qlustar installer and called Boot. Changes to this network are rarely necessary but if changes were made, all nodes must be rebooted to get the updated configuration.

Individual IP addresses and MACs of a host can be configured in the Enclosure View, once the host has been assigned to a network config. As with other settings/configs, this assignment can also be done in the Enclosure View by assigning a network config in one of the usual ways (direct assignment, host or global template).

The GUI has some safeguards to prevent invalid network configurations. For example different networks must not overlap. Attempting to change the network address so that it overlaps another network won’t be accepted: The Save button at the bottom of the window will be disabled and a red LED icon will appear with a tool-tip explaining why the selected configuration is not allowed.

When changing the network IP address or netmask, the IP addresses of all hosts configured to be in that network will be remapped to reflect the changed values. This requires that a new netmask is large enough, so that the resulting network range can include all existing hosts in the cluster. Therefore, the GUI won’t let you pick anything too small. If there are unused address ranges in the existing network and you need a smaller netmask than currently selectable, you will first have to change some host addresses so that all of them combined occupy a small enough subset of the current network.

Changing the network address IP will automatically remap the cluster internal Head IP address as well, while changing the netmask will not. Note, that the Qlustar convention, to use the second last IP of the cluster network as the Head IP, is obviously not a requirement. Hence, this is not done automatically when changing the netmask. Furthermore, changing the Head IP involves some additional steps without which the nodes in the cluster won’t function or even boot. The reason is that the Head IP also appears in the Global DHCP Template and may have been added to other templates too. These templates are simple, freely editable text blobs. A change of the network definitions will not change them, so you need to check and adjust each of them manually.

Changes to the networks definition have wide-ranging effects. To prevent accidental changes or booting hosts while in between configurations any changes to the network are not saved instantly. Instead the Save button at the bottom of the window needs to be clicked to confirm the changes. Alternatively, the Undo button can be used to revert any changes to the last saved values. Any changes to a network must be saved or reverted before switching tabs or closing the window.

If the head-node does not have direct access to the Internet, a HTTP proxy must be configured. QluMan uses this proxy to download packages from the Qlustar repository, when creating a new chroot. The proxy can be configured under Manage Cluster  Global Configs  Network Settings (Other Network Settings).

Configuring slave networks

For convenience, in a cluster with Infiniband and/or IPMI, the corresponding networks are usually setup to mirror the Boot network: If the Boot IP of a host ends in .10 then the Infiniband and IPMI IPs will usually also end in .10. Within QluMan, this relationship can be set up by configuring these special networks as a slave to the Boot network: In the tab of the network under consideration, simply select the network to be slaved to from the drop-down menu.

In a slave network, the IP of a host is always derived from its IP in the network it is slaved to. During the initial part of a node’s boot process, the qluman-execd writes the resulting static slave network information into the relevant configuration file, so that the adapter will later be configured via standard OS methods. Hence, the IPs in the slave network don’t need to be set for each host individually.

The hostnames corresponding to the IPs in the slave network are also under control of this mechanism. The name of a host in a slave network will be auto generated using the QluMan node name of the host as the stem and adding a prefix and/or postfix separated by a dash to it. The default Infiniband network setup for example has a postfix of ib, meaning a host named beo-01 will be reachable on the Infiniband network as beo-01-ib. Such pre-/postfixes may be set/changed in the configuration dialog of the network.

As part of the above mentioned reimplementation of QluMan network configuration management, a new config class Network Config has been added. It allows combining multiple network definitions (as described above) and link each of them to a physical adapter. Like any other config class, such a Network Config may then be assigned to the Global Template, Host Templates, Config Sets or individually to hosts. Every host must have exactly one assigned Network Config which must match its hardware (adapter names).

There can be any number of Network Configs, but only one is shown at a time in the corresponding dialog. To view or edit a different Network Config, select the desired entry from the drop-down menu.

The configuration of the selected Network Config is shown as a tree. The top-level items of the tree list the defined network definitions: Both the name and the network/mask of the corresponding network are shown for each entry. Below each network definition, the NIC information (device name and network type) for that network is displayed. QluMan currently supports three types of NICs: ETHER for Ethernet, IB for Infiniband/OmniPath and IPMI.

A new Network Config can be created by clicking the New button. This opens a dialog asking for the name of the new Config. Entering an unused name and pressing the Ok button will create it and select the new entry in the Networks Configs dialog. Initially this will be empty.

A Network Config may be deleted by clicking the Delete button. It can only be deleted, if it is no longer directly assigned to a host or included in a Config Set. Otherwise an error dialog will pop up describing the locations where it is still in use.

When all the networks have been defined and required Network Configs were created and assigned to a host through a template or directly, the final step of the network configuration involves the host’s individual settings.

They are displayed by selecting the host in the Enclosure View. For each network the host belongs to, the Host IP, MAC address (where applicable) and optional host aliases are shown and can be set or changed.

If the host already got a Network Config assigned at its creation time, either from the Global Template, by setting a Host Template or by copying the config from an existing host, then the boot network will already have a Host IP and MAC address filled in. Both of these are required for the host to be able to boot from the network and to receive the correct configuration at boot.

The Host IP can be entered directly or changed using the up or down arrows. Only the host part of the IP can be changed in this way, its network part is fixed and grayed out: The GUI ensures that only IPs being part of the corresponding network can be entered. If no Host IP has been manually set for the host yet, then the lowest IP in the network will be suggested and the Host IP will be color coded to indicate unsaved changes. More info about color coding and how to save changes can be found in this section.

For networks that use DHCP(boot) to configure the network adapter, the correct MAC address must be given. It must be entered as six hexadecimal bytes separated by ":". For example: 00:25:90:12:fe:cc. Again, color coding of the MAC label will show whether the entered MAC address is valid, although most invalid input is rejected by the GUI outright. For networks defined as DHCP(external), the MAC field is purely informational and not used by QluMan.

The last part of a host’s network settings are optional host aliases. These are simply alternative names under which the host can be reached and which will be added to the NIS database. Aliases are entered as a space-separated list of hostnames and must be unique. For performance reasons, the uniqueness is not fully checked by the GUI, so care must be taken to avoid collisions.

A host can have multiple names. Typically there is at least one name for each network it is connected to. The primary name of a host in QluMan is its Cluster node name, which is its name shown in the Enclosure View tree. By convention and default, the head-node is named beosrv-c and the FrontEnd node login-c. Note, that these are their names in the cluster-internal boot network and not their real hostname (displayed by the hostname command). Per default, compute nodes are named beo-<N> with <N> being a two-digit running number and their Cluster node name will also be used as their real hostname.

A host’s Cluster node name will always resolve to its IP in the boot network. It is also used as the stem, when the name of the host in networks slaved to the boot network is generated with the configured pre-/suffix of the slave. E.g. per default, the name in the IPMI network has a suffix of ipmi, which means that a host with Cluster node name beo-01 will become beo-01-ipmi in the IPMI network.

Sometimes the generated names are inconvenient to remember, or the network does not have generated names at all, when it is neither the boot nor a slave network (e.g. the external network of the FrontEnd node). In such cases, a host can be given additional names by defining host aliases for it.

Even stronger than an alias is the hostname override. The hostname override does not just add an additional name for the host, but also makes it the real hostname that is displayed by the hostname command) and will appear on the shell prompt, in logfiles or outgoing mails from that host. This is commonly used for FrontEnd nodes, so that the visible name matches the external name of the host that is used to connect to it.

For most practical purposes, Infiniband (IB) adapters need to be configured with an IP address (IPoIB) just like Ethernet adapters. If you have chosen to configure an IB network during installation, this section is mostly about how to review or change the initial settings. If not, a network definition for IB has to be created in the Networks dialog. There, a network IP address and a netmask can be chosen for the IBoIB Network.

The Infiniband network must not collide with any other network. This is prevented automatically in the settings dialog. It is convenient to define the IB network as a slave to the boot network. Then the IB IP of each host is computed by mapping the host part of its Boot IP to the IB network and no further configuration is necessary. Example: If a host’s boot network IP address is 192.168.17.100, the corresponding slaved IB IP address will become 192.168.18.100.

In order to have the IB adapter of a node configured correctly during the boot process, the network definition must also be added to the Network Config used by the host. It is not uncommon, that a cluster consists of hosts with IB and hosts without. In such cases, multiple Network Configs must be created (at least one with IB and one without IB) and assigned to the different hosts in one of the standard ways (via templates or directly). If the Network Config for a host includes a NIC of type IB, during its boot process, the necessary Infiniband kernel modules will be loaded and IP-over-IB will be set up with the IP mapping configured in the network definition.

Configuring IPMI is similar to Infiniband and also involves multiple steps, because there are a number of options to set. If you have chosen to configure an IPMI network during installation, a larger part of this section is about how to review or change the initial settings. If not, a network definition for IPMI has to be created in the Networks dialog.

There, an IPMI network address and netmask can be chosen. The IPMI network must not collide with any other network. This is prevented automatically in the settings dialog. By making the network a slave to the boot network, the IPMI IP of each host is computed by mapping the host part of its Boot IP to the IPMI Network. Example: If a host’s boot network IP address is 192.168.17.100, the corresponding slaved IPMI IP address will become 192.168.19.100.

Just as in the case of an IB adapter, the network definition for IPMI must be added to the Network Config used by the host. It is not uncommon, that a cluster consists of hosts with IPMI and hosts without. In such cases, multiple Network Configs must be created (at least one with IPMI and one without IPMI) and assigned to the different hosts in one of the standard ways (via templates or directly). If the Network Config for a host includes a NIC of type IPMI, the node is ready for monitoring its temperature and fan speeds.

Enabling IPMI nodes for remote control involves one more setting: The generic property Initialize IPMI. Per default, the settings of the IPMI cards are not touched by Qlustar as they retain their configuration across boot. However, if the Initialize IPMI generic property is assigned and set to true, the IPMI card network settings of the corresponding host will be set every time it boots. Changing the value of this property to true and after booting back to false, allows a one-time setup of the card’s network properties.

Some network settings can not be assigned to a group of nodes but relate to the cluster as a whole and how it connects to the outside world. This includes the configuration of the DNS and an optional HTTP Proxy. To configure these global network settings, select Manage Cluster  Global Configs  Network Settings.