Installation and configuration

Basic

scalems requires Python 3.8 or higher. We recommend setting up your Python for each project in a separate virtual environment.

The scalems package can then be installed from the root directory of a local copy of the SCALE-MS software repository.

Example:

$ python3 -m venv myproject
$ . myproject/bin/activate
(myproject) $ git clone https://github.com/SCALE-MS/scale-ms.git
(myproject) $ cd scale-ms
(myproject) $ python -m pip install --upgrade pip setuptools
(myproject) $ pip install .

For a more complete set of Python packages, and for the exact versions of dependencies currently used for development, use the requirements-testing.txt file in the root directory of the repository:

pip install -r requirements-testing.txt

RADICAL Pilot (RP)

Execution through the scalems.radical backend requires additional set up.

You must have a functioning RADICAL Pilot (RP) installation.

Python virtual environment

Create a new Python virtual environment. (See, for example, venv.) Activate the environment, then install scalems (and the RP software), along with any software required for your workflow.

Explicitly activate the virtual environment.

Some packages (including LAMMPS and RP) do not behave properly unless their virtual environment is explicitly activated. You must . /path/to/venv/bin/activate before installing or using radical.pilot or scalems.radical. (It is insufficient simply to use /path/to/venv/bin/python -m scalems.radical ...)

Depending on various configuration information and workflow details, RP automatically manages one or more venvs for Pilot agents and Tasks. See More notes on Python virtual environments for details.

Configuration

Many of the configurable aspects of scalems.radical only allow you to refer to resources prepared ahead of time in the filesystem.

Resource

When used in discussion of the scalems.radical module, the term resource refers to the string name of an existing RP resource definition

RP describes its targeted execution environment as a resource definition. Built-in resource definitions are provided with the radical.pilot package. To extend or override the built-in resource definitions, you must add or edit a resource configuration file in your home directory before launching scalems.radical.

Note

Password-less ssh private key is not necessary.

RP documentation for machconf implies a need to set up a password-less ssh key pair, but this is not strictly necessary. It is only necessary that RP is able to make new ssh connections at run time without storing or asking for a password. Refer to the ssh-agent documentation for your SSH client, and unlock your private key before launching your script with scalems.radical.

Setting resource parameters

https://radicalpilot.readthedocs.io/en/stable/machconf.html#writing-a-custom-resource-configuration-file describes the user files for defining new resources or replacing built-in resource definitions.

To override the default logic for a built-in resource definition, copy the JSON object for the resource(s) from your RP version (e.g. https://github.com/radical-cybertools/radical.pilot/tree/devel/src/radical/pilot/configs) to your home directory and then apply updates.

To update parameters for local.localhost:

mkdir $HOME/.radical/pilot/configs/
cp $VIRTUAL_ENV/lib/python3*/site-packages/radical/pilot/configs/resource_local.json $HOME/.radical/pilot/configs/

Then edit the localhost JSON object in $HOME/.radical/pilot/configs/resource_local.json.

Note

The resource definition determines the virtual environment in which remote Pilot agent software runs. See More notes on Python virtual environments for details.

Additional notes

RP creates many processes, threads, and files at run time. Due to its loosely coupled, asynchronous architecture, resources may not be released immediately when RP components shut down. You may have to increase your ulimit allowances, and it may not be possible to rapidly create and destroy execution sessions, especially within a single process.