Skip to content
Snippets Groups Projects
Commit c12bbdfc authored by patmjen's avatar patmjen
Browse files

Simplify Python guide

parent 324f33d9
No related branches found
No related tags found
No related merge requests found
...@@ -23,78 +23,27 @@ sxm2sh ...@@ -23,78 +23,27 @@ sxm2sh
a100sh a100sh
``` ```
## II. First time virtualenv setup ## II. Setup Virtualenv
1. Get to a CPU or GPU node on the cluster (see [Section I](#i-preliminaries)).
1. Navigate to your project folder. 2. Navigate to your project folder.
2. Load modules for Python by entering: 3. Download `scripts/init.sh` and place it in your project folder. This **only** needs to be done the first time.
``` > **Tip:** You can do this by calling
module load python3/3.9.14 > ```bash
module load numpy/1.23.3-python-3.9.14-openblas-0.3.21 > wget https://lab.compute.dtu.dk/patmjen/hcp_tutorials/-/raw/main/scripts/init.sh
module load scipy/1.9.1-python-3.9.14 > ```
module load matplotlib/3.6.0-numpy-1.23.3-python-3.9.14 > in the terminal.
module load cuda/11.6
```
We load `numpy`, `scipy`, and `matplotlib` as modules, because the HPC team have made optimized versions for the HPC cluster.
> __NOTE:__ This guide uses Python 3.9 and CUDA 11.6 but other versions are available.
3. Create a virtualenv by running:
```
virtualenv <VENV_NAME>
```
4. Activate the virtualenv by running:
```
source <VENV_NAME>/bin/activate
```
You should now be able to install packages with pip install <PACKAGE> as normal.
> __Troubleshooting:__ If pip doesn't work, you may need to manually install it with:
> ```
> easy_install pip
> ```
## III. Virtualenv activation
> __NOTE:__ These steps must be done every time. Also if you change from a login node to a GPU node (e.g. by calling `sxm2sh`)
1. Navigate to your project folder.
2. Load modules for Python by entering:
```
module load python3/3.9.14
module load numpy/1.23.3-python-3.9.14-openblas-0.3.21
module load scipy/1.9.1-python-3.9.14
module load matplotlib/3.6.0-numpy-1.23.3-python-3.9.14
module load cuda/11.6
```
We load `numpy` and `scipy` as modules, since the HPC team have made optimized versions for the HPC cluster.
3. Activate the virtualenv by running: 4. Call
``` ```
source <VENV_NAME>/bin/activate source init.sh
``` ```
This will setup and activate your virtualenv. You must **do this every time** you log in or change node (e.g. by calling `sxm2sh`)!
> **Tip:** You can configure the Python version, CUDA version, and name and location of the virtualenv by changing the variables at the top of `init.sh`.
> __Pro tip:__ To make life easy, put these commands in a bash script called `init.sh`: 5. Your are done! You can now install packages with `pip install <PACKAGE>` and run python3 code with `python`.
> ``` > __Troubleshooting:__ If pip doesn't work, you may need to manually install it with: `easy_install pip`
> #!/bin/bash
> module load python3/3.9.14
> module load numpy/1.23.3-python-3.9.14-openblas-0.3.21
> module load scipy/1.9.1-python-3.9.14
> module load matplotlib/3.6.0-numpy-1.23.3-python-3.9.14
> module load cuda/11.6
>
> source <VENV_NAME>/bin/activate
> ```
>
> which you can then run by entering:
> ```
> source init.sh
> ```
>
> and this will prepare everything
## IV. Jupyter notebooks on ThinLinc ## IV. Jupyter notebooks on ThinLinc
...@@ -108,7 +57,7 @@ You should now be able to install packages with pip install <PACKAGE> as normal. ...@@ -108,7 +57,7 @@ You should now be able to install packages with pip install <PACKAGE> as normal.
a100sh -X a100sh -X
``` ```
3. Navigate to your project folder, load modules, and activate the virtualenv. Same steps as in section III. 3. Navigate to your project folder and activate the virtualenv. Same steps as in [section II](#ii-setup-virtualenv).
4. Install Jupyter with: 4. Install Jupyter with:
``` ```
...@@ -135,7 +84,7 @@ You should now be able to install packages with pip install <PACKAGE> as normal. ...@@ -135,7 +84,7 @@ You should now be able to install packages with pip install <PACKAGE> as normal.
1. Open a terminal on the cluster, either through ThinLinc or ssh. 1. Open a terminal on the cluster, either through ThinLinc or ssh.
2. Call `sxm2sh` or `linuxsh`, as described in section I, so you are not on a login node. 2. Call `sxm2sh` or `linuxsh`, as described in [section I](#i-preliminaries), so you are not on a login node.
3. Start a tmux session by running: 3. Start a tmux session by running:
``` ```
...@@ -153,7 +102,7 @@ You should now be able to install packages with pip install <PACKAGE> as normal. ...@@ -153,7 +102,7 @@ You should now be able to install packages with pip install <PACKAGE> as normal.
echo $HOSTNAME echo $HOSTNAME
``` ```
5. Navigate to your project folder, load modules, and activate the virtualenv. Same steps as in section III. 5. Navigate to your project folder, and activate the virtualenv. Same steps as in [section II](#ii-setup-virtualenv).
6. Start a Jupyter lab or Jupyter notebook server by entering one of the following: 6. Start a Jupyter lab or Jupyter notebook server by entering one of the following:
``` ```
......
0% Loading or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment