Using Python in an HPC environment

Using Python in an HPC environment

This course aims to give a brief, but comprehensive introduction to using Python in an HPC environment. You will learn how to use modules to load Python, how to find site installed Python packages, as well as how to install packages yourself. In addition, you will learn how to use virtual environments, write a batch script for running Python, use Python in parallel, and how to use Python for ML and on GPUs. 

The course is a cooperation between UPPMAX and HPC2N. The instructors will use UPPMAX's systems for demos and there will be hands-on exercises for the participants.

This course will consist of lectures interspersed with hands-on sessions where you get to try out what you have just learned.

Remote/online participation: The course will be completely online and we will use Zoom. More information about connecting and such will be sent to the participants close to the course.

The goal for the course is that you will be able to

• Load Python modules and site-installed Python packages
• Create a virtual environment
• Install Python packages with pip and/or conda
• Write a batch script for running Python
• Use Python in parallel
• Use Python for ML
• Use GPUs with Python

Prerequisites: familiarity with the LINUX command line, basic Python

Preliminary Schedule

Tuesday, 2023-05-23, 9:00-16:00

• 09:00  Introduction to Python. Logging in. Cluster info.
•            Loading modules and running Python codes 
•            Dealing with packages 
•            Creating isolated environments
• 12:00 - 13:15  Lunch
•            Batch scripts for Python jobs 
•            Parallelising a simple Python code
•            Using Python for Machine Learning jobs
•            Using GPUs with Python
•            Q&A on-demand and Summary 

Materials: Lectures and notes | Exercises | Recorded lectures and demos

Participation is free and open to all users within NAISS, whether their site has NAISS resources or not.

Please make sure you have an account at SUPR if you want to participate in the hands-on part of the training. There will be a course project that can be used to run the examples in during the hands-ons, if you do not already have an account with access. If you don't have an account at SUPR and/or UPPMAX/HPC2N, you will be contacted with further instructions for how to create those. You are encouraged to sign up to SUPR as soon as possible after registering for the course.

Course project: As part of the hands-on, you will be given temporary access to a course project at some of UPPMAX's systems, which will be used for running the hands-on examples. There are some policies regarding this, that we ask that you follow:

• You may be given access to the project before the course; please do not use the allocation for running your own codes in. Usage of the project before the course means the priority of jobs submitted to it goes down, diminishing the opportunity for you and your fellow participants to run the examples during the course.
• The course project will be open 1-2 weeks after the course, giving the participants the opportunity to test run examples and shorter codes related to the course. During this time, we ask that you only use it for running course related jobs. Use your own discretion, but it could be: (modified) examples from the hands-on, short personal codes that have been modified to test things learned at the course, etc.
• Anyone found to be misusing the course project, using up large amounts of the allocation for their own production runs, will be removed from the course project.
• You will likely also be given access to a storage area connected to the compute project. Any data you store there should be course-related and if you wish to save it you should copy it to somewhere else soon after the course as it will be deleted about a month later.

The course uses computing resources provided by the National Academic Infrastructure for Super­computing in Sweden (NAISS) at UPPMAX, partially funded by the Swedish Research Council through grant agreement no. 2022-06725.

NOTE: Kebnekaise has become a local resource. The accounts at this system are ONLY for people who are at Umeå university, one of HPC2N's partnersites (IRF, LTU, MIUN, SLU), or are in a research group with a PI at one of those. If you do have an account at HPC2N's cluster "Kebnekaise", you can use it for the hands-ons for this course. Everyone else must get an account at the NAISS resources at UPPMAX for this course.

Time and date: 23 May 2023, 09:00-16:00

Location: ONLINE. Zoom link will be sent to participants a few days before the course.

Deadline for registration: 2023-05-15 

Please register by filling in the below form. Fields marked with a * are mandatory.

Registration closed!