HPC2N
High Performance Computing Center North
2021-02-11 Storage system issues causes Kebnekaise login and fileaccess problems
Update: The problems with the storage system was resolved 11:30 and the Kebnekaise login nodes are available again.
Dear PIs and users at HPC2N.
As you hopefully already know our new storage system is in full production since November.
There is still some work to be done by You, the user, to make the transition to Project Storage complete.
All data in your /pfs/nobackup$HOME space must be moved to a Project Storage directory or to your $HOME space depending on the type and amount of data.
We are currently having issues with the CVMFS subsystem on all HPC2N machines. This affects local software and modules, amongst other things.
Fixing is in progress, but might take a while before everything is sorted out.
We apologize for the inconvenience.
2021-01-13 11:59
The problem has been resolved
As a side effect of the recent file system upgrade we are observing a small set of user-installed programs crashing with a bus error when loading dynamic libraries from the PFS file system. We are working with the vendor to find the root cause of these and are running some bulk operations on the file system to mitigate the problem.
The problem is elusive and may only affect a particular set of nodes and may disappear when hashing or otherwise fully reading the affected files on that node, or reinstalling the affected files in another location.
We are currently experiencing filesystem problems on the Kebnekaise cluster and logins will likely fail or hang indefinitely.
The problem is under investigation.
Update 2020-12-09 12:00
The filesystem on the login nodes is now working.
We're currently doing some minor maintenance on the storage file system.
The batch queues on Kebnekaise are stopped, therefor no new jobs will be allowed to start.
Already running jobs will continue to run.
The file system should be available most of the time but may at times access may be stalled for shorter periods.
Logins may be slower than normal.
* UPDATE 2020-12-01 11:30 *
Maintenance done and batch queues now running again.
We have a maintenance window starting 2020-11-12 08:00 for migrating to our new storage system.
This affects batch queues and login nodes for all users.
We have acquired a new storage system which will replace our old center storage.
The new storage is twice as large and will provide better performance overall for both user and project storage.
The old storage will be decommissioned and the data must therefore be migrated to the new system.
Early Saturday morning on October 10th we observed an outage in name resolution (DNS) for HPC2N hosts.
This prevented external and internal access to batch nodes, the website and more.
2020-10-10 11:00
We have restored DNS functionality and HPC2N should once again be reachable from the outside. It may take a little while for the changes to propagate out to internet providers.
Batch jobs that were running may not have run to completion and any queued jobs may have failed on startup.
UPDATE 2020-10-02 07:30 Umeå Energi will perform urgent repairs on the city cooling network between Thursday 2020-10-01 18:00 and Friday 2020-10-02 12:00.
Due to unforeseen events, the end time for Umeå Energi maintenance have been moved to Friday 2020-10-02 12:00.
Initial information
Umeå Energi will perform urgent repairs on the city cooling network between Thursday 2020-10-01 18:00 and Friday 2020-10-02 04:00.
After 9 years and almost 20 million compute jobs, it is time to retire Abisko.
On September 30 2020, Abisko will be closed for public use. From 22 September, no new jobs will be allowed to start. Any remaining jobs in the batch queues of Abisko on September 30 will be deleted.
