system news

2017-06-29: Disruptive Lustre server issues, queues suspended (22:45 queues now re-enabled)

  • Posted on: 29 June 2017
  • By: nikke

Node reboots to activate security updates seems to have triggered strange disruptive behavior in the Lustre service affecting all compute resources.

We are currently diagnosing the issue in order to come up with a solution.

 

*UPDATE 2017-06-29 22:45*

Lustre now looks stable again. Queues have been re-enabled.

2017-06-28: Maintenance on the Lustre file system finished, queues running again

  • Posted on: 20 June 2017
  • By: ake

On Wednesday, June 28 08:00 - 17:00, we will perform yet another maintenance on our Lustre filesystem.

This time two of the internal UPSes need to be replaced.

To be on the safe side we will take the Lustre file system offline during this process.

This means that no jobs will be allowed to run after 08:00.

 

*Update 2017-06-28 15:00*

The maintenance is finished and the queues are running again.

2017-06-13: Resolved: PFS outage. Systems back to normal.

  • Posted on: 13 June 2017
  • By: zao

The PFS file system is having some server problems and is currently not accessible.
Due to this, it is not possible to log in to the login nodes.

The batch queues are suspended as we work on this.

We will update this news entry as we make progress and/or resolve the problem.

2017-06-13 17:45
The failing hardware have been reported to our hardware vendor.

2017-06-14 12:24
We are working with the vendor to resolve the problem.

srun works again

  • Posted on: 24 May 2017
  • By: bbrydsoe

srun works again for parallel jobs on both Abisko and Kebnekaise, so in general you should no longer use mpirun.

Note that srun and mpirun are now equivalent in most cases, but mpirun does not always work when using standard input (mpirun program < file).

2017-06-07 - 09 Major file system maintenance, all systems down

  • Posted on: 24 May 2017
  • By: ake

During 2017-06-07 - 09 we will perform a major upgrade of the disk subsystem serving the /pfs/nobackup file system.

To do this, the file system must be quiet and no jobs can run, nor can anyone access the file system from the login nodes.

Jobs will not be allowed to start unless they can finish before the maintenance window starts at 08:00 on the 7th.

Abisko maintenance window 2017-05-23 08:00 - 17:00

  • Posted on: 16 May 2017
  • By: ake

The final upgrade of Abisko to Ubuntu Xenial will take place on May 23rd.

We will then upgrade the batch server system.

This is expected to take most of the day if all goes well. If there is trouble it will take longer.

During this upgrade there can be no jobs running on the system. We have therefor put a system wide reservation in place starting 2017-05-23 08:00.

Any jobs that can't finish before that time will not be allowed to start.

Pages

Updated: 2024-03-21, 12:31