local port | open to | reason |
---|
2811/tcp | * | gridftp control connection |
20000-25000/tcp | * | Globus port range for gridftp/xrootd data streams |
Remarque : Le caractères dièse [#], le cas échéant peut masquer les numéros de contrat et de garantie ou autres données sensibles. Les informations figurant sur cette page correspondent aux détails développés de : Produit : HP DL380 Gen9 12LFF CTO Server Numéro de série : CZJ5390FSB Numéro de produit : 719061-B21 1. Accord de support: ##### Type de garantie: Contrat Type de service: HP Foundation Care NBD Service Type de service: HP Hardware Maintenance Onsite Support* Statut: Actif Date de début: 7 oct. 2015 Date de fin: 31 oct. 2020 Type de service: HP Software Technical Unlimited Support Statut: Actif Date de début: 7 oct. 2015 Date de fin: 31 oct. 2020 Type de service: HP Collaborative Remote Support Statut: Actif Date de début: 7 oct. 2015 Date de fin: 31 oct. 2020 2. Garantie HP: CZJ5390FSB Les garanties de base avec des composants actifs peuvent être liées à votre profil en visitant la page Lier des garanties. Si votre garantie a expiré, vous pouvez acheter un HP Care Pack post-garantie à ladresse HP Care Pack Services. Type de garantie: Garantie de base Type de service: Wty: HP HW Maintenance Onsite Support* Statut: Actif Date de début: 29 sept. 2015 Date de fin: 28 oct. 2018 Niveau de service: Standard Material Handling Global Coverage NextAvail TechResource Remote Std Office Hrs Std Office Days NextAvail TechResource Onsite No Usage Limitation Next Cov Day Onsite Response Standard Parts Logistics Éléments à livrer: Onsite Support Parts and Material provided Hardware Problem Diagnosis Type de service: Wty: HP Support for Initial Setup Statut: Actif Date de début: 29 sept. 2015 Date de fin: 26 janv. 2016 Niveau de service: NextAvail TechResource Remote Std Office Hrs Std Office Days 2 Hr Remote Response Unlimited Named Callers Éléments à livrer: Initial Setup Assistance *Remarque : Selon les termes de service HP de maintencance du matérial hors site ; HP peut à sa seule discrétion décider si un défaut est réparable : À distance À l'aide d'une pièce de réparation par le client Par une demande d'intervention à l'emplacement de l'appareil défectueux Pour plus de détails consultez le document « Garantie limitée et assistance technique internationales » qui a été livré avec le produit.
[root@t3nfs02 ~]# fdisk -l 2>/dev/null | grep sd | grep -o "/dev/sd[o-z]" | uniq | xargs -iI echo /usr/lib/udev/scsi_id -g -v I | bash -x + /usr/lib/udev/scsi_id -g -v /dev/sdp 3600a098000a87a7b000001005805ae91 + /usr/lib/udev/scsi_id -g -v /dev/sdo 3600a098000a87a7b000001005805ae91 + /usr/lib/udev/scsi_id -g -v /dev/sds 3600a098000a87a7b000001005805ae91 + /usr/lib/udev/scsi_id -g -v /dev/sdu 3600a098000a87a7b000000fe5805ae3a + /usr/lib/udev/scsi_id -g -v /dev/sdq 3600a098000a87a7b000000fe5805ae3a + /usr/lib/udev/scsi_id -g -v /dev/sdr 3600a098000a87a7b000000fe5805ae3a + /usr/lib/udev/scsi_id -g -v /dev/sdt 3600a098000a87a7b000001005805ae91 + /usr/lib/udev/scsi_id -g -v /dev/sdv 3600a098000a87a7b000000fe5805ae3a
Portchannel 201 Cable 13-18908 and 13-18909 Portchannel 202 Cable 13-18910 and 13-18911 Portchannel 203 Cable 13-18912 and 13-18913 Portchannel 204 Cable 13-18914 and 13-18915 Portchannel 205 Cable 13-18916 and 13-18917 VLAN 410
parallel hdparm -t --direct /dev/sd*
[root@t3nfs02 iozone]# lsscsi | grep MB3000FCWDH | awk '{print $6}' | parallel -iI hdparm -t --direct I /dev/sdg: Timing O_DIRECT disk reads: 556 MB in 3.01 seconds = 184.87 MB/sec /dev/sdb: Timing O_DIRECT disk reads: 548 MB in 3.00 seconds = 182.38 MB/sec /dev/sdd: Timing O_DIRECT disk reads: 558 MB in 3.01 seconds = 185.60 MB/sec /dev/sdf: Timing O_DIRECT disk reads: 582 MB in 3.00 seconds = 193.71 MB/sec /dev/sdh: Timing O_DIRECT disk reads: 568 MB in 3.01 seconds = 188.79 MB/sec /dev/sdi: Timing O_DIRECT disk reads: 550 MB in 3.00 seconds = 183.03 MB/sec /dev/sdj: Timing O_DIRECT disk reads: 538 MB in 3.00 seconds = 179.11 MB/sec /dev/sda: Timing O_DIRECT disk reads: 542 MB in 3.01 seconds = 180.31 MB/sec /dev/sdc: Timing O_DIRECT disk reads: 520 MB in 3.00 seconds = 173.29 MB/sec /dev/sde: Timing O_DIRECT disk reads: 546 MB in 3.00 seconds = 181.77 MB/sec /dev/sdk: Timing O_DIRECT disk reads: 554 MB in 3.00 seconds = 184.37 MB/sec /dev/sdl: Timing O_DIRECT disk reads: 538 MB in 3.01 seconds = 178.93 MB/sec
0.6.5.7 > 0.6.5.8
[root@t3nfs02 ~]# yum update --disableplugin=* EGI-trustanchors | 2.5 kB 00:00:00 Tier3 | 2.9 kB 00:00:00 base | 3.6 kB 00:00:00 cern | 4.1 kB 00:00:00 extras | 3.4 kB 00:00:00 updates | 3.8 kB 00:00:00 zfs | 2.9 kB 00:00:00 Resolving Dependencies --> Running transaction check ---> Package libnvpair1.x86_64 0:0.6.5.7-1.el7.centos will be updated ---> Package libnvpair1.x86_64 0:0.6.5.8-1.el7.centos will be an update ---> Package libuutil1.x86_64 0:0.6.5.7-1.el7.centos will be updated ---> Package libuutil1.x86_64 0:0.6.5.8-1.el7.centos will be an update ---> Package libzfs2.x86_64 0:0.6.5.7-1.el7.centos will be updated ---> Package libzfs2.x86_64 0:0.6.5.8-1.el7.centos will be an update ---> Package libzpool2.x86_64 0:0.6.5.7-1.el7.centos will be updated ---> Package libzpool2.x86_64 0:0.6.5.8-1.el7.centos will be an update ---> Package spl.x86_64 0:0.6.5.7-1.el7.centos will be updated ---> Package spl.x86_64 0:0.6.5.8-1.el7.centos will be an update ---> Package spl-dkms.noarch 0:0.6.5.7-1.el7.centos will be updated ---> Package spl-dkms.noarch 0:0.6.5.8-1.el7.centos will be an update ---> Package zfs.x86_64 0:0.6.5.7-1.el7.centos will be updated ---> Package zfs.x86_64 0:0.6.5.8-1.el7.centos will be an update ---> Package zfs-dkms.noarch 0:0.6.5.7-1.el7.centos will be updated ---> Package zfs-dkms.noarch 0:0.6.5.8-1.el7.centos will be an update --> Finished Dependency Resolution Dependencies Resolved ================================================================================================================================================================================================================== Package Arch Version Repository Size ================================================================================================================================================================================================================== Updating: libnvpair1 x86_64 0.6.5.8-1.el7.centos zfs 35 k libuutil1 x86_64 0.6.5.8-1.el7.centos zfs 41 k libzfs2 x86_64 0.6.5.8-1.el7.centos zfs 123 k libzpool2 x86_64 0.6.5.8-1.el7.centos zfs 423 k spl x86_64 0.6.5.8-1.el7.centos zfs 29 k spl-dkms noarch 0.6.5.8-1.el7.centos zfs 443 k zfs x86_64 0.6.5.8-1.el7.centos zfs 334 k zfs-dkms noarch 0.6.5.8-1.el7.centos zfs 1.9 M Transaction Summary ================================================================================================================================================================================================================== Upgrade 8 Packages Total download size: 3.3 M Is this ok [y/d/N]: y Downloading packages: No Presto metadata available for zfs (1/8): libuutil1-0.6.5.8-1.el7.centos.x86_64.rpm | 41 kB 00:00:01 (2/8): libnvpair1-0.6.5.8-1.el7.centos.x86_64.rpm | 35 kB 00:00:01 (3/8): libzfs2-0.6.5.8-1.el7.centos.x86_64.rpm | 123 kB 00:00:00 (4/8): spl-0.6.5.8-1.el7.centos.x86_64.rpm | 29 kB 00:00:00 (5/8): libzpool2-0.6.5.8-1.el7.centos.x86_64.rpm | 423 kB 00:00:01 (6/8): zfs-0.6.5.8-1.el7.centos.x86_64.rpm | 334 kB 00:00:00 (7/8): spl-dkms-0.6.5.8-1.el7.centos.noarch.rpm | 443 kB 00:00:01 (8/8): zfs-dkms-0.6.5.8-1.el7.centos.noarch.rpm | 1.9 MB 00:00:01 ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ Total 693 kB/s | 3.3 MB 00:00:04 Running transaction check Running transaction test Transaction test succeeded Running transaction Updating : libuutil1-0.6.5.8-1.el7.centos.x86_64 1/16 Updating : libnvpair1-0.6.5.8-1.el7.centos.x86_64 2/16 Updating : libzpool2-0.6.5.8-1.el7.centos.x86_64 3/16 Updating : spl-dkms-0.6.5.8-1.el7.centos.noarch 4/16 Loading new spl-0.6.5.8 DKMS files... Building for 3.10.0-327.22.2.el7.x86_64 Building initial module for 3.10.0-327.22.2.el7.x86_64 Done. spl: Running module version sanity check. - Original module - No original module exists within this kernel - Installation - Installing to /lib/modules/3.10.0-327.22.2.el7.x86_64/extra/ splat.ko: Running module version sanity check. - Original module - No original module exists within this kernel - Installation - Installing to /lib/modules/3.10.0-327.22.2.el7.x86_64/extra/ Adding any weak-modules depmod.... DKMS: install completed. Updating : spl-0.6.5.8-1.el7.centos.x86_64 5/16 Updating : zfs-dkms-0.6.5.8-1.el7.centos.noarch 6/16 Loading new zfs-0.6.5.8 DKMS files... Building for 3.10.0-327.22.2.el7.x86_64 Building initial module for 3.10.0-327.22.2.el7.x86_64 Done. zavl: Running module version sanity check. Good news! Module version 0.6.5.8-1 for zavl.ko exactly matches what is already found in kernel 3.10.0-327.22.2.el7.x86_64. DKMS will not replace this module. You may override by specifying --force. znvpair.ko: Running module version sanity check. - Original module - No original module exists within this kernel - Installation - Installing to /lib/modules/3.10.0-327.22.2.el7.x86_64/extra/ zunicode.ko: Running module version sanity check. Good news! Module version 0.6.5.8-1 for zunicode.ko exactly matches what is already found in kernel 3.10.0-327.22.2.el7.x86_64. DKMS will not replace this module. You may override by specifying --force. zcommon.ko: Running module version sanity check. Good news! Module version 0.6.5.8-1 for zcommon.ko exactly matches what is already found in kernel 3.10.0-327.22.2.el7.x86_64. DKMS will not replace this module. You may override by specifying --force. zfs.ko: Running module version sanity check. - Original module - No original module exists within this kernel - Installation - Installing to /lib/modules/3.10.0-327.22.2.el7.x86_64/extra/ zpios.ko: Running module version sanity check. Good news! Module version 0.6.5.8-1 for zpios.ko exactly matches what is already found in kernel 3.10.0-327.22.2.el7.x86_64. DKMS will not replace this module. You may override by specifying --force. Adding any weak-modules modinfo: ERROR: Module /lib/modules/3.10.0-327.10.1.el7.x86_64/weak-updates/ not found. modinfo: ERROR: Module /lib/modules/3.10.0-327.22.2.el7.x86_64/zavl.ko not found. modprobe: FATAL: Module /lib/modules/3.10.0-327.22.2.el7.x86_64/zavl.ko not found. Warning: Module zavl.ko from kernel has no modversions, so it cannot be reused for kernel 3.10.0-327.10.1.el7.x86_64 modinfo: ERROR: Module /lib/modules/3.10.0-327.10.1.el7.x86_64/weak-updates/ not found. modinfo: ERROR: Module /lib/modules/3.10.0-327.22.2.el7.x86_64/zunicode.ko not found. modprobe: FATAL: Module /lib/modules/3.10.0-327.22.2.el7.x86_64/zunicode.ko not found. Warning: Module zunicode.ko from kernel has no modversions, so it cannot be reused for kernel 3.10.0-327.10.1.el7.x86_64 modinfo: ERROR: Module /lib/modules/3.10.0-327.10.1.el7.x86_64/weak-updates/ not found. modinfo: ERROR: Module /lib/modules/3.10.0-327.22.2.el7.x86_64/zcommon.ko not found. modprobe: FATAL: Module /lib/modules/3.10.0-327.22.2.el7.x86_64/zcommon.ko not found. Warning: Module zcommon.ko from kernel has no modversions, so it cannot be reused for kernel 3.10.0-327.10.1.el7.x86_64 modinfo: ERROR: Module /lib/modules/3.10.0-327.10.1.el7.x86_64/weak-updates/ not found. modinfo: ERROR: Module /lib/modules/3.10.0-327.22.2.el7.x86_64/zpios.ko not found. modprobe: FATAL: Module /lib/modules/3.10.0-327.22.2.el7.x86_64/zpios.ko not found. Warning: Module zpios.ko from kernel has no modversions, so it cannot be reused for kernel 3.10.0-327.10.1.el7.x86_64 modinfo: ERROR: Module /lib/modules/3.10.0-327.18.2.el7.x86_64/weak-updates/ not found. modinfo: ERROR: Module /lib/modules/3.10.0-327.22.2.el7.x86_64/zavl.ko not found. modprobe: FATAL: Module /lib/modules/3.10.0-327.22.2.el7.x86_64/zavl.ko not found. Warning: Module zavl.ko from kernel has no modversions, so it cannot be reused for kernel 3.10.0-327.18.2.el7.x86_64 modinfo: ERROR: Module /lib/modules/3.10.0-327.18.2.el7.x86_64/weak-updates/ not found. modinfo: ERROR: Module /lib/modules/3.10.0-327.22.2.el7.x86_64/zunicode.ko not found. modprobe: FATAL: Module /lib/modules/3.10.0-327.22.2.el7.x86_64/zunicode.ko not found. Warning: Module zunicode.ko from kernel has no modversions, so it cannot be reused for kernel 3.10.0-327.18.2.el7.x86_64 modinfo: ERROR: Module /lib/modules/3.10.0-327.18.2.el7.x86_64/weak-updates/ not found. modinfo: ERROR: Module /lib/modules/3.10.0-327.22.2.el7.x86_64/zcommon.ko not found. modprobe: FATAL: Module /lib/modules/3.10.0-327.22.2.el7.x86_64/zcommon.ko not found. Warning: Module zcommon.ko from kernel has no modversions, so it cannot be reused for kernel 3.10.0-327.18.2.el7.x86_64 modinfo: ERROR: Module /lib/modules/3.10.0-327.18.2.el7.x86_64/weak-updates/ not found. modinfo: ERROR: Module /lib/modules/3.10.0-327.22.2.el7.x86_64/zpios.ko not found. modprobe: FATAL: Module /lib/modules/3.10.0-327.22.2.el7.x86_64/zpios.ko not found. Warning: Module zpios.ko from kernel has no modversions, so it cannot be reused for kernel 3.10.0-327.18.2.el7.x86_64 modinfo: ERROR: Module /lib/modules/3.10.0-327.28.3.el7.x86_64/weak-updates/ not found. modinfo: ERROR: Module /lib/modules/3.10.0-327.22.2.el7.x86_64/zavl.ko not found. modprobe: FATAL: Module /lib/modules/3.10.0-327.22.2.el7.x86_64/zavl.ko not found. Warning: Module zavl.ko from kernel has no modversions, so it cannot be reused for kernel 3.10.0-327.28.3.el7.x86_64 modinfo: ERROR: Module /lib/modules/3.10.0-327.28.3.el7.x86_64/weak-updates/ not found. modinfo: ERROR: Module /lib/modules/3.10.0-327.22.2.el7.x86_64/zunicode.ko not found. modprobe: FATAL: Module /lib/modules/3.10.0-327.22.2.el7.x86_64/zunicode.ko not found. Warning: Module zunicode.ko from kernel has no modversions, so it cannot be reused for kernel 3.10.0-327.28.3.el7.x86_64 modinfo: ERROR: Module /lib/modules/3.10.0-327.28.3.el7.x86_64/weak-updates/ not found. modinfo: ERROR: Module /lib/modules/3.10.0-327.22.2.el7.x86_64/zcommon.ko not found. modprobe: FATAL: Module /lib/modules/3.10.0-327.22.2.el7.x86_64/zcommon.ko not found. Warning: Module zcommon.ko from kernel has no modversions, so it cannot be reused for kernel 3.10.0-327.28.3.el7.x86_64 modinfo: ERROR: Module /lib/modules/3.10.0-327.28.3.el7.x86_64/weak-updates/ not found. modinfo: ERROR: Module /lib/modules/3.10.0-327.22.2.el7.x86_64/zpios.ko not found. modprobe: FATAL: Module /lib/modules/3.10.0-327.22.2.el7.x86_64/zpios.ko not found. Warning: Module zpios.ko from kernel has no modversions, so it cannot be reused for kernel 3.10.0-327.28.3.el7.x86_64 depmod.... DKMS: install completed. Updating : libzfs2-0.6.5.8-1.el7.centos.x86_64 7/16 Updating : zfs-0.6.5.8-1.el7.centos.x86_64 8/16 Cleanup : zfs-0.6.5.7-1.el7.centos.x86_64 9/16 Uninstall of zfs module (version 0.6.5.7) beginning: -------- Uninstall Beginning -------- Module: zfs Version: 0.6.5.7 Kernel: 3.10.0-327.10.1.el7.x86_64 (x86_64) ------------------------------------- Status: Before uninstall, this module version was ACTIVE on this kernel. Removing any linked weak-modules rmdir: failed to remove '.': Invalid argument rmdir: failed to remove '.': Invalid argument rmdir: failed to remove '.': Invalid argument zavl.ko: - Uninstallation - Deleting from: /lib/modules/3.10.0-327.10.1.el7.x86_64/ rmdir: failed to remove ‘’: No such file or directory - Original module - No original module was found for this module on this kernel. - Use the dkms install command to reinstall any previous module version. znvpair.ko: - Uninstallation - Deleting from: /lib/modules/3.10.0-327.10.1.el7.x86_64/ rmdir: failed to remove ‘’: No such file or directory - Original module - No original module was found for this module on this kernel. - Use the dkms install command to reinstall any previous module version. zunicode.ko: - Uninstallation - Deleting from: /lib/modules/3.10.0-327.10.1.el7.x86_64/ rmdir: failed to remove ‘’: No such file or directory - Original module - No original module was found for this module on this kernel. - Use the dkms install command to reinstall any previous module version. zcommon.ko: - Uninstallation - Deleting from: /lib/modules/3.10.0-327.10.1.el7.x86_64/ rmdir: failed to remove ‘’: No such file or directory - Original module - No original module was found for this module on this kernel. - Use the dkms install command to reinstall any previous module version. zfs.ko: - Uninstallation - Deleting from: /lib/modules/3.10.0-327.10.1.el7.x86_64/extra/ - Original module - No original module was found for this module on this kernel. - Use the dkms install command to reinstall any previous module version. zpios.ko: - Uninstallation - Deleting from: /lib/modules/3.10.0-327.10.1.el7.x86_64/ rmdir: failed to remove ‘’: No such file or directory - Original module - No original module was found for this module on this kernel. - Use the dkms install command to reinstall any previous module version. depmod.... DKMS: uninstall completed. ------------------------------ Deleting module version: 0.6.5.7 completely from the DKMS tree. ------------------------------ Done. Cleanup : zfs-dkms-0.6.5.7-1.el7.centos.noarch 10/16 Cleanup : libzfs2-0.6.5.7-1.el7.centos.x86_64 11/16 Cleanup : libzpool2-0.6.5.7-1.el7.centos.x86_64 12/16 Cleanup : libnvpair1-0.6.5.7-1.el7.centos.x86_64 13/16 Cleanup : spl-0.6.5.7-1.el7.centos.x86_64 14/16 Uninstall of spl module (version 0.6.5.7) beginning: -------- Uninstall Beginning -------- Module: spl Version: 0.6.5.7 Kernel: 3.10.0-327.10.1.el7.x86_64 (x86_64) ------------------------------------- Status: Before uninstall, this module version was ACTIVE on this kernel. Removing any linked weak-modules rmdir: failed to remove '.': Invalid argument depmod: WARNING: /lib/modules/3.10.0-327.10.1.el7.x86_64/extra/zpios.ko needs unknown symbol dmu_tx_hold_write depmod: WARNING: /lib/modules/3.10.0-327.10.1.el7.x86_64/extra/zpios.ko needs unknown symbol dmu_read depmod: WARNING: /lib/modules/3.10.0-327.10.1.el7.x86_64/extra/zpios.ko needs unknown symbol dmu_tx_assign depmod: WARNING: /lib/modules/3.10.0-327.10.1.el7.x86_64/extra/zpios.ko needs unknown symbol dmu_tx_create depmod: WARNING: /lib/modules/3.10.0-327.10.1.el7.x86_64/extra/zpios.ko needs unknown symbol dmu_object_alloc depmod: WARNING: /lib/modules/3.10.0-327.10.1.el7.x86_64/extra/zpios.ko needs unknown symbol dmu_object_free depmod: WARNING: /lib/modules/3.10.0-327.10.1.el7.x86_64/extra/zpios.ko needs unknown symbol dmu_objset_own depmod: WARNING: /lib/modules/3.10.0-327.10.1.el7.x86_64/extra/zpios.ko needs unknown symbol dsl_destroy_head depmod: WARNING: /lib/modules/3.10.0-327.10.1.el7.x86_64/extra/zpios.ko needs unknown symbol dmu_write depmod: WARNING: /lib/modules/3.10.0-327.10.1.el7.x86_64/extra/zpios.ko needs unknown symbol dmu_objset_disown depmod: WARNING: /lib/modules/3.10.0-327.10.1.el7.x86_64/extra/zpios.ko needs unknown symbol dmu_tx_commit depmod: WARNING: /lib/modules/3.10.0-327.10.1.el7.x86_64/extra/zpios.ko needs unknown symbol dmu_tx_wait depmod: WARNING: /lib/modules/3.10.0-327.10.1.el7.x86_64/extra/zpios.ko needs unknown symbol dmu_tx_abort depmod: WARNING: /lib/modules/3.10.0-327.10.1.el7.x86_64/extra/zpios.ko needs unknown symbol dmu_object_set_blocksize depmod: WARNING: /lib/modules/3.10.0-327.10.1.el7.x86_64/extra/zpios.ko needs unknown symbol dmu_objset_create depmod: WARNING: /lib/modules/3.10.0-327.10.1.el7.x86_64/extra/zpios.ko needs unknown symbol dmu_tx_hold_free depmod: ERROR: fstatat(4, zfs.ko): No such file or directory depmod: WARNING: /lib/modules/3.10.0-327.18.2.el7.x86_64/weak-updates/zpios.ko needs unknown symbol dmu_tx_hold_write depmod: WARNING: /lib/modules/3.10.0-327.18.2.el7.x86_64/weak-updates/zpios.ko needs unknown symbol dmu_read depmod: WARNING: /lib/modules/3.10.0-327.18.2.el7.x86_64/weak-updates/zpios.ko needs unknown symbol dmu_tx_assign depmod: WARNING: /lib/modules/3.10.0-327.18.2.el7.x86_64/weak-updates/zpios.ko needs unknown symbol dmu_tx_create depmod: WARNING: /lib/modules/3.10.0-327.18.2.el7.x86_64/weak-updates/zpios.ko needs unknown symbol dmu_object_alloc depmod: WARNING: /lib/modules/3.10.0-327.18.2.el7.x86_64/weak-updates/zpios.ko needs unknown symbol dmu_object_free depmod: WARNING: /lib/modules/3.10.0-327.18.2.el7.x86_64/weak-updates/zpios.ko needs unknown symbol dmu_objset_own depmod: WARNING: /lib/modules/3.10.0-327.18.2.el7.x86_64/weak-updates/zpios.ko needs unknown symbol dsl_destroy_head depmod: WARNING: /lib/modules/3.10.0-327.18.2.el7.x86_64/weak-updates/zpios.ko needs unknown symbol dmu_write depmod: WARNING: /lib/modules/3.10.0-327.18.2.el7.x86_64/weak-updates/zpios.ko needs unknown symbol dmu_objset_disown depmod: WARNING: /lib/modules/3.10.0-327.18.2.el7.x86_64/weak-updates/zpios.ko needs unknown symbol dmu_tx_commit depmod: WARNING: /lib/modules/3.10.0-327.18.2.el7.x86_64/weak-updates/zpios.ko needs unknown symbol dmu_tx_wait depmod: WARNING: /lib/modules/3.10.0-327.18.2.el7.x86_64/weak-updates/zpios.ko needs unknown symbol dmu_tx_abort depmod: WARNING: /lib/modules/3.10.0-327.18.2.el7.x86_64/weak-updates/zpios.ko needs unknown symbol dmu_object_set_blocksize depmod: WARNING: /lib/modules/3.10.0-327.18.2.el7.x86_64/weak-updates/zpios.ko needs unknown symbol dmu_objset_create depmod: WARNING: /lib/modules/3.10.0-327.18.2.el7.x86_64/weak-updates/zpios.ko needs unknown symbol dmu_tx_hold_free depmod: ERROR: fstatat(4, zfs.ko): No such file or directory depmod: ERROR: fstatat(4, zfs.ko): No such file or directory depmod: WARNING: /lib/modules/3.10.0-327.28.3.el7.x86_64/weak-updates/zpios.ko needs unknown symbol dmu_tx_hold_write depmod: WARNING: /lib/modules/3.10.0-327.28.3.el7.x86_64/weak-updates/zpios.ko needs unknown symbol dmu_read depmod: WARNING: /lib/modules/3.10.0-327.28.3.el7.x86_64/weak-updates/zpios.ko needs unknown symbol dmu_tx_assign depmod: WARNING: /lib/modules/3.10.0-327.28.3.el7.x86_64/weak-updates/zpios.ko needs unknown symbol dmu_tx_create depmod: WARNING: /lib/modules/3.10.0-327.28.3.el7.x86_64/weak-updates/zpios.ko needs unknown symbol dmu_object_alloc depmod: WARNING: /lib/modules/3.10.0-327.28.3.el7.x86_64/weak-updates/zpios.ko needs unknown symbol dmu_object_free depmod: WARNING: /lib/modules/3.10.0-327.28.3.el7.x86_64/weak-updates/zpios.ko needs unknown symbol dmu_objset_own depmod: WARNING: /lib/modules/3.10.0-327.28.3.el7.x86_64/weak-updates/zpios.ko needs unknown symbol dsl_destroy_head depmod: WARNING: /lib/modules/3.10.0-327.28.3.el7.x86_64/weak-updates/zpios.ko needs unknown symbol dmu_write depmod: WARNING: /lib/modules/3.10.0-327.28.3.el7.x86_64/weak-updates/zpios.ko needs unknown symbol dmu_objset_disown depmod: WARNING: /lib/modules/3.10.0-327.28.3.el7.x86_64/weak-updates/zpios.ko needs unknown symbol dmu_tx_commit depmod: WARNING: /lib/modules/3.10.0-327.28.3.el7.x86_64/weak-updates/zpios.ko needs unknown symbol dmu_tx_wait depmod: WARNING: /lib/modules/3.10.0-327.28.3.el7.x86_64/weak-updates/zpios.ko needs unknown symbol dmu_tx_abort depmod: WARNING: /lib/modules/3.10.0-327.28.3.el7.x86_64/weak-updates/zpios.ko needs unknown symbol dmu_object_set_blocksize depmod: WARNING: /lib/modules/3.10.0-327.28.3.el7.x86_64/weak-updates/zpios.ko needs unknown symbol dmu_objset_create depmod: WARNING: /lib/modules/3.10.0-327.28.3.el7.x86_64/weak-updates/zpios.ko needs unknown symbol dmu_tx_hold_free spl.ko: - Uninstallation - Deleting from: /lib/modules/3.10.0-327.10.1.el7.x86_64/ rmdir: failed to remove ‘’: No such file or directory - Original module - No original module was found for this module on this kernel. - Use the dkms install command to reinstall any previous module version. splat.ko: - Uninstallation - Deleting from: /lib/modules/3.10.0-327.10.1.el7.x86_64/extra/ - Original module - No original module was found for this module on this kernel. - Use the dkms install command to reinstall any previous module version. depmod.... DKMS: uninstall completed. ------------------------------ Deleting module version: 0.6.5.7 completely from the DKMS tree. ------------------------------ Done. Cleanup : spl-dkms-0.6.5.7-1.el7.centos.noarch 15/16 Cleanup : libuutil1-0.6.5.7-1.el7.centos.x86_64 16/16 Verifying : libnvpair1-0.6.5.8-1.el7.centos.x86_64 1/16 Verifying : libzfs2-0.6.5.8-1.el7.centos.x86_64 2/16 Verifying : zfs-0.6.5.8-1.el7.centos.x86_64 3/16 Verifying : spl-0.6.5.8-1.el7.centos.x86_64 4/16 Verifying : libuutil1-0.6.5.8-1.el7.centos.x86_64 5/16 Verifying : zfs-dkms-0.6.5.8-1.el7.centos.noarch 6/16 Verifying : libzpool2-0.6.5.8-1.el7.centos.x86_64 7/16 Verifying : spl-dkms-0.6.5.8-1.el7.centos.noarch 8/16 Verifying : spl-0.6.5.7-1.el7.centos.x86_64 9/16 Verifying : zfs-0.6.5.7-1.el7.centos.x86_64 10/16 Verifying : libzfs2-0.6.5.7-1.el7.centos.x86_64 11/16 Verifying : libnvpair1-0.6.5.7-1.el7.centos.x86_64 12/16 Verifying : libuutil1-0.6.5.7-1.el7.centos.x86_64 13/16 Verifying : spl-dkms-0.6.5.7-1.el7.centos.noarch 14/16 Verifying : zfs-dkms-0.6.5.7-1.el7.centos.noarch 15/16 Verifying : libzpool2-0.6.5.7-1.el7.centos.x86_64 16/16 Updated: libnvpair1.x86_64 0:0.6.5.8-1.el7.centos libuutil1.x86_64 0:0.6.5.8-1.el7.centos libzfs2.x86_64 0:0.6.5.8-1.el7.centos libzpool2.x86_64 0:0.6.5.8-1.el7.centos spl.x86_64 0:0.6.5.8-1.el7.centos spl-dkms.noarch 0:0.6.5.8-1.el7.centos zfs.x86_64 0:0.6.5.8-1.el7.centos zfs-dkms.noarch 0:0.6.5.8-1.el7.centos Complete! [root@t3nfs02 ~]#
Uncorrectable Machine Check Exception
More... Close
EVENT (11 Nov 23:19): Uncorrectable Machine Check Exception (Board 0, Processor 1, APIC ID 0x00000000, Bank 0x00000011, Status 0xFE200000'000C110A, Address 0x00000000'80102000, Misc 0xA4FFE016'06100086)
Integrated Management Log Severity: CRITICAL
iLO IP: https://192.168.2.82
iLO Name: ILOCZJ5390FSB
| ProLiant Gen9, P89 07/20/2015
Server UUID: 30393137-3136-5A43-4A35-333930465342
[root@t3nfs02 ~]# hplog -v ID Severity Initial Time Update Time Count ------------------------------------------------------------- ... 0003 Critical 22:19 11/11/2015 22:19 11/11/2015 0001 LOG: Uncorrectable Machine Check Exception (Board 0, Processor 1, APIC ID 0x00000000, Bank 0x00000011, Status 0xFE200000'000C110A, Address 0x00000000'80102000, Misc 0xA4FFE016'06100086)Reaction - case number 4652815076
[235196.883815] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [235196.883815] CR2: 0000000000875e28 CR3: 0000000f46f1a000 CR4: 00000000001407f0 [235196.883816] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000 [235196.883817] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400 [235196.883817] Stack: [235196.883821] 0000000100000000 ffffffff81a68260 ffffffff819c3680 ffffffff81065d30 [235196.883824] 0000000000000000 ffff8804c99a3a00 000000000000ce1c ffff8804c99a3958 [235196.883828] ffffffff810e6f9d ffffffff819c3680 ffff8804c99a39a8 ffff8804c99a39f8 [235196.883828] Call Trace: [235196.883831] [] ? flush_tlb_func+0xb0/0xb0 [235196.883833] [ ] on_each_cpu+0x2d/0x60 [235196.883835] [ ] flush_tlb_kernel_range+0x59/0xa0 [235196.883838] [ ] __purge_vmap_area_lazy+0x1a0/0x210 [235196.883840] [ ] free_vmap_area_noflush+0x7c/0x90 [235196.883842] [ ] remove_vm_area+0x5e/0x70 [235196.883844] [ ] __vunmap+0x2a/0x100 [235196.883847] [ ] vfree+0x36/0x70 [235196.883852] [ ] spl_kmem_free_impl+0x35/0x40 [spl] [235196.883856] [ ] spl_vmem_free+0xe/0x10 [spl] [235196.883874] [ ] dmu_recv_stream+0x145/0xb90 [zfs] [235196.883880] [ ] ? nvlist_common.part.102+0x10a/0x210 [znvpair] [235197.191537] BUG: soft lockup - CPU#20 stuck for 22s! [migration/20:189] [235203.227005] md: delaying data-check of md4 until md3 has finished (they share one or more physical units) [235203.227006] md: delaying data-check of md0 until md3 has finished (they share one or more physical units) [235203.227008] md: delaying data-check of md2 until md3 has finished (they share one or more physical units) [235203.227027] md: delaying data-check of md8 until md3 has finished (they share one or more physical units) [235203.227034] md: delaying data-check of md1 until md3 has finished (they share one or more physical units) [235203.438837] md: delaying data-check of md0 until md3 has finished (they share one or more physical units) [235203.438846] md: delaying data-check of md2 until md3 has finished (they share one or more physical units) [235203.438847] md: delaying data-check of md4 until md3 has finished (they share one or more physical units) [235203.438869] md: delaying data-check of md1 until md3 has finished (they share one or more physical units) [235203.438876] md: delaying data-check of md8 until md3 has finished (they share one or more physical units) [235212.117881] INFO: rcu_sched detected stalls on CPUs/tasks: { 17} (detected by 8, t=121563377 jiffies, g=520797, c=520796, q=0) [235212.117882] sending NMI to all CPUs: [235212.117884] NMI backtrace for cpu 1 [235212.117886] CPU: 1 PID: 93 Comm: migration/1 Tainted: P W OEL ------------ 3.10.0-327.10.1.el7.x86_64 #1 [235212.117886] Hardware name: HP ProLiant DL380 Gen9, BIOS P89 07/20/2015 [235212.117887] task: ffff880853de8b80 ti: ffff880853df0000 task.ti: ffff880853df0000 [235212.117890] RIP: 0010:[ ] [ ] multi_cpu_stop+0x83/0xf0 [235212.117890] RSP: 0000:ffff880853df3d90 EFLAGS: 00000293 [235212.117891] RAX: ffffffff81661260 RBX: ffff88083da97b90 RCX: dead000000200200 [235212.117892] RDX: 0000000000000001 RSI: 0000000000000286 RDI: ffff88083da97b90 [235212.117892] RBP: ffff880853df3db0 R08: 0000000000000000 R09: 0000000000000001 [235212.117893] R10: 0000000000000001 R11: 0000000000000002 R12: 0000000000000001 [235212.117894] R13: ffff88083da97b00 R14: 0000000000000286 R15: ffff880853df3fd8 [235212.117895] FS: 0000000000000000(0000) GS:ffff88085f840000(0000) knlGS:0000000000000000 [235212.117896] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [235212.117897] CR2: 00007ffd4062a108 CR3: 000000104656a000 CR4: 00000000001407e0 [235212.117904] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000 [235212.117905] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400 [235212.117905] Stack: [235212.117910] ffff88083da97bb8 ffff88085f84dd00 ffff88083da97b90 ffffffff81103270 [235212.117914] ffff880853df3e78 ffffffff811034f8 ffff88085f84dd08 0000000000000000 [235212.117917] 0000000000000000 ffff88085f854780 ffff8810511ee400 0000000000000000 [235212.117917] Call Trace: [235212.117920] [ ] ? cpu_stop_should_run+0x50/0x50 [235212.117922] [ ] cpu_stopper_thread+0x88/0x160 [235212.117925] [ ] ? __schedule+0x2d8/0x900 [235212.117928] [ ] smpboot_thread_fn+0xff/0x1a0 [235212.117930] [ ] ? schedule+0x29/0x70 [235212.117932] [ ] ? lg_double_unlock+0x90/0x90 [235212.117935] [ ] kthread+0xcf/0xe0 [235212.117938] [ ] ? kthread_create_on_node+0x140/0x140 [235212.117940] [ ] ret_from_fork+0x58/0x90 [235212.117943] [ ] ? kthread_create_on_node+0x140/0x140 [235212.117960] Code: ed 75 65 f0 ff 4b 24 0f 94 c1 84 c9 44 89 e2 74 0f 8b 43 20 8b 73 10 8d 48 01 89 73 24 89 4b 20 83 fa 04 74 23 f3 90 44 8b 63 20 <41> 39 d4 74 f0 41 83 fc 02 75 c2 fa 66 0f 1f 44 00 00 eb c4 66 [235212.117961] NMI backtrace for cpu 3 [235212.117962] CPU: 3 PID: 103 Comm: migration/3 Tainted: P W OEL ------------ 3.10.0-327.10.1.el7.x86_64 #1 [235212.117963] Hardware name: HP ProLiant DL380 Gen9, BIOS P89 07/20/2015 [235212.117964] task: ffff880853fd0000 ti: ffff880853fc4000 task.ti: ffff880853fc4000 [235212.117966] RIP: 0010:[ ] [ ] multi_cpu_stop+0x7f/0xf0 [235212.117967] RSP: 0000:ffff880853fc7d90 EFLAGS: 00000293 [235212.117968] RAX: ffffffff81661260 RBX: ffff880499d9fb90 RCX: dead000000200200 [235212.117968] RDX: 0000000000000001 RSI: 0000000000000286 RDI: ffff880499d9fb90 [235212.117969] RBP: ffff880853fc7db0 R08: 0000000000000000 R09: 0000000000000001 [235212.117969] R10: 0000000000000001 R11: 0000000000000002 R12: 0000000000000001 [235212.117970] R13: ffff880499d9fb00 R14: 0000000000000286 R15: ffff880853fc7fd8 [235212.117971] FS: 0000000000000000(0000) GS:ffff88085f8c0000(0000) knlGS:0000000000000000 [235212.117972] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [235212.117972] CR2: 00007ffe2555ea58 CR3: 0000000eeab13000 CR4: 00000000001407e0 [235212.117973] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000 [235212.117974] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400 [235212.117974] Stack: [235212.117978] ffff880499d9fbb8 ffff88085f8cdd00 ffff880499d9fb90 ffffffff81103270 [235212.117981] ffff880853fc7e78 ffffffff811034f8 ffff88085f8cdd08 0000000000000000 [235212.117984] 0000000000000000 ffff88085f8d4780 ffff881051305780 0000000000000000 [235212.117985] Call Trace: [235212.117987] [ ] ? cpu_stop_should_run+0x50/0x50 [235212.117990] [ ] cpu_stopper_thread+0x88/0x160 [235212.117992] [ ] ? __schedule+0x2d8/0x900 [235212.117995] [ ] smpboot_thread_fn+0xff/0x1a0 [235212.117997] [ ] ? schedule+0x29/0x70 [235212.117999] [ ] ? lg_double_unlock+0x90/0x90 [235212.118002] [ ] kthread+0xcf/0xe0 [235212.118005] [ ] ? kthread_create_on_node+0x140/0x140 [235212.118007] [ ] ret_from_fork+0x58/0x90 [235212.118010] [ ] ? kthread_create_on_node+0x140/0x140 [235212.118027] Code: 75 05 45 84 ed 75 65 f0 ff 4b 24 0f 94 c1 84 c9 44 89 e2 74 0f 8b 43 20 8b 73 10 8d 48 01 89 73 24 89 4b 20 83 fa 04 74 23 f3 90 <44> 8b 63 20 41 39 d4 74 f0 41 83 fc 02 75 c2 fa 66 0f 1f 44 00 [235212.118027] NMI backtrace for cpu 23 [235212.118028] CPU: 23 PID: 204 Comm: migration/23 Tainted: P W OEL ------------ 3.10.0-327.10.1.el7.x86_64 #1 [235212.118029] Hardware name: HP ProLiant DL380 Gen9, BIOS P89 07/20/2015 [235212.118889] R13: ffff8804923f3b00 R14: 0000000000000286 R15: ffff8808538b7fd8 [235212.119223] RSP: 0018:ffff880853dbbe10 EFLAGS: 00000046 [235212.119224] RAX: 0000000000000020 RBX: 0000000000000008 RCX: 0000000000000001 [235212.119225] RDX: 0000000000000000 RSI: ffff880853dbbfd8 RDI: 000000000000001d [235212.119225] RBP: ffff880853dbbe40 R08: 0000000000000ab6 R09: 0000000000000018 [235212.119226] R10: 0000000000000b44 R11: 0000000000002e00 R12: ffff880853dbbfd8 [235212.119227] R13: 0000000000000004 R14: 0000000000000020 R15: ffffffff819fdeb8 [235212.119228] FS: 0000000000000000(0000) GS:ffff88085fcc0000(0000) knlGS:0000000000000000 [235212.119229] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [235212.119229] CR2: 00000000021b2e78 CR3: 000000000194a000 CR4: 00000000001407e0 [235212.119230] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000 [235212.119231] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400 [235212.119231] Stack: [235212.119235] 0000001d53dbbe40 df3373102ff725f9 ffffe8f8004c0200 ffffffff819fdd40 [235212.119239] 0000d617405ac97e 0000000000000004 ffff880853dbbe78 ffffffff814d4600 [235212.119242] ffffe8f8004c0200 0000000000000004 0000000000000004 ffffffff819fdd40 [235212.119243] Call Trace: [235212.119246] [ ] cpuidle_enter_state+0x40/0xc0 [235212.119249] [ ] cpuidle_idle_call+0xd9/0x210 [235212.119252] [ ] arch_cpu_idle+0xe/0x30 [235212.119254] [ ] cpu_startup_entry+0x245/0x290 [235212.119256] [ ] start_secondary+0x1ba/0x230 [235212.119274] Code: 31 d2 65 48 8b 34 25 b8 b7 00 00 48 89 d1 48 8d 86 38 c0 ff ff 0f 01 c8 48 8b 86 38 c0 ff ff a8 08 75 08 b1 01 4c 89 f0 0f 01 c9 <65> 48 8b 04 25 b8 b7 00 00 f0 80 a0 3a c0 ff ff 7f 85 1d 7a fe [235212.119275] NMI backtrace for cpu 9 [235212.119276] CPU: 9 PID: 133 Comm: migration/9 Tainted: P W OEL ------------ 3.10.0-327.10.1.el7.x86_64 #1 [235212.119277] Hardware name: HP ProLiant DL380 Gen9, BIOS P89 07/20/2015 [235212.119278] task: ffff8808538cdc00 ti: ffff88085392c000 task.ti: ffff88085392c000 [235212.119281] RIP: 0010:[ ] [ ] multi_cpu_stop+0x83/0xf0 [235212.119281] RSP: 0000:ffff88085392fd90 EFLAGS: 00000293 [235212.119282] RAX: ffffffff81661260 RBX: ffff88048801fb90 RCX: dead000000200200 [235212.119283] RDX: 0000000000000001 RSI: 0000000000000286 RDI: ffff88048801fb90 [235212.119283] RBP: ffff88085392fdb0 R08: 0000000000000000 R09: 0000000000000001 [235212.119284] R10: 0000000000000001 R11: 0000000000000002 R12: 0000000000000001 [235212.119284] R13: ffff88048801fb00 R14: 0000000000000286 R15: ffff88085392ffd8 [235212.119285] FS: 0000000000000000(0000) GS:ffff88085fa40000(0000) knlGS:0000000000000000 [235212.119286] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [235212.119287] CR2: 00007ffe93741308 CR3: 0000000e6d5aa000 CR4: 00000000001407e0 [235212.119287] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000 [235212.119288] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400 [235212.119288] Stack: [235212.119292] ffff88048801fbb8 ffff88085fa4dd00 ffff88048801fb90 ffffffff81103270 [235212.119295] ffff88085392fe78 ffffffff811034f8 ffff88085fa4dd08 0000000000000000 [235212.119299] 0000000000000000 ffff88085fa54780 ffff881051306a40 0000000000000000 [235212.119299] Call Trace: [235212.119302] [ ] ? cpu_stop_should_run+0x50/0x50 [235212.119304] [ ] cpu_stopper_thread+0x88/0x160 [235212.119307] [ ] ? __schedule+0x2d8/0x900 [235212.119310] [ ] smpboot_thread_fn+0xff/0x1a0 [235212.119312] [ ] ? schedule+0x29/0x70 [235212.119314] [ ] ? lg_double_unlock+0x90/0x90 [235212.119317] [ ] kthread+0xcf/0xe0 [235212.119319] [ ] ? kthread_create_on_node+0x140/0x140 [235212.119322] [ ] ret_from_fork+0x58/0x90 [235212.119324] [ ] ? kthread_create_on_node+0x140/0x140 [235212.119339] Code: ed 75 65 f0 ff 4b 24 0f 94 c1 84 c9 44 89 e2 74 0f 8b 43 20 8b 73 10 8d 48 01 89 73 24 89 4b 20 83 fa 04 74 23 f3 90 44 8b 63 20 <41> 39 d4 74 f0 41 83 fc 02 75 c2 fa 66 0f 1f 44 00 00 eb c4 66 [235212.119340] NMI backtrace for cpu 17 [235212.119342] CPU: 17 PID: 27871 Comm: md6_resync Tainted: P W OEL ------------ 3.10.0-327.10.1.el7.x86_64 #1 [235212.119343] Hardware name: HP ProLiant DL380 Gen9, BIOS P89 07/20/2015 [235212.119344] task: ffff88056bf12280 ti: ffff880f4ad84000 task.ti: ffff880f4ad84000 [235212.119347] RIP: 0010:[ ] [ ] native_read_tsc+0x6/0x20 [235212.119348] RSP: 0018:ffff880f4ad87a78 EFLAGS: 00000046 [235212.119349] RAX: 00000000961f1d88 RBX: 00000000961f1919 RCX: 0000000000000000 [235212.119349] RDX: 00000000000231b4 RSI: 00000000000002fd RDI: 0000000000000a2a [235212.119350] RBP: ffff880f4ad87a78 R08: ffffffff81a67fe0 R09: 0000000000000000 [235212.119351] R10: 0000000000000000 R11: ffff880f4ad879c6 R12: 0000000000000a2a [235212.119351] R13: 0000000000000011 R14: ffffffff81ca99e4 R15: 0000000000000044 [235212.119353] FS: 0000000000000000(0000) GS:ffff88105f1c0000(0000) knlGS:0000000000000000 [235212.119353] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [235212.119354] CR2: 00007fd88bdc4000 CR3: 000000000194a000 CR4: 00000000001407e0 [235212.119355] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000 [235212.119355] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400 [235212.119356] Stack: [235212.119360] ffff880f4ad87aa0 ffffffff8130053a ffffffff81f17328 0000000000002335 [235212.119364] 0000000000000020 ffff880f4ad87ab0 ffffffff81300488 ffff880f4ad87ad8 [235212.119367] ffffffff813d0a20 ffffffff81f17328 0000000000000070 ffffffff81f17328 [235212.119368] Call Trace: [235212.119370] [ ] delay_tsc+0x4a/0x80 [235212.119373] [ ] __const_udelay+0x28/0x30 [235212.119375] [ ] wait_for_xmitr+0x30/0xa0 [235212.119378] [ ] serial8250_console_putchar+0x1c/0x30 [235212.119380] [ ] ? serial8250_co
[422740.330753] BUG: soft lockup - CPU#13 stuck for 22s! [khugepaged:305] [422740.330777] Modules linked in: binfmt_misc bonding zfs(POE) zunicode(POE) zavl(POE) zcommon(POE) znvpair(POE) intel_powerclamp spl(OE) coretemp vfat fat zlib_deflate intel_rapl kvm_intel kvm ipmi_ssif crc32_pclmul iTCO_wdt ghash_clmulni_intel iTCO_vendor_support aesni_intel lrw ses gf128mul enclosure glue_helper ipmi_si sb_edac ablk_helper hpwdt pcspkr lpc_ich hpilo sg cryptd pcc_cpufreq i2c_i801 ioatdma ipmi_msghandler edac_core mfd_core shpchp wmi acpi_power_meter nfsd auth_rpcgss nfs_acl lockd grace openafs(POE) sunrpc ip_tables xfs libcrc32c raid1 sd_mod crc_t10dif crct10dif_generic mgag200 syscopyarea sysfillrect sysimgblt i2c_algo_bit ixgbe drm_kms_helper mdio ttm tg3 crct10dif_pclmul dca crct10dif_common drm crc32c_intel ptp i2c_core hpsa pps_core [422740.330779] CPU: 13 PID: 305 Comm: khugepaged Tainted: P W OEL ------------ 3.10.0-327.36.1.el7.x86_64 #1 [422740.330779] Hardware name: HP ProLiant DL380 Gen9/ProLiant DL380 Gen9, BIOS P89 06/02/2016 [422740.330780] task: ffff8808515a3980 ti: ffff8808515a8000 task.ti: ffff8808515a8000 [422740.330782] RIP: 0010:[] [ ] smp_call_function_many+0x202/0x260 [422740.330782] RSP: 0018:ffff8808515abbb8 EFLAGS: 00000202 [422740.330783] RAX: 000000000000000a RBX: 000000280000000d RCX: ffff88105f01a9d8 [422740.330783] RDX: 000000000000000a RSI: 0000000000000028 RDI: 0000000000000000 [422740.330784] RBP: ffff8808515abbf0 R08: ffff881053d15000 R09: ffff88105f0d9620 [422740.330938] R10: ffffea0006feb600 R11: ffffffff812f2a59 R12: 000000fc812f29af [422740.330939] R13: 0000000000000296 R14: 0000000000000296 R15: ffff8808515abb68 [422740.330940] FS: 0000000000000000(0000) GS:ffff88105f0c0000(0000) knlGS:0000000000000000 [422740.330941] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [422740.330941] CR2: 00007efd14a97000 CR3: 000000000194a000 CR4: 00000000001407e0 [422740.330941] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000 [422740.330942] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400 [422740.330942] Stack: [422740.330944] 00000001000001fe ffffffff81e86580 ffffffff81e86580 ffffffff81171d80 [422740.330946] 0000000000000000 000000000000000d ffff88087ffda000 ffff8808515abc20 [422740.330948] ffffffff810e71ca 0000000000000027 0000000000000028 0000000000000028 [422740.330949] Call Trace: [422740.330951] [ ] ? drain_pages+0xb0/0xb0 [422740.330953] [ ] on_each_cpu_mask+0x2a/0x60 [422740.330954] [ ] drain_all_pages+0xb5/0xc0 [422740.330956] [ ] __alloc_pages_nodemask+0x8a2/0xba0 [422740.330959] [ ] khugepaged_scan_mm_slot+0x419/0xc60 [422740.330960] [ ] ? schedule_timeout+0x17d/0x2d0 [422740.330962] [ ] khugepaged+0x257/0x480 [422740.330966] [ ] ? khugepaged_scan_mm_slot+0xc60/0xc60 [422768.158422] task: ffff880d0f2a0b80 ti: ffff880d9e458000 task.ti: ffff880d9e458000 [422768.158791] [ ] system_call_fastpath+0x16/0x1b [422768.158808] Code: 48 63 35 96 37 98 00 89 c2 39 f0 0f 8d 86 fe ff ff 48 98 49 8b 0f 48 03 0c c5 20 c8 a5 81 f6 41 20 01 74 cd 0f 1f 44 00 00 f3 90 41 20 01 75 f8 48 63 35 65 37 98 00 eb b7 0f b6 4d cc 4c 89 [422768.184374] BUG: soft lockup - CPU#8 stuck for 22s! [migration/8:128] [422768.184390] Modules linked in: binfmt_misc bonding zfs(POE) zunicode(POE) zavl(POE) zcommon(POE) znvpair(POE) intel_powerclamp spl(OE) coretemp vfat fat zlib_deflate intel_rapl kvm_intel kvm ipmi_ssif crc32_pclmul iTCO_wdt ghash_clmulni_intel iTCO_vendor_support aesni_intel lrw ses gf128mul enclosure glue_helper ipmi_si sb_edac ablk_helper hpwdt pcspkr lpc_ich hpilo sg cryptd pcc_cpufreq i2c_i801 ioatdma ipmi_msghandler edac_core mfd_core shpchp wmi acpi_power_meter nfsd auth_rpcgss nfs_acl lockd grace openafs(POE) sunrpc ip_tables xfs libcrc32c raid1 sd_mod crc_t10dif crct10dif_generic mgag200 syscopyarea sysfillrect sysimgblt i2c_algo_bit ixgbe drm_kms_helper mdio ttm tg3 crct10dif_pclmul dca crct10dif_common drm crc32c_intel ptp i2c_core hpsa pps_core [422768.184391] CPU: 8 PID: 128 Comm: migration/8 Tainted: P W OEL ------------ 3.10.0-327.36.1.el7.x86_64 #1 [422768.184392] Hardware name: HP ProLiant DL380 Gen9/ProLiant DL380 Gen9, BIOS P89 06/02/2016 [422768.184393] task: ffff8808538d2280 ti: ffff8808538f4000 task.ti: ffff8808538f4000 [422768.184396] RIP: 0010:[ ] [ ] multi_cpu_stop+0x83/0xf0 [422768.184397] RSP: 0000:ffff8808538f7d88 EFLAGS: 00000293 [422768.184397] RAX: ffffffff81661260 RBX: 00000000000167c0 RCX: dead000000200200 [422768.184398] RDX: 0000000000000001 RSI: 0000000000000282 RDI: ffff881052ee3b80 [422768.184398] RBP: ffff8808538f7da8 R08: 0000000000000000 R09: 0000000000000001 [422768.184399] R10: 000000000000beec R11: 0000000000000002 R12: 00000000000167c0 [422768.184399] R13: ffff88085345a000 R14: ffff88085f41a000 R15: 0000000000000000 [422768.184400] FS: 0000000000000000(0000) GS:ffff88085fa00000(0000) knlGS:0000000000000000 [422768.184400] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [422768.184401] CR2: 00007ffcaddf1f18 CR3: 000000000194a000 CR4: 00000000001407e0 [422768.184402] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000 [422768.184402] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400 [422768.184402] Stack: [422768.184405] ffff88085fa0fce8 ffff88085fa0fce0 ffff881052ee3b80 ffff881052ee3ba8 [422768.184407] ffff8808538f7e78 ffffffff811036c6 ffff8808538f7fd8 ffff88085fa0fcf0 [422768.184409] 0000000000000000 0000000000000000 ffff88085fa167c0 ffff88105085be80 [422768.184409] Call Trace: [422768.184412] [ ] cpu_stopper_thread+0x96/0x170 [422768.184414] [ ] ? __schedule+0x2d8/0x900 [422768.184416] [ ] smpboot_thread_fn+0xff/0x1a0 [422768.184418] [ ] ? schedule+0x29/0x70 [422768.184420] [ ] ? lg_double_unlock+0x90/0x90 [422768.184422] [ ] kthread+0xcf/0xe0 [422768.184424] [ ] ? kthread_create_on_node+0x140/0x140 [422768.184426] [ ] ret_from_fork+0x58/0x90 [422768.184428] [ ] ? kthread_create_on_node+0x140/0x140 [422768.184438] Code: ed 75 65 f0 ff 4b 24 0f 94 c1 84 c9 44 89 e2 74 0f 8b 43 20 8b 73 10 8d 48 01 89 73 24 89 4b 20 83 fa 04 74 23 f3 90 44 8b 63 20 <41> 39 d4 74 f0 41 83 fc 02 75 c2 fa 66 0f 1f 44 00 00 eb c4 66
reset
to reboot this server, Fabio double checked the CPU C-State in the BIOS but he found it already properly configured We need to observe this system when it will run the 2 * LSI HBAs cards.
BIOS/Platform Configuration (RBSU) Power Management Power Profile [Maximum Performance] Power Regulator [Static High Performance Mode] Minimum Processor Idle Power Core C-State [No C-states] Minimum Processor Idle Power Package C-State [No Package State] Advanced Power Options
[606466.752169] BUG: soft lockup - CPU#2 stuck for 23s! [java:21235] [606466.752194] Modules linked in: binfmt_misc bonding dm_service_time vfat fat zfs(POE) zunicode(POE) zavl(POE) zcommon(POE) znvpair(POE) spl(OE) zlib_deflate intel_powerclamp coretemp intel_rapl kvm_intel kvm crc32_pclmul ghash_clmulni_intel ses aesni_intel iTCO_wdt enclosure lrw gf128mul iTCO_vendor_support sb_edac ipmi_ssif glue_helper ablk_helper lpc_ich hpwdt cryptd sg hpilo edac_core pcspkr mfd_core ioatdma i2c_i801 ipmi_si pcc_cpufreq ipmi_msghandler shpchp wmi acpi_power_meter nfsd auth_rpcgss dm_multipath nfs_acl dm_mod lockd grace openafs(POE) sunrpc ip_tables xfs libcrc32c raid1 sd_mod crc_t10dif crct10dif_generic mgag200 syscopyarea sysfillrect sysimgblt i2c_algo_bit drm_kms_helper ttm ixgbe mdio drm tg3 dca crct10dif_pclmul crct10dif_common ptp crc32c_intel hpsa(OE) i2c_core scsi_transport_sas pps_core [606466.752196] CPU: 2 PID: 21235 Comm: java Tainted: P W OEL ------------ 3.10.0-327.36.3.el7.x86_64 #1 [606466.752196] Hardware name: HP ProLiant DL380 Gen9/ProLiant DL380 Gen9, BIOS P89 06/02/2016 [606466.752197] task: ffff8808538c2e00 ti: ffff880dde614000 task.ti: ffff880dde614000 [606466.752200] RIP: 0010:[] [ ] smp_call_function_many+0x202/0x260 [606466.752200] RSP: 0018:ffff880dde617a10 EFLAGS: 00000202 [606466.752201] RAX: 000000000000001a RBX: 0000000000000002 RCX: ffff88085fc1a820 [606466.752201] RDX: 000000000000001a RSI: 0000000000000028 RDI: 0000000000000000 [606466.752202] RBP: ffff880dde617a48 R08: ffff88085416b000 R09: ffff88085f899620 [606466.752203] R10: ffffea0008ccd000 R11: ffffffff812f2a89 R12: 000000000000f0a8 [606466.752204] R13: 000000fc812f29df R14: 0000000000000282 R15: 0000000000000282 [606466.752204] FS: 00007ff20a8e8700(0000) GS:ffff88085f880000(0000) knlGS:0000000000000000 [606466.752205] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [606466.752206] CR2: 000000067865b000 CR3: 0000001028610000 CR4: 00000000001407e0 [606466.752207] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000 [606466.752207] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400 [606466.752207] Stack: [606466.752211] 0000000100000006 ffffffff81e86580 ffffffff81e86580 ffffffff81171d80 [606466.752214] 0000000000000000 0000000000000002 ffff88087ffda000 ffff880dde617a78 [606466.752218] ffffffff810e71ca 0000000000000027 0000000000000028 0000000000000028 [606466.752218] Call Trace: [606466.752221] [ ] ? drain_pages+0xb0/0xb0 [606466.752223] [ ] on_each_cpu_mask+0x2a/0x60 [606466.752225] [ ] drain_all_pages+0xb5/0xc0 [606466.752227] [ ] __alloc_pages_nodemask+0x8a2/0xba0 [606466.752230] [ ] alloc_pages_current+0xaa/0x170 [606466.752233] [ ] sk_page_frag_refill+0x70/0x160 [606466.752236] [ ] tcp_sendmsg+0x263/0xc20 [606466.752238] [ ] inet_sendmsg+0x64/0xb0 [606466.752240] [ ] sock_aio_write+0x157/0x180 [606466.752243] [ ] do_sync_write+0x8d/0xd0 [606466.752245] [ ] vfs_write+0x1b5/0x1e0 [606466.752247] [ ] SyS_write+0x7f/0xe0 [606466.752249] [ ] system_call_fastpath+0x16/0x1b [606466.752266] Code: 48 63 35 96 37 98 00 89 c2 39 f0 0f 8d 86 fe ff ff 48 98 49 8b 0f 48 03 0c c5 20 c8 a5 81 f6 41 20 01 74 cd 0f 1f 44 00 00 f3 90 41 20 01 75 f8 48 63 35 65 37 98 00 eb b7 0f b6 4d cc 4c 89 [606466.765160] BUG: soft lockup - CPU#3 stuck for 23s! [java:24042] [606466.765185] Modules linked in: binfmt_misc bonding dm_service_time vfat fat zfs(POE) zunicode(POE) zavl(POE) zcommon(POE) znvpair(POE) spl(OE) zlib_deflate intel_powerclamp coretemp intel_rapl kvm_intel kvm crc32_pclmul ghash_clmulni_intel ses aesni_intel iTCO_wdt enclosure lrw gf128mul iTCO_vendor_support sb_edac ipmi_ssif glue_helper ablk_helper lpc_ich hpwdt cryptd sg hpilo edac_core pcspkr mfd_core ioatdma i2c_i801 ipmi_si pcc_cpufreq ipmi_msghandler shpchp wmi acpi_power_meter nfsd auth_rpcgss dm_multipath nfs_acl dm_mod lockd grace openafs(POE) sunrpc ip_tables xfs libcrc32c raid1 sd_mod crc_t10dif crct10dif_generic mgag200 syscopyarea sysfillrect sysimgblt i2c_algo_bit drm_kms_helper ttm ixgbe mdio drm tg3 dca crct10dif_pclmul crct10dif_common ptp crc32c_intel hpsa(OE) i2c_core scsi_transport_sas pps_core [606466.765187] CPU: 3 PID: 24042 Comm: java Tainted: P W OEL ------------ 3.10.0-327.36.3.el7.x86_64 #1 [606466.765187] Hardware name: HP ProLiant DL380 Gen9/ProLiant DL380 Gen9, BIOS P89 06/02/2016 [606466.765188] task: ffff8800203c5c00 ti: ffff88000a0a0000 task.ti: ffff88000a0a0000 [606466.765191] RIP: 0010:[ ] [ ] smp_call_function_many+0x202/0x260 [606466.765191] RSP: 0018:ffff88000a0a3a10 EFLAGS: 00000202 [606466.765192] RAX: 000000000000001a RBX: 0000000000000003 RCX: ffff88085fc1a848 [606466.929071] Stack: [606466.981032] Stack: [606467.072961] Stack: [606467.176880] Stack: [606494.716723] task: ffff8810463a7300 ti: ffff880013cb0000 task.ti: ffff880013cb0000 [606494.717738] 0000000100000006 ffffffff81e86580 ffffffff81e86580 ffffffff81171d80 [606494.717741] 0000000000000000 0000000000000000 ffff88087ffda000 ffff880010307a78 [606494.717745] ffffffff810e71ca 0000000000000027 0000000000000028 0000000000000028 [606494.717745] Call Trace: [606494.717748] [ ] ? drain_pages+0xb0/0xb0 [606494.717750] [ ] on_each_cpu_mask+0x2a/0x60 [606494.717752] [ ] drain_all_pages+0xb5/0xc0 [606494.717755] [ ] __alloc_pages_nodemask+0x8a2/0xba0 [606494.717757] [ ] alloc_pages_current+0xaa/0x170 [606494.717760] [ ] sk_page_frag_refill+0x70/0x160 [606494.717763] [ ] tcp_sendmsg+0x263/0xc20 [606494.717766] [ ] inet_sendmsg+0x64/0xb0 [606494.717768] [ ] sock_aio_write+0x157/0x180 [606494.717770] [ ] do_sync_write+0x8d/0xd0 [606494.717772] [ ] vfs_write+0x1b5/0x1e0 [606494.717775] [ ] ? __schedule+0x2d8/0x900 [606494.717777] [ ] SyS_write+0x7f/0xe0 [606494.717779] [ ] system_call_fastpath+0x16/0x1b [606494.717796] Code: 48 63 35 96 37 98 00 89 c2 39 f0 0f 8d 86 fe ff ff 48 98 49 8b 0f 48 03 0c c5 20 c8 a5 81 f6 41 20 01 74 cd 0f 1f 44 00 00 f3 90 41 20 01 75 f8 48 63 35 65 37 98 00 eb b7 0f b6 4d cc 4c 89 [606494.730685] BUG: soft lockup - CPU#2 stuck for 23s! [java:21235] [606494.730710] Modules linked in: binfmt_misc bonding dm_service_time vfat fat zfs(POE) zunicode(POE) zavl(POE) zcommon(POE) znvpair(POE) spl(OE) zlib_deflate intel_powerclamp coretemp intel_rapl kvm_intel kvm crc32_pclmul ghash_clmulni_intel ses aesni_intel iTCO_wdt enclosure lrw gf128mul iTCO_vendor_support sb_edac ipmi_ssif glue_helper ablk_helper lpc_ich hpwdt cryptd sg hpilo edac_core pcspkr mfd_core ioatdma i2c_i801 ipmi_si pcc_cpufreq ipmi_msghandler shpchp wmi acpi_power_meter nfsd auth_rpcgss dm_multipath nfs_acl dm_mod lockd grace openafs(POE) sunrpc ip_tables xfs libcrc32c raid1 sd_mod crc_t10dif crct10dif_generic mgag200 syscopyarea sysfillrect sysimgblt i2c_algo_bit drm_kms_helper ttm ixgbe mdio drm tg3 dca crct10dif_pclmul crct10dif_common ptp crc32c_intel hpsa(OE) i2c_core scsi_transport_sas pps_core [606494.730712] CPU: 2 PID: 21235 Comm: java Tainted: P W OEL ------------ 3.10.0-327.36.3.el7.x86_64 #1 [606494.730712] Hardware name: HP ProLiant DL380 Gen9/ProLiant DL380 Gen9, BIOS P89 06/02/2016 [606494.730713] task: ffff8808538c2e00 ti: ffff880dde614000 task.ti: ffff880dde614000 [606494.730716] RIP: 0010:[ ] [ ] smp_call_function_many+0x206/0x260 [606494.730716] RSP: 0018:ffff880dde617a10 EFLAGS: 00000202 [606494.730717] RAX: 000000000000001a RBX: 0000000000000002 RCX: ffff88085fc1a820 [606494.730717] RDX: 000000000000001a RSI: 0000000000000028 RDI: 0000000000000000 [606494.730718] RBP: ffff880dde617a48 R08: ffff88085416b000 R09: ffff88085f899620 [606494.730719] R10: ffffea0008ccd000 R11: ffffffff812f2a89 R12: 000000000000f0a8 [606494.730719] R13: 000000fc812f29df R14: 0000000000000282 R15: 0000000000000282 [606494.730720] FS: 00007ff20a8e8700(0000) GS:ffff88085f880000(0000) knlGS:0000000000000000 [606494.730721] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [606494.730722] CR2: 000000067865b000 CR3: 0000001028610000 CR4: 00000000001407e0 [606494.730722] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000 [606494.730723] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400 [606494.730723] Stack: [606494.730727] 0000000100000006 ffffffff81e86580 ffffffff81e86580 ffffffff81171d80 [606494.730730] 0000000000000000 0000000000000002 ffff88087ffda000 ffff880dde617a78 [606494.730734] ffffffff810e71ca 0000000000000027 0000000000000028 0000000000000028 [606494.730734] Call Trace: [606494.730737] [ ] ? drain_pages+0xb0/0xb0 [606494.730739] [ ] on_each_cpu_mask+0x2a/0x60 [606494.730741] [ ] drain_all_pages+0xb5/0xc0 [606494.730743] [ ] __alloc_pages_nodemask+0x8a2/0xba0 [606494.730746] [ ] alloc_pages_current+0xaa/0x170 [606494.730749] [ ] sk_page_frag_refill+0x70/0x160 [606494.730751] [ ] tcp_sendmsg+0x263/0xc20 [606494.730754] [ ] inet_sendmsg+0x64/0xb0 [606494.730756] [ ] sock_aio_write+0x157/0x180 [606494.730759] [ ] do_sync_write+0x8d/0xd0 [606494.730761] [ ] vfs_write+0x1b5/0x1e0 [606494.730763] [ ] SyS_write+0x7f/0xe0 [606494.730765] [ ] system_call_fastpath+0x16/0x1b [606494.730781] Code: 37 98 00 89 c2 39 f0 0f 8d 86 fe ff ff 48 98 49 8b 0f 48 03 0c c5 20 c8 a5 81 f6 41 20 01 74 cd 0f 1f 44 00 00 f3 90 f6 41 20 01 <75> f8 48 63 35 65 37 98 00 eb b7 0f b6 4d cc 4c 89 f2 4c 89 ee [606494.743678] BUG: soft lockup - CPU#3 stuck for 23s! [java:24042] [606494.743702] Modules linked in: binfmt_misc bonding dm_service_time vfat fat zfs(POE) zunicode(POE) zavl(POE) zcommon(POE) znvpair(POE) spl(OE) zlib_deflate intel_powerclamp coretemp intel_rapl kvm_intel kvm crc32_pclmul ghash_clmulni_intel ses aesni_intel iTCO_wdt enclosure lrw gf128mul iTCO_vendor_support sb_edac ipmi_ssif glue_helper ablk_helper lpc_ich hpwdt cryptd sg hpilo edac_core pcspkr mfd_core ioatdma i2c_i801 ipmi_si pcc_cpufreq ipmi_msghandler shpchp wmi acpi_power_meter nfsd auth_rpcgss dm_multipath nfs_acl dm_mod lockd grace openafs(POE) sunrpc ip_tables xfs libcrc32c raid1 sd_mod crc_t10dif crct10dif_generic mgag200 syscopyarea sysfillrect sysimgblt i2c_algo_bit drm_kms_helper ttm ixgbe mdio drm tg3 dca crct10dif_pclmul crct10dif_common ptp crc32c_intel hpsa(OE) i2c_core scsi_transport_sas pps_core [606494.743704] CPU: 3 PID: 24042 Comm: java Tainted: P W OEL ------------ 3.10.0-327.36.3.el7.x86_64 #1 [606494.743705] Hardware name: HP ProLiant DL380 Gen9/ProLiant DL380 Gen9, BIOS P89 06/02/2016 [606494.743706] task: ffff8800203c5c00 ti: ffff88000a0a0000 task.ti: ffff88000a0a0000 [606494.743708] RIP: 0010:[ ] [ ] smp_call_function_many+0x206/0x260 [606494.743709] RSP: 0018:ffff88000a0a3a10 EFLAGS: 00000202 [606494.743710] RAX: 000000000000001a RBX: 0000000000000003 RCX: ffff88085fc1a848 [606494.907590] Stack: [606494.998508] Stack: [606495.077456] Stack: [606495.181379] 0000000100000000 ffffffff81e86580 ffffffff81e86580 ffffffff81171d80 [606522.696252] 0000000100000006 ffffffff81e86580 ffffffff81e86580 ffffffff81171d80 [606522.696255] 0000000000000000 0000000000000000 ffff88087ffda000 ffff880010307a78 [606522.696259] ffffffff810e71ca 0000000000000027 0000000000000028 0000000000000028 [606522.696259] Call Trace: [606522.696262] [ ] ? drain_pages+0xb0/0xb0 [606522.696264] [ ] on_each_cpu_mask+0x2a/0x60 [606522.696266] [ ] drain_all_pages+0xb5/0xc0 [606522.696269] [ ] __alloc_pages_nodemask+0x8a2/0xba0 [606522.696272] [ ] alloc_pages_current+0xaa/0x170 [606522.696274] [ ] sk_page_frag_refill+0x70/0x160 [606522.696277] [ ] tcp_sendmsg+0x263/0xc20 [606522.696280] [ ] inet_sendmsg+0x64/0xb0 [606522.696282] [ ] sock_aio_write+0x157/0x180 [606522.696284] [ ] do_sync_write+0x8d/0xd0 [606522.696286] [ ] vfs_write+0x1b5/0x1e0 [606522.696289] [ ] ? __schedule+0x2d8/0x900 [606522.696291] [ ] SyS_write+0x7f/0xe0 [606522.696293] [ ] system_call_fastpath+0x16/0x1b [606522.696310] Code: 37 98 00 89 c2 39 f0 0f 8d 86 fe ff ff 48 98 49 8b 0f 48 03 0c c5 20 c8 a5 81 f6 41 20 01 74 cd 0f 1f 44 00 00 f3 90 f6 41 20 01 <75> f8 48 63 35 65 37 98 00 eb b7 0f b6 4d cc 4c 89 f2 4c 89 ee [606522.709199] BUG: soft lockup - CPU#2 stuck for 23s! [java:21235] [606522.709224] Modules linked in: binfmt_misc bonding dm_service_time vfat fat zfs(POE) zunicode(POE) zavl(POE) zcommon(POE) znvpair(POE) spl(OE) zlib_deflate intel_powerclamp coretemp intel_rapl kvm_intel kvm crc32_pclmul ghash_clmulni_intel ses aesni_intel iTCO_wdt enclosure lrw gf128mul iTCO_vendor_support sb_edac ipmi_ssif glue_helper ablk_helper lpc_ich hpwdt cryptd sg hpilo edac_core pcspkr mfd_core ioatdma i2c_i801 ipmi_si pcc_cpufreq ipmi_msghandler shpchp wmi acpi_power_meter nfsd auth_rpcgss dm_multipath nfs_acl dm_mod lockd grace openafs(POE) sunrpc ip_tables xfs libcrc32c raid1 sd_mod crc_t10dif crct10dif_generic mgag200 syscopyarea sysfillrect sysimgblt i2c_algo_bit drm_kms_helper ttm ixgbe mdio drm tg3 dca crct10dif_pclmul crct10dif_common ptp crc32c_intel hpsa(OE) i2c_core scsi_transport_sas pps_core [606522.709226] CPU: 2 PID: 21235 Comm: java Tainted: P W OEL ------------ 3.10.0-327.36.3.el7.x86_64 #1 [606522.709227] Hardware name: HP ProLiant DL380 Gen9/ProLiant DL380 Gen9, BIOS P89 06/02/2016 [606522.709228] task: ffff8808538c2e00 ti: ffff880dde614000 task.ti: ffff880dde614000 [606522.709230] RIP: 0010:[ ] [ ] smp_call_function_many+0x206/0x260 [606522.709231] RSP: 0018:ffff880dde617a10 EFLAGS: 00000202 [606522.709231] RAX: 000000000000001a RBX: 0000000000000002 RCX: ffff88085fc1a820 [606522.709232] RDX: 000000000000001a RSI: 0000000000000028 RDI: 0000000000000000 [606522.709233] RBP: ffff880dde617a48 R08: ffff88085416b000 R09: ffff88085f899620 [606522.709233] R10: ffffea0008ccd000 R11: ffffffff812f2a89 R12: 000000000000f0a8 [606522.709234] R13: 000000fc812f29df R14: 0000000000000282 R15: 0000000000000282 [606522.709235] FS: 00007ff20a8e8700(0000) GS:ffff88085f880000(0000) knlGS:0000000000000000 [606522.709236] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [606522.709236] CR2: 000000067865b000 CR3: 0000001028610000 CR4: 00000000001407e0 [606522.709237] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000 [606522.709238] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400 [606522.709238] Stack: [606522.709241] 0000000100000006 ffffffff81e86580 ffffffff81e86580 ffffffff81171d80 [606522.709245] 0000000000000000 0000000000000002 ffff88087ffda000 ffff880dde617a78 [606522.709248] ffffffff810e71ca 0000000000000027 0000000000000028 0000000000000028 [606522.709248] Call Trace: [606522.709251] [ ] ? drain_pages+0xb0/0xb0 [606522.709253] [ ] on_each_cpu_mask+0x2a/0x60 [606522.709255] [ ] drain_all_pages+0xb5/0xc0 [606522.709258] [ ] __alloc_pages_nodemask+0x8a2/0xba0 [606522.709261] [ ] alloc_pages_current+0xaa/0x170 [606522.709263] [ ] sk_page_frag_refill+0x70/0x160 [606522.709266] [ ] tcp_sendmsg+0x263/0xc20 [606522.709269] [ ] inet_sendmsg+0x64/0xb0 [606522.709271] [ ] sock_aio_write+0x157/0x180 [606522.709273] [ ] do_sync_write+0x8d/0xd0 [606522.709275] [ ] vfs_write+0x1b5/0x1e0 [606522.709277] [ ] SyS_write+0x7f/0xe0 [606522.709280] [ ] system_call_fastpath+0x16/0x1b [606522.709296] Code: 37 98 00 89 c2 39 f0 0f 8d 86 fe ff ff 48 98 49 8b 0f 48 03 0c c5 20 c8 a5 81 f6 41 20 01 74 cd 0f 1f 44 00 00 f3 90 f6 41 20 01 <75> f8 48 63 35 65 37 98 00 eb b7 0f b6 4d cc 4c 89 f2 4c 89 ee [606522.722189] BUG: soft lockup - CPU#3 stuck for 23s! [java:24042] [606522.722213] Modules linked in: binfmt_misc bonding dm_service_time vfat fat zfs(POE) zunicode(POE) zavl(POE) zcommon(POE) znvpair(POE) spl(OE) zlib_deflate intel_powerclamp coretemp intel_rapl kvm_intel kvm crc32_pclmul ghash_clmulni_intel ses aesni_intel iTCO_wdt enclosure lrw gf128mul iTCO_vendor_support sb_edac ipmi_ssif glue_helper ablk_helper lpc_ich hpwdt cryptd sg hpilo edac_core pcspkr mfd_core ioatdma i2c_i801 ipmi_si pcc_cpufreq ipmi_msghandler shpchp wmi acpi_power_meter nfsd auth_rpcgss dm_multipath nfs_acl dm_mod lockd grace openafs(POE) sunrpc ip_tables xfs libcrc32c raid1 sd_mod crc_t10dif crct10dif_generic mgag200 syscopyarea sysfillrect sysimgblt i2c_algo_bit drm_kms_helper ttm ixgbe mdio drm tg3 dca crct10dif_pclmul crct10dif_common ptp crc32c_intel hpsa(OE) i2c_core scsi_transport_sas pps_core [606522.722215] CPU: 3 PID: 24042 Comm: java Tainted: P W OEL ------------ 3.10.0-327.36.3.el7.x86_64 #1 [606522.722216] Hardware name: HP ProLiant DL380 Gen9/ProLiant DL380 Gen9, BIOS P89 06/02/2016 [606522.722216] task: ffff8800203c5c00 ti: ffff88000a0a0000 task.ti: ffff88000a0a0000 [606522.722219] RIP: 0010:[ ] [ ] smp_call_function_many+0x206/0x260 [606522.722220] RSP: 0018:ffff88000a0a3a10 EFLAGS: 00000202 [606522.722220] RAX: 000000000000001a RBX: 0000000000000003 RCX: ffff88085fc1a848 [606522.800171] 0000000100000006 ffffffff81e86580 ffffffff81e86580 ffffffff81171d80 [606523.159896] 0000000100000000 ffffffff81e86580 ffffffff81e86580 ffffffff81171d80 [606550.674767] 0000000100000006 ffffffff81e86580 ffffffff81e86580 ffffffff81171d80 [606550.674771] 0000000000000000 0000000000000000 ffff88087ffda000 ffff880010307a78 [606550.674774] ffffffff810e71ca 0000000000000027 0000000000000028 0000000000000028 [606550.674774] Call Trace: [606550.674777] [ ] ? drain_pages+0xb0/0xb0 [606550.674780] [ ] on_each_cpu_mask+0x2a/0x60 [606550.674782] [ ] drain_all_pages+0xb5/0xc0 [606550.674784] [ ] __alloc_pages_nodemask+0x8a2/0xba0 [606550.674787] [ ] alloc_pages_current+0xaa/0x170 [606550.674790] [ ] sk_page_frag_refill+0x70/0x160 [606550.674793] [ ] tcp_sendmsg+0x263/0xc20 [606550.674795] [ ] inet_sendmsg+0x64/0xb0 [606550.674797] [ ] sock_aio_write+0x157/0x180 [606550.674800] [ ] do_sync_write+0x8d/0xd0 [606550.674802] [ ] vfs_write+0x1b5/0x1e0 [606550.674804] [ ] ? __schedule+0x2d8/0x900 [606550.674806] [ ] SyS_write+0x7f/0xe0 [606550.674809] [ ] system_call_fastpath+0x16/0x1b [606550.674825] Code: 48 63 35 96 37 98 00 89 c2 39 f0 0f 8d 86 fe ff ff 48 98 49 8b 0f 48 03 0c c5 20 c8 a5 81 f6 41 20 01 74 cd 0f 1f 44 00 00 f3 90 41 20 01 75 f8 48 63 35 65 37 98 00 eb b7 0f b6 4d cc 4c 89 [606550.687715] BUG: soft lockup - CPU#2 stuck for 22s! [java:21235] [606550.687739] Modules linked in: binfmt_misc bonding dm_service_time vfat fat zfs(POE) zunicode(POE) zavl(POE) zcommon(POE) znvpair(POE) spl(OE) zlib_deflate intel_powerclamp coretemp intel_rapl kvm_intel kvm crc32_pclmul ghash_clmulni_intel ses aesni_intel iTCO_wdt enclosure lrw gf128mul iTCO_vendor_support sb_edac ipmi_ssif glue_helper ablk_helper lpc_ich hpwdt cryptd sg hpilo edac_core pcspkr mfd_core ioatdma i2c_i801 ipmi_si pcc_cpufreq ipmi_msghandler shpchp wmi acpi_power_meter nfsd auth_rpcgss dm_multipath nfs_acl dm_mod lockd grace openafs(POE) sunrpc ip_tables xfs libcrc32c raid1 sd_mod crc_t10dif crct10dif_generic mgag200 syscopyarea sysfillrect sysimgblt i2c_algo_bit drm_kms_helper ttm ixgbe mdio drm tg3 dca crct10dif_pclmul crct10dif_common ptp crc32c_intel hpsa(OE) i2c_core scsi_transport_sas pps_core [606550.687741] CPU: 2 PID: 21235 Comm: java Tainted: P W OEL ------------ 3.10.0-327.36.3.el7.x86_64 #1 [606550.687742] Hardware name: HP ProLiant DL380 Gen9/ProLiant DL380 Gen9, BIOS P89 06/02/2016 [606550.687743] task: ffff8808538c2e00 ti: ffff880dde614000 task.ti: ffff880dde614000 [606550.687745] RIP: 0010:[ ] [ ] smp_call_function_many+0x206/0x260 [606550.687746] RSP: 0018:ffff880dde617a10 EFLAGS: 00000202 [606550.687746] RAX: 000000000000001a RBX: 0000000000000002 RCX: ffff88085fc1a820 [606550.687747] RDX: 000000000000001a RSI: 0000000000000028 RDI: 0000000000000000 [606550.687748] RBP: ffff880dde617a48 R08: ffff88085416b000 R09: ffff88085f899620 [606550.687748] R10: ffffea0008ccd000 R11: ffffffff812f2a89 R12: 000000000000f0a8 [606550.687749] R13: 000000fc812f29df R14: 0000000000000282 R15: 0000000000000282 [606550.687750] FS: 00007ff20a8e8700(0000) GS:ffff88085f880000(0000) knlGS:0000000000000000 [606550.687750] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [606550.687751] CR2: 000000067865b000 CR3: 0000001028610000 CR4: 00000000001407e0 [606550.687752] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000 [606550.687752] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400 [606551.177410] [ ] do_sync_write+0x8d/0xd0 [606551.177412] [ ] vfs_write+0x1b5/0x1e0 [606551.177414] [ ] ? __schedule+0x2d8/0x900 [606551.177416] [ ] SyS_write+0x7f/0xe0 [606551.177419] [ ] system_call_fastpath+0x16/0x1b [606551.177435] Code: 37 98 00 89 c2 39 f0 0f 8d 86 fe ff ff 48 98 49 8b 0f 48 03 0c c5 20 c8 a5 81 f6 41 20 01 74 cd 0f 1f 44 00 00 f3 90 f6 41 20 01 <75> f8 48 63 35 65 37 98 00 eb b7 0f b6 4d cc 4c 89 f2 4c 89 ee [606551.190329] BUG: soft lockup - CPU#36 stuck for 22s! [gmond:6836] [606551.190353] Modules linked in: binfmt_misc bonding dm_service_time vfat fat zfs(POE) zunicode(POE) zavl(POE) zcommon(POE) znvpair(POE) spl(OE) zlib_deflate intel_powerclamp coretemp intel_rapl kvm_intel kvm crc32_pclmul ghash_clmulni_intel ses aesni_intel iTCO_wdt enclosure lrw gf128mul iTCO_vendor_support sb_edac ipmi_ssif glue_helper ablk_helper lpc_ich hpwdt cryptd sg hpilo edac_core pcspkr mfd_core ioatdma i2c_i801 ipmi_si pcc_cpufreq ipmi_msghandler shpchp wmi acpi_power_meter nfsd auth_rpcgss dm_multipath nfs_acl dm_mod lockd grace openafs(POE) sunrpc ip_tables xfs libcrc32c raid1 sd_mod crc_t10dif crct10dif_generic mgag200 syscopyarea sysfillrect sysimgblt i2c_algo_bit drm_kms_helper ttm ixgbe mdio drm tg3 dca crct10dif_pclmul crct10dif_common ptp crc32c_intel hpsa(OE) i2c_core scsi_transport_sas pps_core [606551.190355] CPU: 36 PID: 6836 Comm: gmond Tainted: P W OEL ------------ 3.10.0-327.36.3.el7.x86_64 #1 [606551.190356] Hardware name: HP ProLiant DL380 Gen9/ProLiant DL380 Gen9, BIOS P89 06/02/2016 [606551.190357] task: ffff8808464d5080 ti: ffff88083eb2c000 task.ti: ffff88083eb2c000 [606551.190359] RIP: 0010:[ ] [ ] smp_call_function_many+0x202/0x260 [606551.190360] RSP: 0018:ffff88083eb2f880 EFLAGS: 00000202 [606551.190361] RAX: 000000000000001a RBX: 0000000000000024 RCX: ffff88085fc1ad70 [606551.190361] RDX: 000000000000001a RSI: 0000000000000028 RDI: 0000000000000000 [606551.190362] RBP: ffff88083eb2f8b8 R08: ffff8810534a5000 R09: ffff88105f419620 [606551.190362] R10: ffffea00403f4000 R11: ffffffff812f2a89 R12: 000000000000f0a8 [606551.190363] R13: 000000fc812f29df R14: 0000000000000282 R15: 0000000000000282 [606551.190364] FS: 00007fe0d78ea740(0000) GS:ffff88105f400000(0000) knlGS:0000000000000000 [606551.190365] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [606551.190365] CR2: 000000000062e000 CR3: 0000000852bf3000 CR4: 00000000001407e0 [606551.190366] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000 [606551.190367] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400 [606551.190367] Stack: [606551.190370] 0000000100000006 ffffffff81e86580 ffffffff81e86580 ffffffff81171d80 [606551.190374] 0000000000000000 0000000000000024 ffff88107ffd6000 ffff88083eb2f8e8 [606551.190377] ffffffff810e71ca 0000000000000027 0000000000000028 0000000000000028 [606551.190377] Call Trace: [606551.190380] [ ] ? drain_pages+0xb0/0xb0 [606551.190382] [ ] on_each_cpu_mask+0x2a/0x60 [606551.190384] [ ] drain_all_pages+0xb5/0xc0 [606551.190387] [ ] __alloc_pages_nodemask+0x8a2/0xba0 [606551.190389] [ ] alloc_pages_current+0xaa/0x170 [606551.190391] [ ] new_slab+0x275/0x300 [606551.190394] [ ] __slab_alloc+0x315/0x48f [606551.190396] [ ] ? proc_alloc_inode+0x1d/0xb0 [606551.190398] [ ] ? __get_free_pages+0xe/0x50 [606551.190400] [ ] ? kmem_cache_alloc_trace+0x3c/0x1f0 [606551.190402] [ ] ? seq_open+0xfe/0x170 [606551.190404] [ ] kmem_cache_alloc+0x193/0x1d0 [606551.190406] [ ] ? proc_alloc_inode+0x1d/0xb0 [606551.190408] [ ] proc_alloc_inode+0x1d/0xb0 [606551.190410] [ ] alloc_inode+0x1d/0xa0 [606551.190412] [ ] new_inode_pseudo+0x11/0x60 [606551.190414] [ ] proc_get_inode+0x14/0x130 [606551.190416] [ ] proc_lookup_de+0x86/0xe0 [606551.190418] [ ] proc_lookup+0x1b/0x20 [606551.190421] [ ] proc_root_lookup+0x1c/0x40 [606551.190422] [ ] lookup_real+0x1d/0x50 [606551.190424] [ ] do_last+0xb83/0x1270 [606551.190427] [ ] ? free_one_page+0x165/0x300 [606551.190429] [ ] path_openat+0xc2/0x490 [606551.190431] [ ] ? call_rcu_sched+0x1d/0x20 [606551.190433] [ ] ? evict+0x106/0x170 [606551.190436] [ ] do_filp_open+0x4b/0xb0 [606551.190438] [ ] ? __alloc_fd+0xa7/0x130 [606551.190440] [ ] do_sys_open+0xf3/0x1f0 [606551.190442] [ ] SyS_open+0x1e/0x20 [606551.190444] [ ] system_call_fastpath+0x16/0x1b [606551.190460] Code: 48 63 35 96 37 98 00 89 c2 39 f0 0f 8d 86 fe ff ff 48 98 49 8b 0f 48 03 0c c5 20 c8 a5 81 f6 41 20 01 74 cd 0f 1f 44 00 00 f3 90 41 20 01 75 f8 48 63 35 65 37 98 00 eb b7 0f b6 4d cc 4c 89 [606578.652241] BUG: soft lockup - CPU#1 stuck for 22s! [java:24928] [606578.652265] Modules linked in: binfmt_misc bonding dm_service_time vfat fat zfs(POE) zunicode(POE) zavl(POE) zcommon(POE) znvpair(POE) spl(OE) zlib_deflate intel_powerclamp coretemp intel_rapl kvm_intel kvm crc32_pclmul ghash_clmulni_intel ses aesni_intel iTCO_wdt enclosure lrw gf128mul iTCO_vendor_support sb_edac ipmi_ssif glue_helper ablk_helper lpc_ich hpwdt cryptd sg hpilo edac_core pcspkr mfd_core ioatdma i2c_i801 ipmi_si pcc_cpufreq ipmi_msghandler shpchp wmi acpi_power_meter nfsd auth_rpcgss dm_multipath nfs_acl dm_mod lockd grace openafs(POE) sunrpc ip_tables xfs libcrc32c raid1 sd_mod crc_t10dif crct10dif_generic mgag200 syscopyarea sysfillrect sysimgblt i2c_algo_bit drm_kms_helper ttm ixgbe mdio drm tg3 dca crct10dif_pclmul crct10dif_common ptp crc32c_intel hpsa(OE) i2c_core scsi_transport_sas[606579.155927] [ ] do_sync_write+0x8d/0xd0 [606579.155929] [ ] vfs_write+0x1b5/0x1e0 [606579.155932] [ ] ? __schedule+0x2d8/0x900 [606579.155934] [ ] SyS_write+0x7f/0xe0 [606579.155936] [ ] system_call_fastpath+0x16/0x1b [606579.155953] Code: 21 00 48 63 35 96 37 98 00 89 c2 39 f0 0f 8d 86 fe ff ff 48 98 49 8b 0f 48 03 0c c5 20 c8 a5 81 f6 41 20 01 74 cd 0f 1f 44 00 00 90 f6 41 20 01 75 f8 48 63 35 65 37 98 00 eb b7 0f b6 4d cc [606579.168844] BUG: soft lockup - CPU#36 stuck for 22s! [gmond:6836] [606579.168868] Modules linked in: binfmt_misc bonding dm_service_time vfat fat zfs(POE) zunicode(POE) zavl(POE) zcommon(POE) znvpair(POE) spl(OE) zlib_deflate intel_powerclamp coretemp intel_rapl kvm_intel kvm crc32_pclmul ghash_clmulni_intel ses aesni_intel iTCO_wdt enclosure lrw gf128mul iTCO_vendor_support sb_edac ipmi_ssif glue_helper ablk_helper lpc_ich hpwdt cryptd sg hpilo edac_core pcspkr mfd_core ioatdma i2c_i801 ipmi_si pcc_cpufreq ipmi_msghandler shpchp wmi acpi_power_meter nfsd auth_rpcgss dm_multipath nfs_acl dm_mod lockd grace openafs(POE) sunrpc ip_tables xfs libcrc32c raid1 sd_mod crc_t10dif crct10dif_generic mgag200 syscopyarea sysfillrect sysimgblt i2c_algo_bit drm_kms_helper ttm ixgbe mdio drm tg3 dca crct10dif_pclmul crct10dif_common ptp crc32c_intel hpsa(OE) i2c_core scsi_transport_sas pps_core [606579.168870] CPU: 36 PID: 6836 Comm: gmond Tainted: P W OEL ------------ 3.10.0-327.36.3.el7.x86_64 #1 [606579.168870] Hardware name: HP ProLiant DL380 Gen9/ProLiant DL380 Gen9, BIOS P89 06/02/2016 [606579.168871] task: ffff8808464d5080 ti: ffff88083eb2c000 task.ti: ffff88083eb2c000 [606579.168874] RIP: 0010:[ ] [ ] smp_call_function_many+0x206/0x260 [606579.168874] RSP: 0018:ffff88083eb2f880 EFLAGS: 00000202 [606579.168875] RAX: 000000000000001a RBX: 0000000000000024 RCX: ffff88085fc1ad70 [606579.168876] RDX: 000000000000001a RSI: 0000000000000028 RDI: 0000000000000000 [606579.168877] RBP: ffff88083eb2f8b8 R08: ffff8810534a5000 R09: ffff88105f419620 [606579.168877] R10: ffffea00403f4000 R11: ffffffff812f2a89 R12: 000000000000f0a8 [606579.168878] R13: 000000fc812f29df R14: 0000000000000282 R15: 0000000000000282 [606579.168879] FS: 00007fe0d78ea740(0000) GS:ffff88105f400000(0000) knlGS:0000000000000000 [606579.168880] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [606579.168880] CR2: 000000000062e000 CR3: 0000000852bf3000 CR4: 00000000001407e0 [606579.168881] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000 [606579.168882] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400 [606579.168882] Stack: [606579.168885] 0000000100000006 ffffffff81e86580 ffffffff81e86580 ffffffff81171d80 [606579.168889] 0000000000000000 0000000000000024 ffff88107ffd6000 ffff88083eb2f8e8 [606579.168892] ffffffff810e71ca 0000000000000027 0000000000000028 0000000000000028 [606579.168892] Call Trace: [606579.168895] [ ] ? drain_pages+0xb0/0xb0 [606579.168897] [ ] on_each_cpu_mask+0x2a/0x60 [606579.168899] [ ] drain_all_pages+0xb5/0xc0 [606579.168901] [ ] __alloc_pages_nodemask+0x8a2/0xba0 [606579.168904] [ ] alloc_pages_current+0xaa/0x170 [606579.168906] [ ] new_slab+0x275/0x300 [606579.168909] [ ] __slab_alloc+0x315/0x48f [606579.168911] [ ] ? proc_alloc_inode+0x1d/0xb0 [606579.168913] [ ] ? __get_free_pages+0xe/0x50 [606579.168915] [ ] ? kmem_cache_alloc_trace+0x3c/0x1f0 [606579.168917] [ ] ? seq_open+0xfe/0x170 [606579.168919] [ ] kmem_cache_alloc+0x193/0x1d0 [606579.168921] [ ] ? proc_alloc_inode+0x1d/0xb0 [606579.168923] [ ] proc_alloc_inode+0x1d/0xb0 [606579.168925] [ ] alloc_inode+0x1d/0xa0 [606579.168927] [ ] new_inode_pseudo+0x11/0x60 [606579.168929] [ ] proc_get_inode+0x14/0x130 [606579.168932] [ ] proc_lookup_de+0x86/0xe0 [606579.168934] [ ] proc_lookup+0x1b/0x20 [606579.168936] [ ] proc_root_lookup+0x1c/0x40 [606579.168938] [ ] lookup_real+0x1d/0x50 [606579.168940] [ ] do_last+0xb83/0x1270 [606579.168942] [ ] ? free_one_page+0x165/0x300 [606579.168944] [ ] path_openat+0xc2/0x490 [606579.168947] [ ] ? call_rcu_sched+0x1d/0x20 [606579.168949] [ ] ? evict+0x106/0x170 [606579.168951] [ ] do_filp_open+0x4b/0xb0 [606579.168953] [ ] ? __alloc_fd+0xa7/0x130 [606579.168955] [ ] do_sys_open+0xf3/0x1f0 [606579.168957] [ ] SyS_open+0x1e/0x20 [606579.168960] [ ] system_call_fastpath+0x16/0x1b [606579.168976] Code: 37 98 00 89 c2 39 f0 0f 8d 86 fe ff ff 48 98 49 8b 0f 48 03 0c c5 20 c8 a5 81 f6 41 20 01 74 cd 0f 1f 44 00 00 f3 90 f6 41 20 01 <75> f8 48 63 35 65 37 98 00 eb b7 0f b6 4d cc 4c 89 f2 4c 89 ee [606594.412143] INFO: rcu_sched detected stalls on CPUs/tasks: { 26} (detected by 18, t=69901942 jiffies, g=5097659, c=5097658, q=0) [606594.412148] sending NMI to all CPUs: [606594.413250] NMI backtrace for cpu 0 [606594.413251] CPU: 0 PID: 26706 Comm: java Tainted: P W OEL ------------ 3.10.0-327.36.3.el7.x86_64 #1 [606594.413252] Hardware name: HP ProLiant DL380 Gen9/ProLiant DL380 Gen9, BIOS P89 06/02/2016 [606594.413252] task: ffff881051ad5080 ti: ffff880010304000 task.ti: ffff880010304000 [606594.413253] RIP: 0010:[ ] [ ] smp_call_function_many+0x206/0x260 [606594.413253] RSP: 0018:ffff880010307a10 EFLAGS: 00000202 [606594.413253] RAX: 000000000000001a RBX: 0000000000000028 RCX: ffff88085fc19b08 [606594.413254] RDX: 000000000000001a RSI: 0000000000000028 RDI: 0000000000000000 [606594.413259] [ ] __alloc_pages_nodemask+0x8a2/0xba0 [606594.413514] [ ] sock_aio_write+0x157/0x180 [606618.622597] ffffffff810e71ca 0000000000000027 0000000000000028 0000000000000028 [606618.622597] Call Trace: [606618.622600] [ ] ? drain_pages+0xb0/0xb0 [606618.622602] [ ] on_each_cpu_mask+0x2a/0x60 [606618.622604] [ ] drain_all_pages+0xb5/0xc0 [606618.622607] [ ] __alloc_pages_nodemask+0x8a2/0xba0 [606618.622610] [ ] alloc_pages_current+0xaa/0x170 [606618.622612] [ ] sk_page_frag_refill+0x70/0x160 [606618.622615] [ ] tcp_sendmsg+0x263/0xc20 [606618.622618] [ ] inet_sendmsg+0x64/0xb0 [606618.622620] [ ] sock_aio_write+0x157/0x180 [606618.622622] [ ] do_sync_write+0x8d/0xd0 [606618.622624] [ ] vfs_write+0x1b5/0x1e0 [606618.622627] [ ] ? __schedule+0x2d8/0x900 [606618.622629] [ ] SyS_write+0x7f/0xe0 [606618.622631] [ ] system_call_fastpath+0x16/0x1b [606618.622648] Code: 37 98 00 89 c2 39 f0 0f 8d 86 fe ff ff 48 98 49 8b 0f 48 03 0c c5 20 c8 a5 81 f6 41 20 01 74 cd 0f 1f 44 00 00 f3 90 f6 41 20 01 <75> f8 48 63 35 65 37 98 00 eb b7 0f b6 4d cc 4c 89 f2 4c 89 ee [606618.635538] BUG: soft lockup - CPU#2 stuck for 22s! [java:21235] [606618.635563] Modules linked in: binfmt_misc bonding dm_service_time vfat fat zfs(POE) zunicode(POE) zavl(POE) zcommon(POE) znvpair(POE) spl(OE) zlib_deflate intel_powerclamp coretemp intel_rapl kvm_intel kvm crc32_pclmul ghash_clmulni_intel ses aesni_intel iTCO_wdt enclosure lrw gf128mul iTCO_vendor_support sb_edac ipmi_ssif glue_helper ablk_helper lpc_ich hpwdt cryptd sg hpilo edac_core pcspkr mfd_core ioatdma i2c_i801 ipmi_si pcc_cpufreq ipmi_msghandler shpchp wmi acpi_power_meter nfsd auth_rpcgss dm_multipath nfs_acl dm_mod lockd grace openafs(POE) sunrpc ip_tables xfs libcrc32c raid1 sd_mod crc_t10dif crct10dif_generic mgag200 syscopyarea sysfillrect sysimgblt i2c_algo_bit drm_kms_helper ttm ixgbe mdio drm tg3 dca crct10dif_pclmul crct10dif_common ptp crc32c_intel hpsa(OE) i2c_core scsi_transport_sas pps_core [606618.635565] CPU: 2 PID: 21235 Comm: java Tainted: P W OEL ------------ 3.10.0-327.36.3.el7.x86_64 #1 [606618.635565] Hardware name: HP ProLiant DL380 Gen9/ProLiant DL380 Gen9, BIOS P89 06/02/2016 [606618.635566] task: ffff8808538c2e00 ti: ffff880dde614000 task.ti: ffff880dde614000 [606618.635568] RIP: 0010:[ ] [ ] smp_call_function_many+0x206/0x260 [606618.635569] RSP: 0018:ffff880dde617a10 EFLAGS: 00000202 [606618.635570] RAX: 000000000000001a RBX: 0000000000000002 RCX: ffff88085fc1a820 [606618.635570] RDX: 000000000000001a RSI: 0000000000000028 RDI: 0000000000000000 [606618.635571] RBP: ffff880dde617a48 R08: ffff88085416b000 R09: ffff88085f899620 [606618.635572] R10: ffffea0008ccd000 R11: ffffffff812f2a89 R12: 000000000000f0a8 [606618.635572] R13: 000000fc812f29df R14: 0000000000000282 R15: 0000000000000282 [606618.635573] FS: 00007ff20a8e8700(0000) GS:ffff88085f880000(0000) knlGS:0000000000000000 [606618.635574] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [606618.635574] CR2: 000000067865b000 CR3: 0000001028610000 CR4: 00000000001407e0 [606618.635575] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000 [606618.635576] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400 [606618.635576] Stack: [606618.635580] 0000000100000006 ffffffff81e86580 ffffffff81e86580 ffffffff81171d80 [606618.635583] 0000000000000000 0000000000000002 ffff88087ffda000 ffff880dde617a78 [606618.635586] ffffffff810e71ca 0000000000000027 0000000000000028 0000000000000028 [606618.635587] Call Trace: [606618.635589] [ ] ? drain_pages+0xb0/0xb0 [606618.635592] [ ] on_each_cpu_mask+0x2a/0x60 [606618.635594] [ ] drain_all_pages+0xb5/0xc0 [606618.635596] [ ] __alloc_pages_nodemask+0x8a2/0xba0 [606618.635599] [ ] alloc_pages_current+0xaa/0x170 [606618.635602] [ ] sk_page_frag_refill+0x70/0x160 [606618.635604] [ ] tcp_sendmsg+0x263/0xc20 [606618.635607] [ ] inet_sendmsg+0x64/0xb0 [606618.635609] [ ] sock_aio_write+0x157/0x180 [606618.635611] [ ] do_sync_write+0x8d/0xd0 [606618.635613] [ ] vfs_write+0x1b5/0x1e0 [606618.635615] [ ] SyS_write+0x7f/0xe0 [606618.635618] [ ] system_call_fastpath+0x16/0x1b [606618.635634] Code: 37 98 00 89 c2 39 f0 0f 8d 86 fe ff ff 48 98 49 8b 0f 48 03 0c c5 20 c8 a5 81 f6 41 20 01 74 cd 0f 1f 44 00 00 f3 90 f6 41 20 01 <75> f8 48 63 35 65 37 98 00 eb b7 0f b6 4d cc 4c 89 f2 4c 89 ee [606618.648526] BUG: soft lockup - CPU#3 stuck for 22s! [java:24042] [606618.648551] Modules linked in: binfmt_misc bonding dm_service_time vfat fat zfs(POE) zunicode(POE) zavl(POE) zcommon(POE) znvpair(POE) spl(OE) zlib_deflate intel_powerclamp coretemp intel_rapl kvm_intel kvm crc32_pclmul ghash_clmulni_intel ses aesni_intel iTCO_wdt enclosure lrw gf128mul iTCO_vendor_support sb_edac ipmi_ssif glue_helper ablk_helper lpc_ich hpwdt cryptd sg hpilo edac_core pcspkr mfd_core ioatdma i2c_i801 ipmi_si pcc_cpufreq ipmi_msghandler shpchp wmi acpi_power_meter nfsd auth_rpcgss dm_multipath nfs_acl dm_mod lockd grace openafs(POE) sunrpc ip_tables xfs libcrc32c raid1 sd_mod crc_t10dif crct10dif_generic mgag200 syscopyarea sysfillrect sysimgblt i2c_algo_bit drm_kms_helper ttm ixgbe mdio drm tg3 dca crct10dif_pclmul crct10dif_common ptp crc32c_intel hpsa(OE) i2c_core scsi_transport_sas pps_core [606618.648553] CPU: 3 PID: 24042 Comm: java Tainted: P W OEL ------------ 3.10.0-327.36.3.el7.x86_64 #1 [606618.648553] Hardware name: HP ProLiant DL380 Gen9/ProLiant DL380 Gen9, BIOS P89 06/02/2016 [606618.648554] task: ffff8800203c5c00 ti: ffff88000a0a0000 task.ti: ffff88000a0a0000 [606618.648557] RIP: 0010:[ ] [ ] smp_call_function_many+0x206/0x260 [606618.648557] RSP: 0018:ffff88000a0a3a10 EFLAGS: 00000202 [606618.648558] RAX: 000000000000001a RBX: 0000000000000003 RCX: ffff88085fc1a848 [606618.648559] RDX: 000000000000001a RSI: 0000000000000028 RDI: 0000000000000000 [606618.648559] RBP: ffff88000a0a3a48 R08: ffff88085416bc00 R09: ffff88085f8d9620 [606618.648560] R10: ffffea000404fe00 R11: ffffffff812f2a89 R12: 000000000000f0a8 [606618.812453] ffffffff810e71ca 0000000000000027 0000000000000028 0000000000000028 [606619.138215] [ ] new_slab+0x275/0x300 [606646.601130] [ ] tcp_sendmsg+0x263/0xc20 [606646.601133] [ ] inet_sendmsg+0x64/0xb0 [606646.601135] [ ] sock_aio_write+0x157/0x180 [606646.601138] [ ] do_sync_write+0x8d/0xd0 [606646.601140] [ ] vfs_write+0x1b5/0x1e0 [606646.601142] [ ] ? __schedule+0x2d8/0x900 [606646.601144] [ ] SyS_write+0x7f/0xe0 [606646.601147] [ ] system_call_fastpath+0x16/0x1b [606646.601163] Code: 48 63 35 96 37 98 00 89 c2 39 f0 0f 8d 86 fe ff ff 48 98 49 8b 0f 48 03 0c c5 20 c8 a5 81 f6 41 20 01 74 cd 0f 1f 44 00 00 f3 90 41 20 01 75 f8 48 63 35 65 37 98 00 eb b7 0f b6 4d cc 4c 89 [606646.614051] BUG: soft lockup - CPU#2 stuck for 22s! [java:21235] [606646.614076] Modules linked in: binfmt_misc bonding dm_service_time vfat fat zfs(POE) zunicode(POE) zavl(POE) zcommon(POE) znvpair(POE) spl(OE) zlib_deflate intel_powerclamp coretemp intel_rapl kvm_intel kvm crc32_pclmul ghash_clmulni_intel ses aesni_intel iTCO_wdt enclosure lrw gf128mul iTCO_vendor_support sb_edac ipmi_ssif glue_helper ablk_helper lpc_ich hpwdt cryptd sg hpilo edac_core pcspkr mfd_core ioatdma i2c_i801 ipmi_si pcc_cpufreq ipmi_msghandler shpchp wmi acpi_power_meter nfsd auth_rpcgss dm_multipath nfs_acl dm_mod lockd grace openafs(POE) sunrpc ip_tables xfs libcrc32c raid1 sd_mod crc_t10dif crct10dif_generic mgag200 syscopyarea sysfillrect sysimgblt i2c_algo_bit drm_kms_helper ttm ixgbe mdio drm tg3 dca crct10dif_pclmul crct10dif_common ptp crc32c_intel hpsa(OE) i2c_core scsi_transport_sas pps_core [606646.614078] CPU: 2 PID: 21235 Comm: java Tainted: P W OEL ------------ 3.10.0-327.36.3.el7.x86_64 #1 [606646.614079] Hardware name: HP ProLiant DL380 Gen9/ProLiant DL380 Gen9, BIOS P89 06/02/2016 [606646.614079] task: ffff8808538c2e00 ti: ffff880dde614000 task.ti: ffff880dde614000 [606646.614082] RIP: 0010:[ ] [ ] smp_call_function_many+0x206/0x260 [606646.614083] RSP: 0018:ffff880dde617a10 EFLAGS: 00000202 [606646.614083] RAX: 000000000000001a RBX: 0000000000000002 RCX: ffff88085fc1a820 [606646.614084] RDX: 000000000000001a RSI: 0000000000000028 RDI: 0000000000000000 [606646.614085] RBP: ffff880dde617a48 R08: ffff88085416b000 R09: ffff88085f899620 [606646.614085] R10: ffffea0008ccd000 R11: ffffffff812f2a89 R12: 000000000000f0a8 [606646.614086] R13: 000000fc812f29df R14: 0000000000000282 R15: 0000000000000282 [606646.614087] FS: 00007ff20a8e8700(0000) GS:ffff88085f880000(0000) knlGS:0000000000000000 [606646.614088] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [606646.614088] CR2: 000000067865b000 CR3: 0000001028610000 CR4: 00000000001407e0 [606646.614089] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000 [606646.614090] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400 [606646.614090] Stack: [606646.614093] 0000000100000006 ffffffff81e86580 ffffffff81e86580 ffffffff81171d80 [606646.614097] 0000000000000000 0000000000000002 ffff88087ffda000 ffff880dde617a78 [606646.614100] ffffffff810e71ca 0000000000000027 0000000000000028 0000000000000028 [606646.614100] Call Trace: [606646.614103] [ ] ? drain_pages+0xb0/0xb0 [606646.614105] [ ] on_each_cpu_mask+0x2a/0x60 [606646.614108] [ ] drain_all_pages+0xb5/0xc0 [606646.614110] [ ] __alloc_pages_nodemask+0x8a2/0xba0 [606646.614113] [ ] alloc_pages_current+0xaa/0x170 [606646.614116] [ ] sk_page_frag_refill+0x70/0x160 [606646.614118] [ ] tcp_sendmsg+0x263/0xc20 [606646.614121] [ ] inet_sendmsg+0x64/0xb0 [606646.614123] [ ] sock_aio_write+0x157/0x180 [606646.614125] [ ] do_sync_write+0x8d/0xd0 [606646.614128] [ ] vfs_write+0x1b5/0x1e0 [606646.614129] [ ] SyS_write+0x7f/0xe0 [606646.614132] [ ] system_call_fastpath+0x16/0x1b [606646.614149] Code: 37 98 00 89 c2 39 f0 0f 8d 86 fe ff ff 48 98 49 8b 0f 48 03 0c c5 20 c8 a5 81 f6 41 20 01 74 cd 0f 1f 44 00 00 f3 90 f6 41 20 01 <75> f8 48 63 35 65 37 98 00 eb b7 0f b6 4d cc 4c 89 f2 4c 89 ee [606646.627042] BUG: soft lockup - CPU#3 stuck for 22s! [java:24042] [606646.627066] Modules linked in: binfmt_misc bonding dm_service_time vfat fat zfs(POE) zunicode(POE) zavl(POE) zcommon(POE) znvpair(POE) spl(OE) zlib_deflate intel_powerclamp coretemp intel_rapl kvm_intel kvm crc32_pclmul ghash_clmulni_intel ses aesni_intel iTCO_wdt enclosure lrw gf128mul iTCO_vendor_support sb_edac ipmi_ssif glue_helper ablk_helper lpc_ich hpwdt cryptd sg hpilo edac_core pcspkr mfd_core ioatdma i2c_i801 ipmi_si pcc_cpufreq ipmi_msghandler shpchp wmi acpi_power_meter nfsd auth_rpcgss dm_multipath nfs_acl dm_mod lockd grace openafs(POE) sunrpc ip_tables xfs libcrc32c raid1 sd_mod crc_t10dif crct10dif_generic mgag200 syscopyarea sysfillrect sysimgblt i2c_algo_bit drm_kms_helper ttm ixgbe mdio drm tg3 dca crct10dif_pclmul crct10dif_common ptp crc32c_intel hpsa(OE) i2c_core scsi_transport_sas pps_core [606646.627068] CPU: 3 PID: 24042 Comm: java Tainted: P W OEL ------------ 3.10.0-327.36.3.el7.x86_64 #1 [606646.627069] Hardware name: HP ProLiant DL380 Gen9/ProLiant DL380 Gen9, BIOS P89 06/02/2016 [606646.627070] task: ffff8800203c5c00 ti: ffff88000a0a0000 task.ti: ffff8800
offline
a E2760 RAID Controller and we observe the multipath reaction ; regreatably even if we put back online
that same RAID controller the multipath driver will never recover
[87389.683943] scsi 2:0:8:0: alua: port group 00 state N non-preferred supports TolUsNA [87389.683944] scsi 2:0:8:0: alua: Attached [87389.684099] sd 2:0:8:0: Attached scsi generic sg25 type 0 [87389.684100] sd 2:0:8:0: Embedded Enclosure Device [87391.104496] sd 2:0:8:0: [sdv] 223620156621 512-byte logical blocks: (114 TB/104 TiB) [87391.104787] sd 1:0:4:0: alua: rtpg failed with 8000002 [87391.104983] sd 1:0:4:0: alua: port group 01 state A non-preferred supports TolUsNA [87392.091326] sd 2:0:8:0: [sdv] 4096-byte physical blocks [87392.345337] sd 2:0:8:0: [sdv] Write Protect is off [87392.577500] sd 2:0:8:0: [sdv] Mode Sense: 83 00 10 08 [87392.577956] sd 2:0:8:0: [sdv] Write cache: enabled, read cache: enabled, supports DPO and FUA [87392.580431] sds: unknown partition table [87392.583077] sd 1:0:3:0: alua: rtpg failed with 8000002 [87392.583426] sd 1:0:3:0: alua: port group 01 state A non-preferred supports TolUsNA [87392.583983] sd 1:0:4:0: alua: rtpg failed with 8000002 [87392.584294] sd 1:0:4:0: alua: port group 01 state A non-preferred supports TolUsNA [87392.584403] sdt: unknown partition table [87392.587752] sdu: unknown partition table [87392.590777] sd 1:0:1:0: alua: rtpg failed with 8000002 [87392.591164] sd 1:0:1:0: alua: port group 00 state A non-preferred supports TolUsNA [87392.591827] sd 1:0:2:0: alua: rtpg failed with 8000002 [87392.591981] sds: unknown partition table [87392.592133] sd 1:0:2:0: alua: port group 00 state A non-preferred supports TolUsNA [87392.600123] sdt: unknown partition table [87392.605096] sdu: unknown partition table [87396.619717] sdv: unknown partition table [87396.815220] sd 2:0:8:0: [sdv] Attached SCSI disk [87397.047678] sd 1:0:1:0: alua: rtpg failed with 8000002 [87397.296956] sd 1:0:1:0: alua: port group 00 state A non-preferred supports TolUsNA [87397.660170] sd 1:0:2:0: alua: rtpg failed with 8000002 [87397.909081] sd 1:0:2:0: alua: port group 00 state A non-preferred supports TolUsNA [87945.133985] hpsa 0000:84:00.0: CDB 880000000001985e14b0000002000000 was aborted with status 0x0 [87945.555234] hpsa 0000:84:00.0: CDB 880000000001985e16b0000002000000 was aborted with status 0x0 [87947.499349] device-mapper: multipath: Failing path 65:80. [87947.777697] sd 1:0:1:0: [sdo] FAILED Result: hostbyte=DID_OK driverbyte=DRIVER_SENSE [87948.101420] sd 1:0:3:0: Parameters changed [87948.101454] sd 1:0:4:0: Parameters changed [87948.549230] sd 1:0:1:0: [sdo] Sense Key : Illegal Request [current] [87948.856690] sd 1:0:1:0: [sdo] Add. Sense: Logical unit not supported [87949.164304] sd 1:0:1:0: [sdo] CDB: Read(16) 88 00 00 00 00 34 10 cc cc 00 00 00 00 08 00 00 [87949.568990] blk_update_request: I/O error, dev sdo, sector 223620156416 [87949.889148] device-mapper: multipath: Failing path 8:224. [87950.150840] sd 1:0:2:0: [sdp] FAILED Result: hostbyte=DID_OK driverbyte=DRIVER_SENSE [87950.525757] sd 1:0:2:0: [sdp] Sense Key : Illegal Request [current] [87950.833541] sd 1:0:2:0: [sdp] Add. Sense: Logical unit not supported [87951.141320] sd 1:0:2:0: [sdp] CDB: Read(16) 88 00 00 00 00 34 10 cc cc 00 00 00 00 08 00 00 [87951.547279] blk_update_request: I/O error, dev sdp, sector 223620156416 [87951.869052] device-mapper: multipath: Failing path 8:240. [87952.131972] sd 2:0:7:0: alua: rtpg failed with 8000002 [87952.381715] sd 2:0:7:0: alua: rtpg sense code 05/25/00 [87952.631811] device-mapper: multipath: Failing path 65:64. [87954.282018] hpsa 0000:84:00.0: Acknowledging event: 0x80000012 (HP SSD Smart Path configuration change) [87954.739264] hpsa 0000:88:00.0: Acknowledging event: 0x80000012 (HP SSD Smart Path configuration change) [87955.204886] hpsa 0000:84:00.0: removed scsi 1:0:1:0: Direct-Access NETAPP INF-01-00 PHYS DRV SSDSmartPathCap- En- Exp=1 qd=58 [87955.852274] hpsa 0000:84:00.0: removed scsi 1:0:2:0: Direct-Access NETAPP INF-01-00 PHYS DRV SSDSmartPathCap- En- Exp=1 qd=58 [87956.498097] hpsa 0000:84:00.0: replaced scsi 1:0:3:0: Direct-Access NETAPP INF-01-00 PHYS DRV SSDSmartPathCap- En- Exp=1 qd=58 [87957.143959] hpsa 0000:84:00.0: replaced scsi 1:0:4:0: Direct-Access NETAPP INF-01-00 PHYS DRV SSDSmartPathCap- En- Exp=1 qd=58 [87957.797809] hpsa 0000:88:00.0: removed scsi 2:0:7:0: Direct-Access NETAPP INF-01-00 PHYS DRV SSDSmartPathCap- En- Exp=1 qd=58 [87958.442633] hpsa 0000:88:00.0: removed scsi 2:0:8:0: Direct-Access NETAPP INF-01-00 PHYS DRV SSDSmartPathCap- En- Exp=1 qd=58 [87959.088123] sd 1:0:1:0: [sdo] Synchronizing SCSI cache [87959.336990] sd 1:0:1:0: [sdo] Synchronize Cache(10) failed: Result: hostbyte=DID_NO_CONNECT driverbyte=DRIVER_OK [87960.087922] sd 2:0:7:0: [sdu] Synchronizing SCSI cache [87960.338126] sd 2:0:7:0: [sdu] Synchronize Cache(10) failed: Result: hostbyte=DID_NO_CONNECT driverbyte=DRIVER_OK [87961.832018] sd 2:0:8:0: [sdv] Synchronizing SCSI cache [87962.080709] sd 2:0:8:0: [sdv] Synchronize Cache(10) failed: Result: hostbyte=DID_NO_CONNECT driverbyte=DRIVER_OK
[root@t3nfs02 hp-firmware-smartarray-ea3138d8e8-3.52-1.1]# ./hpsetup Supplemental Update / Online ROM Flash Component for Linux (x64) - Smart Array H240ar, H240nr, H240, H241, H244br, P240nr, P244br, P246br, P440ar, P440, P441, P542D, P741m, P840, P840ar, and P841 (3.52), searching... 1) Smart Array P440 Smart Array P440 in Slot 3 (3.00) 2) Smart Array P841 Smart Array P841 in Slot 4 (4.52) 3) Smart Array P841 Smart Array P841 in Slot 5 (4.52) Select which devices to flash [#,#-#,(A)ll,(N)one]> 1 Flashing Smart Array P440 in Slot 3 [ 3.00 -> 3.52 ] Deferred flashes will be performed on next system reboot ============ Summary ============ Smart Component Finished Summary Messages ================ User opted to not flash 2 devices Reboot needed to activate 1 new FW image Exit Status: 1 Deferred flashes will be performed on next system reboot A reboot is required to complete update. [root@t3nfs02 hp-firmware-smartarray-ea3138d8e8-3.52-1.1]#
NodeTypeForm | |
---|---|
Hostnames | t3nfs02 |
Services | NFSv4 service based on ZoL + dCache Pool |
Hardware | HP DL380 G9 |
Install Profile | t3nfs |
Guarantee/maintenance until | 31-10-2020 |
I | Attachment | History | Action | Size | Date | Who | Comment |
---|---|---|---|---|---|---|---|
png | HP_G9_SN_CZJ5390FSB.png | r1 | manage | 263.8 K | 2015-11-12 - 09:28 | FabioMartinelli | HP_G9_SN_CZJ5390FSB |
3 | hpssacli.slot.3 | r1 | manage | 13.4 K | 2016-11-04 - 21:25 | FabioMartinelli | hpssacli slot 3 |
4 | hpssacli.slot.4 | r1 | manage | 4.3 K | 2016-11-04 - 21:25 | FabioMartinelli | hpssacli slot 4 |
5 | hpssacli.slot.5 | r1 | manage | 4.3 K | 2016-11-04 - 21:26 | FabioMartinelli | hpssacli slot 5 |
sh | shows.netapp.luns.sh | r1 | manage | 0.1 K | 2016-11-04 - 21:26 | FabioMartinelli | |
t3nfs01-2-10GbsCard-SASController.pdf | r1 | manage | 142.6 K | 2016-04-14 - 09:12 | FabioMartinelli | ||
zfs_last_presentation.pdf | r1 | manage | 3122.4 K | 2016-12-19 - 10:29 | FabioMartinelli | ZFS Slides |