Hello
I'm trying to run EBS 12.2 with Oracle database 12.1.0.2 on OEL7.7.
However intermittently but very frequently CPU usage and load average goes up to incredible numbers.
Sometimes it happens when I just start database. Sometimes, when I run some EBS job connecting to the database, like adop.
This virtual machine runs on Xen Server, so in management GUI I observe increased usage for one of the CPUs
After a while error messages appear in the terminal:
After a reboot systems reports that something went wrong
I tried to google by error messages in /var/log/messages, but it didn't help me much.
Update to latest kernel also didn't help.
Can anyone please share some ideas?
Thank you
Dec 20 07:57:06 demo kernel: watchdog: BUG: soft lockup - CPU#10 stuck for 22s! [kworker/10:244:14107]
Dec 20 07:57:06 demo kernel: Modules linked in: tcp_lp fuse xt_CHECKSUM ipt_MASQUERADE nf_nat_masquerade_ipv4 tun xenfs xen_privcmd cmac arc4 ecb md4 nls_utf8 cifs ccm dns_resolver fscache scsi_transport_iscsi ip6t_rpfilter ip6t_REJECT nf_reject_ipv6 ipt_REJECT nf_reject_ipv4 xt_conntrack ebtable_nat ebtable_broute bridge stp llc ip6table_nat nf_conntrack_ipv6 nf_defrag_ipv6 nf_nat_ipv6 ip6table_mangle ip6table_security ip6table_raw iptable_nat nf_conntrack_ipv4 nf_defrag_ipv4 nf_nat_ipv4 nf_nat iptable_mangle iptable_security iptable_raw nf_conntrack ip_set nfnetlink ebtable_filter ebtables ip6table_filter ip6_tables iptable_filter sunrpc crct10dif_pclmul crc32_pclmul ghash_clmulni_intel pcbc ppdev aesni_intel crypto_simd glue_helper cryptd sg parport_pc parport i2c_piix4 pcspkr binfmt_misc ip_tables xfs libcrc32c
Dec 20 07:57:06 demo kernel: sr_mod cdrom ata_generic pata_acpi bochs_drm drm_kms_helper xen_netfront syscopyarea xen_blkfront sysfillrect sysimgblt fb_sys_fops ttm ata_piix drm libata crc32c_intel floppy serio_raw dm_mirror dm_region_hash dm_log dm_mod
Dec 20 07:57:06 demo kernel: CPU: 10 PID: 14107 Comm: kworker/10:244 Tainted: G L 4.14.35-1902.8.4.el7uek.x86_64 #2
Dec 20 07:57:06 demo kernel: Hardware name: Xen HVM domU, BIOS 4.11.1-7.3 05/09/2019
Dec 20 07:57:06 demo kernel: Workqueue: events netstamp_clear
Dec 20 07:57:06 demo kernel: task: ffff9c97389e1840 task.stack: ffffbf75509e0000
Dec 20 07:57:06 demo kernel: RIP: 0010:smp_call_function_many+0x21c/0x27d
Dec 20 07:57:06 demo kernel: RSP: 0018:ffffbf75509e3cc8 EFLAGS: 00000202 ORIG_RAX: ffffffffffffff0c
Dec 20 07:57:06 demo kernel: RAX: 0000000000000008 RBX: ffffffffab0366c0 RCX: ffff9c9c77029be0
Dec 20 07:57:06 demo kernel: RDX: 0000000000000001 RSI: 0000000000000080 RDI: ffff9c988406c700
Dec 20 07:57:06 demo kernel: RBP: ffffbf75509e3d00 R08: 000000000000fbff R09: 0000000000000000
Dec 20 07:57:06 demo kernel: R10: ffff9c988406c700 R11: 0000000000000000 R12: 0000000000000000
Dec 20 07:57:06 demo kernel: R13: ffff9c9c770a3d80 R14: 0000000000000001 R15: 0000000000000080
Dec 20 07:57:06 demo kernel: FS: 0000000000000000(0000) GS:ffff9c9c77080000(0000) knlGS:0000000000000000
Dec 20 07:57:06 demo kernel: CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
Dec 20 07:57:06 demo kernel: CR2: 00005644e4887320 CR3: 000000041e40a004 CR4: 00000000003606e0
Dec 20 07:57:06 demo kernel: DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
Dec 20 07:57:06 demo kernel: DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
Dec 20 07:57:06 demo kernel: Call Trace:
Dec 20 07:57:06 demo kernel: ? netif_receive_skb_internal+0x20/0x117
Dec 20 07:57:06 demo kernel: ? cpumask_weight+0x20/0x11
Dec 20 07:57:06 demo kernel: ? netif_receive_skb_internal+0x21/0x117
Dec 20 07:57:06 demo kernel: on_each_cpu+0x2d/0x59
Dec 20 07:57:06 demo kernel: ? netif_receive_skb_internal+0x20/0x117
Dec 20 07:57:06 demo kernel: text_poke_bp+0x6a/0xe2
Dec 20 07:57:06 demo kernel: ? netif_receive_skb_internal+0x20/0x117
Dec 20 07:57:06 demo kernel: __jump_label_transform.isra.0+0x140/0x19b
Dec 20 07:57:06 demo kernel: arch_jump_label_transform+0x32/0x43
Dec 20 07:57:06 demo kernel: __jump_label_update+0x77/0x83
Dec 20 07:57:06 demo kernel: jump_label_update+0x57/0xa9
Dec 20 07:57:06 demo kernel: static_key_enable_cpuslocked+0x55/0x7a
Dec 20 07:57:06 demo kernel: static_key_enable+0x1a/0x22
Dec 20 07:57:06 demo kernel: netstamp_clear+0x2d/0x37
Dec 20 07:57:06 demo kernel: process_one_work+0x169/0x399
Dec 20 07:57:06 demo kernel: worker_thread+0x4d/0x3e5
Dec 20 07:57:06 demo kernel: kthread+0x105/0x138
Dec 20 07:57:06 demo kernel: ? rescuer_thread+0x380/0x375
Dec 20 07:57:06 demo kernel: ? kthread_bind+0x20/0x15
Dec 20 07:57:06 demo kernel: ret_from_fork+0x24/0x49
Dec 20 07:57:06 demo kernel: Code: 08 89 c7 e8 e7 d0 71 00 3b 05 f5 a1 51 01 0f 83 6b fe ff ff 48 63 d0 49 8b 4d 00 48 03 0c d5 00 97 27 ac 8b 51 18 83 e2 01 74 0a <f3> 90 8b 51 18 83 e2 01 75 f6 eb c6 0f b6 4d cc 4c 89 e2 48 89
Dec 20 07:57:07 demo sh: abrt-dump-oops: Found oopses: 1
Dec 20 07:57:07 demo sh: abrt-dump-oops: Creating problem directories
Dec 20 07:57:07 demo sh: abrt-dump-oops: Not going to make dump directories world readable because PrivateReports is on
Dec 20 07:57:08 demo abrt-dump-oops: Reported 1 kernel oopses to Abrt