Hello, my ODC friends ,
A couple of days ago received a notice from the security department, we have been replaced with new PASSWORD from production user with all of our Oracle DB. At the same time, we also asked for our Dev colleagues to update their App Servers (who is connecting to oracle db) to be new password.
Afterwards they gave us a feedback about almost all of App Servers' password has been updated. But now we're observing the situation (97.8% session connect and disconnect and 92.4% Shared Pool Latches) from Oracle EMCC platform.

Hence we wrote a trigger detecting login failed/denied from real IP (which App Server) meanwhile writing the corresponding message to ALERT log file.
create or replace trigger logon_denied_to_alert
after servererror on database
declare
message varchar2(120);
IP varchar2(15);
v_os_user varchar2(80);
v_module varchar2(50);
v_action varchar2(50);
v_pid varchar2(10);
v_sid number;
v_username varchar2(50);
v_suser varchar2(50);
begin
IF (ora_is_servererror(1017)) THEN
if sys_context('userenv', 'network_protocol') = 'tcp' then
IP := sys\_context('userenv', 'ip\_address');
else
select distinct sid into v\_sid from sys.v\_$mystat;
SELECT p.SPID
into v\_pid
FROM V$PROCESS p, V$SESSION v
WHERE p.ADDR = v.PADDR
AND v.sid = v\_sid;
end if;
v_os_user := sys_context('userenv', 'os_user');
v_username := sys_context('userenv', 'CURRENT_USER');
v_suser := SYS_CONTEXT('USERENV','SESSION_USER');
dbms_application_info.READ_MODULE(v_module, v_action);
message := to_char(sysdate, 'Dy Mon dd HH24:MI:SS YYYY') ||
' logon denied from '||v\_username||' '||v\_suser||' '|| nvl(IP, v\_pid) || ' ' || v\_os\_user ||
' with ' || v\_module || ' ' || v\_action;
sys.dbms_system.ksdwrt(2, message);
end if;
end;
/
This approach is very nice. Immediately we found an App IP address and then told Dev colleague. Finally he also updated the password.
-- alert_xxxx.log
......
Thu Jul 02 09:51:50 20201
Thu Jul 02 09:51:50 2020 logon denied from SYS 10.1.0.8 with
Thu Jul 02 09:51:50 2020
Thu Jul 02 09:51:50 2020 logon denied from SYS 10.1.0.8 with
Thu Jul 02 09:51:50 2020
Thu Jul 02 09:51:50 2020 logon denied from SYS 10.1.0.8 with
Thu Jul 02 09:51:51 2020
Thu Jul 02 09:51:51 2020 logon denied from SYS 10.1.0.8 with
Thu Jul 02 09:51:51 2020
Thu Jul 02 09:51:51 2020 logon denied from SYS 10.1.0.8 with
Thu Jul 02 09:51:51 2020
Thu Jul 02 09:51:51 2020 logon denied from SYS 10.1.0.8 with
Thu Jul 02 09:51:51 2020
Thu Jul 02 09:51:51 2020 logon denied from SYS 10.1.0.8 with
Thu Jul 02 09:51:51 2020
Thu Jul 02 09:51:51 2020 logon denied from SYS 10.1.0.8 with
Thu Jul 02 09:51:51 2020
Thu Jul 02 09:51:51 2020 logon denied from SYS 10.1.0.8 with
......
Now the trigger "logon_denied_to_alert" has not found other IP but oracle db has still plenty of frequent session connecting and disconnecting. On the other hand, next to check the wait event "library cache lock". They are appearing a little while, disappearing a little while. So strange!!!
09:53:45 SYS@xxxx> select sid, saddr from v$session where event='library cache lock';
SID SADDR
---------- ----------------
911 00000056C3CF6570
1065 0000005583D44320
1711 0000005543E61EE0
2308 00000055C3FD8AB0
3654 00000055841BDB40
4850 00000055A4466BB0
5462 00000056C44D3610
6334 00000055C46B82B0
9172 00000055A4BD8DB0
9732 00000055A4CC71F0
9830 0000005544C48A40
12488 00000055850D8F00
13777 00000055852F59E0
14325 00000055A5491D50
14816 00000055854B08C0
14822 00000055654A9A48
16 rows selected.
09:53:51 SYS@xxxx> /
no rows selected
......
10:00:56 SYS@xxxx> /
SID SADDR
---------- ----------------
122 00000056C3B9D290
171 0000005523BC5CD0
584 0000005543C88740
729 00000055A3D6BBD0
961 0000005543D3CAE0
2377 0000005543FABD60
2459 0000005563FB8AA8
2564 00000055A407CCB0
2565 00000055C40375D0
2625 00000055C4062210
2633 0000005523FECD50
3967 00000055C42ACA10
4047 00000055A4313A90
4064 0000005544273940
4299 00000055242BDBD0
5063 000000558442FEA0
5218 00000055844762C0
5418 00000055C450C830
5813 00000055845552A0
5892 0000005544587B00
5936 00000055C460D1B0
5939 00000055645A3CA8
6447 0000005584668160
6586 00000055446C5600
7212 00000055247BDA70
7555 00000055A48F2910
7684 000000552488D5F0
8380 00000055649D3FC8
8484 00000056C49E8AD0
8772 0000005584A85F40
8803 00000055C4ADC250
9434 0000005584B8CA80
9507 0000005564BAD768
10006 0000005544C85BC0
10132 0000005524CBA830
10430 0000005564D56108
10455 00000055C4DB3290
10998 0000005584E45200
11338 00000055A4F6D430
11559 00000056C4F2BCB0
11883 00000056C4FB2330
11981 0000005524FC5750
12586 00000055C5156D70
12660 00000055C517B7F0
12688 00000055C516F470
12807 00000055A520A3D0
13194 0000005545221700
13463 00000055C52CE910
13703 00000055852D0F60
13708 00000055452DED40
14414 00000055C546AF30
14492 000000558542A240
14617 000000554548D8A0
14870 00000056C54D6C50
15056 0000005525508930
10:00:58 SYS@xxxx> /
no rows selected
Now, how to continue to troubleshooting it for me? Could you give me some new ideas? Thanks beforehand!
Best Regards
Quanwen Zhao