Database Software

Hi,

I have created an asm instance using raw devices. However, my asm instance hangs after opening. The problem started after I added a new disk to ASM diskgroup. It was all working fine till that time.

I have created the asm instance on Oracle linux 32 bit platform.

FYI, the permissions on the new disk look good. Request someone to please help. Here is part of the alert log...

Thanks in advance.

-------------------------------
SYS auditing is disabled
Starting up ORACLE RDBMS Version: 11.1.0.6.0.
Using parameter settings in server-side spfile /app/oracle/product/11.1.0/dbs/spfile+ASM.ora
System parameters with non-default values:
large_pool_size = 12M
instance_type = "asm"
asm_diskstring = "/dev/sd*"
asm_diskgroups = "DATA"
asm_power_limit = 4
diagnostic_dest = "/app/oracle"
Sat Jul 18 14:16:20 2009
PMON started with pid=2, OS id=25603
Sat Jul 18 14:16:21 2009
VKTM started with pid=3, OS id=25605 at elevated priority
VKTM running at (20)ms precision
Sat Jul 18 14:16:21 2009
PSP0 started with pid=5, OS id=25611
Sat Jul 18 14:16:21 2009
DIAG started with pid=4, OS id=25609
Sat Jul 18 14:16:22 2009
DSKM started with pid=6, OS id=25613
Sat Jul 18 14:16:22 2009
DIA0 started with pid=7, OS id=25615
Sat Jul 18 14:16:22 2009
MMAN started with pid=8, OS id=25617
Sat Jul 18 14:16:22 2009
DBW0 started with pid=6, OS id=25619
Sat Jul 18 14:16:23 2009
LGWR started with pid=9, OS id=25621
Sat Jul 18 14:16:23 2009
SMON started with pid=11, OS id=25625
Sat Jul 18 14:16:23 2009
CKPT started with pid=10, OS id=25623
Sat Jul 18 14:16:24 2009
RBAL started with pid=12, OS id=25627
Sat Jul 18 14:16:24 2009
GMON started with pid=13, OS id=25629
Sat Jul 18 14:16:24 2009
ORACLE_BASE from environment = /app/oracle
Sat Jul 18 14:16:26 2009
SQL> ALTER DISKGROUP ALL MOUNT
WARNING: Deprecated privilege SYSDBA for command 'ALTER DISKGROUP MOUNT'
NOTE: cache registered group DATA number=1 incarn=0x9fc8267c
WARNING::lib=/opt/oracle/extapi/32/asm err:9 rc:Directory does not exist
location:skgdllOpenDi
errbuf=2
msgbuf=No such file or directory
NOTE: Assigning number (1,0) to disk (/dev/sdb1)
NOTE: Assigning number (1,1) to disk (/dev/sdc1)
NOTE: start heartbeating (grp 1)
kfdp_query(): 3
kfdp_queryBg(): 3
NOTE: cache opening disk 0 of grp 1: DATA_0000 path:/dev/sdb1
NOTE: F1X0 found on disk 0 fcn 0.13795
NOTE: cache opening disk 1 of grp 1: DATA_0001 path:/dev/sdc1
NOTE: cache mounting (first) group 1/0x9FC8267C (DATA)
NOTE: cache recovered group 1 to fcn 0.90671
Sat Jul 18 14:16:33 2009
NOTE: opening chunk 1 at fcn 0.90671 ABA
NOTE: seq=7 blk=10543
NOTE: cache mounting group 1/0x9FC8267C (DATA) succeeded
kfdp_query(): 4
kfdp_queryBg(): 4
NOTE: Instance updated compatible.asm to 10.1.0.0.0 for grp 1
SUCCESS: diskgroup DATA was mounted
SUCCESS: ALTER DISKGROUP ALL MOUNT
Sat Jul 18 14:16:56 2009
Starting background process ASMB
Sat Jul 18 14:16:56 2009
ASMB started with pid=16, OS id=25660
Sat Jul 18 14:17:24 2009
NOTE: ASMB process exiting due to lack of ASM file activity
Sat Jul 18 14:38:04 2009
Starting background process ASMB
Sat Jul 18 14:38:04 2009
ASMB started with pid=15, OS id=26481
Sat Jul 18 14:38:14 2009
NOTE: ASMB process exiting due to lack of ASM file activity
Sat Jul 18 17:50:04 2009
Starting background process ASMB
Sat Jul 18 17:50:04 2009
ASMB started with pid=18, OS id=30910
Sat Jul 18 17:50:15 2009
NOTE: ASMB process exiting due to lack of ASM file activity
Sun Jul 19 17:52:17 2009
Starting background process ASMB
Sun Jul 19 17:52:17 2009
ASMB started with pid=15, OS id=8760
Sun Jul 19 17:52:32 2009
NOTE: ASMB process exiting due to lack of ASM file activity
Fri Jul 24 13:43:44 2009
Starting ORACLE instance (normal)
WARNING: Deprecated privilege SYSDBA for command 'STARTUP'
LICENSE_MAX_SESSION = 0
LICENSE_SESSIONS_WARNING = 0
Picked latch-free SCN scheme 2
Using LOG_ARCHIVE_DEST_1 parameter default value as /app/oracle/product/11.1.0/dbs/arch
Autotune of undo retention is turned on.
IMODE=BR
ILAT =0
LICENSE_MAX_USERS = 0
SYS auditing is disabled
Fri Jul 24 13:43:55 2009
Starting up ORACLE RDBMS Version: 11.1.0.6.0.
Using parameter settings in server-side spfile /app/oracle/product/11.1.0/dbs/spfile+ASM.ora
System parameters with non-default values:
large_pool_size = 12M
instance_type = "asm"
asm_diskstring = "/dev/sd*"
asm_diskgroups = "DATA"
asm_power_limit = 4
diagnostic_dest = "/app/oracle"
Fri Jul 24 13:43:56 2009
PMON started with pid=2, OS id=6413
Fri Jul 24 13:43:56 2009
VKTM started with pid=3, OS id=6415 at elevated priority
VKTM running at (20)ms precision
Fri Jul 24 13:43:56 2009
DIAG started with pid=4, OS id=6419
Fri Jul 24 13:43:56 2009
DSKM started with pid=6, OS id=6423
Fri Jul 24 13:43:57 2009
DIA0 started with pid=7, OS id=6425
Fri Jul 24 13:43:57 2009
PSP0 started with pid=5, OS id=6421
Fri Jul 24 13:43:57 2009
MMAN started with pid=8, OS id=6427
Fri Jul 24 13:43:57 2009
DBW0 started with pid=6, OS id=6429
Fri Jul 24 13:43:57 2009
CKPT started with pid=10, OS id=6433
Fri Jul 24 13:43:57 2009
LGWR started with pid=9, OS id=6431
Fri Jul 24 13:43:58 2009
RBAL started with pid=12, OS id=6437
Fri Jul 24 13:43:58 2009
SMON started with pid=11, OS id=6435
Fri Jul 24 13:43:58 2009
GMON started with pid=13, OS id=6439
ORACLE_BASE from environment = /app/oracle
Fri Jul 24 13:44:00 2009
SQL> ALTER DISKGROUP ALL MOUNT
WARNING: Deprecated privilege SYSDBA for command 'ALTER DISKGROUP MOUNT'
NOTE: cache registered group DATA number=1 incarn=0x4efd22f8
WARNING::lib=/opt/oracle/extapi/32/asm err:9 rc:Directory does not exist
location:skgdllOpenDi
errbuf=2
msgbuf=No such file or directory
NOTE: Assigning number (1,0) to disk (/dev/sdb1)
NOTE: Assigning number (1,1) to disk (/dev/sdc1)
NOTE: start heartbeating (grp 1)
kfdp_query(): 3
kfdp_queryBg(): 3
NOTE: cache opening disk 0 of grp 1: DATA_0000 path:/dev/sdb1
NOTE: F1X0 found on disk 0 fcn 0.13795
NOTE: cache opening disk 1 of grp 1: DATA_0001 path:/dev/sdc1
NOTE: cache mounting (first) group 1/0x4EFD22F8 (DATA)
NOTE: starting recovery of thread=1 ckpt=8.1129 group=1
NOTE: advancing ckpt for thread=1 ckpt=8.1129
NOTE: cache recovered group 1 to fcn 0.100636
NOTE: opening chunk 1 at fcn 0.100636 ABA
NOTE: seq=9 blk=1130
NOTE: cache mounting group 1/0x4EFD22F8 (DATA) succeeded
kfdp_query(): 4
kfdp_queryBg(): 4
NOTE: Instance updated compatible.asm to 10.1.0.0.0 for grp 1
Fri Jul 24 13:44:10 2009
SUCCESS: diskgroup DATA was mounted
SUCCESS: ALTER DISKGROUP ALL MOUNT
Fri Jul 24 13:44:41 2009
Starting background process ASMB
Fri Jul 24 13:44:41 2009
ASMB started with pid=16, OS id=6460
Fri Jul 24 13:45:02 2009
NOTE: ASMB process exiting due to lack of ASM file activity
Fri Jul 24 14:25:19 2009
Starting background process ASMB
Fri Jul 24 14:25:19 2009
ASMB started with pid=18, OS id=12216
Fri Jul 24 14:25:38 2009
NOTE: ASMB process exiting due to lack of ASM file activity
Fri Jul 24 14:34:42 2009
SQL> alter diskgroup '+DATA' add disk '/dev/sdd1'
ORA-15032: not all alterations performed
ORA-15001: diskgroup "+DATA" does not exist or is not mounted
ERROR: alter diskgroup '+DATA' add disk '/dev/sdd1'
Fri Jul 24 14:37:17 2009
SQL> alter diskgroup DATA add disk '/dev/sdd1'
WARNING: Deprecated privilege SYSDBA for command 'ALTER DISKGROUP ADD/DROP/REBALANCE'
NOTE: Assigning number (1,2) to disk (/dev/sdd1)
NOTE: requesting all-instance membership refresh for group=1
NOTE: initializing header on grp 1 disk DATA_0002
NOTE: cache opening disk 2 of grp 1: DATA_0002 path:/dev/sdd1
NOTE: requesting all-instance disk validation for group=1
Fri Jul 24 14:37:20 2009
NOTE: disk validation pending for group 1/0x4efd22f8 (DATA)
SUCCESS: validated disks for 1/0x4efd22f8 (DATA)
NOTE: initiating PST update: grp = 1
kfdp_update(): 5
Fri Jul 24 14:37:23 2009
kfdp_updateBg(): 5
NOTE: PST update grp = 1 completed successfully
NOTE: membership refresh pending for group 1/0x4efd22f8 (DATA)
kfdp_query(): 6
kfdp_queryBg(): 6
kfdp_query(): 7
kfdp_queryBg(): 7
SUCCESS: refreshed membership for 1/0x4efd22f8 (DATA)
NOTE: starting rebalance of group 1/0x4efd22f8 (DATA) at power 4
Starting background process ARB0
SUCCESS: alter diskgroup DATA add disk '/dev/sdd1'
Starting background process ARB1
Starting background process ARB2
Fri Jul 24 14:37:27 2009
ARB1 started with pid=24, OS id=12923
Fri Jul 24 14:37:27 2009
ARB0 started with pid=23, OS id=12921
Starting background process ARB3
Fri Jul 24 14:37:27 2009
ARB2 started with pid=30, OS id=12925
NOTE: assigning ARB1 to group 1/0x4efd22f8 (DATA)
NOTE: assigning ARB0 to group 1/0x4efd22f8 (DATA)
NOTE: assigning ARB2 to group 1/0x4efd22f8 (DATA)
Fri Jul 24 14:37:27 2009
ARB3 started with pid=31, OS id=12927
NOTE: assigning ARB3 to group 1/0x4efd22f8 (DATA)
Sun Jul 26 20:27:08 2009