Thread: Question on dbms_stats.auto_invalidate (Solved?)


Permlink Replies: 9 - Pages: 1 - Last Post: Feb 2, 2008 6:08 PM Last Post By: Dion_Cho
Dion_Cho

Posts: 648
Registered: 10/05/07
Question on dbms_stats.auto_invalidate (Solved?)
Posted: Dec 3, 2007 10:04 PM
Click to report abuse...   Click to reply to this thread Reply
As you know already, Oracle has changed the default action of "NO_INVALIDATE" parameter of DBMS_STATS package to "DBMS_STATS.AUTO_INVALIDATE".

But I found that Oracle's explanation on the meaning of auto invalidation is too vague. Like following...

no_invalidate Does not invalidate the dependent cursors if set to TRUE.
The procedure invalidates the dependent cursors immediately if set to FALSE.
Use DBMS_STATS.AUTO_INVALIDATE. to have Oracle decide when to nvalidate
dependent cursors. This is the default. The default can be changed using the
SET_PARAM Procedure.

I've tested a few cases to find out how AUTO_INVALIDATE works, but with no reasonable explanation.

After googling for a couple of mintutes, I found Tanel's comment. He said that "_optimizer_invalidation_period" hidden parameter determines the period when the invalidation occurs. It's default value is 18000 == 5 hour.
http://www.freelists.org/archives/oracle-l/10-2006/msg00352.html

But he also found that "_optimizer_invalidation_period" parameter is not working as expected as of 10.2.0.2, which meets my experiment.

http://www.freelists.org/archives/oracle-l/10-2006/msg00358.html

Does anyone has reasonable explaination on how DBMS_STATS.AUTO_INVAVLIDATE works and with some sound test cases?
orawarebyte

Posts: 3,648
Registered: 10/02/00
Re: Question on dbms_stats.auto_invalidate
Posted: Dec 4, 2007 12:57 AM   in response to: Dion_Cho in response to: Dion_Cho
Click to report abuse...   Click to reply to this thread Reply
dion how did you measure that either auto invalidate is occurring or not if you set the hidden parameter,are you checking it with tkprof result?

Khurram
Dion_Cho

Posts: 648
Registered: 10/05/07
Re: Question on dbms_stats.auto_invalidate
Posted: Dec 4, 2007 2:54 AM   in response to: orawarebyte in response to: orawarebyte
Click to report abuse...   Click to reply to this thread Reply
Before i show my test result, i should tell you the fact that you shouldn't use sql trace & tkprof to identify whether your SQL cursor is invalidated. Why? SQL trace-ed query generates additional child cursor. SQL trace is a typical case of unshareable SQL statements. V$SQL.invalidations is the more appropriate way to do it.

I tested more and had following conclusion:

- DBMS_STATS.NO_INVALIDATE == false
Change in statistics always invalidates the dependent SQL cursors.

- DBMS_STATS.NO_INVALIDATE == true
Change in statistics never invalidates the dependent SQL cursros.

- DBMS_STATS.NO_INVALIDATE == auto_invalidate
This is the strangest part of the story. My test shows that original cursors is
not invalidated but new child cursor is generated. I'm not sure this is expected behavior and it's my question.


I'm not sure my interpretation is right, and think that different behavior may be observed in different versions.

See following test case...
connect id/pwd;

-- Change
alter system set "_optimizer_invalidation_period" = 1;

alter system flush shared_pool;

-- Initialize table
drop table stat_table;

create table stat_table(id int, name varchar2(10));

insert into stat_table values(1, 'name1');

commit;

-- Gather statistics
exec dbms_stats.gather_table_stats(user, 'STAT_TABLE', cascade=>true, no_invalidate=>false);

-- Select query
select * from stat_table where id = 1;

-- Check v$sql
select * from v$sql where sql_text like 'select * from stat_table where id = 1%';

-- Change table a lot
insert into stat_table select rownum, substr(object_name, 1, 10) from all_objects where rownum <= 10000;

commit;

-- Gather statistics
exec dbms_stats.gather_table_stats(user, 'STAT_TABLE', cascade=>true, no_invalidate=>dbms_stats.auto_invalidate);

--exec dbms_stats.gather_table_stats(user, 'STAT_TABLE', cascade=>true, no_invalidate=>true);

--exec dbms_stats.gather_table_stats(user, 'STAT_TABLE', cascade=>true, no_invalidate=>false);

-- Select query
select * from stat_table where id = 1;

-- Check v$sql
select * from v$sql where sql_text like 'select * from stat_table where id = 1%';

orawarebyte

Posts: 3,648
Registered: 10/02/00
Re: Question on dbms_stats.auto_invalidate
Posted: Dec 4, 2007 3:35 AM   in response to: Dion_Cho in response to: Dion_Cho
Click to report abuse...   Click to reply to this thread Reply
Before i show my test result, i should tell you the
fact that you shouldn't use sql trace & tkprof to
identify whether your SQL cursor is invalidated. Why?
SQL trace-ed query generates additional child cursor.
SQL trace is a typical case of unshareable SQL
statements. V$SQL.invalidations is the more
appropriate way to do it.

dion i dont know much in detail about it why not to use sql trace and tkprof for invalidated.

i have done a small test case where i found invalidation but its not auto.

session 1
---------
SQL> drop table t
  2  / 
 
Table dropped.
 
SQL> create table t as
  2  select 1 rn,object_name from all_objects
  3  / 
 
Table created.
 
SQL> update t set rn=2 where rownum<=10
  2  / 
 
10 rows updated.
 
SQL> commit
  2  / 
 
Commit complete.
 
SQL> select count(*),rn
  2    from t
  3   group by rn
  4  / 
 
  COUNT(*)         RN
---------- ----------
     38962          1
        10          2
 
SQL> create index t_rn_ndx on t (rn)
  2  / 
 
Index created.
 
SQL> exec dbms_stats.gather_table_stats(user,'T',cascade=>true,estimate_percent=>null,method_opt=>'f
or all indexed columns')
 
PL/SQL procedure successfully completed.
 
SQL> variable v1 number
SQL> exec :v1:=2
 
PL/SQL procedure successfully completed.
 
SQL> alter session set sql_trace=true
  2  / 
 
Session altered.
 
SQL> select *
  2    from t
  3   where rn=:v1
  4  / 
 
        RN OBJECT_NAME
---------- ------------------------------
         2 DUAL
         2 DUAL
         2 SYSTEM_PRIVILEGE_MAP
         2 SYSTEM_PRIVILEGE_MAP
         2 TABLE_PRIVILEGE_MAP
         2 TABLE_PRIVILEGE_MAP
         2 STMT_AUDIT_OPTION_MAP
         2 STMT_AUDIT_OPTION_MAP
         2 MAP_OBJECT
         2 RE$NV_LIST
 
10 rows selected.
 
SQL> exec :v1:=1
 
PL/SQL procedure successfully completed.


before executing i know if i collect stats from other session then parsed one plan will be invalidate and it should parse again but it will not due to default true invalidation.

gathering stats from other session with default

SQL> exec dbms_stats.gather_table_stats(user,'T',cascade=>true,estimate_percent=>null,method_opt=>'f
or all indexed columns')
 
PL/SQL procedure successfully completed.


back again to session 1

SQL> select *
  2    from t
  3   where rn=:v1
  4  / 
 
.
.
.
.
.
.
 
38962 rows selected.
 
 
 
SQL> alter session set sql_trace=false
  2  / 
 
Session altered.
 
SQL> disconnect
Disconnected from Oracle Database 10g Enterprise Edition Release 10.1.0.2.0 - Production
With the Partitioning, OLAP and Data Mining options


BEGIN :v1:=1; END;

call count cpu elapsed disk query current rows


------
----------
----------

Parse 1 0.00 0.02 0 0 0 0
Execute 1 0.00 0.01 0 0 0 1
Fetch 0 0.00 0.00 0 0 0 0

------
----------
----------

total 2 0.00 0.03 0 0 0 1

Misses in library cache during parse: 1
Misses in library cache during execute: 1
Optimizer mode: ALL_ROWS
Parsing user id: 57
********************************************************************************

select *
from t
where rn=:v1

call count cpu elapsed disk query current rows


------
----------
----------

Parse 2 0.04 0.47 0 0 0 0
Execute 2 0.00 0.07 0 0 0 0
Fetch 2601 0.17 0.09 0 5447 0 38972

------
----------
----------

total 2605 0.21 0.64 0 5447 0 38972

Misses in library cache during parse: 1<---reusing the plan
Optimizer mode: ALL_ROWS
Parsing user id: 57

Rows Row Source Operation


38962 TABLE ACCESS BY INDEX ROWID T (cr=5442 pr=0 pw=0 time=116941 us)
38962 INDEX RANGE SCAN T_RN_NDX (cr=2675 pr=0 pw=0 time=38979 us)(object id 49256)

********************************************************************************

what we see stats collection from other session does not in validate the parsed one plan

Next scenario
-------------
SQL> conn scott/tiger
Connected.
SQL> drop table t
  2  / 
 
Table dropped.
 
SQL> create table t as
  2  select 1 rn,object_name from all_objects
  3  / 
 
Table created.
 
SQL> update t set rn=2 where rownum<=10
  2  / 
 
10 rows updated.
 
SQL> commit
  2  / 
 
Commit complete.
 
SQL> select count(*),rn
  2    from t
  3   group by rn
  4  / 
 
  COUNT(*)         RN
---------- ----------
     38962          1
        10          2
 
SQL> create index t_rn_ndx on t (rn)
  2  / 
 
Index created.
 
SQL> exec dbms_stats.gather_table_stats(user,'T',cascade=>true,estimate_percent=>null,method_opt=>'f
or all indexed columns')
 
PL/SQL procedure successfully completed.
 
SQL> alter session set sql_trace=true
  2  / 
 
Session altered.
 
SQL> variable v2 number
SQL> exec :v2:=2
 
PL/SQL procedure successfully completed.
 
SQL> select *
  2    from t
  3   where rn=:v2
  4  / 
 
        RN OBJECT_NAME
---------- ------------------------------
         2 DUAL
         2 DUAL
         2 SYSTEM_PRIVILEGE_MAP
         2 SYSTEM_PRIVILEGE_MAP
         2 TABLE_PRIVILEGE_MAP
         2 TABLE_PRIVILEGE_MAP
         2 STMT_AUDIT_OPTION_MAP
         2 STMT_AUDIT_OPTION_MAP
         2 MAP_OBJECT
         2 RE$NV_LIST
 
10 rows selected.


now i go to other session and collect stats with

exec dbms_stats.gather_table_stats(user,'T',cascade=>true,estimate_percent => null,method_opt=>'for all indexed columns', no_invalidate=>FALSE);

SQL> exec dbms_stats.gather_table_stats(user,'T',cascade=>true,estimate_percent => null,method_opt=>
'for all indexed columns', no_invalidate=>FALSE); 
 
PL/SQL procedure successfully completed.
 
Come to session 1 again
-----------------------
 
SQL> exec :v2:=1
 
PL/SQL procedure successfully completed.
 
SQL> select *
  2    from t
  3   where rn=:v2
  4  .
SQL> / 
.
.
.
.
.
 
 
.
.
.
.
.
.
 
38962 rows selected.
 
 
 
SQL> alter session set sql_trace=false
  2  / 
 
Session altered.
 
SQL> disconnect
Disconnected from Oracle Database 10g Enterprise Edition Release 10.1.0.2.0 - Production
With the Partitioning, OLAP and Data Mining options
SQL> 



BEGIN :v2:=2; END;

call count cpu elapsed disk query current rows


------
----------
----------

Parse 1 0.00 0.00 0 0 0 0
Execute 1 0.00 0.00 0 0 0 1
Fetch 0 0.00 0.00 0 0 0 0

------
----------
----------

total 2 0.00 0.00 0 0 0 1

Misses in library cache during parse: 1
Misses in library cache during execute: 1
Optimizer mode: ALL_ROWS
Parsing user id: 57
********************************************************************************

select *
from t
where rn=:v2

call count cpu elapsed disk query current rows

------
----------
----------

Parse 2 0.04 0.40 0 0 0 0
Execute 2 0.00 0.18 0 0 0 0
Fetch 2601 0.04 0.06 0 2774 0 38972

------
----------
----------

total 2605 0.09 0.65 0 2774 0 38972

Misses in library cache during parse: 2<----not reusing 2 time parsing
Optimizer mode: ALL_ROWS
Parsing user id: 57
********************************************************************************

m i missing something?

Khurram
Dion_Cho

Posts: 648
Registered: 10/05/07
Re: Question on dbms_stats.auto_invalidate
Posted: Dec 4, 2007 4:56 AM   in response to: orawarebyte in response to: orawarebyte
Click to report abuse...   Click to reply to this thread Reply
Sorry, i should have been more clear about my comment on sql trace. What i wanted to say was that same SQL statement has different child cursor when executed with sql trace enabled.

See following:
alter system flush shared_pool;

select * from stat_table where id = 1; -- SQL trace is not enabled

alter session set sql_trace = true;

select * from stat_table where id = 1; -- SQL trace is enabled

alter session set sql_trace = false;

-- Two child cursors are created
select sql_id, child_number, sql_text from v$sql where sql_text like 'select * from stat_table where id = 1%';
==>
SQL_ID CHILD_NUMBER SQL_TEXT
1urxy5smf0fwf 3 select * from stat_table where id = 1
1urxy5smf0fwf 4 select * from stat_table where id = 1

-- Why? stats_row_mismatch happend, which means sql trace-enabled SQL statement is treated as different child cursor
select child_number, stats_row_mismatch from v$sql_shared_cursor where sql_id = '1urxy5smf0fwf' and child_number in (3, 4);
==>
CHILD_NUMBER STATS_ROW_MISMATCH
3 N
4 Y

Yes, your test case is flawless. Same SQL statements within same SQL traced block have same child cursor.

In addition, what we're talking about has no relationship with my original OP. :)

Typo...
Message was edited by:
Dion_Cho
Dion_Cho

Posts: 648
Registered: 10/05/07
Re: Question on dbms_stats.auto_invalidate (Solved?)
Posted: Dec 4, 2007 6:31 PM   in response to: Dion_Cho in response to: Dion_Cho
Click to report abuse...   Click to reply to this thread Reply
Why the gurus do not respond to my question? :)

Anyway, I did more intensive research and think that i've found answer.

Let me make the mechanims of cursor invalidation by statisitics change clearer.

1. when no_invalidate == false (default as of 9i)
Change in statistics makes all dependent cursors invalidated immediately.

2. when no_invalidate == true
Change in statistics makes no invalidation. New execution plan is generated only after the cursor is flushed out and reloaded.

3. when no_invalidate == dbms_stats.auto_invalidate (default as of 10g)
The most interesting case.
Change in statistics makes all dependent cursors invalidated after some specific period of time. This time is determined by the hidden parameter "_optimizer_invalidation_period".
The default value of this parameter is 18000(s) == 5 hour.
The weird thing is that this does not make the dependent cursor "really invalidated".
Instead, new child cursor is generated when the cursor is accessed after time has expired. This seems quite tricky.
v$sql_shared_cursor.roll_invalid_mismatch is 'Y' on those child cursors.

I want my comment make the situation clear and would like someone to point out the validity or invalidity of my conclusion.
Fairlie Rego

Posts: 25
Registered: 04/21/00
Re: Question on dbms_stats.auto_invalidate (Solved?)
Posted: Jan 29, 2008 4:48 PM   in response to: Dion_Cho in response to: Dion_Cho
Click to report abuse...   Click to reply to this thread Reply
This parameter works as expected.

Let us assume we set _optimizer_invalidation_period is 120 (2 minutes)
If at time t1 stats are gathered and at time t2 the sql statement of interest is re-executed then within the time period of t2+120 a new child cursor will be created with roll_invalid_mismatch.

This parameter however does not work for queries using PQ slaves.
I hope to put the results on my blog when I get around to it. So I am not sure what your exact concern is

Thanks
Fairlie
Fairlie Rego

Posts: 25
Registered: 04/21/00
Re: Question on dbms_stats.auto_invalidate (Solved?)
Posted: Jan 30, 2008 1:22 AM   in response to: Dion_Cho in response to: Dion_Cho
Click to report abuse...   Click to reply to this thread Reply
Just to add your observations are absolutely spot on.

Oracle does a hard parse when ROLL_INVALID_MISMATCH flag/bit is set.
The fact that the parent cursor is not set to INVALID is a non-issue since the behaviour is correct
as only the executions of the child cursor and the last_active_time of the child cursor are updated
on subsequent executions. I have pasted my test here

http://el-caro.blogspot.com/2008_01_01_archive.html
Tanel Poder

Posts: 178
Registered: 07/06/98
Re: Question on dbms_stats.auto_invalidate (Solved?)
Posted: Feb 2, 2008 8:59 AM   in response to: Fairlie Rego in response to: Fairlie Rego
Click to report abuse...   Click to reply to this thread Reply
Yeah, btw Oracle uses a pseudorandom number generator for distributing the invalidation timestamps across dependent objects needing invalidation (within allowed timeframe specified usng _optimizer_invalidation_period). That way the potentially large number of hard parses is distributed across a long period of time.
Dion_Cho

Posts: 648
Registered: 10/05/07
Re: Question on dbms_stats.auto_invalidate (Solved?)
Posted: Feb 2, 2008 6:08 PM   in response to: Tanel Poder in response to: Tanel Poder
Click to report abuse...   Click to reply to this thread Reply
Great. Seems that we have a whole picture now.
Legend
Guru Guru : 2500 - 1000000 pts
Expert Expert : 1000 - 2499 pts
Pro Pro : 500 - 999 pts
Journeyman Journeyman : 200 - 499 pts
Newbie Newbie : 0 - 199 pts
Oracle ACE Director
Oracle ACE Member
Oracle Employee ACE
Helpful Answer (5 pts)
Correct Answer (10 pts)

Point your RSS reader here for a feed of the latest messages in all forums