Wednesday, January 19, 2011

More on Interested Transaction Lists

How Oracle Locking Works
When a Lock is NOT a Lock!

In the last installment of this series "100 Thing you Probably Didn't Know About Oracle" you learned how Oracle locks the rows of a table. Here is what you learned in a nutshell:

(1) When a transaction modifies a record, the pre-change image is stored in the undo segments, which is required for various things; the most important of which is to provide a read consistent version of the row when another session wants it.

(2) The transaction is assigned a transaction identifier that shows the undo segment number, slot# and record of the undo information.

(3) The transaction locks the rows (since it did not commit) by placing a special type of data in the block header known as Interested Transaction List (ITL) entry. The ITL entry shows the transaction ID and other information.

(4) When a new transaction wants to update the same rows (locked by the previous transaction) it checks the ITL entries in the block first, to check if there is a lock.

(5) Since the lock information of rows is stored in the block itself, and the ITL entries in the block refer to the locks on the rows in that block alone, there is no need to have a central lock manager to dispense and handle the release of the locks. This makes the locking process not only immensely scalable but feasible as well since there is no theoretical limit to the number of locks.

[Updated Jan 22, 2011] [Thank you, Randolph Geist (info@www.sqltools-plusplus.org) for pointing it out. I follow his blog http://oracle-randolf.blogspot.com/, which is a treasure trove of information.
(6) The information that a row is locked is stored along with the row in the form of a lock byte.
[End of Update Jan 22, 2011]

While the article might have answered some of the vexing questions you may have had or needed some clarity on the concepts you were somewhat familiar with, I sincerely hope it has piqued you curiosity to learn even more about these concepts. If I was successful in explanation, now you should not be satisfied, you should have more questions. If you don’t have any, then I completely failed in my explanation.

So, what are the questions? For starters, how do you know what objects being locked in the transaction? It’s actually quite trivial. The view V$LOCK has provided that information for years, albeit in a convoluted form. A new view V$LOCKED_OBJECT is a bit more user-friendly. Let’s examine that with an example. First, update a row:

SQL> update itltest set col2 = 'CHANGED BY SESSION AGAIN' where col1 = 221
  2  /

1 row updated.

We can check the transaction ID:

SQL> select dbms_transaction.local_transaction_id from dual'

LOCAL_TRANSACTION_ID
--------------------------------------------------------------------------------
2.16.41316

1 row selected.

As you learned from the previous installment in this series, the transaction ID is a series of numbers denoting undo segment number, slot# and record# (also known as sequence#) respectively, separated by periods.

Now, check the view V$LOCKED_OBJECT:

SQL> select * from v$locked_object
  2  /

    XIDUSN    XIDSLOT     XIDSQN  OBJECT_ID SESSION_ID
---------- ---------- ---------- ---------- ----------
ORACLE_USERNAME                OS_USER_NAME
------------------------------ ------------------------------
PROCESS                  LOCKED_MODE
------------------------ -----------
         2         16      41316      95263         56
ARUP                           oracle
13181                              3

The view shows Undo Segment# (XIDUSN), Undo Slot# (XIDSLOT) and Undo Rec# (XIDSQN), which can be used to construct the transaction ID to be joined with the V$TRANSACTION to get the details. The view contains the column OBJECT_ID. Another important column is LOCKED_MODE, which shows the mode the rows are locked. In this case, it’s “3”, which means Row Exclusive. Here is a script that decodes the modes as well as reports the object name.

select
    owner               object_owner,
    object_name         object_name,
    session_id          oracle_sid,
    oracle_username     db_user,
    decode(LOCKED_MODE,
        0, 'None',
        1, 'Null',
        2, 'Row Share',
        3, 'Row Exclusive',
        4, 'Share',
        5, 'Sub Share Exclusive',
        6, 'Exclusive',
        locked_mode
    )                   locked_mode
    from v$locked_object lo,
        dba_objects do
    where
        (xidusn||'.'||xidslot||'.'||xidsqn)
            = ('&transid')
    and
        do.object_id = lo.object_id
/

Save this script and execute it when you need further details on the transaction. The script will ask for the transaction ID which you can pass in the format reported by dbms_transaction.local_transaction_id.

Next, you may draw my attention to the point #3 above. If there are 10 records in the block and a transaction updated (and therefore locked) all ten of them, how many ITL entries will be used – one or ten?

Good question (I have to say that, since I asked that :) I suppose you can answer that yourself. Ten ITL slots may be feasible; but what if the block has 10,000 records? Is it possible to have that many ITL slots in the block header? Let’s ponder on that for a second. There will be two big issues with that many ITL slots.

First, each ITL slot, by the way, is 24 bytes long. So, 10000 slots will take up 240,000 bytes or almost 22 KB. A typical Oracle block is 8KB (I know, it could be 2K, 4K or 16K; but suppose it is the default 8K). Of course it can’t accommodate 22KB.

Second, even if the total size of the ITL slots is less than the size of the block, where will be the room to hold data? In addition, there should be some space for the data block overhead; where will that space come from?

Obviously, these are genuine problems that make one ITL slot per row impractical. Therefore Oracle does not create an ITL entry for each locked row. Instead, it creates the ITL entry for each transaction, which may have updated a number of rows. Let me repeat that – each ITL slot in the block header actually refers to a transaction; not the individual rows. That is the reason why you will not find the rowid of the rows locked in the ITL slot.  Here is the ITL entry from the block header, again:

Itl           Xid                  Uba         Flag  Lck        Scn/Fsc
0x01   0x000a.019.00007c05  0x00c00288.1607.0e  ----    1  fsc 0x0000.00000000
0x02   0x0003.017.00009e24  0x00c00862.190a.0f  C---    0  scn 0x0000.02234e2b

There is a reference to a transaction ID; but not rowid. When a transaction wants to update a row in the block, it checks the ITL entries. If there is none, it means rows in that block are unlocked. However, if there are some ITL entries, does it mean that some rows in the block are locked? Not necessarily. It simply means that the rows the block were locked earlier; but that lock may or may not be active now. To check if a row is locked, the transaction checks for the lock byte stored along with the row.

That brings up an interesting question. If presence of an ITL slot does not mean a record in the block is locked, when does the ITL slot get cleared so that it can be reused, or when does that ITL slot disappear? Shouldn’t that ITL slot disappear when the transaction ends by commit or rollback? That should be the next burning question throbbing in your head right now.

Clearing of ITL Slots

To answer that question, consider this scenario: a transaction updates 10000 records, on 10000 different blocks. Naturally there will be 10000 ITL slots, one on each block, all pointing to the same transaction ID. The transaction commits; and the locks are released. Should Oracle revisit each block and remove the ITL entry corresponding to the transaction as a part of the commit operation?

If that were the processing logic, the commit would have taken a very long time. Acquiring the buffers of the 10000 blocks and updating the ITL entry will not be quick; it will take a very long time, prolonging the commit processing. From part 1 of the series, you learned that the commit processing is actually very quick, with a flush of the log buffer to redo logs and the writing of the commit marker in the redo stream. Even a checkpoint to the datafiles is not done as a part of commit processing – all the effort going towards making the process fast, very fast. Had Oracle added the logic of altering ITL slots, the commit processing would have been potentially long, very long. Therefore Oracle does not remove the ITL entries after that transaction ends (by committing, or rolling back); the slots are just left behind as artifacts.

The proof, as they say, is in the pudding. Let’s see with an example:

SQL> create table itltest (col1 number, col2 varchar2(200));

Table created.

SQL> begin
  2     for i in 1..1000 loop
  3             insert into itltest values (
  4                     i,'INITIAL VALUE OF COLUMN');
  5     end loop;
  6  end;
  7  /

PL/SQL procedure successfully completed.

SQL> commit;

Commit complete.

This inserts 1000 records. Let’s find out the file and block these records go to:

  1  select
  2     dbms_rowid.rowid_relative_fno(rowid) File#,
  3     dbms_rowid.rowid_block_number(rowid) Block#,
  4     count(1)
  5  from itltest
  6  group by
  7     dbms_rowid.rowid_relative_fno(rowid),
  8     dbms_rowid.rowid_block_number(rowid)
  9  order by
 10*    1,2
SQL> /

     FILE#     BLOCK#   COUNT(1)
---------- ---------- ----------
         7       4027        117
         7       4028        223
         7       4029        220
         7       4030        220
         7       4031        220

5 rows selected.

Let’s identify the rows in a specific block, block# 4028, for instance.

SQL> select min(col1), max(col1)
  2  from itltest
  3  where dbms_rowid.rowid_block_number(rowid) = 4028
SQL> /

 MIN(COL1)  MAX(COL1)
---------- ----------
1 223

1 row selected.

Block 4028 has the rows 1 through 223. That’s all we need to know for now. We will limit our activity to this block alone. We will need to update a single row in this block from a session:

SQL> update itltest set col2 = ‘Changed’ where col1 = 1;

Do NOT commit; just keep the session at this point. Open a different session, and update a different row, e.g. one with col1 = 2. Since this is a different row, there will be no lock contention. Similarly update 20 other rows on this block. There will be 20 different transactions on the rows of this table.

Let’s examine the innards of the block by dumping it. Before that, we should flush the block to the disk.

SQL> alter system checkpoint;

System altered.

SQL> alter system dump datafile 7 block min 4028 block max 4028;

System altered.

The information will be written to a tracefile. We have to know the SPID of the process to identify the tracefile:

SQL> select p.spid
  2  from v$session s, v$process p
  3  where s.sid = (select sid from v$mystat where rownum < 2)
  4* and p.addr = s.paddr
SQL> /

SPID
------------------------
9537

We will locate a file called D112D2_ora_9537.trc in the trace directory. Please note, this tracefile is named OracleSID_ora_ProcessID.trc; so the exact name will be different your system. Open the file and search for “Itl”. Here is an excerpt from the file:

Block header dump:  0x01c00fbc
 Object id on Block? Y
 seg/obj: 0x1741f  csc: 0x00.235a849  itc: 36  flg: E  typ: 1 - DATA
     brn: 0  bdba: 0x1c00fb8 ver: 0x01 opc: 0
     inc: 0  exflg: 0

 Itl           Xid                  Uba         Flag  Lck        Scn/Fsc
0x01   0x0008.00d.0000a1eb  0x00c015d1.1d3c.28  ----    1  fsc 0x0005.00000000
0x02   0x0007.018.00007fab  0x00c01246.180b.21  ----    1  fsc 0x0005.00000000
0x03   0x0003.004.0000a1b0  0x00c005ef.1a18.07  ----    1  fsc 0x0005.00000000
0x04   0x0010.010.00000004  0x00c011ee.0001.10  ----    1  fsc 0x0005.00000000
0x05   0x000e.00e.00000003  0x00c011cb.0001.0f  ----    1  fsc 0x0005.00000000
0x06   0x000c.00e.00000003  0x00c011ab.0001.1b  ----    1  fsc 0x0005.00000000
0x07   0x0013.011.00000004  0x00c00f9c.0001.0f  ----    1  fsc 0x0005.00000000
0x08   0x0002.00a.0000a166  0x00c014d8.1c06.12  ----    1  fsc 0x0005.00000000
0x09   0x0001.010.00007f65  0x00c00cd3.16ae.14  ----    1  fsc 0x0005.00000000
0x0a   0x0014.01b.00000008  0x00c00faa.0003.67  ----    1  fsc 0x0005.00000000
0x0b   0x000f.00f.00000003  0x00c011db.0001.20  ----    1  fsc 0x0005.00000000
0x0c   0x000d.00f.00000004  0x00c011bb.0001.1d  ----    1  fsc 0x0005.00000000
0x0d   0x0012.010.00000003  0x00c00f8b.0001.1c  ----    1  fsc 0x0005.00000000
0x0e   0x000a.00c.00007f76  0x00c003d7.16ea.31  ----    1  fsc 0x0005.00000000
0x0f   0x0011.010.00000004  0x00c011fb.0001.10  ----    1  fsc 0x0005.00000000
0x10   0x0009.000.0000a236  0x00c00e91.1bbe.17  ----    1  fsc 0x0005.00000000
0x11   0x0006.00e.0000a1fc  0x00c0035c.1c24.2d  ----    1  fsc 0x0005.00000000
0x12   0x000b.012.00000003  0x00c01193.0001.1d  ----    1  fsc 0x0005.00000000
0x13   0x0004.00e.00007ff7  0x00c00d01.1771.0a  ----    1  fsc 0x0005.00000000
0x14   0x0005.002.0000a19f  0x00c00f1a.1bd6.1d  ----    1  fsc 0x0005.00000000
0x15   0x0015.000.00000002  0x00c00fba.0000.02  ----    1  fsc 0x0005.00000000
0x16   0x0016.000.00000002  0x00c00fca.0000.02  ----    1  fsc 0x0005.00000000
0x17   0x0017.000.00000002  0x00c00fda.0000.02  ----    1  fsc 0x0005.00000000
0x18   0x0018.000.00000002  0x00c00fea.0000.02  ----    1  fsc 0x0005.00000000
0x19   0x0019.000.00000002  0x00c00ffa.0000.02  ----    1  fsc 0x0005.00000000
0x1a   0x001a.000.00000002  0x00c0100a.0000.02  ----    1  fsc 0x0005.00000000
0x1b   0x001b.000.00000002  0x00c0101a.0000.02  ----    1  fsc 0x0005.00000000
0x1c   0x001c.000.00000002  0x00c0102a.0000.02  ----    1  fsc 0x0005.00000000
0x1d   0x001d.000.00000002  0x00c0103a.0000.02  ----    1  fsc 0x0005.00000000
0x1e   0x001e.002.00000002  0x00c0104a.0000.03  ----    1  fsc 0x0005.00000000
0x1f   0x001f.002.00000002  0x00c0105a.0000.03  ----    1  fsc 0x0005.00000000
0x20   0x0020.000.00000002  0x00c0106a.0000.02  ----    1  fsc 0x0005.00000000
0x21   0x0021.005.00000002  0x00c0107a.0000.08  ----    1  fsc 0x0000.00000000
0x22   0x0022.000.00000002  0x00c0108a.0000.02  ----    1  fsc 0x0005.00000000
0x23   0x0023.000.00000002  0x00c0109a.0000.02  ----    1  fsc 0x0005.00000000
0x24   0x0024.000.00000002  0x00c010aa.0000.02  ----    1  fsc 0x0005.00000000
bdba: 0x01c00fbc
data_block_dump,data header at 0xeb6994

Note the Itl entries – there is an entry for each transaction, marked by its transaction ID, as expected. When the block was created, there were two ITL slots. As the demand for locks increased, additional slots were created and used for these new transactions.

Now go to all these sessions and either commit or rollback to end the transactions. Dump the block and search for “Itl”. The ITL slots are still there, even though the transactions have ended and the locks released. Oracle does not update the ITL entries.

So, when does the ITL entry gets cleared? When block’s buffer is written to the disk, the unneeded ITL entries are checked and cleared out. Let’s force a block flushing:

SQL> alter system checkpoint;
Now dump the data block once again and examine the ITLs. Here is an excerpt from the tracefiles.

Object id on Block? Y
 seg/obj: 0x1741f  csc: 0x00.235a849  itc: 36  flg: E  typ: 1 - DATA
     brn: 0  bdba: 0x1c00fb8 ver: 0x01 opc: 0
     inc: 0  exflg: 0

 Itl           Xid                  Uba         Flag  Lck        Scn/Fsc
0x01   0x0014.016.00000008  0x00c00fb3.0002.11  C---    0  scn 0x0000.0235a524
0x02   0x0000.000.00000000  0x00000000.0000.00  ----    0  fsc 0x0000.00000000
0x03   0x0000.000.00000000  0x00000000.0000.00  ----    0  fsc 0x0000.00000000
0x04   0x0000.000.00000000  0x00000000.0000.00  ----    0  fsc 0x0000.00000000
0x05   0x0000.000.00000000  0x00000000.0000.00  ----    0  fsc 0x0000.00000000
0x06   0x0000.000.00000000  0x00000000.0000.00  ----    0  fsc 0x0000.00000000
0x07   0x0000.000.00000000  0x00000000.0000.00  ----    0  fsc 0x0000.00000000
0x08   0x0000.000.00000000  0x00000000.0000.00  ----    0  fsc 0x0000.00000000
0x09   0x0000.000.00000000  0x00000000.0000.00  ----    0  fsc 0x0000.00000000
0x0a   0x0000.000.00000000  0x00000000.0000.00  ----    0  fsc 0x0000.00000000
0x0b   0x0000.000.00000000  0x00000000.0000.00  ----    0  fsc 0x0000.00000000
0x0c   0x0000.000.00000000  0x00000000.0000.00  ----    0  fsc 0x0000.00000000
0x0d   0x0000.000.00000000  0x00000000.0000.00  ----    0  fsc 0x0000.00000000
0x0e   0x0000.000.00000000  0x00000000.0000.00  ----    0  fsc 0x0000.00000000
0x0f   0x0000.000.00000000  0x00000000.0000.00  ----    0  fsc 0x0000.00000000
0x10   0x0000.000.00000000  0x00000000.0000.00  ----    0  fsc 0x0000.00000000
0x11   0x0000.000.00000000  0x00000000.0000.00  ----    0  fsc 0x0000.00000000
0x12   0x0000.000.00000000  0x00000000.0000.00  ----    0  fsc 0x0000.00000000
0x13   0x0000.000.00000000  0x00000000.0000.00  ----    0  fsc 0x0000.00000000
0x14   0x0000.000.00000000  0x00000000.0000.00  ----    0  fsc 0x0000.00000000
0x15   0x0000.000.00000000  0x00000000.0000.00  ----    0  fsc 0x0000.00000000
0x16   0x0000.000.00000000  0x00000000.0000.00  ----    0  fsc 0x0000.00000000
0x17   0x0000.000.00000000  0x00000000.0000.00  ----    0  fsc 0x0000.00000000
0x18   0x0000.000.00000000  0x00000000.0000.00  ----    0  fsc 0x0000.00000000
0x19   0x0000.000.00000000  0x00000000.0000.00  ----    0  fsc 0x0000.00000000
0x1a   0x0000.000.00000000  0x00000000.0000.00  ----    0  fsc 0x0000.00000000
0x1b   0x0000.000.00000000  0x00000000.0000.00  ----    0  fsc 0x0000.00000000
0x1c   0x0000.000.00000000  0x00000000.0000.00  ----    0  fsc 0x0000.00000000
0x1d   0x0000.000.00000000  0x00000000.0000.00  ----    0  fsc 0x0000.00000000
0x1e   0x0000.000.00000000  0x00000000.0000.00  ----    0  fsc 0x0000.00000000
0x1f   0x0000.000.00000000  0x00000000.0000.00  ----    0  fsc 0x0000.00000000
0x20   0x0000.000.00000000  0x00000000.0000.00  ----    0  fsc 0x0000.00000000
0x21   0x0021.002.00000002  0x00c0107a.0000.05  C---    0  scn 0x0000.0235a807
0x22   0x0000.000.00000000  0x00000000.0000.00  ----    0  fsc 0x0000.00000000
0x23   0x0000.000.00000000  0x00000000.0000.00  ----    0  fsc 0x0000.00000000
0x24   0x0000.000.00000000  0x00000000.0000.00  ----    0  fsc 0x0000.00000000
bdba: 0x01c00fbc
data_block_dump,data header at 0x484994

Note the Xid columns – the transaction Id, which shows 0’s, meaning there is no transaction using the ITL slots. These ITL slots are eligible for reuse. Update two rows from two different sessions, checkpoint and dump the block once again. Here is the ITL information again:

Itl           Xid                  Uba          Flag  Lck        Scn/Fsc
0x01   0x0014.016.00000008  0x00c00fb3.0002.11  C---    0  scn 0x0000.0235a524
0x02   0x0005.009.0000a1a5  0x00c00f21.1bd6.04  ----    1  fsc 0x0016.00000000
0x03   0x000a.002.00007f7a  0x00c003d8.16ea.13  ----    1  fsc 0x0016.00000000
0x04   0x0000.000.00000000  0x00000000.0000.00  ----    0  fsc 0x0000.00000000
0x05   0x0000.000.00000000  0x00000000.0000.00  ----    0  fsc 0x0000.00000000
0x06   0x0000.000.00000000  0x00000000.0000.00  ----    0  fsc 0x0000.00000000
... and so on ...

The first two Itl slots are now used. Note, only the ITL slots in this specific block will be created. All other blocks will continue to have the same number of ITL slots.

ITL Waits

From the part 2 of this series you learned that the ITL slots are not preallocated, at least not all of them. When a transaction needs to lock rows in the block, and it does not find an unused ITL slot, Oracle creates a new ITL slot for the transaction. Consider the figure below. There is no more room in the block for a new ITL entry. A new transaction comes in to update Record3. What will happen?



The transaction will have to wait. This is not the same wait as a row lock; because there is no lock on the row marked Record3.Instead, session will wait on a special wait event. You can check the wait event from the V$SESSION view.

SQL> select event
  2  from v$session
  3  where sid = 78
  4  /

EVENT
----------------------------------------------------------------
enq: TX - allocate ITL entry

The moment one of the transactions – from either Session1 or Session2 end by commit or rollback, the new transaction can grab that ITL slot and complete the locking operation. You will see that wait event disappear.

Since the ITL waits come and go, how do you capture them; or more specifically how will you know which objects are being subjected to this wait? It’s fairly trivial. Since Oracle 9.2 a new view – V$SEGMENT_STATSTICS – shows various segment related statistics on segments. Here is an example:

SQL> select statistic_name, value from v$segment_statistics
  2* where object_name = 'ITLTEST';

STATISTIC_NAME                                                        VALUE
---------------------------------------------------------------- ----------
logical reads                                                          7216
buffer busy waits                                                         3
gc buffer busy                                                            0
db block changes                                                       5600
physical reads                                                            0
physical writes                                                          39
physical read requests                                                    0
physical write requests                                                   9
physical reads direct                                                     0
physical writes direct                                                    0
optimized physical reads                                                  0
gc cr blocks received                                                     0
gc current blocks received                                                0
ITL waits                                                                 2
row lock waits                                                            1

Various stats on the segment named ITLTEST are listed here. Of the lot, the one interesting to our discussion here is “ITL waits”, which shows “2”. It means the table ITLTEST has waited 2 times for ITL waits (not for a legitimate row locking, which shown in the stats immediately afterwards).

Conversely, you may want to find out what have been subjected to ITL waits. The following query shows you that:

SQL> select owner, object_name
  2  from v$segment_statistics
  3  where statistic_name = 'ITL waits'
  4* and value > 0;

OWNER                          OBJECT_NAME
------------------------------ ------------------------------
ARUP                           ITLTEST

1 row selected.

The view has many more columns for making filtering easier:

SQL> desc v$segment_statistics
 Name                                      Null?    Type
 ----------------------------------------- -------- ----------------------------
 OWNER                                              VARCHAR2(30)
 OBJECT_NAME                                        VARCHAR2(30)
 SUBOBJECT_NAME                                     VARCHAR2(30)
 TABLESPACE_NAME                                    VARCHAR2(30)
 TS#                                                NUMBER
 OBJ#                                               NUMBER
 DATAOBJ#                                           NUMBER
 OBJECT_TYPE                                        VARCHAR2(18)
 STATISTIC_NAME                                     VARCHAR2(64)
 STATISTIC#                                         NUMBER
 VALUE                                              NUMBER

Actually selecting from the above view is a bit expensive on the database. The base view is V$SEGSTAT, shown below:

SQL> desc v$segstat
 Name                                      Null?    Type
 ----------------------------------------- -------- ----------------------------
 TS#                                                NUMBER
 OBJ#                                               NUMBER
 DATAOBJ#                                           NUMBER
 STATISTIC_NAME                                     VARCHAR2(64)
 STATISTIC#                                         NUMBER
 VALUE                                              NUMBER

While V$SEGMENT_STATISTICS show much more information, it’s a little slow due to all those joins. If you don’t need all that information, you may want to select instead from V$SEGSTAT, which is usually faster. The columns are self explanatory; but here they are in any case:

TS# - the tablespace number. You can use this to get the tablespace name from TS$ table joined by TS# column
OBJ# - the object_id, from dba_objects. You can get the rest of the details from that view
DATAOBJ# - the data_object_id, from dba_objects. This is usually the same as object_id; except in case of sub-objects such as partitions in which case they differ.

One important point about this view: like all V$ views, it shows information from the start of the instance. When the instance recycles, the values are reset to 0. To get a historical information, you should periodically select from this view and store in a regular table. If you have AWR enabled, you can check the historical records from there. Here is an example:

SQL> select snap_id, ITL_WAITS_TOTAL, ITL_WAITS_DELTA
  2  from DBA_HIST_SEG_STAT
  3  where obj# = 95263
  4* order by snap_id;

   SNAP_ID ITL_WAITS_TOTAL ITL_WAITS_DELTA
---------- --------------- ---------------
      5014               2               2

1 row selected.

Solution

Well, so far I talked about a problem. Is there a solution? Of course there is.

Remember, the cause of ITL waits is simply space inside a block. If there is no space inside the block to grow the ITL list to add more slots, the sessions will wait with the ITL wait event. So the solution is to reserve same space for that growth. There are two basic alternatives to solve the ITL wait problem:

(1) INITRANS.

Remember the little clause during table or index creation? Have you ever explicitly set it to its non-default value? Most likely you haven’t. It specifies the number of ITL slots that must be initially created on a block. If you specify 10, then 10 ITL slots are created on the block, guaranteeing the slot for 10 transactions. The 11th transaction will need to extend the ITL list; or wait if that is not possible.

To check for the INITRANS value of tables, use:

SQL> select ini_trans
  2  from dba_tables
  3  where table_name = 'T';

 INI_TRANS
----------
        10

(2) Less Space for Data

The other option is to make sure that you have less data inside a data block to allow the ITL sufficient free space. You can do it by several ways – by setting a high value of PCTFREE and by setting MINIMIZE_RECORDS_PER_BLOCK clause.

Obviously, both these options waste space inside the block; so you should use these only on those segments that experience high ITL waits, as you can see from AWR reports or your homegrown data collectors. To increase the INITRANS of an existing table, you should issue:

ALTER TABLE ITLTEST INITRANS 10;

Remember, the setting affects the new blocks only; not the existing ones. You can issue ALTER TABLE … MOVE command to relocate the blocks to new blocks, and thereby effecting the new settings.

What is the upper limit of the ITL slots? They are set by a parameter of the object called MAXTRANS. The default is 256. If you set it to 20, the ITL slots will go up to that much only. However, the parameter has no effect in Oracle 10gR2. It’s ignored and the ITL slots can go up to 256.

Takeaways

In this installment you learned:

(1) ITL itself does not say whether a row is locked or not. The lock byte stored in the row tells that.
(2) When a transaction ends, the corresponding ITL entry is not removed or altered. It gets cleared during flush to the disk.
(3) When the ITL can’t grow due to the lack of space in the block, the session waits will the event “enq: TX - allocate ITL entry
(4) You can identify the segments that have suffered from this wait by checking the view V$SEGSTAT.
(5) To reduce the possibility of these waits, you should have sufficient space inside the data block for ITL expansion, either by defining higher number of initial ITL slots, or forcing less data inside the blocks.


I hope you enjoyed this installment. As always, I will appreciate if you drop in a line on how you liked it.

45 comments:

Yogi said...

Interesting insight.

Since you have started discussion on ITL, I hope your next article covers delayed block cleanout.

Regards
Yogi

Alberto said...

Very interesting article and very clear,i've followed it step by step.
I'm already waiting the next one...
Great job, Arup.
Alberto

jean-michel said...

Dear Arup,
When I read your first article on the Oracle myths, I decided to make an electronic folder on my PC as well as a paper folder where I can easily find all your interesting developments on the Oracle database software and then share those nuggets with my colleagues here at work. Thank you very much for your work. I really need to understand the fondamentals of Oracle to then be able to work in a better way.
Greetings
Jean-michel, France

Yasin Baskan said...

Hi Arup,

You can do it by several ways – by setting a high value of PCTUSED and by setting MINIZE_RECORDS_PER_BLOCK clause.

Should it not be PCTFREE? We can reserve free space in a block by setting PCTFREE higher.

Arup said...

@Yashin. You are correct. Thank you for pointing it out. It was a typo error. I have fixed it.

Anonymous said...

Hi Sir,

Very interesting.Can you provide some details on "lock byte" which is stored along with the row during dml.

Regards,
Anand

Arup said...

@Anand

The lock byte (or is it, "bit"?) is stored along with the row. Here is the block dump

Block header dump: 0x01c00fd4
Object id on Block? Y
seg/obj: 0x1753f csc: 0x00.238a937 itc: 2 flg: E typ: 1 - DATA
brn: 0 bdba: 0x1c00fd0 ver: 0x01 opc: 0
inc: 0 exflg: 0

Itl Xid Uba Flag Lck Scn/Fsc
0x01 0x0008.019.0000a20a 0x00c01342.1d45.29 --U- 2 fsc 0x0000.0238a979
0x02 0x0000.000.00000000 0x00000000.0000.00 ---- 0 fsc 0x0000.00000000
bdba: 0x01c00fd4
data_block_dump,data header at 0x645664
===============
tsiz: 0x1f98
hsiz: 0x16
pbl: 0x00645664
76543210
flag=--------
ntab=1
nrow=2
frre=-1
fsbo=0x16
fseo=0x1f6e
avsp=0x1f58
tosp=0x1f58
0xe:pti[0] nrow=2 offs=0
0x12:pri[0] offs=0x1f83
0x14:pri[1] offs=0x1f6e
block_row_dump:
tab 0, row 0, @0x1f83
tl: 21 fb: --H-FL-- lb: 0x1 cc: 2
col 0: [ 2] c1 02
col 1: [14] 54 68 69 73 20 69 73 20 61 20 74 65 73 74
tab 0, row 1, @0x1f6e
tl: 21 fb: --H-FL-- lb: 0x1 cc: 2
col 0: [ 2] c1 03
col 1: [14] 54 68 69 73 20 69 73 20 61 20 74 65 73 74
end_of_block_dump
End dump data blocks tsn: 4 file#: 7 minblk 4052 maxblk 4052

Note the highlighted section that shows lock bytes. However, note that the lock byte (or bit) may not be cleared on commit. So, the system of record for a row being locked is still the transaction; not the lock byte.

Vishal said...

Arup,

This series is very interesting! Thanks for sharing this great stuff!

Kashif said...

Hi Arup, really enjoying your articles so far.

I've noticed when performing an update (or delete) which affects 0 rows, the session gets a row-x lock on the table. Dumping the block shows that there is now corresponding ITL entry, as you'd expect.

Interestingly I noticed that after performing update:

update itltest set col2='Changed' where col1=-1;

v$locked_object shows values for XIDUSN/XIDSLOT and XIDSQN are all 0. After performing 'select dbms_transaction.local_transaction_id from dual' from the session which performed the update, the corresponding TX_ID gets reflected in the v$ view.

So why does Oracle obtain a lock in this scenario?

Regards
Kashif

Arup said...

@Kashif

I was trying to reproduce the problem described by you; but I couldn't. So, obviously I probably misinterpret your question somehow. Could you post your questions with what specifically you see (with copy/paste)?

Kashif said...

Hi Arup

Session 1: perform an update which changes no rows:
sess1>update itltest set col2='Changed' where col1=-1;

0 rows updated.

sess2 >select XIDUSN,XIDSLOT,XIDSQN,LOCKED_MODE from v$locked_object;

XIDUSN XIDSLOT XIDSQN LOCKED_MODE
---------- ---------- ---------- -----------
0 0 0 3

--Sess1 has acquired a row exclusive lock

sess1 > select dbms_transaction.local_transaction_id from dual;

LOCAL_TRANSACTION_ID
----------------------------------------------------------------
6.42.768738

sess2 >select XIDUSN,XIDSLOT,XIDSQN,LOCKED_MODE from v$locked_object;

XIDUSN XIDSLOT XIDSQN LOCKED_MODE
---------- ---------- ---------- -----------
6 42 768738 3

--Now the transaction info is set in the v$ view.

Even though no actual row is locked we have a lock on the table


sess2 > ALTER TABLE ITLTEST ADD(COL3 NUMBER);
ALTER TABLE ITLTEST ADD(COL3 NUMBER)
*
ERROR at line 1:
ORA-00054: resource busy and acquire with NOWAIT specified

Kashif

Arup said...

@Kashif - Now I understand what you were asking. Well, I have been doing research on this since yesterday; and I can't explain. I definitely do not want to misspeak; but if I could make an educated guess, it would be that the selection from v$locked_object forced population of some memory structure, which were selected by the view (exposed via some x$ table, of course).

I will continue to have some more research on this and update, if I have some more information.

Arup said...

@Kashif,

Sorry, as I mentioned I couldn't find much on that; so I asked for help. Jonathan Lewis (need I tell who he is?) chimed in with some answers.

As to why you would see a row lock, here is his answer:


This makes sense to me in terms of "lazy" processing.

Imagine your code does a three minute tablescan before finding a row to lock.
Oracle doesn't want to find that it does the three minute scan and then cannot
update the row because of a conflicting lock, so it makes sense to take the
TM object (x$ktadm) before starting the update. Similarly, since the TX object
(x$ktcxb) is far more limited in size than the number of transaction table slots
(typically) it makes sense to grab a TX object and link the ktadm before starting
the update. Since acquiring a transaction table slot does some real work (undo
and redo) Oracle postpones that action until the first real data change takes place -
remember the previous transaction table slot information goes into the first undo
record - and your update hasn't yet got to the point of generating undo).

The anomaly, therefore, is that Oracle doesn't release the two x$ items when the
update fails to find any rows - but that's just an example of "lazy processing".

Arup said...

As to why the undo information was not visible until the dbms_transaction.local_transaction_idwas called, nothing is a clear answer. It could be as simple as a bug in the package, as Riyaj Shamsudeen (http://orainternals.wordpress.com) pointed out. Thank you, Riyaj. In any case it does not really affect anything in real life.

Kashif said...

Hi Arup,

OK, it makes sense to grab a lock before beginning the update. Thanks for chasing this up, I appreciate it. Thank you Jonathan Lewis too!

Look forward in reading more of your posts

Thanks again
Kashif

Aravind said...

Hi Arup,

This is a very interesting blog series. Could you please explain on the scenario when multiple transactions try to update the same row as well. Will there be any ITL allocated? Yes, I am talking about the real locking scenario.

Regards,
Aravind

Arup said...

@Aravind,

Thank you for the question and the comment. I was going to address your question, then I realized that it will be probably impossible to fit within the small answer window. So I expanded the scope a little and created a new blog post to address your questions. Please let me know what you think. http://arup.blogspot.com/2011/02/question-came-up-on-my-blog-entry.html

അരവിന്ദ് said...

Hi Arup,

Thanks for the detailed explanation.

Aravind

Pradip said...

Thank you very much on such insights. This is valuable information, might be available , but taking the time to write and giving it in such a crisp manner is deeply appreciated.

rafeek said...

aroop,

These are the actual things dba's should know, tons of thanks to you for brought up to the attention.

Really apriciated, thank to kasif to pointing out.

looking for next post to read.

Rafeek.

Rafeek.

Sanblazer said...
This comment has been removed by the author.
JAMSHER KHAN said...

Very deep study and nicely explain thanks for sharing

--Jamsher

sewa mobil said...

Nice article, thanks for the information.

nicholas mihai said...

Question from a newbie:
in oracle documentation is stated that : "The database uses a transaction table, also called an interested transaction list (ITL), to determine if a transaction was uncommitted when the database began modifying the block" but in the glossary at the end of the concepts book i found the following explination for the transaction table :"The data structure within an undo segment that holds the transaction identifiers of the transactions using the undo segment" , i found this a little bit confusing! where is actualy located this transaction table and what info it contains? The ITL it's located in the header of the data block and it cannot be the same with the transaction table "withing an undo segment".
Many thanks in advance!

Arup said...

@Nicholas that's a good question. I can't speak for the Oracle documentation; but I can see why it could be confusing. It seems the words "transaction table" has been used in two different contexts. In one manual it is referred to as a synonym for ITL and in another the transaction identifiers. These are two very different entities. ITL is on a data block; transaction entries (session address, what value, SCN#, etc.) is inside the undo segment block. They are different entities and located at very different places.

nicholas mihai said...

Thank you for this quick answer but i have some questions regarding the arhitecture links between ITL,undo info, redo logs, restore/recover from a situation:

I)does this transaction entries of transaction table from undo segments keep the information about the commit state of a transaction?
i understand now that the entries inside an ITL points to this undo segments transaction tables using the XID, the ITL also keeps information about the location uf the segments and of the blocks with undo info from the undo tablespace.
II)
as i know undo tablespce data also generates redo logs.(regarding the changes from undo blocks)!
when a restore/recovery is need to be done using a backup and some archived redo log(an restore/recover until SCN for expl),the followings occurs if a transaction has entries in some archived redo logs):
1) the backup is restored
2) the data file raw pieces are updated with the info found in the first arch. redo log(the transaction info with the before and after images of the first statement modifications) in a rollforward process
--in this time also the ITL entries are updated in the data files blocks???--
3) in the recovery process the undo blocks are also updated with the info from archived logs(because undo TBS also generates redo logs)
4)the blocks are updated with the last info from the last archive log , let's assume that this archived log contains the info about the commit state of the transaction started to be recovered in the first logs
5) the recovery process is successfull regarding this transaction and the data is now permanent

-q: does the transaction table info (scn, xid , commit state of the transaction -if this info is stored here) is recovered in the rollfwd process?
7) when the undo blocks and data blocks are finally recovered the ITL correctly points to the undo entries from undo segments (is this true?)
7) does the ITL also contain the info about the state of transactions(commit or not) that has locks on raws from data block(or it only points to undo segment transaction table)?
8) does the transaction table of undo segment keep the locking state from ITLs?
Hope i make myself understud and thank you once again!

nicholas mihai said...

what i ment here "the followings occurs if a transaction has entries in some archived redo logs" is the case when a transaction have more statements and the info about this statements (that modified block datas) is spread on more than one arhived log.

dorobel said...
This comment has been removed by the author.
Amit Kapila said...

1. After transaction T-1 updated row-1 in block, when other transaction T-2 came to update the same row, it can check lock bit and if lock bit indicates row is locked, T-2 has to verify from ITL entry of T-1 whether T-1 is still active to conclude row is locked.
How T-2 will reach ITL entry, is there a transaction id stamp in row header? What does lock byte exactly contain?

2. Where is the particular row's undo address is stored, so that during running transaction when a select comes to read a particular row, it can go to undo to read that info?

Baskar said...

Hi Arup,

Very interesting. Understand the hard work behind it to explain the concept in a clear and simple way.
Many a thanks for sharing the information.

Thanks,
Baskar.l

Ellis Barrett said...

Usually I do not post on articles, but I would like to say that this blog really forced me to do so! Thanks, really nice post.

Anonymous said...

Dear Arup,

Thank you for yr explanation.

But I'm not understand about some parameter INITTRANS, MAXTRANS, PCTFREE. These parameters is used for tables, but ITLs is used in block header. What happen if there is 2 (or more) tables with difference values of these parameters in the same bolck ?

Please help me to explain.
Thanks
K.P

Arup Nanda said...

@K.P. - it is not possible to have two different tables in the same block. Each table has its own block. There are exceptions, such as when hash clusters are used; but in those cases the ITLs will be the same.

Anonymous said...

Dear Arup,

Thank you so much. I have mistake when reading about "Table Directory".
Now I understand this article.

Can you write a post to analyze LRU - Touch count, IMU (in-memory undo) ?

I'm already waiting ...

Thanks
K.P

Puremelda said...

Our pre written essays for sale and online research paper writer work collaboratively with our team of researchers to ensure that every order is thoroughly researched before drafting.

SEO Services said...

Are you running a small business in your local area? Do you want to widen it at a large scale? If yes, we are a complete SEO guide in the USA and offering small business SEO services at very affordable rates. Let us see your SEO strategy. We identify that you’re using an outdated SEO strategy or not, so we make a complete audit of your website’s performance.

Outlook Support Phone Number said...

Microsoft Outlook is a well-known email service accessed by thousands of people. But often, it is obvious that some technical glitches related to Outlook account could encounter by one and need an instant quality tech-support assistance to troubleshoot them. So, here you go! Place a call at Outlook Tech Support Phone Number simply and get united with our team without any fear or doubt.
Microsoft 365 support
Office 365 support
Outlook Support Phone Number
Microsoft Outlook Support
Outlook Support

HP Printer Troubleshooting said...

It is very easy to resolve HP printers’ problem with the help of HP Printer Support team. You will get complete guidance to fix HP printer paper jam, printer offline, printhead error and all other print errors. A team of certified experts is available round the clock to offer instant service to the customers.

HP Printer Troubleshooting said...

It is very easy to resolve HP printers’ problem with the help of HP Printer Support team. You will get complete guidance to fix HP printer paper jam, printer offline, printhead error and all other print errors. A team of certified experts is available round the clock to offer instant service to the customers.

hpprintercare said...

Most of the printers are configured and assembled in such a way that there is no issue with the operating system for attaining the most suitable printing outcome. As soon as you install HP printer with diverse range of operating system, you might be meet slew range of network in order to recognize the system. Certain time, you go through prompt message WPS pin printer for perceiving the quality throughput within date. It is not easy for any user what is WPS and how to achieve the WPS pin printer code. Any user does not have any idea how to settle WPS pin so that it becomes easy to resume the printing issue. No HP customer used to be fret while getting connect in such difficulties. A number of destinations are available to sort out this complexity and however, it is suggested that one should have to come on our third party professional team to get the solution of all difficulties. It would be better to dial our toll free number in emergency case for getting the solution of all abnormalities. View our website to know more information. https://www.hpprintersupportpro.com/blog/find-wps-pin-on-hp-printer-and-establish-connection/
wps pin
wps pin hp printer
wps pin on printer
wps pin printer
wps pin on hp printer


rickypauls said...

Assignment Help is the best answer to solve all the problems related to assignment writing and project submission. Make your work more effective and valuable with the help of academic experts of greatassignmenthelp For any help, talk to academic expert.
Assignment Help Online
Online Assignment Help
Assignment Help Online Services
Assignment Helper
Assignment Assistance
Assignment Help Experts
Online Assignment Help Services

rickypauls said...

Assignment Help is the best answer to solve all the problems related to assignment writing and project submission. Make your work more effective and valuable with the help of academic experts of greatassignmenthelp For any help, talk to academic expert.
Assignment Help Online
Online Assignment Help
Assignment Help Online Services
Assignment Helper
Assignment Assistance
Assignment Help Experts
Online Assignment Help Services

John King said...

The mentioned points in this guide are very helpful and with the help of these, I am able to resolve my Avast Renewal issues in a very proficient way. In one step, I got stuck and don’t know what to do next, then I called on the given number. The technician, who received my phone call, helped me to fix my problem and guide me until the renewal process didn’t get completed.

Epson Support said...

Epson Printer Error Code 0x97

Epson Error Code 0x97

Epson windows service disabled error

Epson error code 0xf1

Epson wireless printer setup

Epson printer offline

Epson Printer in error state

Epson Support

Epson Printer Support
Epson Printer Not Printing
Epson Printer Setup

MarksonJoy said...

Very nice!!! This is really good blog information thanks for sharing. We are a reliable third party Quickbooks Help company offering technical support for various any types of technical errors.

Translate