Wednesday, September 30, 2015

UNDO_RETENTION behaviour in autoextend undo tabelspace vs. fixed-size undo tablespace

Introduction

In this post I'd like to clarify a common misconception regarding the UNDO_RETENTION initialization parameter but first, let's start with a short introduction.
The Undo Tablespace is a key component in Oracle Databases. It consists of undo segments which hold the "before" images of the data that has been changed by users running transactions. The undo is essential for Rollback operations, Data Concurrency, Read Consistency and is being used by several flashback features (Flashback Query, Flashback Table, Flashback Version Query, Flashback Transaction Query).
As you probably know, the UNDO_RETENTION initialization parameter specifies the low threshold (in seconds) for the undo retention period. Obviously, this parameter is important and you should bear in mind the different behaviour of this parameter in fixed-size undo tablespace and in autoextend undo tablespace.

Undo Extent Types - Active/Expired/Unexpired

Active undo extents are used by running transactions. These extents will never be overwritten as they are needed in order to perform ROLLBACK operations. 
Expired Undo Extents hold committed information that is older than the UNDO_RETENTION period.
Unexpired undo Extents hold committed information that its age is less than the UNDO_RETENTION period.

You can use the following query in order to calculate the distribution of the different extent types:
select status,
       ROUND (sum_bytes / (1024 * 1024), 0)  MB,
       ROUND ((sum_bytes / undo_size) * 100, 0)  "%"
  from (  select status, SUM (bytes) sum_bytes
            from dba_undo_extents
        group by status),
       (select SUM (a.bytes) undo_size
          from dba_tablespaces c
               join v$tablespace b
                  on (b.name = c.tablespace_name)
               join v$datafile a
                  using (ts#)
         where c.contents = 'UNDO')

Sample Output:

STATUS                                 MB        %
------------------------------    ----------  ----------
ACTIVE                               1040         70
EXPIRED                                27          3
UNEXPIRED                             418         27
If the ACTIVE percent is 100 it means that all the undo extents contain undo images of running transaction and needed for performing a ROLLBACK operation and none of them can be overwritten therefore you'll get "ORA-30036: unable to extend segment by string in undo tablespace" so you need to make sure that the ACTIVE percent is not close to 100.
Also, if the UNEXPIRED pecent is close to 100 it means that almost all of the undo extents hold data of committed information that its age is less than the UNDO_RETENTION period so these undo extents may be overwritten and then the chances of encountering "ORA-01555: Snapshot Too Old" will increase. Other option is that you just set you UNDO_RETENTION to be too high and you can verify it by comparing the UNDO_RETENTION to the longest running query (MAXQUERYLEN column in V$UNDOSTAT).

Fixed-Size Undo Tablespace

For fixed-size undo tablespaces this parameter is being ignored (unless retention guarantee is enabled) and Oracle will automatically tune for the maximum possible undo retention period, based on the undo tablespace size and undo usage history.

Autoextend Undo Tablespace

For autoextend undo tablespace this parameter specifies the minimum retention period the Oracle will attempt to honor. When space in the undo tablespace becomes low (due to running transactions which generate undo records) Oracle will increase the tablespace size (up to the MAXSIZE limit) and once it will reach to the upper limit of the MAXSIZE it will begin to overwrite unexpired undo information therefore the retention period defined in the UNDO_RETENTION period is not guaranteed. This is why the actual period might be lower or even higher. The actual undo retention period can be obtained by querying the TUNED_UNDORETENTION column in V$UNDOSTAT dynamic performance view.
Note: You can specify RETENTION GUARANTEE (in the CREATE UNDO TABLESPACE or ALTER TABLESPACE commands) and then Oracle never overwrites unexpired undo data even if it means that transactions fail due to lack of space in the undo tablespace.

Below is an example of a database that I have configured with UNDO_RETENTION of 15 minutes (i.e. UNDO_RETENTION = 900) but because there are queries with higher elapsed time than 900 (in my case the longest query duration is ~23 minutes (=1358 seconds) and therefore V$UNDOSTAT will report on a TUNED_UNDORETENTION which is higher than 900 in many cases (558 to be more specific). Following is a screenshot of my example:


















Useful Links:

Wednesday, September 16, 2015

Wondering why you can't view data with a COMMON USER in Oracle 12c? You probably didn't use the CONTAINER_DATA clause

Introduction
Let's say you've created a common user in your Oracle 12c instance and you granted the user permissions to connect and select some specific dynamic views (e.g. V$PDBS, V$CONTAINERS).
Afterwards, you connect with that common user to the root container (CDB$ROOT), you query V$PDBS but no rows are returned.
This issue was raised in the OTN forum by a user and my answer to that user was  very simple - use the CONTAINER_DATA clause (See: https://community.oracle.com/message/13301017#13301017).

Basically, when a common user is connected to the ROOT and it executes a query on a container data object (As per Oracle Doc, container data objects include: V$, GV$, CDB_, and some Automatic Worklaod Repository DBA_HIST* view), then that query will only dispay data for the PDBs which are visible for that common user, and this is what you can set using the CONTAINER_DATA clause.

Demonstration
In the first step I'll create a common user, grant him permissions to connect and query V$PDBS and then I'll connect with that user and try to query V$PDBS.
SQL> select name,open_mode from v$pdbs;
NAME                           OPEN_MODE
------------------------------ ----------
PDB$SEED                       READ ONLY
PDBTEST                        MOUNTED
PINIDB                         READ WRITE

SQL> create user C##TEST identified by test;
User created.

SQL> grant connect to C##TEST;
Grant succeeded.

SQL> grant select on sys.v_$pdbs to C##TEST;
Grant succeeded.

SQL> connect C##TEST/test@isrvmrh541-cdb.world
Connected.

SQL> select * from v$pdbs;
no rows selected
As you can see, no rows are returned from the query.
Let's connect again with SYS and verify which PDBs are visible for user C##TEST using the CDB_CONTAINER_DATA dictionary view which displays information about the user-level and object-level CONTAINER_DATA attributes specific in the CDB:
SQL> connect sys@isrvmrh541-cdb.world as sysdba
Enter password:
Connected.

SQL> SELECT username,
  2         owner,
  3         object_name,
  4         all_containers container_name
  5    FROM CDB_CONTAINER_DATA
  6   WHERE username = 'C##TEST';
no rows selected
As you can see, user C##TEST has no container data attributes.
Let's specify that user C##TEST can see the data of V$PDBS from all the containers:
SQL> alter user C##TEST set container_data=all for sys.v_$pdbs container = current;
User altered.

SQL> SELECT username,
  2         owner,
  3         object_name,
  4         all_containers,
  5         container_name
  6    FROM CDB_CONTAINER_DATA
  7   WHERE username = 'C##TEST'
  8   ;

USERNAME           OWNER     OBJECT_NAME   ALL_CONTAINERS CONTAINER_NAME
--------------- ---------- --------------- ------------- --------------------
C##TEST            SYS      V_$PDBS           Y
As you can see, the CONTAINER_NAME is NULL because I sepcific in the aler command that the container_data will be visible for all containers.
If we would like to specify that the data of every container data object that the user has SELECT permissions to access, you can remove the "for" clause and execute the following command:
SQL> alter user C##TEST set container_data=all container = current;
User altered.

SQL> SELECT username,
  2         owner,
  3         object_name,
  4         all_containers,
  5         container_name
  6    FROM CDB_CONTAINER_DATA
  7   WHERE username = 'C##TEST';

USERNAME           OWNER    OBJECT_NAME    ALL_CONTAINERS CONTAINER_NAME
--------------- ---------- --------------- ------------- --------------------
C##TEST            SYS      V_$PDBS            Y
C##TEST                                        Y
As you can see, now user C##TEST has an access for all the objects which he will granted permissions to SELECT from. 

Summary
  • Once you create a COMMON USER, you should also specify which PDBs data are visible to that common user for which objects using the CONTAINER_DATA clause
  • You can specify the object-level CONTAINER_DATA attributes for a user using the ALTER USER command
  • You can view the information about the user-level and object-level CONTAINER_DATA attributes via CDB_CONTAINER_DATA dictionary view

Useful Links:

Friday, September 11, 2015

The new Oracle Edition - "Standard Edition 2" has been released!

The new Oracle Edition, Standard Edition 2 has been released (on September 1st) and is available for download as version 12.1.0.2 at: http://www.oracle.com/technetwork/database/enterprise-edition/downloads/index.html

So far 12.1.0.2 was available only for the Enterprise Edition so with this new edition, customers who are using 12.1.0.1 with the Standard Edition, can finally upgrade to 12.1.0.2

This new edition will replace the Standard Edition and Standard Edition one, therefore 12.1.0.1 is the last version that offers the Standard Edition and Standard Edition One. Standard Edition 2 will cost exactly the same as Standard Edition (as per Oracle global price list, it costs 17,500$ per processor), therefore customers who are using Standard Edition One (as per Oracle global price list, it costs 5,800$ per processor) and would like to upgrade to Standard Edition 2, will need to pay an extra charge in order to benefit the advantages of Standard Edition 2 (such as using Real Application Clusters). For customers who are using 12.1.0.1 Standard Edition or Standard Edition One, the permier support is available until september 1st, 2016

Notes:

  1. Standard Edition 2 is limited to 2 CPU sockets per system (Standard Edition was limited to 4)
  2. RAC with Standard Edition 2 is limited to 2 nodes (1 CPU socket per node)
  3. Standard Edition 2 is limited to 16 CPU threads per system (Standard Edition was unlimited)
  4. RAC with Standard Edition 2 is limited to 8 CPU threads per node


Useful Links:

Monday, September 7, 2015

Table point in-time recovery using RMAN

Introduction
In this post I'd like to present a new 12c feature - RMAN table point in time recovery.
This feature allows to perform a recovery for a specific table or table partition via RMAN, and it can be very useful in the following cases :
  • The table was truncated - so you can't use the 10g Flashback Table feature as the undo is not available
  • The table was dropped with the purge clause -so you can't use the 10g Flashback Drop feature
  • Table was logically corrupted and no undo is available - so you can't use the 10g Flashback Table feature
  • Table was logically corrupted and a DDL operation modified the structure of the table - so you can't use the 10g Flashback Table feature in order to flashback the table prior to the DDL
In order to overcome this scenarios before Oracle 12c you could recover the entire database to a point in time recovery - however, you will also recover all the other objects to the same point in time which will result in a data loss.
A better option would be to perform a tablespace point-in-time-recovery (only available for the Enterprise Edition) which means that you can recovery the entire tablespace of that table to an auxiliary instance and then manually export and import the table to the target database.
The new feature in Oracle 12c performs all this automatically.

Prerequisites
In order to use this feaute you must have a backup of the tablespace/s for the selected table/tables because during the recover process Oracle will automatically perform point in time recovery using those tablespaces and then it will export the table/s from the auxiliary (temporary) instance and then it will import the data into the target (source) instance.

Demonstration:
In this demonstration I'm connected with user "pini", creating a table "emp" and inserts 1 row and then commit it:
SQL> show user;
USER is "PINI"
SQL> create table emp (id number, name varchar2(20));
Table created.
SQL> insert into emp values (1, 'DAVID');
1 row created.
SQL> commit;
Commit complete.
Now, I check what's the current SCN so I can use it later in the RMAN RECOVER command (RMAN supports point-in-time-recovery until SCN or TIME or SEQUENCE)
SQL> select current_scn from v$database;
CURRENT_SCN
-----------
   25427549
Now I will drop it with the "purge" option (so the flashback drop feature is irrelevant):
SQL> drop table emp purge;
Table dropped.
The next step would be to connect to RMAN and execute the RECOVER TABLE command.
You should also specify the auxiliary destination which is the location of the auxiliary database files and also the Data Pump Export Dump File:
RMAN> recover table pini.emp until scn 25427549 auxiliary destination '/oravl01/recover';

Starting recover at 07-SEP-15
using channel ORA_DISK_1
RMAN-05026: WARNING: presuming following set of tablespaces applies to specified Point-in-Time

List of tablespaces expected to have UNDO segments
Tablespace SYSTEM
Tablespace UNDOTBS1

Creating automatic instance, with SID='Fkyk'

initialization parameters used for automatic instance:
db_name=O12102NP
db_unique_name=Fkyk_pitr_O12102NP
compatible=12.1.0.2.0
db_block_size=8192
db_files=200
diagnostic_dest=/oravl01/oracle
_system_trig_enabled=FALSE
sga_target=1024M
processes=200
db_create_file_dest=/oravl01/recover
log_archive_dest_1='location=/oravl01/recover'
#No auxiliary parameter file used
...
...
...
...
Performing import of tables...
   IMPDP> Master table "SYS"."TSPITR_IMP_Fkyk_kvas" successfully loaded/unloaded
   IMPDP> Starting "SYS"."TSPITR_IMP_Fkyk_kvas":
   IMPDP> Processing object type TABLE_EXPORT/TABLE/TABLE
   IMPDP> Processing object type TABLE_EXPORT/TABLE/TABLE_DATA
   IMPDP> . . imported "PINI"."EMP"                                5.476 KB       1 rows
   IMPDP> Processing object type TABLE_EXPORT/TABLE/STATISTICS/TABLE_STATISTICS
   IMPDP> Processing object type TABLE_EXPORT/TABLE/STATISTICS/MARKER
   IMPDP> Job "SYS"."TSPITR_IMP_Fkyk_kvas" successfully completed at Mon Sep 7 14:13:05 2015 elapsed 0 00:00:11
Import completed


Removing automatic instance
Automatic instance removed
auxiliary instance file /oravl01/recover/O12102NP/datafile/o1_mf_temp_bytw2co8_.tmp deleted
auxiliary instance file /oravl01/recover/FKYK_PITR_O12102NP/onlinelog/o1_mf_3_bytw4md4_.log deleted
auxiliary instance file /oravl01/recover/FKYK_PITR_O12102NP/onlinelog/o1_mf_2_bytw4m1t_.log deleted
auxiliary instance file /oravl01/recover/FKYK_PITR_O12102NP/onlinelog/o1_mf_1_bytw4lhq_.log deleted
auxiliary instance file /oravl01/recover/FKYK_PITR_O12102NP/datafile/o1_mf_users_bytw4gos_.dbf deleted
auxiliary instance file /oravl01/recover/O12102NP/datafile/o1_mf_sysaux_bytvy598_.dbf deleted
auxiliary instance file /oravl01/recover/O12102NP/datafile/o1_mf_undotbs1_bytvy590_.dbf deleted
auxiliary instance file /oravl01/recover/O12102NP/datafile/o1_mf_system_bytvy59h_.dbf deleted
auxiliary instance file /oravl01/recover/O12102NP/controlfile/o1_mf_bytvxw4w_.ctl deleted
auxiliary instance file tspitr_Fkyk_10996.dmp deleted
Finished recover at 07-SEP-15
That's it!
The EMP table is now available in the source instance with the data which is relevant to the selected time period (in my case to SCN 25427549):
SQL> select * from emp;
        ID NAME
---------- --------------------
         1 DAVID

Few Notes:
  • You can specify a different location for the dump file using the DATAPUMP DESTINATION clause
  • During the last step of this process, i.e. during the import step, you may choose to import the table to a new table name and not to its original name using the REMAP TABLE clause
  • During the last step of this process, i.e. during the import step, you may choose to import the table to a new tablespace and not to its original tablespace using the REMAP TABLESPACE clause
  • If you prefer to have a dump file point in time export backup without actually importing it backup to the target (source) database, use the NOTABLEIMPORT clause
Conclusions
  • This feature requires minimum work and effort from the DBA
  • Due to the fact that as part of the recovery process RMAN needs to restore an auxiliary instance, and also export and import the relevant table/s, this whole process has an overhead (mainly in terms of I/O load), therefore If you can use other options for recovering the table (such as Flashback Table/Flashback Drop) you should use them and avoid using this feature. Also, table-recovery using this feature takes much longer than using the Flashback drop/Flashback table
  • When you can't use Flashback drop/Flashback table to recover the table, you may find this feature very useful.
  • This feature is available with the Enterprise Edition only

Thursday, September 3, 2015

Simple Performance Tuning Methodology

In this post I'd like to show a simple methodology for performance tuning using Wait-Event Analysis.
Logically, the process is very simple:
  1. Determine the most significant bottleneck
  2. Improve/Fix it
  3. Repeat it until the performance is good enough
The question is, how can we find the most significant bottleneck for a specific session or for a specific SQL, or even for the entire instance?

Basically, as process in Oracle can be in one of the following states:
  • Consuming CPU - which is usually fine and useful
  • Non-Idle Wait event - Waiting for something (lock, latch, I/O)
    • For example: whenever a process is reading multi blocks in a single I/O operation (usually in cases of a Full Table Scan or Index Fast Full Scan), Oracle will report on a "db file scattered read" wait event
    • In order to get a list of all the Non-Idle wait events, execute the following query:
SELECT name
  FROM v$event_name
 WHERE wait_class <> 'Idle'
  • Idle Wait Event - Doesn't do anything, indication that the user session is inactive.
    • For example: whenever a server process is waiting for the user client to do something, Oracle will report on a "SQL*Net message from client"
    • In order to get a list of all the Idle wait events, execute the following query:
SELECT name
  FROM v$event_name
 WHERE wait_class = 'Idle'

Note: when a process is waiting for something it can also consume CPU, for example, in some cases of latch contentions, there will also be a high CPU consumption (due to the latch spinning).

So how can you measure the top non-idle wait events in the database?

There are severl moethods:

Oracle Dynamic Views:

  • V$SYSTEM_EVENT - Provides information on all the wait-events which occured in the instance since-startup. You can get all the non-idle wait events, ordered by the time (in seconds) using the following query:
SELECT event, time_waited / 100 "Seconds Waited"
   FROM v$system_event
   WHERE wait_class <> 'Idle'
 ORDER BY 2 DESC
  • V$SESSION_EVENT - Provides information on all the wait-events which occudes in a specific session (relevant only for connected sessions, not historical sessions). You can get all the non-idle wait events for a specific session, ordered by the time (in seconds) using the following query:
SELECT event, time_waited / 100 "Seconds Waited"
  FROM v$session_event
 WHERE wait_class <> 'Idle' AND SID = <SID>
ORDER BY 2 DESC

    AWR:
    The AWR report also holds lots wait-events related information. However, usually the "Top 5  Timed Foreground Events" section in the beginning of the report will provide most of the important information. Following is a screenshot of this section in one of our Oracle instances:















    So how can you measure the overall performance?
    The answer is, "DB Time" - a statistic which represents the total amount of time that Oracle spent during CPU processing and waiting for non-idle wait events.
    DB Time = CPU Time + Non Idle wait-events

    Notes:

    1. DB Time doesn't take into account background sessions. It takes into account foreground sessions only.
    2. It doesn't take into account idle wait-events.
    The DB Time could also be measured in a several ways:

    Oracle Dynamic Views:
    • V$SYSSTAT - Displays instance-level statistics. You can view the value of the DB Time (since instance startup), using the following query:
    SELECT value
      FROM v$sysstat
     WHERE name = 'DB time'
    • V$SESSTAT - Displays session-level statistics. You can view the value of the DB Time for a specific session (connected session only), using the following query:
    SELECT value
      FROM v$sesstat JOIN v$statname USING (statistic#)
     WHERE name = 'DB time' AND sid= <SID>


    AWR:

    Obviously, the AWR report also hold information about the DB Time.
    In the beginning of the report there's a summary of the DB Time for the AWR report duarion:






















    As you can see, this AWR report is based on a 2 snapshots which their distance is 1 hour - the first snapshot is taken at 13:00, and the second snapshot is taken at 16:00, therefore the value of "Elapsed" is 1 hour. You may now ask yourself, how is it possible that the value of DB time (3 hours) was higher than the enitre duration of the report (which is 1 hour). Is it possible that during a 1-hour period, Oracle spent 3 hours on cpu and non-idle wait events?!
    The answer is: absolutely!
    Since the DB Time equals to the total amount of CPU time + non-idle wait events, so in machines where you have more than 1 CPU core (in this particular example there are 4 CPU cores), the DB Tme can definitely be higher than the Elapsed Time.

    For example:

    If during a 10 second-period, all the 4 CPU cores were fully utilized, then the DB Time value will be 40 for those 10 second-period.

    After we determined the most significant Wait-Events, i.e. those which impact the database mostly, how can we reduce the wait time or even totally avoid them?

    The answer is that it mainly depends on your experience with performance tuning.
    There are over 1000 wait-events in Oracle but the good news are the eventually there's a limited number of common wait events, and this number is much smaller than 1000, so eventually, you will be familiar with most of the common wait-events, and you'll know which actions you should take in order to solved them.
    If you're encountering performance issues that are related to a specific wait-event which you are not familiar with, you can always ask our best friend, google :)

    Why use Performance Dianostics tools?

    Some tools can make our life easier by visualizing the workload of the instance.
    Some tools can even display the workload filtered by a specific dimension or a combination of dimensions (e.g. SQL Statment, User, Program, Machine, PDB, Session, etc.).
    Having the ability to filter the workload for a set of combinations allows performing advanced performance diagnostics research.
    For example: viewing all the wait-events and statistics for all the SQL Statements that were executed by user HR using TOAD under PDB (Pluggable Database) PROD.

    OEM ASH Analytics was introduced in OEM 12c and it allows you to perform OLAP operations on the database dimensions which makes the performance tuning process much simpler and faster.

    Please note that in order to benefit this feature you must have the Enterprise Edition and also the tuning and diagnostics packs.

    Foglight for Oracle also provides the ability to view a list of all the dimensions and drill-down to any combination of dimensions, as you can see in the below screenshot:









    Foglight for Oracle comes with a full dimension stack:




    Here is an example of how easily you can drill-down to all the SQL Statements that were executed from TOAD by the user SALES:

























    Conclusions:
    1. DB Time is an important measurement which represents the total CPU + Non-Idle wait events of foreground sessions.
    2. When a DBA wants to improve the performance, the best approach is to focus on the major/top wait-events and reduce/fix them in order to reduce the total DB time until the performance will be good enough.
    3. Some tools (Oracle Enterprise Manager, Dell Foglight) allow visualizing the workload of the entire instance or the activity of a specific dimension/set of dimensions. This can reduce dramatically(!) the time it takes for the DBA identifying and solving these issues.