Table of Contents

< Day Day Up >
Next Section

Oracle High Performance Tuning for 9i and 10g
Oracle High Performance Tuning for 9i and 10g
by Gavin Powell ISBN:1555583059
Digital Press © 2004 (738 pages)

This text covers the three parts of tuning an Oracle database: data modeling, SQL code tuning, and physical database configuration, and explains both problem detection and resolution.

Table of Contents
 | Oracle High Performance Tuning for 9i and 10g
 | Preface
 | Introduction
 | Part I - Data Model Tuning
 | Chapter 1 | - | The Relational Database Model
 | Chapter 2 | - | Tuning the Relational Database Model
 | Chapter 3 | - | Different Forms of the Relational Database Model
 | Chapter 4 | - | A Brief History of Data Modeling
 | Part II - SQL Code Tuning
 | Chapter 5 | - | What is SQL?
 | Chapter 6 | - | The Basics of Efficient SQL
 | Chapter 7 | - | Common Sense Indexing
 | Chapter 8 | - | Making SQL Efficient in Oracle Database
 | Chapter 9 | - | How to Find Problem Queries
 | Chapter 10 | - | Tuning SQL with Oracle Enterprise Manager
 | Part III - Physical and Configuration Tuning
 | Chapter 11 | - | Installing Oracle and Creating a Database
 | Chapter 12 | - | Tuning Oracle Database File Structures
 | Chapter 13 | - | Object Tuning
 | Chapter 14 | - | Low-Level Physical Tuning
 | Chapter 15 | - | Hardware Resource Usage Tuning
 | Chapter 16 | - | Tuning Network Usage
 | Chapter 17 | - | Oracle Partitioning and Parallelism
 | Chapter 18 | - | Ratios: Possible Symptoms of Problems
 | Chapter 19 | - | Wait Events
 | Chapter 20 | - | Latches
 | Chapter 21 | - | Tools and Utilities
 | Chapter 22 | - | Tuning with the Wait Event Interface and STATSPACK
 | Appendix A | - | Sample Databases
 | Appendix B | - | Sample Scripts
 | Appendix C | - | Sources of Information
 | Index
 | List of Figures

 < Day Day Up >
Next Section


BackCover

Previous Section
< Day Day Up >
Next Section

Back Cover

Master the three parts of tuning an Oracle database: data modeling, SQL code tuning and physical database configuration.

A data model contains tables and relationships between tables. Tuning a data model involves nNormalization and dDe-normalization. Different approaches are required depending on the application, such as OLTP or a Data Warehouse. Inappropriate database design can make SQL code impossible to tune. Poor data modeling can have a most profound effect on database performance since all SQL code is constructed from the data model.

Poorly written SQL code is often a culprit in performance problems and is expensive to rectify. However, tuning of SQL code is generally cheaper than changing the data model. SQL code tend to be contained inside independent blocks within applications or stored procedures.

Physical database tuning involves hardware resource usage, networking and various other Oracle components such as configuration and file distribution. Physical configuration is often a culprit of poor performance where Oracle is installed with defaults, and never altered by an expert.

  • Includes all three aspects of Oracle database tuning: data model tuning, SQL & PL/SQL code tuning, physical plus configuration tuning
  • Contains experienced guidance and real-world examples using large datasets
  • Emphasizes development as opposed to operating system perspective

About the Author

Gavin Powell BSc., Comp.Sci., OCP has fifteen years of diverse experience in database administration and database development in both relational and object databases. His applications development experience is procedural and object-oriented and he also has some systems administration experience. He consults for software vendors, Internet .COMs (some of which unfortunately have met with their demise), accounting, banking, and financial services, the travel industry, construction, retail, mining, shipping, education and in a general advisory capacity.

Previous Section


Oracle High Performance Tuning for 9i and 10g

Gavin Powell

Digital Press is an imprint of Elsevier
200 Wheeler Road, Burlington, MA 01803, USA
Linacre House, Jordan Hill, Oxford OX2 8DP, UK

Copyright © 2004, Elsevier Inc. All rights reserved.

No part of this publication may be reproduced, stored in a retrieval system, or transmitted in any form or by any means, electronic, mechanical, photocopying, recording, or otherwise, without the prior written permission of the publisher.

Permissions may be sought directly from Elsevier's Science & Technology Rights Department in Oxford, UK: phone: (+44) 1865 843830, fax: (+44) 1865 853333, e-mail: permissions@elsevier.com.uk. You may also complete your request on-line via the Elsevier homepage (http://elsevier.com), by selecting "Customer Support" and then "Obtaining Permissions."

Recognizing the importance of preserving what has been written, Elsevier prints its books on acid-free paper whenever possible.

Library of Congress Cataloging-in-Publication Data

British Library Cataloguing-in-Publication Data

A catalogue record for this book is available from the British Library.

ISBN: 1-55558-305-9

For information on all Digital Press publications visit our website at www.digitalpress.com and www.bh.com/digitalpress

03 04 05 06 07 08 10 9 8 7 6 5 4 3 2 1

There are three parts to tuning an Oracle database: data modeling, SQL code tuning followed by physical and configuration tuning.

A data model contains tables and relationships between tables. Tuning a data model involves Normalization and Denormalization. Different approaches are required depending on the application, such as for an Internet OLTP or data warehouse database. Inappropriate database design can make SQL code impossible to tune. Poor data modeling can have a most profound effect on data base performance since all SQL code is constructed from the data model. Changes to the data model are often the most difficult and expensive changes to implement.

Poorly written SQL code is often a culprit of performance prob lems and is expensive to rectify. However, tuning of SQL code is gen erally cheaper than changing the data model. SQL code tends to be contained inside independent blocks within applications or stored procedures.

Physical database tuning involves hardware resource usage, net working, and various other facets of Oracle such as configuration and file distribution. Inappropriate physical and configuration set tings are often a culprit of poor performance where Oracle is installed with defaults, and never altered by an expert.


What is in this book?

Previous Section
< Day Day Up >
Next Section

Preface

This book is about tuning Oracle databases. Three areas of Oracle Database tuning are data model tuning, SQL code tuning plus physical, and configuration tuning. The author began his career as an applications developer and not a systems or network administrator. As a result this book is written from an applications rather than an operating system perspective.

The objective of this book is to cover all three areas of Oracle Database tuning. Currently there is no title on the market completely covering all of these. This book will cover all three by explaining both problem detection and resolution.

The approach in this book is to present something which appears to be immensely complex in a simplistic and easy-to-understand manner. Both reference material and examples are utilized appropriately in order to expedite understanding for the reader.

Reference material is not overused. Oracle software has been in general use commercially for many years and is intended to provide for a very large and diverse customer base. Features are often not removed from Oracle software between versions and new features are continuously being added. The result is that Oracle software contains a plethora of available options and features. Using only reference information to explain Oracle Database tuning would therefore be difficult to read, contrary to the approach of this book and would not provide the reader with much practical advice. This book contains a lot of examples, with realistic databases and data; sometimes even very large amounts of data. After all, if your production database needs tuning, you probably have more data than you first expected. This book is written from not only an applications perspective but from the expert knowledge of the author, both in development and database administration roles.

A broad-based tutorial on the subject of tuning Oracle Database is much needed. Most database administrators have operating system administration experience and little SQL code or data-modeling experience. On the other hand, developers have the opposite. This book will target both developers and database administrators since it includes all three areas essential to tuning Oracle installations effectively. The important factor is that all tuning skills are required for best performance, both administration and development skills sets.

Being a broad-based tutorial this title is written to reach the widest possible audience, including data modelers, developers, database administrators, and system administrators. Each of these audiences is very specialized, but all are related and interdependent. There are no existing titles including tuning for data models, tuning of SQL code plus physical and configuration tuning, all in one book.

People who would benefit from reading this book would be database administrators, developers, data modelers, systems or network administrators and technical managers. Technical people with these different skills are often ignorant of the skills of each other. This is a great pity because all skill sets are very much dependent on each other for well-constructed databases and applications. Let's take a common example situation. Developers cannot simply hand off SQL code tuning to database administrators when application coding is complete. Database administrators more often than not know absolutely nothing about SQL code tuning. The result is no SQL code tuning is ever done and too much time is spent squeezing out an extra 10% of performance, with the database administrator doing physical and configuration tuning. Targeting a few hard- hitting SQL statements will probably result in much more than a 10% performance improvement, which is much more productive.

What is in this book?

Data Model Tuning

What is the data model?

The data model is the table structure and the relationships between those tables. Tuning the data model for performance involves Normalization and Denormalization. Different approaches are required depending on the type of database installation such as OLTP or data warehouse type databases. Inappropriate database design can make SQL code impossible to tune. If the data model is poor, changing the data model can have the most profound effect on database performance. All SQL code is constructed from the underlying tables. The big problem is that altering the data model after completion of development is expensive since application code may require extensive rework.

Tip

OLTP refers to Online Transaction Processing. OLTP generally implies the Internet. Within the scope of this book OLTP is used to represent both OLTP architectures and perhaps Client-Server architectures as well.
What in the data model causes problems and what is data model tuning?

Data model tuning is most effectively performed by a combination of both database administrators and developers. It is seldom the case that both skill sets are involved. The result is that table structures are often either development centric (top-down) or administration centric (bottom-up) designed. Java development is often top-down and attempts to impose an object structure over a relational framework. Bottom-up design often results in over-Normalization and too much granularity. People with different skills should be working together.

What are the benefits of data model tuning?

Tuning the data model can often provide performance improvements in excess of 100% but it is expensive because application code can be drastically affected.

SQL Code Tuning

What is SQL code?

SQL code is the code directly accessing the database, either embedded in applications or in stored procedures. Sometimes generic SQL code is used, which is SQL code generated by an application on an ad hoc basis. Generic SQL code can cause serious performance issues.

What causes SQL code performance problems and what is SQL code tuning?

As with data modeling it is often confusing which personnel skill sets are responsible for SQL code tuning. This is one of the causes of poorly performing SQL code. Performance is served most effectively when developers and database administrators work together to a certain extent.

Poorly written SQL code is often the biggest culprit of performance problems; it is expensive to rectify but cheaper than changing the data model. SQL code tends to be contained inside independent blocks within applications or stored procedures. This containment is commonly known as embedded SQL code. Tuning SQL code is in general a two-step process.

  • Isolation and recoding of the worst-performing SQL statements, perhaps the slowest-performing 10% of SQL code.

  • General tuning of SQL code involving changes to SQL statements throughout applications and the database, plus adjustments to alternate (secondary) indexing. Alternate indexing is not specifically part of the steps of Normalization and Denormalization but can be designated as data modeling or SQL code tuning. It is important that database administrators have some involvement with respect to alternate indexing, in the very least in an advisory capacity. Too many or inappropriately constructed alternate indexes can completely destroy performance.

What are the benefits of SQL code tuning?

SQL code tuning can increase performance between 25% and 100%, sometimes much more. In rare situations I have seen enormous performance improvements when tuning SQL code. One or two projects I have worked on in the past have been sped up 30 to 500 times, for both individual SQL code statements and sometimes even the applications in general. Additionally SQL code is often embedded in applications in such a way that changing the SQL code does not affect application functionality.

Physical Database and Configuration Tuning

What is physical and configuration tuning?

Physical database tuning involves hardware resource usage, networking, and various other administration tasks such as configuration and file distribution.

What causes physical and configuration performance problems and what is physical and configuration tuning?

Physical configuration is usually a culprit of poor performance where Oracle software is installed with defaults and never altered by an expert. Quite often developers build table structures and SQL code. In this case physical tuning is relied upon to solve performance problems. This is usually a mistake since physical configuration tuning usually only provides at most 10-20% performance improvement.

What are the benefits of physical and configuration tuning?

Physical and configuration tuning usually only results in at most a 25% performance improvement, and usually a lot less. The relative cost of using physical and configuration tuning only is usually not cost effective. Hardware upgrades are common in these situations.

Hardware Upgrades

As a side issue there are always potential hardware upgrades, sometimes a short-term solution because this approach does not necessarily tackle the real problems. Sometimes a combination of hardware upgrades and Oracle installation tuning is the most cost effective option. Hardware upgrades can often be more expensive than tuning. Three months of SQL code tuning by an expert may be much more cost effective than purchasing new machines, RAID arrays and all the "other bits-and-pieces that go with it". Additionally an expert can teach developers and database administrators to build properly tuned databases and applications in the first place.

Sample Databases Used in this Book

A number of sample databases are utilized in this publication. Some are simple and some are highly complex, depending on their use when explaining aspects of tuning. All details and diagrams are included in full in Appendix A.

Please note that this book does not cover operating system tuning or data warehouse tuning, even though they may be mentioned or alluded to in many places. I do not claim to know everything about Oracle Database performance tuning. I do have 15 years of custom applications development and database administration experience. What I do know is shared in this book.

Let's get started.


Previous Section
< Day Day Up >
Next Section

Introduction

Let's begin by taking a general look at what we need to examine in order to tune Oracle installations.

A Tuning Environment

What is a tuning environment? A tuning environment is an environment in which your tuning efforts can be productive.

What is Required when Tuning Oracle Database?

  • Good software tools.

  • Skilled personnel.

  • Staging (testing) environments.

  • A realistic duplication of the production environment.

    • Actual and expected production environments. These can often be different if growth is rapid or requirements change.

    • If possible databases should be of the same size and content. If this is impractical then at least development and testing databases should be proportionate to production.

    • Are the statistics the same? Statistics can be copied or executed using the same time intervals as production.

What Tools are Available?

Excellent software tools for tuning and monitoring Oracle databases are numerous. Oracle Enterprise Manager has many very useful bells and whistles. Spotlight is excellent for visual and informative real- time monitoring of busy production systems. Both Oracle Enterprise Manager and Spotlight are very useful as tuning aids for both physical and SQL code performance analysis.

There are many other tools available. The most important tools in the tuning process are the developers and the administrators. That is why you are reading this book. The best software tools are usually the most expensive but that does not mean that the less expensive tools are useless. In general, the more expensive tools tend to do more for you. However, when something is being done for you automatically and you do not understand the internals, it is unlikely that your tools set can do better than well-trained, experienced database administrators and developers.

Skilled Personnel

Good skills have their correct places. Database administrators tend to have roots as either systems administrators or developers. Each skills set has its pros and cons. Developers tend to know a lot more about coding SQL and building data models. Systems administrators have extensive knowledge of operating systems such as Unix and tend to concentrate tuning efforts on the physical aspects of Oracle Database. Developers tend to concentrate on tuning the data model and building efficiently performing SQL code. Unfortunately this is not always the case because there is sometimes a tendency for developers to place the burden of tuning SQL code and the data model into the area of responsibility of the database administrators. Confusion can be the result and perhaps nothing gets done.

Staging (Testing) Environments

You need as many testing and staging environments as you can get. As the resident DBA you should not be expected to perform database tuning on an active production database which is required to be up and usable 24 × 7 × 365. Tuning on a production database in this situation will limit your scope and could cost you your job! Insist on at least one extra machine and always test anything you do, no matter how trivial. This is the most important difference between production and development. Developers do everything quickly because they have to. Production database administrators are expected to get their tasks done just as fast but additionally everything must be perfect all the time. So make sure you insist on extra hardware and extra time.

Duplicating Production Databases for Effective Tuning

It is absolutely essential to have the most recent and most realistic copy of a production database for tuning purposes. Tuning on a production database is risky and using a development database for tuning can be completely useless. It is extremely rare that development and production databases are alike. Testing databases can be useful when development and production databases cannot be made the same.

Statistics are also important. In a production environment such as an online transactional database the data in that production database could be changing constantly. Even if your database is not changing too much statistics could change or rapidly become out-of-date. The more dynamic the data is the more often statistics should be updated. The SQL code Optimizer utilizes statistics to compile the most efficient methods of executing SQL statements. Statistics are measurements of the data itself, such as how large a table is and how useful an index is. When an SQL statement accesses a table both the table and index states are important. States of database objects such as tables and indexes are contained within statistics. If statistics are out-of-date the Optimizer is not functioning realistically. Out-of-date statistics would have the same effect on all types of databases. It is very important to duplicate statistics from production to tuning environments, either by copying or by executing statistics gathering on a tuning database, consistent with the production database.

Making a copy of a production database to a development database is not an issue when the production database is small. When the production database is large continuous copying to a development database could be very time consuming. Be aware that using the database import utility for even single schema imports on even a small database can take a much longer time than the production database export.

Tip

The DBMS_STATS package can be used to copy statistics between databases.


When to Tune

When should you tune? Constantly and throughout the life cycle of your product, perhaps only when performance is a problem. On the other hand, the database should be configured and organized properly in the first place to avoid continuously putting out fires. If you are always putting out fires the chances are you will make little progress with anything else. It is much better to be proactive and preempt potential problems rather than to react when they occur. It is preferable to tune your database during development rather than after development in production.

Tuning in development will lengthen the life cycle of any system. Why is this the case? If different tasks can be compartmentalized during distinctly separate phases of a project, such that those phases do not overlap each other, a better product will result. For instance, any developer knows that changing code in order to tune SQL code, after completion of development, could change the integrity and neatness of that code. This may not seem too important. However, the more times a piece of code is altered the more it will deteriorate, not only because the code is changing but because different coders may be changing that code. Every coder has a different style and approach and quite often subsequent coding is confused by code written by other people and vice versa. Errors can be introduced into application code when it is changed. The more changes the more potential errors.

We do not live in a perfect world. Distinct lines cannot be drawn between development and production. Changing requirements often cause changes in different project phases making gray areas. There is often overlap between development and production.

It is best to tune in development, particularly data models and SQL code. When tuning after development in a post-development requirements change phase, or in production, tuning should be done carefully, especially when it comes to changing application code. If tuning cannot be done fully in the development cycle, which it probably cannot, take the following approach when tuning in production:

  • Set performance targets.

  • Use test environments.

  • Use test environments matching production as closely as possible.

  • Tune with care.


What to Tune in Production

There are five general stages in tuning an Oracle production database:

  • Resolve obvious bottlenecks.

  • Configuration is often inappropriate due to initial Oracle installation and database creation. Examine basic configuration.

  • Physical space can be wasted. Often a database can be better organized and become much smaller, even as much as one-tenth of current size, and sometimes more.

    Note

    Oracle Database 10 Grid Oracle Database is becoming more automated with respect to physical configuration.

  • Poorly written SQL code in both applications and in the database can only be counteracted partially by physical and configuration tuning. SQL code tuning can help performance immensely but should be performed in development and testing environments first.

  • Data model tuning may be required if SQL code tuning does not resolve production performance problems. As with SQL code tuning, data model tuning should be done in development and testing environments first.

The easiest approach to tuning in production is Oracle Database physical and configuration tuning. In general, tuning the data model is the most expensive because it will require changes to SQL code, application code, and the production database. If SQL code is not tunable then your data model may be too granular or not properly Normalized. Physical and configuration tuning on your database server itself will not require SQL code and data model tuning changes. However, the purely database-oriented tuning approach may eventually lead to expensive hardware upgrades. Hardware upgrades can often be the best tuning option. However, costs can sometimes be more when the hardware becomes so complex that highly skilled, expensive administrators are required. Hardware upgrades are often a short-term solution and their cost effectiveness can be good if the software life cycle is short or money is tight, assuming the hardware is reusable or sellable.


When to Stop Tuning in Production

When do you stop tuning? This is always debatable. You could stop tuning when performance targets are met, depending on what needs to be tuned. The resolution of obvious bottlenecks is a very clear indicator that no more tuning is required. It is often the case that physical tuning, that is, configuration, physical database structure, networking and hardware bottleneck issues can only amount to as little as 10% of total effective tuning activity, both in development and production.

The simple approach to when to stop tuning is to teach your developers to build properly tuned SQL code from the word go and make sure that the data model is sound before you do even that. This is probably impossible but the more SQL code tuning that is done during the development process then the fewer problems you will have later on. Many software projects are discarded because they take too long to develop. However, many other software projects are thrown out or rewritten because they do not meet acceptable performance standards, rendering them useless. Tuning data models and SQL code after completion of development can sometimes simply be too expensive.

When to stop tuning depends on your situation and the skills you have. If the company and your database size and activity grow you will have performance problems anyway, but you can be better prepared.

Let us examine the steps in tuning production databases in order to decide when to stop tuning the different areas.

Bottlenecks

Solving performance bottlenecks is usually reactive rather than proactive. The term bottleneck is technical computer jargon and usually deals with a particular facet of your environment, which is overloaded, be it within or outside your database.

Stop tuning when the bottleneck is removed.

Configuration

If there are immense configuration and physical problems some downtime may be required. Configuration issues are easier to resolve than physical problems and both are easier than tuning data models and SQL code.

Configuration can be as simple as setting parameters correctly in the Oracle Database configuration parameters file and Oracle networking software configuration files. Make absolutely sure configuration parameters are completely understood before changing. Firstly, incorrectly formed configuration can prevent the database from starting and perhaps cause a crash. Secondly, some parameters have very specific functions; incorrect settings can cause a totally different and probably undesired effect.

Stop tuning when configuration parameters are correct. Experimenting with changing configuration parameters on a production database is risky; test, test, test!

Physical Space Usage

Physical space usage and growth tuning for performance includes tuning of datafiles, redo logs, archive logs, and rollback segments.

Note

Oracle Database 10 Grid Manual rollback is deprecated. Use automatic undo.

Resolving physical problems with space usage and growth can cause a lot of downtime but may be the only solution. Small databases have small initial configuration parameter settings. If those small databases suddenly start to get very busy then immense problems can occur. The sooner the physical issues are resolved for a rapidly growing database the better. If the database continues growing the temptation is often to spend enormous amounts of money on hardware. Organizing a database physically can essentially save a lot of disk space.

Note

Oracle Database 10 Grid The current trend in Oracle Database is veering towards automated management of physical space. As a result physical space management to save disk space is becoming less important.

The smaller a database is the less disk space to search through when finding data, thus the faster your data searches will be. Additionally highly fragmented data can cause a lot of "bouncing around" when retrieving data.

Stop tuning when performance is acceptable as long as requirements for uptime are not compromised. Only extreme situations of database growth cause problems with use of physical space.

SQL Code Tuning

Poorly constructed SQL code usually causes most database performance problems. When SQL code is poorly tuned there is little that database administrators can do to improve performance using physical and configuration tuning alone. Database administrators can tune SQL code contained in PL/SQL stored procedures. The ideal approach to SQL code tuning is to teach your developers to tune SQL code as they build applications.

Sometimes developers will build applications rapidly without much consideration to building efficient SQL code. Developers do not really have extra time to make sure the SQL code is as efficient as possible. Most production database administrators tend to have roots as either operating system or network administrators. These skills are essential for production database administration. There's the rub! The administrators sometimes do not know how to tune SQL code and they cannot change the application code since the code belongs to the developers. The developers do not have time or the skills to produce efficient SQL code. Most Oracle Database tuning experts perform physical tuning on the database and the operating system, not SQL code tuning. The result is often only a 10% performance improvement. I have seen SQL code tuned for an application in its entirety and performance increases of 30 to 500 times. That is 500 times faster. One hundred percent is twice as fast. This is an extreme from a consulting job I worked on a few years ago. There were other projects in the past with similar performance issues.

Stop tuning SQL code when development stops if you can. Teach and encourage your developers to write efficient SQL code during development. It is much more difficult and expensive to tune SQL in production, especially when SQL is embedded in applications.

Data Model Tuning

SQL code performance depends largely on the data model, especially if the data model is poorly structured. Beware of copying older, previously "invented" data models or using a data model because it is the accepted standard for a particular application. Relational databases, Normalization, and Denormalization have been in existence for many years. However, the structure of the applications overlaying those relational databases has changed recently and is still changing. What in the past was COBOL and C is now C++, Java, Perl, and even object Perl.

Object-oriented design application development languages such as Java tend to throw sand in the eyes of what used to be accepted relational database design. Object applications tend to impose an unwilling structure on top of a relational database. The result is a data model which is a hybrid between relational and object database methodologies. This hybrid structure can be an advantage for OLTP and client-server transactional applications but a performance hog for any type of reporting or data warehouse applications.

I have seen many OLTP applications with Java object structures imposed onto relational databases. Sometimes this top-down application to data model approach works extremely well, sometimes very poorly. Object methodology promotes breaking things down into their smallest manageable parts. This approach can be a complete disaster for efficiency in a relational database. Imposing an object structure onto a relational database in its purest form is relational database Normalization in its extreme. In these situations 3rd, 4th, and 5th Normal Forms are common. A partial solution is often two databases, one OLTP and the other a data warehouse, doubling costs. The second database is a Denormalized data warehouse database; Denormalizing a database can enhance performance from a reporting perspective.

Tuning the data model is the most difficult and most expensive option because SQL code depends on the structure of the data model; extensive application code changes can result. Tuning the data model is more effective than physical and configuration tuning but can easily escalate into a full rewrite.

If further data model tuning is required after production release you may want to look at data warehouse type options. Tuning the data model for an OLTP database after production release will generally involve a partial or complete rewrite. A data warehouse type approach generally involves duplicating a database and restructuring that duplicate for output performance in terms of processing many rows at once. Quite often transactional databases in OLTP and client- server environments are required to be tuned for small response reaction time to keep your customers happy; very few rows retrieved at a time. If a web page takes more than 7 s to load then your customers may lose interest and go elsewhere, straight to the competition. Data warehouse databases, on the other hand, are designed for rapid throughput of large amounts of information for analytical and reporting purposes.

Stop tuning your data model preferably before development starts or before the first production release. Do not build your relational database to mimic the object structure of a Java application because relational and object methodologies are completely opposed to each other with respect to methodology. Object structures break down for simplicity and relational structures are efficient when summarizing information into groupings. These structural types are completely contrary to each other.

So when should you stop tuning in general? As long as there are problems or if users are not satisfied, never! Tune whenever something changes if there are problems or when you have not reached a performance target. There is no harm in going past a target level of performance. Then again time might be better utilized doing other tasks. You should stop tuning the data model before SQL code development starts. You should stop tuning SQL code when development finishes and the code goes into production.


Tuning from Development to Production

Remember this. Probably most Oracle client installations are very small databases, even for some large companies. For instance, many databases for a lot of the Silicon Valley based dot-com companies of the late nineties were well under 10 Gb, sometimes even as small as being less than a single gigabyte. They expected growth. The point is this. Many databases are small. A lot of Oracle clients are small companies. The result is that Oracle installation software and database creation tools tend to cater to those small companies. Oracle Corporation's strategy in this respect is perfectly sensible since the smaller companies lack the funds for highly skilled staff. The smaller companies need more done for them. Large end user software vendor companies often take this approach. The result is that in the Oracle installation software, most of the configuration parameters and physical settings for files are much too small for any database which experiences a reasonable amount of growth. If a database is over 10 Gb, is highly active or is growing rapidly then configuration created by Oracle installation software and database creation tools is probably inappropriate.

Tuning Oracle Database is not just tuning the database. As we can see from the stages of tuning already discussed tuning the database also includes tuning the data model, SQL code, and thus applications. Tuning Oracle Database is a combination of tuning both the Oracle database server and the applications accessing that database. There is a process of tuning an Oracle Database environment including a large number of separate steps or phases. These phases are repetitive but should be performed as closely as possible to the sequential order as shown in Figure 1.

The Steps in Tuning

Tuning is a set of distinct steps, which can be repeated in any order but are preferably completed in the order shown in Figure 1. The order and existence of these steps could vary depending on the application type, skills resources, time for development, and the capacity of available hardware.

Click To expand
Figure 1: The Steps in Tuning an Oracle Installation

The steps in tuning an Oracle installation should more or less follow the same path as in the development cycle of software development; namely analyze, design, build, test, implement, and verify.

Data Model Tuning

  • A data model is used to represent a set of business rules. Business rules are implemented using entities (tables) and the enforcement of relationships between those entities. Additionally business rules can be implemented using database encapsulated stored procedures plus event or state change triggers.

    Tip

    Using triggers can cause serious performance problems.

  • Business rules can be tuned into a more mathematically correct design using Normalization and Referential Integrity. Referential Integrity ensures that relationships between data items are conformed to.

  • Denormalization is the removal of the more granular results of Normalization. Granularity causes complex mutable multiple table joins. Multiple table joins can be difficult to tune effectively.

  • Alternate or secondary indexing to cater for SQL code not complying directly with a Normalized structure can cause problems. This is common for object applications. This step is part of both the data modeling and the applications coding stages, not either. Alternate indexing is generally enhanced in the applications development stage but should be strictly controlled. Creating too many indexes can cause as many problems as they resolve.

  • Constraints, PL/SQL stored procedures, functions and event or state change triggers should be tuned for performance in accordance with entities, relationships, and alternate indexing structures. Triggers are best avoided. Database level coded PL/SQL will perform better in some situations than others. PL/SQL should only cover business rule functionality and database access code, not applications functionality. Business rules sometimes match relational database structure where applications functionality often does not.

  • Implementing Referential Integrity.

    • Should Referential Integrity be implemented? Not necessarily. Referential Integrity can ensure the correctness of data but it will slow down data changes somewhat due to verification.

    • How should Referential Integrity be implemented? It can be implemented using constraints or triggers. Constraints are the faster and much more reliable method. All sorts of things can go wrong with triggers and the performance of triggers is highly questionable.

    • Can Referential Integrity be partially implemented? Yes it can. Very small tables containing static, referential data can often have their foreign keys removed. Additionally noncritical tables or highly used tables can avoid Referential Integrity to help performance. Generic static tables, when used, probably should avoid Referential Integrity as well. An example of a generic static table is shown in Figure 2. These types of tables, in the interests of performance at the database level, are best avoided.

      Click To expand
      Figure 2: Using Static Generic Entities

    • Where should Referential Integrity be implemented? Referential Integrity can be enforced in the database or at the application level. The benefit of implementing Referential Integrity at the database level is simple implementation in one place and one place to change in the future. Developers may not necessarily agree with this philosophy but database administrators generally would.

SQL Code Tuning

SQL coding requirements should fit the specifications of the data model based on entities, relationships, and alternate indexing. SQL code can be in both database based PL/SQL coding and applications embedded SQL code. What are the steps in tuning SQL code?

  • Identify the worst-performing SQL code and tune only those SQL statements.

  • When the worst-performing code is tuned SQL code can be tuned in general if there is still a problem.

  • Create, remove, and tune alternate indexing to suit SQL code performance requirements without jeopardizing the data model. When matching SQL code to indexing it is best to attempt to map SQL code filtering, sorting and joining to primary and foreign key index structures (Referential Integrity). Using already-existing Referential Integrity keys will reduce the number of alternate indexes. More indexes on tables will speed up data selection but will slow down data updates, sometimes drastically. The fact is if SQL code is not utilizing Referential Integrity indexing there may be a mismatch between the data model and application requirements or the data model is simply inappropriate.

  • How are individual SQL code statements tuned in Oracle Database?

    • Often the most effective method is to examine the SQL code by hand and make sure that the SQL statements conform to indexing in the database, namely in selecting, filtering, sorting, and grouping clauses. The more complex SQL code is, the more difficult it will be to tune in this manner. In fact complex SQL code is sometimes beyond the capabilities of the Optimizer.

    • Use the Oracle Database EXPLAIN PLAN command to examine the Optimizer's best execution path for SQL code. EXPLAIN PLAN will show where potential improvement can be made. A simple query plan is shown in Figure 3.

      Click To expand
      Figure 3: A Simple Query Plan from EXPLAIN PLAN

    • Trace files and TKPROF can be used to tune SQL code but tracing produces excessive amounts of information. Tracing should be a last resort method of tuning SQL code.

    • Make sure coders use bind variables in both PL/SQL and applications embedded SQL code. Bind variables are not as significant after Oracle8i Database Release 1 (8.1.6) because of the cursor sharing configuration parameter but forcing cursor sharing lowers statistical accuracy and deteriorates Optimizer performance.

    • Beware of generic or generated SQL code, common at the applications level. This type of embedded SQL code can be very difficult to tune.

Configuration and Physical Tuning

Configuration Tuning

Possibly one of the most effective tuning practices, especially as far as configuration tuning is concerned, is proper initial Oracle installation and configuration of your databases. Do not leave your Oracle installation as Oracle installation software and database creation tools create it. It might be expedient to use the Oracle Database creation tool for a first experimental database creation only.

Tip

Oracle Database 10 Grid The database creation tool (Database Configuration Assistant) has become more sophisticated over time. However, most database administrators still prefer to use scripting since it can be modified at will and executed in parts. This approach is no longer necessary except for highly complex installations.

The database creation tool creates a fairly well-organized physical and configured structure but its default settings are geared toward very small databases. Take time for properly organizing the database and its configuration initially and then you will be less likely to have large amounts of downtime later on.

Physical Tuning

Physical tuning involves the removal of competition for resources. Physical tuning covers the following areas:

  • Physical space usage and proper storage structure usage in terms of how blocks are used and reused. Tuning at the block level depends on the application type.

  • Growth management of the database and capacity planning must be monitored.

  • Setting files such as log files and rollback segments to sizes and frequencies as appropriate to database usage.

    Note

    Oracle Database 10 Grid Manual rollback is deprecated. Use Automated undo.

  • There can be contention between processes, memory usage, and data. Some of this type of tuning falls within the scope of both configuration and physical tuning.

  • I/O contention can be dealt with in the file system by spreading the I/O load using multiple disks or RAID arrays. RAID arrays are most efficiently used using random access on data spaces and sequential access on index spaces. In non-RAID systems it is sensible to separate data and index spaces since both are read more or less concurrently. Oracle Partitioning is a classic example of this type of tuning.


How is this Book Organized?

This book is divided into three separate parts. The three parts cover relational data model tuning in Part I, SQL code tuning in Part II, and finally physical and configuration tuning in Part III.

The object in this book is to cover all aspects of tuning and Oracle installation. Each chapter focuses on a very particular aspect of tuning Oracle Database. As you read through each part, each chapter will build on the previous chapter, digging deeper step by step.

Part I. Data Model Tuning

Chapters 1 and 2 introduce and offer tuning solutions for relational data modeling. Chapter 3 looks at alternative data modeling methodologies and Chapter 4 provides some historical background.

Chapter 1. The Relational Database Model

Introduces Normalization and Referential Integrity.

Chapter 2. Tuning the Relational Database Model

Tuning using efficient Normalization, Referential Integrity indexing, alternate indexing, Denormalization, and some useful tricks.

Chapter 3. Different Forms of the Relational Database Model

This chapter provides a contrast of relational, object, and object- relational concepts.

Chapter 4. A Brief History of Data Modeling

This is a brief history of relational data modeling and the roots of SQL.

Part II. SQL Code Tuning

The format in Part II is to first describe and then offer solutions and tools to help find those solutions. Chapters 5 and 6 present SQL and describe how to tune SQL code without having to dig into the depths of the Oracle Optimizer. Chapter 7 looks at indexing. Chapter 8 digs into the Oracle Optimizer. Chapters 9 and 10 present tools used to help find SQL code performance problems. Solving as opposed to finding SQL code problems is found in Chapters 6, 7, and 8.

Chapter 5. What is SQL?

A brief description of SQL and the basics that Oracle SQL can accomplish. Everything is explained in detail and using practical examples.

Chapter 6. The Basics of Efficient SQL

This chapter is the most important chapter in Part II SQL Code Tuning. This chapter will teach step by step how to write properly tuned SQL code. Robust realistic data sets and Oracle Database schemas are used to prove all aspects of tuning. Some tables in schemas are in excess of 1 million rows. There is plenty of data to tune with.

Tip

The number of rows in tables varies as the book progresses.

Chapter 7. Common Sense Indexing

Teaches how to build indexes properly, building on tuning of SQL code taught in Chapter 6. Indexing and index types are analyzed, explained, and compared for speed in relation to various database activities.

Chapter 8. Making SQL Efficient in Oracle Database

Covers very deep level tuning of SQL code in Oracle Database with emphasis on using the Optimizer, without wasting time on too many nitty-gritty details. This chapter describes how the Oracle SQL code Optimizer functions and what can be done to influence the Optimizer. If you have read, understood, and implemented the advice and pointers in Chapters 6 and 7, you may never need to understand the Optimizer on this level.

Tip

Chapter 6 should be read prior to reading Chapter 8.

Chapter 9. How to Find Problem Queries

Describes the INS and OUTS of Optimizer query plans using EXPLAIN PLAN, Tracing, and TKPROF. Additional details covering Oracle V$ performance views for SQL code tuning are also provided.

Chapter 10. Tuning SQL with Oracle Enterprise Manager

A brief picture of SQL code tuning tools contained within Oracle Enterprise Manager.

Part III. Physical and Configuration Tuning

As in Part II, the format in Part III is to first describe and then offer solutions and tools to help find those solutions. Chapters 11 to 17 describe how to build cleanly structured Oracle installations. Chapters 18 to 20 present symptoms and indicators of problems. Chapter 21 present tools which can be used to solve problems and Chapter 22 offers up bottleneck tuning methods using the Oracle Wait Event Interface and STATSPACK.

Chapter 11. Installing Oracle and Creating a Database

Shows how to create a cleanly structured Oracle installation and database.

Chapter 12. Tuning Oracle Database File Structures

Tuning Oracle Database File Structures encompasses both the physical and logical layers of tuning from the Oracle Instance and Oracle Database file system layers through to logical layers and tablespaces.

Chapter 13. Object Tuning

Object Tuning covers database objects such as tables, indexes, sequences, and synonyms. There are numerous tweaks to be made to various database objects to tune for performance.

Chapter 14. Low-Level Physical Tuning

This chapter covers physical block and extent structures, and how to build physical structure from the ground up.

Chapter 15. Hardware Resource Usage Tuning

Hardware resource usage tuning is introduced, covering tuning CPU usage, Oracle Database memory cache buffers, and finally tuning of I/O activity.

Chapter 16. Tuning Network Usage

This chapter covers the Listener, network naming methods, and shared server configuration. There are numerous ways in which Oracle network connectivity can be tuned for better performance, depending on requirements.

Chapter 17. Oracle Partitioning and Parallelism

Oracle Partitioning allows for breaking up of large tables into separate objects, which can have SQL code executed on them in parallel for excellent performance improvements.

Chapter 18. Ratios: Possible Symptoms of Problems

Traditionally Oracle Database practitioners use large numbers of ratios to assess and provide pointers as to what and where to tune. The secret is to not tune the ratios themselves. Use the ratios as indicators of and pointers to potential problems.

Chapter 19. Wait Events

A wait event occurs when a process or session is forced to wait for another to complete using something that both processes require.

Chapter 20. Latches

Latches are used to protect Oracle Database memory cache buffers from simultaneous access.

Chapter 21. Tools and Utilities

There are many tools usable for physical and configuration tuning including Oracle Enterprise Manager and STATSPACK. Additionally there are numerous non-Oracle vendor tools such as Spotlight. Various tuning aspects with utilities are briefly mentioned.

Chapter 22. Tuning with the Wait Event Interface and STATSPACK

This "interface" is a group of inter-related statistical performance views, which can be used to isolate bottlenecks. Oracle Enterprise Manager may be the best tool to use when drilling down into the Oracle Database Wait Event Interface, when trying to isolate the source of performance problems. STATSPACK is currently the official Oracle Corporation recommended tuning and performance monitoring tool. However, isolating performance problems by drilling down into the Oracle Database Wait Event Interface is most easily done using the Oracle Enterprise Manager.

Appendices

The appendices contain sample database entity relationship diagrams, scripting examples, and some reference material. Please note that one particular database, the Accounts schema, had active processing executing on it throughout the writing of this book. As a result the quantities of data in the tables are not constant. If row numbers in tables vary between different chapters and sometimes within chapters then it is nothing to be concerned about.


Some Final Points

Oracle software is highly complex, fully featured, fully loaded, and an excellent piece of software. There are a very large number of options and different ways of doing things in Oracle Database software. The result is that the right way to do something in Oracle Database can sometimes be a matter of different opinions between different database administrators, designers, and developers. Since Oracle Database is so very large and versatile many aspects of Oracle Database can sometimes be subject to those opinions.

Perhaps one of the most important things to note about tuning Oracle installations is that there is no hard and fast set of rules for tuning. On the other hand, it is sometimes best to plan a set number of steps and follow those guidelines when tuning in order to avoid a random approach to tuning. Guesswork should never be the order of the day. When tuning Oracle installations focus on critical problem areas. How can critical areas be found? Talk to the users. Talk to developers. Talk to everyone. Find out what problems exist before attempting to solve anything.

Numerous elements discussed in this book have been desupported from Oracle software or deprecated or both. Desupported is a term Oracle Corporation uses for an option soon to be removed. Deprecated is a term used to describe something no longer available in Oracle software. Desupported and deprecated elements have been retained in this book for two reasons. Firstly, multiple versions of Oracle Database are installed out in industry at present. Secondly, it is sometimes necessary to include details of older less favored options within Oracle software in order to facilitate explaining differences between old and new facets of the database.


What is Oracle Database 10g?

Oracle8 Database and before were just Oracle Database. Oracle8i Database and Oracle9i Database were Oracle Internet Database. Oracle Database 10g is Oracle Database 10 Grid. Oracle Database 10g is designed for use on computer grids, incorporating all sorts of different types of applications, including integrated messaging, enhanced scalability and performance plus automated storage management.

This book can be read in part, or in whole by reading each of the separate parts. However, it is not a reference manual so it is best read from start to finish because there is a lot of crossover between the three different disciplines. To start learning from a development perspective and pass through to a production perspective later on, I would suggest starting with data model tuning in Part I. Let's get started with relational data model tuning.


Part I: Data Model Tuning

Chapter List

Chapter 1: The Relational Database Model
Chapter 2: Tuning the Relational Database Model
Chapter 3: Different Forms of the Relational Database Model
Chapter 4: A Brief History of Data Modeling


Chapter 1: The Relational Database Model

Overview

Describing how to tune a relational database model would not be complete without a description of Normalization.

I have attempted to simplify Normalization. When I was at University I simply ignored the accepted, stated rules of Normalization and did as I had been doing for the 3 years before. I worked with a relational database data dictionary based application. I found the rules of the five Normal Forms to be obtuse, over complicated, and simply impossible to decipher at first glance. The response of a college student first reading the descriptions of each Normal Form is usually, "Huh"? I probably read those rules two or three times and wondered what all the fuss was about.

So let's try to simplify Normalization.


1.1 A Layman's Approach to Normalization

Application of the relational database model to a data set involves the removal of duplication. Removal of duplication is performed using a process called Normalization. Normalization comprises a set of rules called Normal Forms. Normalization is applied to a set of data in a database to form tables or entities. Tables are for placing directly associated data into. Tables can be related or linked to each other through the use of key or index identifiers. A key or index identifier identifies a row of data in a table much like an index is used in a book. An index in a book is used to locate an item of interest without having to read the whole title.

There are five levels or layers of Normalization called 1st, 2nd, 3rd, 4th, and 5th Normal Forms. Each Normal Form is a refinement of the previous Normal Form. 4th and 5th Normal Forms are rarely applied. In designing tables for performance it is common practice to ignore the steps of Normalization and jump directly to 2nd Normal Form. 3rd Normal Form is often not applied either, unless many-to-many joins cause an absolute need for unique values at the application level.

Tip

Over-Normalization using 4th and 5th Normal Forms can lead to poor performance in both OLTP and data warehouse type databases. Over-Normalization is common in top-down designed Java object applications. In this situation an object structure is imposed onto a relational database. Object and relational data structures are completely different methodologies.

That is far too much jargon. Let us make the understanding of Normalization very simple. "Forget about it!" Normalization is for academics and in its strictest form is generally impractical due to its adverse effect on performance in a commercial environment, especially 3rd, 4th, and 5th Normal Forms. The simplest way to describe what Normalization attempts to achieve can be explained in three ways:

  1. Dividing the whole into smaller more manageable parts.

  2. Removal of duplicated data into related subsets.

  3. Linking of two indirectly related tables by the creation of a new table. The new table contains indexes (keys) from the two indirectly related tables. This is commonly known as a many-to-many join.

These three points are meaningless without further explanation of Normalization. So let us go through the rules and try to explain it in a non-academic fashion. Let us start with some relational database buzzwords.

  • A table contains many repetitions of the same row. A table defines the structure for a row. An example of a table is a list of customer names and addresses.

  • A row is a line of data. Many rows make up the data in a table. An example of a row is a single customer name and address within a table of many customers. A row is also known as a record or a tuple.

  • The structure of a row in a table is divided up into columns. Each column contains a single item of data such as a name or address. A column can also be called a field or attribute.

  • Referential Integrity is a process of validation between related tables where references between different tables are checked against each other. A primary key is placed on a parent or superset table as the primary identifier or key to each row in the table. The primary key will always point to a single row only and it is unique within the table. A foreign key is a copy of a primary key value in a subset or child table. An example of a function of Referential Integrity is that it will not allow the deletion of a primary key table row where a foreign key value exists in a child table. Primary keys are often referred to as PK and foreign keys as FK. Note that both primary and foreign keys can consist of more than one column. A key consisting of more than one column is known as a composite key.

  • An index is used to gain fast access to a table. A key is a special form of an index used to enforce Referential Integrity relationships between tables. An index allows direct access to rows by duplicating a small part of each row to an additional (index) file. An index is a copy of the contents of a small number of columns in a table, occupying less physical space and therefore faster to search through than a table. The most efficient indexes are made up of single columns containing integers.

    Tip

    Primary and foreign keys are special types of indexes, applying Referential Integrity. Oracle Database automatically indexes primary keys but not foreign keys.

1.1.1 1st Normal Form

1st Normal Form removes repetition by creating one-to-many relationships. Data repeated many times in one table is removed to a subset table. The subset table becomes the container for the removed repeating data. Each row in the subset table will contain a single reference to each row in the original table. The original table will then contain only nonduplicated data.

In the example in Figure 1.1 a 1st Normal Form transformation is shown. The sales order table on the left contains customer details, sales order details, and descriptions of multiple items on the sales order. Application of 1st Normal Form removes the multiple items from the sales order table by creating a one-to-many relationship between the sales order and the sales order item tables. This has three benefits:

  1. Saves space.

  2. Reduces complexity.

  3. Ensures that every sales order item will belong to a sales order.

Click To expand
Figure 1.1: 1st Normal Form

In Figure 1.1 the crow's-foot pointing to the sales order item table indicates that for a sales order to exist, the sales order has to have at least one sales order item. The line across the pointer to the sales order table signifies that at least one sales order is required in this relationship. The crow's-foot is used to denote an inter-entity relationship.

Tip

Inter-entity relationships can be zero, one or many TO zero, one or many.
Click To expand
Figure 1.2: 1st Normal Form Rows for Figure 1.1

The relationship shown in Figure 1.1 between the sales order and sales order item table is that of one to one-or-many.

Example rows for the 1st Normal Form structure in Figure 1.1 are shown in Figure 1.2. Notice how the master and detail rows are now separated.

1.1.2 2nd Normal Form

2nd Normal Form creates not one-to-many relationships but many-toone relationships, effectively separating static from dynamic information. Static information is potentially repeatable. This repeatable static information is moved into separate tables. In Figure 1.3 the customer information is removed from the sales order table.

Click To expand
Figure 1.3: 2nd Normal Form

Customer information can be duplicated for many sales orders or have no sales orders; thus the one-and-only-one to zero-one-or-many relationship between customers and sales orders.

Example rows for the 2nd Normal Form structure in Figure 1.3 are shown in Figure 1.4. Now we have separation of master and detail rows and a single entry for our customer name; there is no duplication of information. On creation of the Customer entity one would create a primary key on the CUSTOMER_NUMBER column, as shown on the right side of the diagram in Figure 1.3. Figure 1.4 does not show a CUSTOMER_NUMBER but merely a customer name for explanatory purposes. In Figure 1.4 a primary key would be created on the name of the Customer table.

Click To expand
Figure 1.4: 2nd Normal Form Rows for Figure 1.3

1.1.3 3rd Normal Form

3rd Normal Form is used to resolve many-to-many relationships into unique values. In Figure 1.5 a student can be enrolled in many courses, and a course can have many students enrolled. It is impossible to find a unique course-student item without joining every student with every course. Therefore, each unique item can be found with the combination of values. Thus the CourseStudent entity in Figure 1.5 is a many-to-many join resolution entity. In a commercial environment it is very unlikely that an application will ever need to find this unique item, especially not a modern-day Java object web application where the tendency is to drill down through list collections rather than display individual items. Many-to-many join resolutions should only be created when they are specifically required by the application. It can sometimes be better to resolve these joins in the application to improve database performance, and not create new entities at all.

Click To expand
Figure 1.5: 3rd Normal Form
Tip

Be very careful using 3rd Normal Form and beyond.

Example rows for the 3rd Normal Form structure in Figure 1.5 are shown in Figure 1.6. Notice how the containment of both students within courses and courses within students is provided by the application of 3rd Normal Form. The question you should ask yourself when using 3rd Normal Forms is this: Does your application need both these one-to-many relationships? If not then do not create the new entity because more entities lead to more complex joins and thus slower SQL statements. Theoretically application of 3rd Normal Form under these circumstances is correct. However, in a commercial application you will not necessarily need to access the information in both orders.

Click To expand
Figure 1.6: 3rd Normal Form Rows for Figure 1.5

Now let's understand this a little further. Take a look at Figure 1.7. That many-to-many relationship we had between the two tables on the left in Figure 1.6 has disappeared. That is because the two entities on the left in Figure 1.6 are the same entity; they contain the same data, only in a different order. The courses and students with the contained intra-relationships can be retrieved from a single table simply by applying a different sort order. Figure 1.7 shows the entities and the number of rows increasing with the application of 3rd Normal Form. So not only will we have more complex joins but we will also have more rows and thus more physical space usage, also leading to slower SQL statements.

Click To expand
Figure 1.7: A More Concise Form of the Rows in Figure 1.6

1.1.4 4th Normal Form

4th Normal Form is intended to separate duplications independent of each other, and those values can be nulls; thus removing repeating, potentially null values into separate entities. In Figure 1.8 employee skill and certification collections are removed into separate entities. An employee could have skills or certifications, or both. Thus there is no connection between the attributes of the Employees table, other than the employee number, and the details of skills or certifications for each employee.

Click To expand
Figure 1.8: 4th Normal Form

Example rows for the 4th Normal Form structure in Figure 1.8 are shown in Figure 1.9. The rows are divided from one into three separate entities. Since skills and certifications are duplicated in the 4th Normal Form application there is further Normalization possible but it is not advisable. Once again creation of too many entities creates a propensity for complex and slow SQL statements.

Click To expand
Figure 1.9: 4th Normal Form Rows for Figure 1.8
Tip

Oracle Database will allow creation of object collections within tables. The Skills and Certifications attributes shown on the left of Figure 1.8 are candidates for a contained object collection structure, namely a TABLE or VARRAY object. However, since object and relational structures do not always mix well this is not necessarily the most efficient storage method for later fast SQL access, even if it is an elegant solution.

1.1.5 5th Normal Form

5th Normal Form divides related columns into separate tables based on those relationships. In Figure 1.10 product, manager, and employee are all related to each other. Thus, three separate entities can be created to explicitly define those inter-relationships. The result is information that can be reconstructed from smaller parts. An additional purpose of 5th Normal Form is to remove redundancy or duplication not covered by the application of 1st to 4th Normal Forms of Normalization.

Click To expand
Figure 1.10: 5th Normal Form
Tip

All the columns in the 5th Normal Forms entities on the right of the diagram in Figure 1.10 are composed of composite primary keys.

Example rows for the 5th Normal Form structure in Figure 1.10 are shown in Figure 1.11. Note how there are two one-to-many relationships on the right side of Figure 1.11; this is caused by the nature of the data. In Figure 1.11 it should be clear that entities divide up data in an elegant and logical way. Once again this level of Normalization will be detrimental to SQL performance due to more complex SQL statement joins and more physical storage space used.

Click To expand
Figure 1.11: 5th Normal Form Rows for Figure 1.10

1.1.6 Let's Recap on Normalization

  • 1st Normal Form removes repetition by creating one-to-many relationships.

  • 2nd Normal Form creates not one-to-many relationships but many-to-one relationships, effectively separating static from dynamic information. 2nd Normal Form removes items from tables independent of the primary key.

  • 3rd Normal Form is used to resolve many-to-many relationships into unique values. 3rd Normal Form allows for uniqueness of information by creation of additional many-to-many join resolution tables. These tables are rarely required in modern-day applications.

  • 4th Normal Form is intended to separate duplications independent of each other, thus removing repeating, potentially null values into separate entities. 5th Normal Form divides related columns into separate tables based on those relationships. 4th and 5th Normal Forms will minimize nulls and composite primary keys by removing null capable fields and subset composite primary key dependent fields to new tables. 4th and 5th Normal Forms are rarely useful.


1.2 Referential Integrity

We know that Referential Integrity is a mechanism used to validate data between primary and foreign key columns in related tables. The primary key is placed on the parent table and the foreign key is placed on a child table. The primary and foreign key values must be the same. Referential Integrity is a set of rules that a relational database uses to ensure that data does not go missing.

What is all this in English? That is very simple to explain. What happens if we have a customer database with associated, or "related", invoices. Now let's assume for an instant that we have no Referential Integrity and a customer is deleted from the database. Well, since we cannot physically get inside the bits and electronics of our database the invoices are essentially, electronically lost! Why? Because it is highly likely that our applications will only ever find invoices by looking at the customers first. Even if this is not the case the invoices will only have a reference to the customer and we won't know whom to mail the invoices to and thus whom to bill. Not billing is likely to persuade the customer not to pay, not on time at least: certainly a sorry state of affairs.

That is what Referential Integrity is and why we should use it.

There is a small but distinct difference between Normalization and Referential Integrity. Normalization is a process of building entities to represent organized data and joining them together with relationships. Referential Integrity is the process of enforcing entity relationships using constraints or rules. Oracle Database rules are called triggers. Triggers are "fired" based on the occurrence of an event.

The next chapter examines how to tune with the relational database model.


Chapter 2: Tuning the Relational Database Model

So how do we go about tuning a relational database model?

2.1 Normalization and Tuning

What does Normalization have to do with tuning? There are a few simple guidelines to follow and things to watch out for:

  • Too little Normalization leads to too much duplication of data and thus your database will get too big. Too much data will lead to slow access times due to having too much disk space to search through.

  • Incorrect Normalization should be obvious: convoluted and complex application code with nowhere to go but a rewrite or the garbage pile, the final resting place of many failed commercial applications.

  • Too much Normalization leads to overcomplex SQL code which can be difficult, if not impossible, to tune. I tend to pass by 4th and 5th Normal Forms and think very carefully about the usefulness of 3rd Normal Form derived entities.

  • Quite often databases are designed without knowledge of applications. The data model could be built on a purely theoretical basis. Later in the development cycle, the application not only has difficulty using a highly granular data model but the data may be structured totally differently to that of the application. One possible answer is that both development and administration people should be involved in data modeling. Another possibility is that the data model supports the application and thus should be built with all knowledge in hand. This is of course impossible. Thus, it should be acceptable to alter the data model at least during the development process, possibly substantially. Most of the problems with relational database model tuning are Normalization related. Normalization should be simple because it is simple!

    Tip

    This is why I attempt to simplify Normalization. Many data models are designed and built from a theoretical perspective. Theory often does not work perfectly commercially. Normalization is based on simple set theory. Normalization in its most basic form is akin to those little circles and intersections and things you used to draw and color at school (in America that would probably be in junior high).

  • Watch out for excessive use of outer joins in SQL code. If you have a lot of outer joins it could mean that your data model is too granular; you could have overused 4th and 5th Normal Forms. 4th and 5th Normal Forms are rarely needed in the name of efficiency. An outer join is akin to: "Go and get this. Oh! Go and get that too because this doesn't quite cover it."

The other side of Normalization is of course Denormalization, which we will get to later. Let's start with taking a look at Referential Integrity.


2.2 Referential Integrity and Tuning

How Referential Integrity and tuning are related is twofold. Firstly, decide whether to use or implement Referential Integrity at all and secondly, if implementing, how?

2.2.1 Using Referential Integrity or Not

Should Referential Integrity be implemented? If so, how? The question should be, "Should you implement Referential Integrity at all"? You should but for the sake of performance sometimes it is not needed. For the sake of maintaining the integrity or correctness of your data you should implement Referential Integrity. It is always advisable to implement Referential Integrity partially, at least on critical relationships. For instance, some static entities do not necessarily need to have Referential Integrity checks even if it is advisable. Continual Referential Integrity checking will affect performance.

Use Referential Integrity where it is needed. When data is changing constantly we continually need to check that the data changes are valid. This is where Referential Integrity comes into play. Consider for instance a data warehouse. Typically data warehouse databases or reporting databases are used to report data or store large amounts of data and hence require throughput. Throughput implies large amounts or processing and database access at once. Reporting databases report, they read data. If they are updated they are often mass updated in batches that can be recovered and executed again. Is Referential Integrity required in data warehouse databases with no OLTP or client-server like updating activity? That is often a very resounding "No!" Referential Integrity is required to validate changes to the database.

If generic static entities are used Referential Integrity is inadvisable on these entities. A generic static entity is a table including multiple types of static data items, which are unrelated to each other in any way other than structurally. Repeating the diagram from the Introduction, as shown in Figure 2.1, we can see an example of a generic static entity. Please be aware that the creation of generic static entities is not anything to do with Normalization. Generic static entities allow for more modular application coding and not Referential Integrity. Efficient data modeling and Normalization does not always benefit when created from the perspective of creating better application code. If you have generic static entities get rid of them if possible!

Click To expand
Figure 2.1: Static Generic Entities
Tip

Generic static entities are often created to allow for ease of programming. Structuring the data model from the perspective of creating generic application code does not necessarily lend itself to an efficient data model. Keep generic code in the application and out of the database if possible. If some aspect of a data model makes application coding easier it is probably detrimental to database performance.

If generic static entities must be used it might be best to enforce Referential Integrity in the application and not in the database since those generic aspects stem from the application.

2.2.2 How to Implement Referential Integrity

Referential Integrity can be implemented at the database level or in the application. The chances are that coding referential data checks in the application will produce messier code, which is less granular or modular than is desirable. This is because application code will be performing tasks probably meaningless to application functionality. The functionality of an application can sometimes have little to do with Normalization of data. Including Referential Integrity in application code might perform better than at the database level since processing is distributed. However, distribution does not necessarily mean better response time for the customer since client or middletier machines will be performing extra processing.

At the database level Referential Integrity can best be implemented using primary and foreign key constraints. Using triggers should be avoided like the plague! Triggers are notoriously slow. Oracle Database triggers usually perform at speeds up to at least 10 times slower than using constraints for Referential Integrity. Triggers are written in interpretive PL/SQL code, which is slow to execute. Additionally triggers do not allow COMMIT or ROLLBACK commands because they are event driven and can drive other events; they can as a result produce large transactions. It is commonplace that Referential Integrity is implemented using triggers when an organization has a lack of data modeling skills and programming skills are abundant. Obviously the reverse will be true when programming skills are lacking, but problems in other areas could arise.

Changes later on, such as disabling some referential checks, will be much easier to find and change in the database when implemented with constraints. Sifting through a lot of trigger code or application code in order to change Referential Integrity constraints could be very time consuming. Also any technical manager knows that each programmer has an individual style and approach to writing code. Since personnel in the computer industry are notorious for job-hopping, new coders working on someone else's code can be very problematic indeed, especially if the original coder was very messy. Building Referential Integrity with constraints is much more manageable. Skills required are an understanding of Normalization plus simple constraint creation using DDL administration commands or Oracle Enterprise Manager. These skills are easily learned with a little practice.

To summarize, changing application code will always introduce new bugs, some completely unexpected! Trigger code is less likely to cause problems than application code in this respect because it is centralized and in one place in the database, not in multiple applications. Changing constraints involves simple changes to the Oracle Database metadata. What could be easier? In short, the more Referential Integrity is implemented without using constraints the more problems you will have. Changes to the data model, for instance, will probably not be possible if Referential Integrity is implemented at the application level. On the other hand, changing simple database-level constraints will not effect the application in any way whatsoever. The only possible effect on the application is if the application is using the tables in a way which is violating Referential Integrity. This of course is positive since data integrity errors will be detectable and can be corrected, probably in development and testing.

Tip

Every form of implementation depends on the current situation and circumstances, both from a technical and business perspective. In other words, everything is relative to the resources available: time plus personnel and financial resources. If changes cannot be made easily to poorly built software then try resolving potentially business-threatening problems in little steps. You do not have to do everything at once.

So what's the "skinny" on implementing Referential Integrity?

  • Implement Referential Integrity at the database level using primary and foreign key constraints. This is usually the best way both for the short term and the long term.

  • Avoid placing Referential Integrity code into the application because later changes for the purposes of helping performance could be problematic, both for tuning the data model and SQL code. The result is likely to be application bugs and little potential for tuning. Coding Referential Integrity into application code will not only over-code but introduce more complexity into application code and will ultimately hurt speed and maintainability.

  • Remove or disable any Referential Integrity constraints from the database at a later stage as required and if necessary. There is much which can be done to keys, indexes, tables, and all sorts of other things in the database itself, easily and in one place.

Using Constraints (Primary and Foreign Keys)

Now we want to examine using constraints to implement Referential Integrity. Referential Integrity establishes relationships between entities. In the case of the relational database model those relationships are most commonly one-to-many in nature. The result is what could be perceived as a hierarchical structure where an entity could contain links to collections of other entities, such as that shown in Figure 2.2.

Click To expand
Figure 2.2: A Hierarchy of Relational Entities

In Figure 2.2 note that many divisions are in a company, many departments are in each division, and each department has many employees. Each successive child entity, passing from companies down through to employees, inherits the primary key of its parent entity. Thus the Company table has a primary key called Company, the Division table has a primary key made up of a concatenation of both the company name and the division name, and so on.

The primary and foreign keys, denoted in Figure 2.2 as PK and FK, are what apply Referential Integrity. Thus in order for a division to exist it must be part of a company. Additionally, a company cannot be removed if it has an existing division. What Referential Integrity does is to verify that these conditions exist whenever changes are attempted in these tables. If a violation occurs an error will be returned. It is also possible to cascade or pass changes down through the hierarchy.

Tuning the structure of the keys will be covered in a later chapter. However, it will suffice at this stage to state that the primary key structures represented in Figure 2.2 are not as efficient as they could be. How would we make these primary keys more efficient?

Efficient Keys

In a purist's or traditional relational data model keys are created on actual values such as those shown in Figure 2.2. The primary key for the Company table is created on the name of the company, a variable length string. Always try to create keys on integer values. Integer values make for more efficient keys than alphanumeric values since both range and exact searches through numbers is mathematically much easier than with alphanumeric values: there are 10 possible digits to compare and many more alphanumeric characters. Sorting and particularly hashing is much more efficient with numbers.

Always try to avoid creating keys on large fixed or variable length strings such as Oracle Database VARCHAR2 data types. Dates can also cause problems due to implicit conversion requirements, and differences between dates and timestamps. Creating keys on anything other than integers or short, fixed length string values is generally asking for trouble. Numbers require less storage and thus shorter byte lengths. Oracle Database NUMBER field values occupy a maximum number of bytes based on the number of digits in a value. The only negative aspect is that as the numbers grow, predicting space occupied becomes more difficult. Sometimes it may be useful to create keys using short fixed length character strings. An Oracle Database CHAR(3) datatype will use exactly 3 bytes, a known and predictable value. However, short fixed length character strings were often used in the past as substitute coded values for large variable length strings. This is rare in modern relational data models due to the object nature of application development tools such as Java.

Some applications create primary and foreign key structures using alphanumeric object pointer representations. This is generally not efficient because the representations of these pointers will require variable length strings using Oracle Database VARCHAR2 datatypes. This approach should be avoided in a relational data model and is much more appropriate to an object data model.

A possibly more efficient key structure for the data model in Figure 2.2 would be as shown in Figure 2.3 where all variable length strings are relegated as details of the entity by the introduction of integer primary keys.

Click To expand
Figure 2.3: A Hierarchy of Relational Entities Using Integer Keys

Another variation on the data models in both Figures 2.2 and 2.3 would be the data model shown in Figure 2.4 where the relationships become what is known as nonidentifying. In Figure 2.4 each entity contains its own unique integer identifier as a primary key and all parent unique integer identifiers are removed from the primary key, becoming foreign keys where applicable. Foreign keys not directly related to immediate parent entities are removed altogether. Thus entities in Figure 2.4 allow unique identification of every row within each table based on the primary key. This is a little closer to the object model and is more consistent with modern object application development environments such as Java. Consistency is always beneficial. However, the Employee table in Figure 2.4 could additionally include the COMPANY_ID and DIVISION_ID columns as foreign keys to allow for better flexibility. The same would apply by including the COMPANY_ID column in the Department table.

Click To expand
Figure 2.4: A Hierarchy of Relational Entities Using Unique Integer Identifier Keys

Unique integer identifiers are most efficiently created in Oracle Database using sequence number generators. Never create unique identifiers for new rows in tables using something like SELECT MAX(identifier) FROM table. This approach will execute a full table scan whenever a new row is added, which is extremely inefficient. Maximum integer identifier values stored in a separate central table of sequence numbers would create hot blocks and concurrency issues but might be more efficient than using a MAX function for every new row required, especially for large tables. Oracle Database sequence objects are by far the most efficient method for integer identifier value generation.

Tip

In Referential Integrity primary keys and foreign keys are known as "keys." Keys are not explicitly called indexes because the two are not quite the same thing.

Indexing Foreign Keys and Locking Issues

Oracle Database automatically creates indexes on primary keys and not on foreign keys. An index is automatically created on a primary key because it is required to be unique. Uniqueness implies that when a new primary key is inserted into a table the whole table must be searched to make sure that the new insertion is not duplicating an already existing key.

If indexes are required on foreign keys they must be created manually. Generally it is essential to create indexes on foreign keys to avoid locking problems. These locking problems occur due to full table scans and potential shared or exclusive locks on the child table when doing updates or deletes in the primary key table.

In a highly concurrent environment, when changes constantly occur to rows in parent and child tables, serious wait times for lock releases on the child table could result when changing primary key parent tables. Why? Oracle Database searches for the existence of child table foreign keys when update or deletion occurs on a parent table.

Tip

Updates should not occur to primary keys because they are static in nature, especially in the case of abstracted integer identifier primary keys. Primary key updates are more likely in more traditionally designed relational databases but should not happen very often.

When Oracle Database searches into the child table, if no key exists on a foreign key, a full table scan will result on the child table because every foreign key must be checked. This can cause locking on the child table. Other concurrent DML activity occurring on the child table would exacerbate the problem. Concurrency issues can be addressed but only up to a point.

The result of creating indexes on foreign keys will be much better performance, particularly in highly concurrent environments. It reduces locking and contention on the child table, since only the index on the foreign key is checked. SQL which is continuously table scanning as a result of no foreign key indexes could kill the performance of your system.

The more sound your data model is the more foreign key indexes will help performance, since the data model key structure should match application requirements. Build indexes on foreign keys to prevent locking issues, not only for Referential Integrity constraint checking but also for SQL performance.

New primary key insertions do not require indexes on child table foreign keys since no activity occurs on the child table. Figure 2.5 attempts to show a picture of potential foreign key locking activity.

Click To expand
Figure 2.5: Avoiding Locking on Foreign Keys

Sacrificing Referential Integrity for Performance

Can performance be increased by removal of primary and foreign keys plus foreign key indexes? Yes sometimes, but it is extremely risky in relation to maintaining the integrity of rows between parent and child tables.

Creating indexes on foreign keys is required when parent tables are updated or deleted from, not inserted into. It may be difficult for a database administrator to be certain of this given limited knowledge of applications and any future changes. In fact the database administrator's knowledge is probably limited to being outside the scope of development. Database administrators are often not included in the development process. Some companies may not even hire a database administrator until the development process is complete or nearing completion.

In cases where child tables are static and very small, perhaps Referential Integrity checks between the parent and child tables are less necessary. In this case one must assume applications always validate the existence of foreign key child rows when adding or changing primary keys. Again, the child table has to be small! The Optimizer will usually full table scan small tables, regardless of the presence of indexes. This further builds the case where small static tables can simply be excluded from Referential Integrity checks by removing or disabling any child table foreign key constraints. Under these rare circumstances it may also be expedient to remove or disable the primary key constraint from the child table as well. Since the table is static there may never be a need to ensure that the primary key remains unique.

Once again this strategy is risky and inadvisable but could be utilized under extreme circumstances when creating indexes on foreign keys has not fully resolved concurrency and locking performance issues. The full strategy is one of two options:

  1. Always create indexes on foreign keys for all types of tables.

  2. For small static tables, if indexes are not created on foreign keys, it may be best to exclude a child table from Referential Integrity altogether by disabling child table foreign key constraints. If this is acceptable the child table primary key constraint may as well be disabled.

Coding Business Rules in the Database

We have already established that Referential Integrity can be implemented in the database by using either constraints or triggers. Constraints are usually more easily maintained than triggers, depending on the skills available. Using triggers to implement Referential Integrity will be much slower. Triggers may be more easily maintainable during development, especially if the data model is continually changing. Constraints are more easily maintainable in production since changes are less likely to cause errors. Code changes are always likely to introduce unexpected bugs. If constraints are continuously being changed then the system should still be under development. If the data model is not correct at the time of your first production release then you may have all sorts of other problems too.

So what are business rules? This term is used offhandedly where it could quite possibly have different meanings. Some technical people will insist on placing all business rules into the database and some will insist on the opposite. This is quite often because of a lack of skilled personnel in development, database administration or data modeling. Many development projects are top heavy with developers and lacking in administration and modeling skills. A technical manager will often quite sensibly tend to guide system development in the direction of the skill sets available, be those skills development or otherwise. An ideal situation is having all skills. Coding business rules in the database or not is often a problem with terminology, namely the term "business rules". Therefore, let's categorically avoid a definition for the meaning of that term. We will concentrate not on what should or should not be in the database but what could be in the database.

There are two important areas:

  1. Referential Integrity: can be implemented in the database using constraints or triggers.

  2. General database access using SQL code: stored procedures and functions.

Using Triggers for Referential Integrity

The case often made for triggers rather than constraints is that triggers are more easily managed. This is often the case when programmers rather than database administrators or data modelers are building those constraints. Programmers are usually frustrated with having to enable, disable, and perhaps even temporarily drop constraints in order to change tables. In fact most table changes do not require constraint changes unless the primary or foreign keys are being changed directly; primary and foreign keys restrict data in tables not the structure of those tables. Where a database administrator would carefully alter a table and never drop a table it is quite normal for developers, during the development process, to simply drop tables. Dropping tables and recreating tables in a production environment is very risky. This sort of practice is common during the development process.

Programmers generally make poor database administrators because the approach to database administration is generally a slower and more careful one. In defense of the programming community, database administrators often make terrible programmers. I have seen some database administrators with Unix backgrounds construct and write entire systems using Unix scripting, even avoiding a primitive programming language such as PL/SQL. This little story I hope makes it more palatable for a developer to understand that database administration is a much more meticulous process, particularly in production. The skills of coding and administration are very different in approach but both are required, even during the development process. In fact database administrators tend to subdivide themselves into one of two camps, Production DBAs and Development DBAs. The acronym "DBA" stands for database administrator.

Tip

Older relational databases and perhaps very ancient versions of Oracle Database required dropping of tables in order to make structural columnar changes: adding, changing, or deleting columns. For many years Oracle Database has been extravagantly equipped with both object CREATE and ALTER commands, allowing more or less any change to a table even when tables are online and actively being used.

So what does all this mean? It means triggers are a hopeless substitute for constraints as a method of enforcing Referential Integrity. Triggers make implementation easier for a programmer but will seriously affect performance in the long run, perhaps even making your application as much as 10 times slower than it could be.

Using Triggers for Event Trapping

In some relational databases triggers can be used to trap specific database events in addition to SQL insert, update, and delete events. In Oracle Database this is not the case. Triggers are a less efficient substitute for constraints when implementing Referential Integrity.

Trigger code is slow because coded PL/SQL will execute much slower than Oracle Database internal constraint checks. Trigger code must be parsed and compiled before execution and uses explicit SQL. Explicit SQL competes with all other SQL in the database for use of parsing and execution resources.

Triggers may require too much coding maintenance. Constraints should be created when the data model is first built such that a database administrator is in a position during production to be able to switch off specific constraints in order to help performance. This type of tuning can be difficult with triggers if individual triggers contain multiple aspects of functionality such as a trigger detecting all insert, update, and delete events for a table. A better approach could be a set of three triggers on that same table: one for each of insert, update, and delete event detection. Coders tend to amalgamate functionality and make things generic. Generic code is not consistent with an efficient data model, as in the case of the static generic entity shown in Figure 2.1.

I have worked with an application in the past where replacement of triggers with constraints gave a general performance increase of between five and tenfold. It is also unlikely that a single trigger cannot deal with all potential problems. The simple fact is that the more code an application has the more convoluted and difficult to maintain it will become over time. If you are creating triggers because they are easier to maintain then the data model is probably being built during the coding phase, which has its disadvantages.

Using Stored Procedures and Functions

Stored procedures and functions can be used for SQL activity as opposed to embedding SQL code in applications. I have encountered varying opinions on this subject and have seen both work and both fail. Stored procedures can include only SQL code or they can include some processing, either in the form of groups of SQL statements, to form transactional blocks, or even as far as including application logic; logic which should probably be at the application level but not necessarily.

In short PL/SQL is not a fully capable programming language. PL/SQL has programming language capabilities and a few bells and whistles, even some object and abstraction capability.

From my experience my recommendation is this. PL/SQL can be used to store database SQL code in one place. This has the effect of only allowing people with PL/SQL and Oracle database administration skills to build and change that code. However, since PL/SQL is essentially primitive as a programming language and developers make better programmers than database administrators, there is a strong case for minimizing use of PL/SQL.

Do not write your entire system using PL/SQL. When using PL/SQL to create database access SQL code think rather in terms of granularity, simple maintenance, and central control as good reasons for using PL/SQL. Modern application environments are very effective at managing and allowing creation of embedded SQL code. Using stored database procedures is probably faster but this is not always the case. Use of stored procedures is very much dependent on skills and the function of applications. Programmers should not be doing too much database administration and modeling because that is not a programmer's area of expertise. In terms of overcoding and perhaps overloading a database with stored procedures you may also overload your database administrators. Additionally, most Oracle database administrators have Unix backgrounds and will quite probably write PL/SQL stored procedures that look like scripts. HTML and JavaScript are scripting languages. Java is a programming language as PL/SQL is supposed to be.

Oracle Database will allow the creation of stored procedures using Java with a JVM embedded in the Oracle Database kernel. There is also a compiler called the Java Accelerator which will allow compilation of Java coded stored procedures into binary, allowing for fast execution times. Performance of Java coded stored procedures is unknown to myself as I have never used Java in Oracle Database as such. Java is an extremely rich coding environment. If even more complexity is required external binary compilations can be used, coded in high execution performance languages such as C.


2.3 Optimizing with Alternate Indexes

Alternate indexes are sometimes known as secondary indexes. The use of the terms alternate and secondary is somewhat confusing because they mean different things in English. In the context of a relational database they are one and the same. Alternate implies another option and secondary implies in addition to. This means the same thing so now we are all completely confused. The precise meaning of the terms in this context is really immaterial.

The problem is this. We create alternate indexes because the primary and foreign key indexes in our data model do not cater for everything our applications require for filtering and sorting. Filtering and sorting SQL statements is what uses indexes to access the database quickly by accessing much smaller amounts of physical space, in a required sorted order.

Having a lot of alternate indexes means your database has a lot of indexes to maintain. Whenever a table is changed every index created on that table must be changed as well. Thus, inserting a single row into a table with four indexes actually requires five additions and not one. That will hurt performance, sometimes very badly. If your database has a large number of alternate indexes a number of possibilities could be responsible.

  • The most obvious is imposition of reporting-type functionality over the top of OLTP-type functionality. More traditional relational database structures such as those shown in Figures 2.2 and 2.3 include composite primary keys and are more compatible with reporting efficiency. Structures such as that shown in Figure 2.4 do not have composite primary keys. Imposition of reporting requirements on the structure in Figure 2.4 would probably require composite alternate indexes.

  • When creating alternate indexing essential to reporting, SQL statements may require filtering and sorting, different to primary composite keys such as those shown in Figures 2.2 and 2.3. When existing composite primary keys do not cater for requirements then perhaps either further Normalization or Denormalization is a possibility, or current structures simply do not match application requirements. However, changing the data model at a later stage is difficult and further Normalization can cause other problems, particularly with recoding of application and stored procedure code.

  • A less obvious and often overlooked possibility is that the database has been ported directly from development to production. Developers have a habit of creating a lot of indexes whilst coding. Quite often they will not clean up and remove indexes no longer in use.

  • Sometimes in object structures such as that in Figure 2.4, since abstract unique identifiers are used for primary keys, items such as names of people or departments may be required to be unique. These unique keys are not part of Referential Integrity but they can be important, and are best maintained at the database level using constraints.

In summary, there is no cut and dried answer to which alternate keys should be allowed and how many should be created. The only sensible approach is to keep control of the creation of what are essentially "extra" keys because they were not thought about in the first place or because an application designed for one thing is expanded to include new functionality such as reporting.

Alternate indexing is not part of the Normalization process. It should be somewhat included at the data model design stage, if only to avoid difficulties when coding. Programmers may swamp database administrators with requests for new indexes if they were not added in the data modeling process. Typically alternate keys added in the data modeling stage are those most obvious as composites where foreign keys are inherited from parent tables, such as the composite primary keys shown in Figures 2.2 and 2.3. Note that this does not apply in the object-like hierarchical structure represented in Figure 2.4 since the parent of the parent is not present on the child table as a foreign key; it is not required to be.

The structure in Figure 2.4 is very typical of modern Java object application environments where every table's primary key is unique to the entire database, within that table. In other words, a row in a child table can be accessed using its primary key without the necessity for obtaining the parent table primary key first; there is no direct dependency between the parent and child tables, at least in an abstract sense. Figure 2.4 is more like an object database than a relational database. Figure 2.4 represents an object to relational mapping solution and functions very well with OLTP systems and very poorly for reporting. This is one reason why Java application driven Oracle database designs often result in two databases, the first an OLTP database and the second a data warehouse. The data warehouse database takes care of the reporting and contains all the alternate and composite indexing required for reporting. The only likely composite indexes sometimes in existence in OLTP object-like relational databases are the primary keys of many-to-many join resolution entities, something like the Actor entity in Figure 2.6.

Click To expand
Figure 2.6: An Object-Like Many-to-Many Join Resolution Entity

Figure 2.7 contains an equivalent traditional, non-object-like representation of the schema snippet shown in Figure 2.6.

Click To expand
Figure 2.7: A Traditional Relational Version of Figure 2.6

The sheer scale and size of OLTP Internet databases can sometimes be horrifying to even the most seasoned of database administrators. Many hundreds of gigabytes are common and when OLTP and reporting processing is mixed in the same database. Enormous amounts of money can be spent on very expensive hardware to maintain performance at a level acceptable to customer satisfaction. Additionally, Development cannot possibly be coded and tested against databases of such magnitude. The result is often applications that are coded to small-scale databases and unexpected if not disappointing performance in production. Extensive tuning is often required. The larger the database the more likely that a dual database architecture has to be adopted, of OLTP plus data warehouse architecture.


2.4 Undoing Normalization

Undoing Normalization, or removal of too much data model granularity, is essential to the performance of data retrieval from a relational data model. This is true not only for reporting and data warehouse databases but also, quite significantly, even for OLTP small transactional unit databases.

I am calling removal of Normalization for performance a process of undoing because Denormalization in its purest form, being Normalization in reverse, does not cover all practical possibilities. I have a number of areas of interest when it comes to undoing of granularity.

  • Denormalization: undoing various Normal Forms, some more than others.

  • Some useful tricks: various tricks to speed up data access.

  • Denormalization using unusual Oracle Database objects: creation of special types of database objects used to cluster, pre-sort and pre-construct data, avoiding excessive amounts of complex repetitive SQL.

The reason for removal or degradation of Normalization is for the purpose of improving performance. As already stated, it is very important to note that removal of Normalization granularity is not only required in data warehousing and reporting environments. It has often been the case, in my experience, where even OLTP transactional data models have required reduction in granularity produced by over-Normalization. Any application retrieving data from an over-Normalized database can potentially have performance problems. A very deeply Normalized database is only most effective for single-row insert, update, and deletion transactions, i.e. changing single data items.

In the past I have even seen some very poorly performing OLTP applications, even those with small transactional units. The reason for this poor performance is often the result of severely granular data models. These systems did not contain large numbers of rows or occupy a lot of physical space. Even so, brief data selection listings into one page of a browser more often than not performed join queries comprising many tables. These types of applications sometimes fail, their creators fail or both. In these cases there is a very strong case to be made for the necessity of designing a data model with the functionality of the application in mind, and not just the beauty of the granularity of an enormously over-Normalized relational data model structure.

One particular project I have in mind had some queries with joins containing in excess of 15 tables, a database under 10 Mb and some query return times of over 30 s on initial testing. These time tests were reduced to a few seconds after extensive application code changes of embedded SQL code. However, it was too late to change the data model without extensive application recoding and thus some very highly tuned and convoluted embedded SQL code remained. Maintenance will probably be a nightmare if it is even possible.

The conclusion is as follows. Normalization granularity should be degraded in order to speed up SQL data retrieval performance. How do we do this? The easiest and often the most effective method, from the perspective of the data model, is a drastic reduction in the number of tables in joins. It is possible to tune SQL statements joining 15 tables but it is extremely difficult. If the data model can possibly be changed then begin with that data model. Changing the data model obviously affects any SQL code, be it in stored procedures or embedded in applications.

So now we know why we might need to do the unexpected with a nicely Normalized data model. Let's start with Denormalization.

2.4.1 Denormalization

What is Denormalization? Simply put Denormalization is the opposite of Normalization. Where Normalization is an increase of granularity removing duplication, Denormalization is an attempt to remove granularity by reintroducing duplication, previously removed by Normalization. Denormalization is usually required in order to assist performance because a highly granular structure is only useful for retrieving very precise, small amounts of information rather than large amounts of information. Denormalization is used to analyze and report rather than facilitate changing specific data items. In simple terms Denormalize to decrease the number of tables in joins; joins are slow! Simple SQL statements are fast and easy to tune and should be the order of the day wherever possible.

It is sometimes the case that table structure is much too granular or possibly even incompatible with structure imposed by applications. This particularly occurs when the data model is designed with perfection of Normalization in mind without knowledge of or perhaps even consideration for realistic application requirements. It is a lucky development team that understands application requirements completely when the data model is being built. Denormalization is a solution in this case.

Denormalization is not rocket-science. Denormalization is synonymous with Normalization except the opposite; both are common sense.

Reminding Ourselves about Normalization

A relational database relates subsets of a dataset to each other. How do we relate these subsets? What do we mean by a subset and what is a data set?

  • A data set is the equivalent of a database in Sybase or Ingres and a schema in Oracle Database. A data set is a set of tables.

  • A subset is a table. A table defines the structure and contains the row and column data for each subset.

  • Tables are related to each other by linking them based on common items and values between two tables, namely primary and foreign keys.

At this point if you have not as yet read the previous chapter on Normalization then please do so. I have an unorthodox approach to Normalization and much of that approach is simply reversed in my approach to Denormalization. My perspective of Normalization is one of it being mostly much too complex for commercial practicality.

Why Denormalize?

Denormalize to speed up a poorly performing database by removing potential for mutable and complex SQL table joins. Joins are difficult and sometimes even impossible to tune.

What to Look for to Denormalize

Mutable and Complex Joins

A mutable join is a join of more than two tables. A complex join is a mutable join including filtering. The more tables in a join the slower it will execute and the more difficult it will be to tune. If these types of joins are heavily executed, simplification by Denormalizing offending tables could help performance. Following is an ANSI format SQL statement showing a mutable join of eight tables in the Accounts data model (see Appendix A). This query is completely ridiculous but it is the sort of complexity that you might want to search for.

SELECT cu.customer_id, o.order_id, ol.seq#, ca.category_id
FROM customer cu JOIN orders o ON (cu.customer_id =
   o.customer_id)
  JOIN transactions t ON (o.order_id = t.order_id)
   JOIN transactionsline tl ON (t.transaction_id =
      tl.transaction_id)
   JOIN ordersline ol ON (o.order_id = ol.order_id)
      JOIN stockmovement sm ON (tl.stockmovement_id =
         sm.stockmovement_id
      AND ol.stockmovement_id = sm.stockmovement_id)
         JOIN stock s ON (s.stock_id = sm.stock_id)
          JOIN category ca ON (ca.category_id =
             s.category_id)
WHERE ca.text = 'Software';

The WHERE clause adds filtering, making the mutable join a complex mutable join. The ANSI format of this query is no more difficult to tune than the Oracle Proprietary format. This example shows the number of tables joined very clearly.

Mutable Joins to Find Few Columns

When constructing SQL statement joins are you finding many tables in joins where those tables are scattered throughout the entity relationship diagram? When finding those columns are you passing through one or more entities from which no columns are retrieved? This is inefficient because every table passed through adds another table to the join. This problem can be resolved in two ways. The first is by Denormalization, which may be difficult, because there could possibly be so much Denormalization that it may affect functionality in too much application code. The second possible solution is to maintain copies of the offending column values in both entities. Refer to the section later in this chapter entitled Copying Columns between Entities (p. 75).

Tip

This type of joining is a potential indicator that a single data model services multiple loosely connected applications. Loosely connected applications should not always be placed into the same dataset (Oracle Database schema).

Following is the same ANSI format query as above but with a small change where only the CUSTOMER_ID and CATEGORY_ID columns are retrieved. Of course the join is still ridiculous and probably would never be a requirement for an application, but this is the only way that the stock category can be linked to a customer. This is definitely a problem. In this example Denormalization would be nonsensical but some type of a relationship could possibly be established between the Customer and the Stock tables. On the other hand, new entity relationships would only serve to complicate the data model further.

SELECT cu.customer_id, ca.category_id
FROM customer cu JOIN orders o ON (cu.customer_id =
   o.customer_id)
  JOIN transactions t ON (o.order_id = t.order_id)
   JOIN transactionsline tl ON (t.transaction_id =
      tl.transaction_id)
     JOIN ordersline ol ON (o.order_id = ol.order_id)
        JOIN stockmovement sm ON (tl.stockmovement_id =
          sm.stockmovement_id
        AND ol.stockmovement_id = sm.stockmovement_id)
         JOIN stock s ON (s.stock_id = sm.stock_id)
          JOIN category ca ON (ca.category_id =
            s.category_id)
WHERE ca.text = 'Software';

Adding Composite Keys

Do entities have composite keys? It is possible to partially Denormalize by adding composite key elements to the primary keys of subset tables. Composite keys are totally contrary to object structure and more compatible with reporting. Java applications may perform poorly when data is accessed using composite keys. Object applications perform best when accessing precise objects containing collections of other precisely defined objects.

Tip

Composite indexes not constructed as primary keys fall into the realm of alternate indexing. Refer to the section earlier in this chapter entitled Optimizing with Alternate Indexes (p. 59).

A purist's relational database structure will contain composite primary key columns in subset tables for all parent tables above in the hierarchy. Java applications typically add a single column unique identifier to each table as a primary key and exclude parent table foreign key columns from the primary key. The only problem with this type of object to relational mapping, top-down Java design is as follows. Reporting and even on-screen listings of less than a page can result in mutable joins, extremely hard-hitting SQL join statements. This is where those 15 table joins start to appear. The root cause and reason for the existence of these types of mutable joins is usually extreme over-Normalization, particularly with 3rd, 4th, and 5th Normal Forms. Normal Forms are a relational and not an object methodology. Figures 2.2 and 2.3 contain composite primary keys. Figure 2.4 contains a Java top-down object-relational mapping structure.

One-to-One Relationships

Look for one-to-one relationships. These may be unnecessary if the required removal of null values causes costly joins. Disk space is cheap. Mutable SQL join statements can destroy performance. These one-to-one relationships are typical of 4th and 5th Normal Forms. An example of this problem is shown in Figure 2.8.

Click To expand
Figure 2.8: 4th Normal Form Denormalization

Many-to-Many Join Resolution Entities

Do you have many-to-many join resolution entities? Are they all necessary? Are they all used by applications? Many-to-many join resolution entities are typical of 3rd Normal Form. Quite often when a data set is first designed there may be overusage of 3rd Normal Form leaving some superfluous, performance-hitting entities which can become more and more difficult to remove due to required application code changes. Two things are important about 3rd Normal Form created entities:

  1. Are the new entities used by the application? Quite often these entities are created from the perspective of the data model and may never be used by the application and may not need to be used either.

  2. A clue as to the usefulness of these new entities is do they have meaning? Are the names of these entities a combination of the parent entity names or do they have a name, which is distinct only to the new entity? The Assignment entity in Figure 2.10 is a good example of a meaningful many-to-many relationship entity and the CourseStudent entity in Figure 2.9 is a good example of a potentially meaningless many-to-many join resolution entity.

    Click To expand
    Figure 2.9: 3rd Normal Form Denormalization

Application Functions versus Entities

Compare the number of functions in the application and the number of tables. If you have far more tables than functional units you might want to Denormalize or simply remove some of the entities because they may have become redundant.

Static Data in Multiple Entities

When searching for static data items such as customer details are you querying single or multiple tables? Querying a single table is much more efficient than multiple tables. Static data does not need to be Normalized into separate entities unless said separate entities are accessed individually using separate SQL statements. Even so static tables are often small and generally will be full table scanned by the Optimizer, regardless of any indexing. A good example of this is represented by the Listed and Exchange entities in Figure 2.11. The Listed and Exchange entities do not need to be separate entities.

Intermediary Tables Covering Summary Groupings and Calculations

Extra tables can be created, perhaps regenerated periodically, to contain summaries of larger tables. These types of tables are more akin to reporting and data warehouse read-only functionality but they sometimes can be useful in OLTP applications. These tables are useful as long as less than real-time response is acceptable for the summarized information. The script shown creates a table in the Accounts schema (see Appendix A) which is a grouping summary of a join between the COA and GeneralLedger tables.

CREATE TABLE ChartOfAccounts AS
  SELECT coa.coa#, coa.type, coa.subtype, coa.text,
     sum(gl.dr), sum(gl.cr)
  FROM coa, generalledger gl
  WHERE coa.coa# = gl.coa#
GROUP BY coa.coa#, coa.type, coa.subtype, coa.text;

Denormalizing by Reversing Normal Forms

Let's explain Denormalization by example. Let's take a look at the reverse of the 3rd, 4th, and 5th Normal Forms. It is unlikely you will ever want to reverse 1st and 2nd Normal Forms. The only case would be where the difference between the number of rows in master and detail tables is extremely small; even so this strategy is not advisable.

A very common cause of performance problems is caused by the over-application of 4th Normal Form to remove null values from entities. Examine the diagram in Figure 2.8. Note the one-to-one or zero relationships between Customer to Listed and Customer to Address entities. The zero part of these relationships implies that a customer does not have to have either a stock ticker symbol or an address. Thus 4th Normal Form allows the removal of these two sections to new entities, the Listed and Address entities.

Removal of null values from entities to new entities can save space but this is a debatable issue. Since null values are empty and Oracle Database has variable record lengths the space saving element may be negligible. SQL statements to retrieve customer details from these three entities would require a three-table join to retrieve all of the customer information. When it is taken into account that other information will be retrieved with customers, such as orders and invoices, very large joins could result. The 4th Normal Form detail shown in the top part of Figure 2.8 could cause very complex mutable joins, difficult to tune, and thus lead to severe performance problems. These types of performance problems are often detected only when applications are placed onto realistically sized production databases at the end of the development cycle. Making data model changes in the maintenance cycle can be so costly as to make them impossible. It is best not to do this sort of thing in the first place.

Now let's take a look at 3rd Normal Form. One cannot really present the argument that 3rd Normal Form is for academics and is commercially unusable because 3rd Normal Form is often usable. However, the point to make about 3rd Normal Form is that every 3rd Normal Form many-to-many join resolution entity should be closely examined. Is it really needed and is it meaningful?

Look at the diagram in Figure 2.9. Would you really need to locate the details of a single course for a single student, or vice versa? Think about it from the perspective of how your application retrieves this type of data and how it interfaces with the reader. Does it ever require you to list the details of a single course on which a single student is enrolled? Perhaps yes. If so then that information may have already been pre-selected from the database. The point is this: 3rd Normal Form many-to-many join resolution entities should only be created when they have meaning or when they are absolutely required.

So 3rd Normal Form should create meaningful entities from many-to-many relationships. 3rd Normal Form is best not used to create many-to-many join resolution entities such as the CourseStudent entity in Figure 2.9. Why?

Look at the schema shown in Figure 2.10. Note how the Assignment entity does two things. Firstly, it does resolve a many-to-many join between the Task and Employee entities. Secondly, and more importantly, the Assignment entity has meaning in itself since it defines the assignment of tasks to employees.

Click To expand
Figure 2.10: The Employees Schema

The practical application of 3rd Normal Form is that many-to-many joins resolved into new entities should produce a meaningful entity, not meaningless unique identifiers which may or may not ever be used. More often than not CourseStudent type many-to-many join resolution entities as shown in Figure 2.9 are not used and if they are, perhaps should not be.

Now let's retrogress a little. Let's go back to the 4th Normal Form entity structure as presented in Figure 2.8 but include an extra entity. Examine the diagram in Figure 2.11. Note the new entity Exchange and the addition to the Listed table of the EXCHANGE_ID foreign key column. The relationship between the Listed and Exchange entities is a many-to-one relationship, placing the tables into 2nd Normal Form. Why is this 2nd Normal Form? This is because the Exchange table will contain items such as NYSE (New York Stock Exchange) or NASDAQ. The names of exchanges are static relative to the companies listed on those exchanges. Ticker symbols are used to represent companies on stock exchanges.

Click To expand
Figure 2.11: 2nd Normal Form Denormalization

The big arrow in Figure 2.11 shows the removal of the Exchange entity by placing the name of the exchange into the Listed entity. This is effective Denormalization since there is no information on the Exchange entity other than its unique identifier (EXCHANGE_ID) and the name of the exchange. In short, the Exchange entity in this situation is completely unnecessary and will probably ultimately hurt performance by requiring more complex mutable join SQL statements.

2.4.2 Some Useful Tricks

There are many tricks to Denormalize data that are not effectively reversals of the steps of Normalization. Here are some of them.

  • Copying columns between entities: making copies of columns between unrelated entities to avoid mutable joins.

  • Placing summary columns into parent entities: avoid grouping joins but continual real-time updating of these types of columns can cause hot block issues, sometimes better updated in batch mode during low activity cycles.

  • Separating active and inactive data: avoiding searching through data no longer often used in order to reduce the amount of physical space searched through.

  • Mixing heavily and lightly accessed columns: similar to the above, separating heavily and lightly accessed columns into separate entities, avoiding unnecessary access to rarely used data.

  • Focus on heavily used functionality: tune specific areas of a data model.

  • Using views: using views is often useful for application coding but often detrimental to database performance.

  • Local application caching: store static values on client or middle-tier machines to avoid repetitive database access.

    Tip

    Any other related useful Denormalization tricks that readers would like to see added to this section can be included in the next edition of this book. Please email the author with details.

Copying Columns between Entities

This is a case where two entities, generally somewhat disassociated from each other in a data model, can be made to contain a copy of the same value. This would be done in order to avoid joining between multiple tables, through a hierarchy, to obtain a single value.

Figure 2.12 shows a section of the Accounts data model (see Appendix A). The date field DTE should be maintained only in the StockMovement entity but it is duplicated on the Orders and Transactions entities as well. This helps to avoid joins from Orders and Transactions through StockMovement entries simply to retrieve the date.

Click To expand
Figure 2.12: Duplicating Individual Columns Between Entities

One could use event triggers to copy column values between entities but this activity is better implemented at the application level. Use of triggers can have a dire effect on performance in highly active concurrent databases.

Placing Summary Columns into Parent Entities

Summary columns can be placed into parent entities to summarize information held in child entities. Obviously these summaries would be commonly accessed summary amounts; we would not want to be continually joining many tables and re-executing grouping functions simply to find a simple summary. The problem with using summary columns is potential locking and concurrency hot block problems on the summary column table rows, if those summary columns are updated in real time. Typically these types of Denormalized structures are batch updated at times of low activity.

Figure 2.13 shows another section of the Accounts data model (see Appendix A). The balance column on the Customer and Supplier entities summarize their respective transaction and cash-book entity amount entries; maintenance of summary columns of this nature in real time may be best updated in batch mode. Using event triggers on child tables will cause even greater performance problems.

Click To expand
Figure 2.13: Summary Columns in Parent Entities

Separating Active and Inactive Data

Separation of active and inactive data effectively creates further granularity but it has the benefit of assuring unused data does not interfere with highly accessed data. This can help immensely with performance, especially when the amount of historical or archived data far outweighs that of current data, as is often the case. This tactic is a primitive form of partitioning or data warehousing without the data warehouse. A prime candidate for this type of separation in the Accounts schema (see Appendix A) would be the GeneralLedger entity; general ledger entries in accounting systems can often become large very rapidly. Splitting an entity such as this into active and archived entities based on some sort of accounting period such as years, quarters, or even months would be expedient.

Separation of active and inactive data could affect application design. Since data is separated the application would have to be constructed such that it can access both types of data separately or together. Obviously if data is being accessed together at the application level then there is probably not much point in separating it in the first place. An effective application-level solution is having multiple layers of functionality, thus multiple programs. As an example the GeneralLedger table in the Accounts schema could have separate reporting programs for dealing with current and previous periods when executing reports against the GeneralLedger table. At the application level a report such as this could be written with date or year input parameters allowing access to active or archived data, or both. This leads to the fact that the GeneralLedger table would be a perfect example for Oracle Partitioning, which will be covered in Part III on physical and configuration tuning.

Mixing Heavily and Lightly Accessed Columns

Mixing heavily and lightly accessed columns in SQL statements can cause problems. Perhaps separate those columns into different entities. However, be very careful with this sort of thing. This kind of tuning is more or less the same as 4th Normal Form Normalization, which causes its own problems as well.

Focus on Heavily Used Functionality

This perspective simply stresses focusing on specific areas of the data model or providing better performance in heavily used areas of applications. In other words, specific tuning of particular tables or indexes. Sometimes this approach can involve use of exotic object types such as clusters, index-organized tables, partitioning, materialized views, amongst other Oracle software bells and whistles.

Using Views

Views are frequently used by developers to simplify application design and coding, a similar approach to that of using generic static entities as shown in Figure 2.1. In general, any type of simplification of application coding leads to poor data model structure. Views often lead to too many database objects and thus too much data model complexity. Views are effectively overlays producing more complexity than is necessary at the database level due to the constant resolving of view contents. Some applications even use multiple layers of views, making problems even worse. Views absolutely do not help SQL code and database performance.

Application Caching

Heavily used data which is not constantly changing can be cached on client or middle-layer machines. For instance, a date such as today's date not requiring a time as well only requires refreshing every 24 h. Some of the highest execution quantities of SQL code are often for statements performing the simplest of tasks such as retrieving today's date. A common SQL statement of this type in Oracle SQL is a SELECT SYSDATE FROM DUAL statement. Even the simplest of SQL statements can affect performance if executed often enough. With Java object applications application-level caching is often effectively utilized. Object-relational mappings function best using pre-loading of static data at application startup.

2.4.3 Denormalization using Special Purpose Oracle Database Objects

Denormalization using unusual Oracle Database objects allows creation of special types of database objects used to cluster, presort and pre-construct data, avoiding excessive amounts of complex repetitive SQL. In effect Denormalization of a relational data model is a way of undoing granular complexity. The reason for this "undoing" process is in order to increase performance by minimizing of hard-hitting SQL statements. There are numerous types of logical data objects available in Oracle Database, which do what could be called a form of Denormalization into pre-constructed physically organized data sets. These objects are often used to precreate data sets to be retrieved at a later date, in the structure and form required by an application. These object types are generally as follows:

  • Clusters. Places most commonly accessed index and some data columns together in the same place physically, in the desired order.

  • Index-Organized Tables. Sorts both the index and the data columns of a table into a required sorted order.

  • Materialized Views. Pre-constructs and stores the results of an SQL statement avoiding repetitive SQL code execution in order to produce the desired result. A Materialized View "materializes" the data. In other words, it stores data physically separately from the source tables. Unlike a materialized view a view is simply an overlay of another entity or entities, re-executing an SQL statement every time the view is accessed. Simple views are not helpful for performance, whereas materialized views are. Materialized views are often used in data warehouses.

All these solutions are a physical rather than a logical form of Denormalization. These objects do not reduce logical complexity but increase complexity by creating more data object entities. What they do accomplish is to place data sets into new purpose-built constructs, which can be used to access data in the required order without jeopardizing access speed to highly active concurrently used entities.

The next chapter will briefly discuss various different forms of the Relational Database model.


Chapter 3: Different Forms of the Relational Database Model

Overview

Over the years since the relational database model was first invented the art of data modeling in relational databases has evolved somewhat. From the perspective of commercial applications this evolving process has often been the result of the changing methodologies used by software application development kits or SDKs.

The most significant change in recent years to the way relational database models are built has been the advent of objects. Data structures most efficiently used by object-oriented Java and C++ are generally very different to those that would have been used by a procedural programming language such as C or Modula-2. Let's start from the first ideas of relational database modeling and work our way up to data models used by modern online Java object applications.


3.1 The Purist's Relational Database Model

What I call the Purist's relational data model is what the relational database looked like in its original, most pure form. Figure 3.1 shows a Purist's form of the Employees schema introduced in the previous chapter. Note how all the primary keys are composite indexes. All of these composite indexes are concatenations of all keys inherited or passed down from all parent tables above in the hierarchy. The unique identifying key in the entity concerned is placed at the end of each primary key.

Click To expand
Figure 3.1: A Purist's Form of the Employees Schema

The type of primary key structure shown in Figure 3.1 provides effective performance for reporting or retrieval of groups of rows where it is assumed the report output is sorted based on primary key values.

SQL was originally designed to retrieve sets of rows not single rows. Note that the size of the keys is large and in some cases the indexes would occupy as much physical space as the tables themselves. If tables were to be joined, the tables would be joined on the basis of all the elements of each primary key. Following is an example script of a join that selects rows from four tables; note the size of the WHERE clause used to execute the join.

SELECT de.name, pr.name, pt.name, prd.revenue - prd.cost
FROM department de, project pr, projecttype pt,
   product prd
WHERE de.division = pr.division
AND de.department = pr.department`
AND pr.projecttype = pt.projecttype
AND pr.division = prd.division
AND pr.department = prd.department
AND pr.projecttype = prd.projecttype
AND pr.project = prd.project;

Large WHERE clauses would be used for SQL statements using a data model as shown in Figure 3.1. The primary keys would be shortened coded versions of the name fields in each entity. Thus if Division.NAME values are "North East", "South East", "Mid-West" and "Pacific Coast" their related division code values could be NE, SE, MW, and PC, respectively. Without the coded values indexes would be created on the name of each row such as Division.NAME. Coded values were used to make coding easier and to reduce the physical size of indexes.

So that is the Purist's form of the relational database model. Let's now take a look at how object SDKs have affected how relational database models have changed and evolved as a result.


3.2 Object Applications and the Relational Database Model

How do object applications affect the structure of relational database models? The point is that object application SDKs do not have to change the structure of a data model but in general they probably should. In order to impose an object-oriented architecture on top of a relational database certain small changes can be made to a relational data model. We will examine these in due course. Let us first take a look at a very simple description of the object database model in order to help explain recent trends in relational data modeling.

3.2.1 The Object Database Model

The object model is somewhat different to the relational model. Object databases are much more capable at handling complex data structures. What are the parts making up the object model?

  • Class: defines a structure. Equivalent to a relational entity.

  • Attribute: specific slots within classes. Equivalent to a relational entity column.

  • Methods: processing executed on objects declared as repetitions of classes. Vaguely equivalent to relational stored procedures, far more powerful in an object database.

  • Unlike a relational model an object model supports relationships between objects through the structure of objects and the classes defining those objects. Classes are defined as containing collections of pointers to other classes, as being inherited from other classes above in a hierarchy or as being abstractions of other classes below in the hierarchy.

  • This leads us to observing that classes can be both specialized and abstracted. A specialized class is a more specific form of a class inheriting everything from its parent class but allowing local overriding changes and additions. An abstracted class is a more general form of a class containing common aspects of inherited classes.

  • It should be understood that there is a distinct difference between a class and an object. An object only exists at run time. As a result the structure of classes can often be very different to the structure of objects created from a class hierarchy. Many object models are designed incorrectly because the difference between a class and an object is not understood. In general, class structure will never look like the object structure. Class structure is an abstraction of object structure.

Figure 3.2 shows an object database class structure on the left and a relational database entity structure on the right. Note a number of differences:

  1. The object model has no types (TypeCode). Types are represented by the specializations of the Person class, Employee class, and Manager class.

  2. The relationships between the different classes in the object model are represented in the actual class structure, those relationships being both collection inclusion and inheritance. Thus the object model contains objects containing pointers to other contained objects.

  3. An invisible difference is the power of black-box processing using methods in an object database. PL/SQL can perform some of these tasks but inadequately.

The Person class is a collection included within the Payroll class, as is the Task class in both the Employee class and the Manager class. The Employee class and the Manager class are both specializations of the Person class. The Person class is an abstraction where the Employee class and the Manager class are inherited from the Person class. The run time object structure is shown in Figure 3.3 for the object model in Figure 3.2. As you can see the object structure in Figure 3.3 is not the same as the object model class structure in Figure 3.2.

Click To expand
Figure 3.2: The Object versus the Relational Model

Figure 3.3: The Run Time Objects

The object model uses data abstraction to conceptualize both data and encapsulated functionality using methods. Inheritance allows the passing of data definition and functionality onto specialized objects and extensive coding reuse. The crux and greatest benefit of objects is in their simplicity. The more that objects are broken down the more powerful and efficient the structure becomes. A relational database cannot be broken down to the level of an object structure. A relational structure can only be made more granular using Normalization, a certain distance, before the amount of Normalization becomes detrimental to performance. Since an object structure can be broken down into an extreme of granularity it is highly efficient at handling small, precise transactions. However, an object structure is dreadful at handling any kind of reporting. A relational structure can do both fairly well.

So what is the answer? Perhaps the object-relational model is the solution.

3.2.2 The Object-Relational Database Model

The object-relational model attempts to combine the relational and the object worlds. The problem is that the underlying storage structure of an object database is totally different to that of a relational database. The relational database storage method uses tables and rows (tuples) and is not terribly compatible with an object structure of containment of objects within other objects in a hierarchy, allowing exact searching anywhere into the entire object database.

The best product of the object-relational model is the ability to store and reference large binary objects such as multimedia. Most of the collection aspects such as table contained array types are hopelessly complex and inefficient. Additionally methods can be created in PL/SQL, whose coding implementation is far too complex and convoluted to be of any real use.

So the object-relational model simply includes some aspects of the object model, more successful in some areas than in others.

My experience of working with both multiple relational and object databases is that the two should not be mixed to any great extent. Relational and object methodologies are completely opposed to each other and the two simply do not mix very easily.

3.2.3 The Benefits of Overlaying Objects onto Relations

So far in this chapter we have examined an example object model and briefly discussed what the object-relational model attempts to achieve. Many ideas have been put forth in attempting to map object structures from an SDK such as Java onto a relational model. The best option I have come across has been that as shown in the previous chapter in Figure 2.10. This diagram is repeated in this chapter in Figure 3.4 for convenience. The data model in Figure 3.4 is a partially abstracted form of the data model shown in Figure 3.1. In Figure 3.4 every entity has a unique identifier, generated using Oracle Database sequences. The only problem with a structure of the form in Figure 3.4 is that, as with an object database, re-ordering the data to generate reports can be exceedingly slow without extensive alternate indexing. Too much indexing uses too much physical space and will slow down database change activity.

Click To expand
Figure 3.4: The Employees Schema

The data model in Figure 3.4 would allow a different version of the join statement shown previously in this chapter. Note how this SQL join statement shown has a much smaller WHERE clause since fewer matching conditions are required.

SELECT de.name, pr.name, pt.name, prd.revenue - prd.cost
FROM department de, project pr, projecttype pt, product prd
WHERE de.department_id = pr.department_id
AND pr.projecttype_id = pt.projecttype_id
AND pr.project_id = prd.project_id;

That covers just about everything we need to look at for relational data model tuning, including a brief glimpse at the object data model. Overall there is not much that can be said about tuning a relational data model other than changes to granularity. Increase granularity using Normalization and decrease granularity using Denormalization, amongst other simple tricks. The secret about relational data model design is that it is probably simpler than it appears. The object model, on the other hand, allows inclusion of extensive functionality in an object database directly. As a result the object model is probably much more difficult to design. Objects by their very nature demand more up-front design and planning work. It should be said however that most of the work required in building a relational data model is analysis and understanding of requirements, particularly with regards to potential applications. Understand applications fully prior to sitting down and drawing out entity relationship diagrams. Putting more time into data model design prior to beginning coding will absolutely benefit performance immensely at a later stage. The data model is the most important part in the development of any custom application because every other piece of the puzzle depends on it. Applications rely on the integrity, applicability to application usage and serviceability of the data model.

The next chapter will present a brief history of data modeling and gradually lead up to using and tuning of SQL in Part II.


Chapter 4: A Brief History of Data Modeling

Overview

A brief history of data modeling is important because it will lead us into the introduction to and tuning of SQL code in the next part of this book. With respect to Oracle software it is important to understand the roots and reasons for the existence of both the relational and object data models. SQL was created in the very beginning of the relational data model. Additionally Oracle Database is effectively an object-relational database. An object-relational database attempts to meld aspects of object data modeling into the relational data model. It is fundamental to the tuning of SQL to understand how SQL was devised in order to comprehend how SQL can be tuned in its most simplistic forms.

We will start our short history of data modeling with a look at all types of data models, progressing on to the history of relational databases and the Oracle Database, finally taking a look at the roots of SQL. This will lead us into the next major part of this book, SQL code tuning.


4.1 The History of Data Modeling

A database is a term used to describe an ordered set of information stored on a computer. This ordered set of data or data set is often structured using a data modeling solution in such a way as to make the retrieval of that data more efficient. Depending on the type of applications using the database, the database structure can be modified to allow for efficient changes to that data as well. It is appropriate to discover how different data modeling techniques have developed over the past 50 years to accommodate efficiency, in terms of both data retrieval and data changes.

Click To expand
Figure 4.1: The History and Evolution of Data Modeling

Databases started out stored quite simply in a file system and have evolved through hierarchical, network, relational, object, and object-relational data models, as shown in Figure 4.1.

4.1.1 The Different Types of Data Models

  • A database stored as a file system structure simply utilizes the structure of an operating system file system.

  • A hierarchical data model as shown in Figure 4.2 is a treelike structure where child tables can only have a single parent. In a hierarchical data model child tables are completely dependent on the existence of parent tables, supporting one-to-many but not many-to-many relationships.

    Click To expand
    Figure 4.2: A Hierarchical Model for Company Employees

  • In a network data model as shown in Figure 4.3, a refinement of the hierarchical model, many-to-many relationships are catered for by allowing a child table to have more than one parent, effectively creating a networked structure of tables.

    Click To expand
    Figure 4.3: A Network Model for Company Employees

  • A relational data model as shown in Figure 4.4 allows links between any two tables in the data model, regardless of position in any hierarchy, plus access to any table without having to access parent tables. A relational database is essentially a two-dimensional structure of loosely linked tables containing rows, helping to remove duplication and allowing for easy access to groups of data. A relational data model is intended for data throughput and thus is most efficient for retrieving sets of data.

    Click To expand
    Figure 4.4: A Relational Model for Company Employees

  • An object data model applies an object collection and unique pointer structure giving a somewhat three-dimensional or spherical structure to data. Object databases are best used for high-speed access to very small and precise amounts of data within a larger dataset. Unlike a relational database an object database is not efficient for reporting. Figure 3.2 in the previous chapter shows a good demonstration of a comparison between a relational and an object model. This diagram is repeated in this chapter in Figure 4.5 for convenience.

    Click To expand
    Figure 4.5: The Object versus the Relational Model

  • An object-relational database is a relational database with some object database aspects added, with varying degrees of success.


4.2 The History of Relational Databases

The diagram in Figure 4.6 shows the evolution of relational databases. Notice on the left how SQL was first devised as being a data access language with the development of System R from IBM, used today as SQL or Structured Query Language.

Click To expand
Figure 4.6: The Evolution of the Relational Databases
Tip

Arrows in Figure 4.6 represent both a flow of development and a flow of people and their skills. Oracle Corporation created Oracle software. Ingres Corporation did not create Oracle Database. However, the skills and personnel were passed between the different database vendor software companies. Ingres Corporation is now defunct and owned by Computer Associates.


4.3 The History of the Oracle Database

Now let's take a more detailed look at the history of the Oracle Database. Examine the diagram in Figure 4.7. Many of the items listed are relevant to tuning Oracle installations. Many items are specifically relevant to SQL code tuning, some physical and configuration tuning, some both.

Click To expand
Figure 4.7: The History of Oracle Database

  • Most of the hardware distribution elements such as clustering, client/server, thin client, and multiple-tier environments, using application servers, helped performance with respect to distribution of processing.

  • 4GL and PL/SQL programming tools and languages helped with creation of better, more efficient, and more maintainable source code. It could be said that the easier that code is to write then the more time will be available for tuning. However, in relation to coding of fast-performing C code this theory is probably complete hogwash! Then again planning and coding efficiency may very well be relative to how much time is spent building code. C code development is notoriously slow and complex.

  • Cost-based optimization and statistics helped with SQL code tuning in relation to actual data space utilization. Statistics allows for two aspects of physical tuning. Firstly, better use of space. Statistics help to assess space usage and storage structure from datafiles down to the block-level. Secondly, better analytical information for using in capacity planning. Capacity planning is the process of estimating physical space usage and growth rates.

  • There are a multitude of aspects introduced in both the Oracle8i Database and Oracle9i Database, Internet versions of Oracle Database software. These releases helped the performance of modern-day Internet and Intranet database environments.

    Note

    Oracle Database 10 Grid Oracle Database 10g has better performance, scalability, is more highly available, is easier to manage, is better automated, has tighter security, amongst other aspects.


4.4 The Roots of SQL

Why are we going backwards in time looking at data models and relational databases that are no longer in use? In order to tune SQL code effectively we need to understand the most basic forms of SQL and why newer data modeling techniques are better. In short, database access coding (SQL coding in the case of a relational database) has evolved with data modeling techniques. Object database data access coding has been called ODQL. ODQL stands for Object Definitional Query Language. ODQL was developed for object database access.

SQL in actuality stems from an idea of a reporting language devised in theory by the inventor of the relational database. It is very important to remember that the roots of SQL lie in retrieval of sets of data. In other words, SQL is intended as a language to retrieve many rows from one or many tables at once, as a result set. Retrieval of sets or groups of rows is consistent with the original relational data model. SQL was not originally intended to retrieve individual rows from a relational database as exact row matches. SQL can now be used to do exactly that, and fairly efficiently.

The next chapter is the first chapter in Part II of this book, SQL code tuning. We will begin the process of explaining SQL code tuning by introducing the very basics of SQL. In subsequent chapters of Part II we will progress onto the most fundamental aspects of SQL code tuning, then later to specifics of using and tuning Oracle SQL.


Part II: SQL Code Tuning

Chapter List

Chapter 5: What is SQL?
Chapter 6: The Basics of Efficient SQL
Chapter 7: Common Sense Indexing
Chapter 8: Making SQL Efficient in Oracle Database
Chapter 9: How to Find Problem Queries
Chapter 10: Tuning SQL with Oracle Enterprise Manager


Chapter 5: What is SQL?

Overview

Let's start the examination of tuning SQL with descriptions of the various basic SQL commands used in relational databases, leading on to the Oracle database specifically.

SQL is pronounced "Es-Queue-Ell" or "Sequel", depending on the source and location of your training. The meaning of the acronym SQL is Structured Query Language. SQL is the language used to access data from a relational database such as Oracle Database. SQL was originally devised as a computer language with the development of the IBM relational database called System R. The idea of SQL originates from the relational data model. SQL is used to access subsets of data in the form of rows or tuples from a larger dataset. A dataset is represented by a related group of entities in a data model of a set of tables in an Oracle Database schema.


5.1 DML and DDL

Unlike other relational databases Oracle Database classifies and separates SQL commands into two distinct groups as shown in Figure 5.1. These two groups are DML or Data Manipulation Language commands, and DDL or Data Definition Language commands. There are two differences between DML and DDL commands:

Click To expand
Figure 5.1: DML Versus DDL

DML commands manipulate data. A DML command is a command that changes column values in rows in tables. Thus the INSERT, UPDATE, DELETE, and MERGE commands are by definition DML commands because they change data. The SELECT statement and the TRUNCATE command are not DML commands. The SELECT statement is not a DML command, it does not change data and as a result does not produce rollback or redo log entries. The TRUNCATE command on the other hand does change data but automatically executes a COMMIT command. The TRUNCATE command is therefore a DDL command.

Now let's take a quick look at syntax for all the Oracle SQL DML statements.


5.2 DML Command Syntax

The first thing to be done is to make some changes and expand the Employees schema as shown in Figure 5.2. The original is shown in Figure 2.10 (p. 73).

Click To expand
Figure 5.2: The Employees Schema

Now I am going to describe the syntax for all of the Oracle SQL DML statements with explanatory examples of each using the schema in Figure 5.2.

5.2.1 The SELECT Statement

Here is a simplistic form of the Oracle SQL syntax for the SELECT statement.

SELECT { [alias.]column | expression | [alias.]* [ , ¼ ] }
FROM [schema.]table [alias]
[ WHERE [ [schema.]table.|alias.] { column | expression }
   comparison { ¼ }
      [ { AND | OR } [ NOT ] ¼ ]
]
[ GROUP BY
   { expression | rollup-cube | grouping-sets }
      [, { expression | rollup-cube | grouping-sets } ¼ ]
   [ HAVING condition ]
]
[ ORDER BY
   { { column | expression | position }
      [ ASC | DESC ] [ NULLS { FIRST | LAST } ]
   [, ¼ ] }
]
;

rollup-cube: ROLLUP | CUBE ( expression [, expression ¼ ] )
GROUPING SETS
      ( rollup-cube | expression [, expression ¼ ])

The WHERE clause is used to filter out or remove unwanted rows. The GROUP BY clause is used to generate summary rows of sets of rows and the ORDER BY clause is used to sort the returned rows. ROLLUP, CUBE, and GROUPING SETS clauses are extensions to the GROUP BY clause.

Logical Operators

Logical operators allow for Boolean logic in WHERE clause filtering and various other places. Mathematically the sequence of precedence is NOT, followed by AND, and finally OR. Precedence can be altered using parentheses.

This first query finds the CEO and the second employee. The CEO has no manager and thus his Employee.MANAGER_ID is NULL.

SELECT * FROM employee WHERE employee_id = 2 OR manager_id
   IS NULL;

This query returns no rows because EMPLOYEE_ID is unique and the MANAGER_ID of that employee is not NULL, that person is not the CEO.

SELECT * FROM employee WHERE employee_id = 2 AND manager_id
   IS NULL;

This query returns a spurious result because precedence of executing AND in conjunction with OR is not implemented using parentheses.

SELECT employee_id, manager_id FROM employee
WHERE manager_id IS NULL AND employee_id = 1 OR
employee_id = 2;

This query enforces precedence using parentheses.

SELECT employee_id, manager_id FROM employee
WHERE manager_id IS NULL AND (employee_id = 1 OR
   employee_id = 2);

Comparison Conditions

Comparison conditions allow for different methods of comparison between expressions.

  • expression { [!] = | > | < | >= | <= } expression

    SELECT * FROM projecttype WHERE projecttype_id = 1;
SELECT * FROM projecttype WHERE projecttype_id != 1;
SELECT * FROM projecttype WHERE projecttype_id < 5;
SELECT * FROM projecttype WHERE projecttype_id >= 8;

  • expression [ NOT ] LIKE expression LIKE is a pattern matcher.

    SELECT name FROM projecttype WHERE name LIKE '%a%';
SELECT name FROM projecttype WHERE name LIKE 'R%';
SELECT name FROM projecttype WHERE name LIKE 'R%' OR
   name LIKE 'S%';
SELECT name FROM projecttype WHERE name LIKE 'R%' AND
   name LIKE '%a%';

  • expression [ NOT ] IN expression IN is used to check for set inclusion.

    SELECT projecttype_id FROM projecttype
   WHERE projecttype_id IN (1,2,3,4,5);
SELECT projecttype_id FROM projecttype
   WHERE projecttype_id NOT IN (1,2,3,4,5);

  • [ NOT ] EXISTS (subquery) EXISTS checks for "existence of" giving a Boolean, TRUE or FALSE result.

    SELECT projecttype_id FROM projecttype
      WHERE EXISTS (SELECT * FROM DUAL);
SELECT p.projecttype_id FROM projecttype p
      WHERE EXISTS
         (SELECT projecttype_id FROM projecttype
            WHERE projecttype_id < 10
               AND projecttype_id = p.projecttype_id);

SELECT p.projecttype_id FROM projecttype p
      WHERE EXISTS
         (SELECT project_id FROM project
            WHERE projecttype_id = p.projecttype_id);

  • expression [ NOT ] BETWEEN expression AND expression BETWEEN does a range check much the same as x >= 1 and x <= 10.

    SELECT projecttype_id FROM projecttype
WHERE projecttype_id BETWEEN 1 AND 5;

  • expression [ = | != | > | < | >= | <= ] [ ANY | SOME | ALL ] expression

ANY and SOME verify if any items match and ALL verifies if all items match.

SELECT name FROM projecttype
WHERE projecttype_id = ANY (SELECT projecttype_id FROM
   project);
SELECT name FROM projecttype
WHERE projecttype_id = SOME (SELECT projecttype_id FROM
   project);

The two queries following will produce no rows since every item must match or not match, respectively.

SELECT name FROM projecttype
WHERE projecttype_id = ALL (SELECT projecttype_id FROM
   project);
SELECT name FROM projecttype
WHERE projecttype_id != ALL (SELECT projecttype_id FROM
   project);

Types of SELECT Statements

There are numerous ways in which the SELECT statement can be used. These different methods are best shown by example.

Simple query

SELECT * FROM projecttype;

Filtering Queries using the WHERE Clause

SELECT * FROM projecttype WHERE name LIKE 'R%';

Sorting Queries using the ORDER BY Clause

SELECT * FROM projecttype ORDER BY name;

Joining Tables

The ANSI format has been introduced in order to comply with ANSI standards. Tuning ANSI standard format SQL join statements is an issue that will be covered in Chapter 6.

The syntax of the Oracle SQL proprietary join format is as shown.

SELECT { [ [schema.]table. | [alias.] ]
   { column | expression [, ... ] } | * }
FROM [schema.]table [alias] [, …]
[ WHERE
   [ [schema.]table.|alias.] { column | expression [(+)] }
      comparison condition
   [ [schema.]table.|alias.] { column | expression [(+)] }
   [ { AND | OR } [ NOT ] … ]
]
[ GROUP BY … ]
[ ORDER BY … ];

Note in the syntax diagram above that the (+) or outer join operator is always placed on the side deficient in information.

SELECT di.name, de.name
FROM division di, department de
WHERE di.division_id = de.division_id(+);

The Department table is the table deficient in information and thus rows in the Divisions table without entries in the Department table will be included in the returned result as a left outer join. The term left outer join implies that if the table on the left (the first table in the FROM clause above) contains rows not in the table on the right (the second table in the FROM clause) then rows from the first table will be returned in addition to the intersection.

This next syntax diagram shows the syntax for the ANSI standard join format.

SELECT {{[[schema.]table.|alias.] {column|expression} [, ¼
   ]}|*}
FROM [schema.]table [alias]
[
      CROSS JOIN [schema.]table [alias]
   | NATURAL [INNER | [ LEFT | RIGHT | FULL] OUTER]
         JOIN [schema.]table [alias]
   |{
      [INNER | [LEFT | RIGHT | FULL] OUTER] JOIN
         [schema.]table [alias]
      {
            ON (column = column [{AND | OR} [NOT] column =
               column … ])
         | USING (column [, column … ])
      }
   }
]
[ WHERE … ]
[ GROUP BY … ]
[ ORDER BY … ];

The query following is the equivalent ANSI standard syntax for the Oracle SQL proprietary format of the left outer join query shown above.

SELECT di.name, de.name
FROM division di LEFT OUTER JOIN department de USING
   (division_id);

Types of Joins

  • Cross Join. A cross join is a merge of all rows in both tables where each row in one table is matched with every other row in the second table. A cross join is a Cartesian product.

    SELECT * FROM division, managerof;

  • Inner or Natural Join. An inner join is an intersection between two tables, joining based on a column or column names.

    SELECT * FROM division NATURAL JOIN managerof;

  • Outer Join. An outer join joins rows from two tables. Rows joined are those both in the intersection plus rows in either or both tables, and not in the other table. It is important to remember that for an outer join rows found in one table and not in the other have their columns replaced with NULL values in the joined rows.

    • Left Outer Join. A left outer join joins all intersecting rows plus rows only in the left table.

      SELECT * FROM division NATURAL LEFT OUTER JOIN managerof;

    • Right Outer Join. A right outer join is the opposite of the left outer join: the intersection plus all rows in the right table only.

      SELECT * FROM division NATURAL RIGHT OUTER JOIN
   managerof;

    • Full Outer Join. A full outer join retrieves all rows from both tables.

      SELECT * FROM division NATURAL FULL OUTER JOIN managerof;

  • Self Join. A self join joins a table to itself.

    SELECT manager.name, employee.name
FROM employee manager JOIN employee employee
   ON (employee.manager_id = manager.employee_id);

  • Equi/Anti/Range Joins. These joins use the appropriate comparison conditions to join rows in tables.

    SELECT * FROM division di JOIN department de
   ON (di.division_id = de.division_id);

  • Mutable and Complex Joins. A mutable join is a join of two or more tables and a complex join is a mutable join with extra filtering.

    SELECT di.name, de.name, prj.name
FROM division di
   JOIN department de ON(di.division_id =
      de.division_id)
   JOIN project prj ON(de.department_id =
      prj.department_id)
WHERE di.division_id = 5;

Subqueries

There are specific types of subqueries:

  • Single Row Subquery. A single row subquery returns a single row. Some comparison conditions require a single row with a single column.

    SELECT * FROM project
WHERE projecttype_id =
   (SELECT projecttype_id FROM projecttype
      WHERE projecttype_id = 1);

  • Multiple Row Subquery. A multiple row subquery returns one or more rows.

    SELECT project_id FROM project
WHERE projecttype_id IN
   (SELECT projecttype_id FROM projecttype);

  • Multiple Column Subquery. A multiple column subquery returns many columns.

    SELECT COUNT(*) FROM(SELECT * FROM project);

  • Regular Subquery. A regular subquery executes a subquery in its entirety with no communication with the calling query.

    SELECT * FROM department WHERE division_id IN
   (SELECT division_id FROM division);

  • Correlated Subquery. A correlated subquery can use a value passed from a calling query as a parameter to filter specific rows in the subquery. Values can only be passed from calling to subquery and not the other way around.

    SELECT * FROM division WHERE EXISTS
   (SELECT division_id FROM department
      WHERE division_id = division.division_id);

There are numerous places where subqueries can be used:

  • SELECT clause

    SELECT p.project_id,
   (SELECT projecttype_id FROM projecttype
   WHERE projecttype_id = p.projecttype_id)
FROM project p;

  • WHERE clause (filtering)

    SELECT project_id FROM project
WHERE projecttype_id IN
   (SELECT projecttype_id FROM projecttype);
SELECT division_id FROM division
WHERE division_id IN
   (SELECT division_id FROM department
  WHERE department_id IN
   (SELECT department_id FROM department));

  • ORDER BY clause (sorting)

    SELECT name FROM projecttype
ORDER BY (SELECT name FROM DUAL);

  • HAVING clause (filtering groups)

    SELECT name, COUNT(name) FROM projecttype
GROUP BY name
HAVING name = ANY (SELECT name FROM projecttype);

  • FROM clause

    SELECT COUNT(*) FROM (
   SELECT di.name, de.name, pr.name
   FROM division di
   JOIN department de ON (di.division_id =
      de.division_id)
   JOIN project pr ON (de.department_id =
      pr.department_id));

  • INSERT statement VALUES clause

    INSERT INTO projecttype(projecttype_id, name)
VALUES((SELECT MAX(projecttype_id)+1 FROM projecttype)
   ,'A new project type');

  • UPDATE statement SET = (subquery)

    UPDATE department SET name = (SELECT name FROM
   division
      WHERE division_id = department.division_id);

  • CASE statement expression

This is PL/SQL and PL/SQL syntax is not part of tuning Oracle SQL code.

  • A function parameter

    SELECT LPAD('xxx',10,(SELECT * FROM DUAL)) FROM DUAL;

Table and View Creation

This example uses a subquery to create a new table.

CREATE TABLE namesTable AS
   SELECT di.name AS Division, de.name AS Department,
      pr.name AS Project
   FROM division di
   JOIN department de ON (di.division_id = de.division_id)
      JOIN project pr ON (de.department_id =
         pr.department_id);

This example uses a subquery to create a new view.

CREATE VIEW namesView AS
   SELECT di.name AS Division, de.name AS Department,
         pr.name AS Project
   FROM division di
   JOIN department de ON (di.division_id = de.division_id)
         JOIN project pr ON (de.department_id =
            pr.department_id);

Hierarchical Query

A hierarchical query allows display of hierarchical data in a single table.

SELECT name, employee_id, manager_id FROM employee
   CONNECT BY PRIOR employee_id = manager_id;

Set Operators and Composite Queries

Set operators are used to combine two separate queries into a composite query. Both queries must have the same data types for each column.

  • UNION ALL retrieves all rows from both queries including duplicates. Duplicate rows are rows returned by both queries.

  • UNION is as UNION ALL but duplicate rows are only returned once.

  • INTERSECT returns distinct rows from both queries. An intersection is a little like an inner join.

  • MINUS returns one query less than the other, a little like a left outer join where only distinct rows in the first query are returned.

In order to demonstrate a sensible use of composite queries Figure 5.3 shows a split of the Employees schema Employee table into two tables.

Click To expand
Figure 5.3: Separating the Employee Table into Manager and Employee

Using the two Manager and Employee tables we can use a set operator to join the two tables together into a composite query.

SELECT manager_id, name FROM manager
UNION ALL
SELECT employee_id, name FROM employee;

This query will not work because the sequence of column data types is different between the two queries.

SELECT name, manager_id FROM manager
UNION ALL
SELECT employee_id, name FROM employee;

Flashback

Flashback queries are not particularly relevant to tuning but important enough to include. Flashback simply returns rows at a particular point in time using a system change number (SCN) or a timestamp.

SELECT ¼ AS OF { SCN | TIMESTAMP } expression

Flashback Versions Queries

Oracle Database 10 Grid Flashback queries are improved in Oracle Database 10g such that they can be used to return multiple versions of a query. A flashback versions query can return multiple iterations of the same rows within a period of time.

SELECT ¼ [ VERSIONS BETWEEN { SCN | TIMESTAMP }
   { expression | MINVALUE } AND { expression | MAXVALUE }
AS OF { SCN | TIMESTAMP } expression

Flashback Database

Oracle Database 10 Grid This has little to do with tuning apart from the fact that it might make restoration of a previous database state much easier and faster. The FLASHBACK DATABASE command allows database recovery by allowing the undoing of all changes back to an SCN or timestamp.

Using DISTINCT

DISTINCT retrieves the first value from a repeating group. When there are multiple repeating groups DISTINCT will retrieve the first row from each repeating group. DISTINCT can operate on single or multiple columns.

SELECT COUNT ( [ DISTINCT | ALL ] expression ) ...
SELECT DISTINCT column [, column ... ] ...
SELECT DISTINCT (column [, column ... ]) ...

The DUAL Table

Every DML statement creates an implicit cursor. A cursor is an area in memory allocated for the results of an SQL statement. SELECT statements require a source table for the implicit cursor to operate on. Some SELECT statements do not retrieve from any particular table and thus the DUAL table is a repository for an expression result applied to a single value. The DUAL table acts as a temporary repository for the result of an expression. The result of the expression is selected from the DUAL table.

SELECT * FROM DUAL;
SELECT a.*, b.* FROM DUAL a, projecttype b;
SELECT 'This is a string' FROM DUAL;

The DUAL table also can be used to retrieve Oracle Database constants in a SELECT statement where a constant value is applied to every row.

SELECT SYSDATE FROM DUAL;
SELECT USER FROM DUAL;

NULLs

There are a number of things to remember about NULL:

  • NULL represents nothing.

  • Not even a space character is NULL.

  • NULL is not the same as zero.

  • NULL values are not indexed.

  • Most functions return NULL when passed NULL.

  • Test for NULL using IS [ NOT ] NULL.

  • An expression containing a NULL returns NULL.

  • Use NVL (value, replace) to account for NULL values in expressions.

  • NULL values sort as the highest value.

Pseudocolumns

A pseudocolumn is a virtual column producing a specific value when used in DML. In actuality a pseudocolumn is an expression calculator or a window into some part of the Oracle database.

  • Sequences.

    • CURRVAL (sequence.CURRVAL).

    • NEXTVAL (sequence.NEXTVAL).

  • ROWID provides the fastest access method but can change since it is a relative address. A ROWID is unique across a table and used as the pointer in an index, pointing to a table row. A ROWID is made up of:

    • A tablespace identifier.

    • A datafile block.

    • A data block row.

    • A tablespace datafile number.

  • ROWNUM is an order sequence of number of rows returned by a SELECT statement.

  • LEVEL provides a hierarchical level number in a hierarchical query.

  • XMLDATA allows access to XMLTYPE datatypes.

Using Functions

Functions can be used in most parts of different DML statements. Using functions can affect performance. Functions can be divided into the following general categories:

  • Single row functions

  • Datatype conversion functions

  • Group functions

  • Object reference functions

  • User defined functions

5.2.2 The INSERT Statement

There are two forms of the INSERT statement, which can be used to add rows to a single table or multiple tables. There are various syntax options.

INSERT INTO [schema.]table [ ( column [, column …] ) ]
VALUES ( { expression | DEFAULT | (subquery) }
         [, { expression | DEFAULT | (subquery) } ... ] )
[ RETURNING expression [, expression ] INTO variable
   [, variable ] ];

The specified columns are optional if the VALUES clause specifies columns in existing physical column order.

INSERT INTO projecttype
      VALUES (projecttype_seq.NEXTVAL, 'A new project type');

The RETURNING clause is PL/SQL and will be covered later on in this book.

Multiple Table INSERT Statements

INSERT { FIRST | ALL }
[ WHEN condition THEN ] INTO table [ VALUES (¼) ]
[ WHEN condition THEN ] INTO table [ VALUES (¼) ]
ELSE INTO table [ VALUES (…) ]subquery;

All WHEN conditions are executed on every row of the table. The WHEN condition determines if an INTO table VALUES clause is executed. The ELSE option is executed if all WHEN conditions have failed. Processing is moved to the next row retrieved after processing a passed WHEN condition.

FIRST will cause only the first WHEN condition to add rows. ALL will cause all passing WHEN conditions to be executed thus potentially adding a row to more than a single table, thus a multiple table insertion.

5.2.3 The UPDATE Statement

UPDATE [schema.]table
SET column = expression | (subquery) | DEFAULT
   | ( column [, column … ] ) = (subquery)
[ WHERE ... ]
[ RETURNING expression [, expression ] INTO variable
   [, variable ] ) ];

5.2.4 The DELETE and TRUNCATE Statements

DELETE [ FROM ] [schema.]table
[ WHERE ... ]
[ RETURNING expression [, expression ] INTO variable
   [, variable ] ) ];

The TRUNCATE statement is a DDL command, executes an automatic COMMIT command, and is therefore not undoable.

TRUNCATE [schema.]table table;

5.2.5 The MERGE Statement

The MERGE statement is used to add new rows and update existing rows between two tables as shown in Figure 5.4.

Click To expand
Figure 5.4: The MERGE Command
MERGE [ hint ] INTO [schema.]table [alias]
USING { [schema.] table | view | (subquery) } [ alias ]
ON ( column = column [, column = column … ] )
WHEN MATCHED THEN
   UPDATE SET { column = { column | expression | DEFAULT }
      [ , column = … ]}
WHEN NOT MATCHED THEN
   INSERT ( column [, column … ] )
      VALUES ( { column | expression | DEFAULT }
         [ , column … ] );

The MERGE Statement in Oracle Database 10g

Oracle Database 10 Grid New syntax for the MERGE statement in Oracle Database 10g is highlighted in the following syntax diagram.

MERGE [ hint ] INTO [schema.]table [alias]
USING { [schema.] table | view | (subquery) } [ alias ]
ON ( column = column [, column = column ... ] )
{
WHEN MATCHED THEN
      UPDATE SET { column = { column | expression | DEFAULT }
         [ , column = ... ]}
      [ WHERE filter ]
      [ DELETE WHERE filter ]
|
   WHEN NOT MATCHED THEN
      INSERT ( column [, column ... ] )
         VALUES ( { column | expression | DEFAULT }
            [ , column ... ] )
      [ WHERE filter ]
|
   both UPDATE and INSERT
};

Three changes are apparent:

  1. The addition of WHERE filtering clauses.

  2. Updated rows can be optionally deleted after updating of each row is complete.

  3. The Oracle9i Database syntax required that both INSERT and UPDATE clauses must exist. Both INSERT and UPDATE are now optional where at least one or both must be specified.

    Note

    Oracle Database 10 Grid The MERGE statement is now much more flexible.


5.3 Transaction Control

SQL DML can be controlled for rollback and DDL commands cannot. A transaction is a sequence of SQL DML commands. This sequence of commands can be stored or undone using a COMMIT or ROLLBACK command respectively. A SAVEPOINT command can be used to execute a partial rollback on a transaction back to a SAVEPOINT label.

SAVEPOINT label;
¼
sql commands
¼
ROLLBACK TO SAVEPOINT label;

The SET TRANSACTION command is used to set certain aspects for a transaction or sequence of SQL statements.

SET TRANSACTION
{
      NAME 'transaction'
   | {
            READ { WRITE | ONLY }
         | ISOLATION LEVEL { READ COMMITTED | SERIALIZABLE }
         | USE ROLLBACK SEGMENT rollback segment
      } [ NAME 'transaction' ]
};

The defaults for the SET TRANSACTION command are READ WRITE and ISOLATION LEVEL READ COMMITTED. ISOLATION LEVEL READ COMMITTED will cause the transaction to wait for any competing locks to be released before the transaction is completed. ISOLATION LEVEL SERIALIZABLE will cause a transaction to immediately fail if a lock is encountered.

5.3.1 COMMIT versus ROLLBACK

COMMIT stores changes made in a transaction to the database. ROLLBACK will undo those changes. A quick examination of exactly what the COMMIT and ROLLBACK commands do is warranted, since COMMIT is in general a faster process than a ROLLBACK operation. The reason for this is that it is assumed, and quite sensibly so, that COMMIT is executed much more often than ROLLBACK.

Why is a COMMIT command faster than a ROLLBACK command? What occurs in the database during the processing of these commands?

  • Before COMMIT or ROLLBACK

    • The database is changed.

    • Redo logs are written.

    • Rollback is written.

  • On COMMIT

    • Rollback is deleted.

  • On ROLLBACK

    • Rollback is applied to the database.

    • Rollback is recorded in the redo logs.

    • Rollback is deleted.

Before a COMMIT or ROLLBACK is executed any changes are physically stored in the database. The COMMIT command simply removes any recourse to undoing the changes and ROLLBACK executes everything in reverse. Figure 5.5 shows a vague relationship among the database, logs, and rollback.

Click To expand
Figure 5.5: COMMIT versus ROLLBACK

5.3.2 Transaction Control between Multiple Sessions

Locking can occur between different sessions when changes are made to the same data. If one user changes a row using an UPDATE or DELETE command, without terminating the transaction, then that row or rows will be locked until a COMMIT or ROLLBACK command is executed. A second session attempting to change the same row or rows will not respond until the first session terminates the transaction. Multiple sessions can access the database as shown in Figure 5.6.

Click To expand
Figure 5.6: Locking Rows Before Transaction Completion


5.4 Parallel Queries

In an ideal world parallel queries are useful on multiple CPU platforms when Oracle Partitioning is being used with separate disks or RAID arrays. Generally parallel queries are only an advantage for very large tables or in very large databases such as data warehouses.

Using parallel queries on small highly active concurrent OLTP databases can cause rather than solve performance problems. There will be more on parallel queries and Oracle Partitioning in Chapter 17. Certain types of SQL can be executed in parallel.

  • Any query with at least a single table scan using SELECT, INSERT, UPDATE, and DELETE commands.

  • The CREATE INDEX and ALTER INDEX REBUILD commands.

  • Using the CREATE TABLE command generating a table from a SELECT command.

  • Partitions with local indexes where a local index is an index created on each separate partition.

That is the basics of Oracle SQL DML Data Manipulation Language. In the next chapter we will start to examine the concepts of creating efficient SQL code.


Chapter 6: The Basics of Efficient SQL

Overview

In the previous chapter we examined the basic syntax of SQL in Oracle Database. This chapter will attempt to detail the most simplistic aspects of SQL code tuning. In other words, we are going to discuss what in SQL statements is good for performance and what is not. The approach to performance in this chapter will be based on a purely SQL basis. We want to avoid the nitty-gritty and internal processing occurring in Oracle Database at this stage. It is essential to understand the basic facts about how to write well-performing SQL code first, without considering specific details of Oracle software.

The most important rule of thumb with SQL statements, and particularly SELECT statements, those most subject to tuning, is what is commonly known as the "KISS" rule "Keep It Simple Stupid!" The simpler your SQL statements are the faster they will be. There are two reasons for this. Firstly, simple SQL statements are much more easily tuned and secondly, the Optimizer will function a lot better when assessing less complex SQL code. The negative effect of this is granularity but this negative effect depends on how the application is coded. For instance, connecting to and disconnecting from the database for every SQL code statement is extremely inefficient.

Part of the approach in this chapter is to present SQL performance examples without bombarding the reader with the details of too much theory and reference material. Any reference items such as explanations of producing query plans will be covered later on in this book.

So what this chapter will cover is mostly a general type of SQL code tuning. Thus the title of this chapter: "The Basics of Efficient SQL." Let's start with a brief look at the SELECT statement.


6.1 The SELECT Statement

It is always faster to SELECT exact column names. Thus using the Employees schema

SELECT division_id, name, city, state, country FROM division;

is faster than

SELECT * FROM division;

Also since there is a primary key index on the Division table

SELECT division_id FROM division;

will only read the index file and should completely ignore the table itself. Since the index contains only a single column and the table contains five columns, reading the index is faster because there is less physical space to traverse.

In order to prove these points we need to use the EXPLAIN PLAN command. Oracle Database's EXPLAIN PLAN command allows a quick peek into how the Oracle Database Optimizer will execute an SQL statement, displaying a query plan devised by the Optimizer.

The EXPLAIN PLAN command creates entries in the PLAN_TABLE for a SELECT statement. The resulting query plan for the SELECT statement following is shown after it. Various versions of the query used to retrieve rows from the PLAN_TABLE, a hierarchical query, can be found in Appendix B. In order to use the EXPLAIN PLAN command statistics must be generated. Both the EXPLAIN PLAN command and statistics will be covered in detail in Chapter 9.

EXPLAIN PLAN SET statement_id='TEST' FOR SELECT * FROM
   division;
Query                                Cost     Rows     Bytes
-----------------------------------  ----     ----     -----
SELECT STATEMENT on                     1       10       460
  TABLE ACCESS FULL on DIVISION         1       10       460

One thing important to remember about the EXPLAIN PLAN command is it produces a listed sequence of events, a query plan. Examine the following query and its query plan. The "Pos" or positional column gives a rough guide to the sequence of events that the Optimizer will follow. In general, events will occur listed in the query plan from bottom to top, where additional indenting denotes containment.

EXPLAIN PLAN SET statement_id='TEST' FOR
      SELECT di.name, de.name, prj.name,
         SUM(prj.budget-prj.cost)
      FROM division di JOIN department de USING(division_id)
            JOIN project prj USING(department_id)
      GROUP BY di.name, de.name, prj.name
      HAVING SUM(prj.budget-prj.cost) > 0;
Query                               Pos   Cost    Rows    Bytes
----------------------------------  ---   ----    ----    -----
SELECT STATEMENT on                  97     97     250    17500
 FILTER on                            1
  SORT GROUP BY on                    1     97     250    17500
   HASH JOIN on                       1     24   10000   700000
    TABLE ACCESS FULL on DIVISION     1      1      10      170
    HASH JOIN on                      2      3     100     3600
     TABLE ACCESS FULL on DEPARTMENT  1      1     100     1900
     TABLE ACCESS FULL on PROJECT     2     13   10000   340000

Now let's use the Accounts schema. The Accounts schema has some very large tables. Large tables show differences between the costs of data retrievals more easily. The GeneralLedger table contains over 700,000 rows at this point in time.

In the next example, we explicitly retrieve all columns from the table using column names, similar to using SELECT * FROM GeneralLedger. Using the asterisk probably involves a small overhead in re-interpretation into a list of all column names, but this is internal to Oracle Database and unless there are a huge number of these types of queries this is probably negligible.

EXPLAIN PLAN SET statement_id='TEST' FOR
      SELECT generalledger_id,coa#,dr,cr,dte FROM
         generalledger;

The cost of retrieving 752,740 rows is 493 and the GeneralLedger table is read in its entirety indicated by "TABLE ACCESS FULL".

Query                                 Cost     Rows      Bytes
-----------------------------------   ----   ------   --------
SELECT STATEMENT on                    493   752740   19571240
  TABLE ACCESS FULL on GENERALLEDGER   493   752740   19571240

Now we will retrieve only the primary key column from the GeneralLedger table.

EXPLAIN PLAN SET statement_id='TEST' FOR
      SELECT generalledger_id FROM generalledger;

For the same number of rows the cost is reduced to 217 since the byte value is reduced by reading the index only, using a form of a full index scan. This means that only the primary key index is being read, not the table.

Query                                       Cost     Rows      Bytes
------------------------------------------------   ------   --------
SELECT STATEMENT on                          217   752740    4516440
  INDEX FAST FULL SCAN on XPKGENERALLEDGER   217   752740    4516440

Here is another example using an explicit column name but this one has a greater difference in cost from that of the full table scan. This is because the column retrieved uses an index, which is physically smaller than the index for the primary key. The index on the COA# column is consistently 5 bytes in length for all rows. For the primary key index only the first 9,999 rows have an index value of less than 5 bytes in length.

EXPLAIN PLAN SET statement_id='TEST' FOR
      SELECT coa# FROM generalledger;

Query                                Cost     Rows     Bytes
----------------------------------   ----   ------   -------
SELECT STATEMENT on                     5   752740   4516440
  INDEX FAST FULL SCAN on XFK_GL_COA#   5   752740   4516440

Following are two interesting examples utilizing a composite index. The structure of the index is built as the SEQ# column contained within the CHEQUE_ID column (CHEQUE_ID + SEQ#) and not the other way around. In older versions of Oracle Database this probably would have been a problem. The Oracle9i Database Optimizer is now much improved when matching poorly ordered SQL statement columns to existing indexes. Both examples use the same index. The order of columns is not necessarily a problem in Oracle9i Database.

Note

Oracle Database 10 Grid Oracle Database 10g has Optimizer improvements such as less of a need for SQL code statements to be case sensitive.

EXPLAIN PLAN SET statement_id='TEST' FOR
      SELECT cheque_id, seq# FROM cashbookline;

Query                                     Cost     Rows     Bytes
---------------------------------------   ----   ------   -------
SELECT STATEMENT on                         65   188185   1505480
  INDEX FAST FULL SCAN on XPKCASHBOOKLINE   65   188185   1505480

It can be seen that even with the columns selected in the reverse order of the index, the index is still used.

EXPLAIN PLAN SET statement_id='TEST' FOR
      SELECT seq#,cheque_id FROM cashbookline;

Query                                Cost     Rows     Bytes
----------------------------------   ----   ------   -------
SELECT STATEMENT on                    65   188185   1505480
  INDEX FAST FULL SCAN on
    XPKCASHBOOKLINE                    65   188185   1505480

The GeneralLedger table has a large number of rows. Now let's examine the idiosyncrasies of very small tables. There are some differences between the behavior of SQL when dealing with large and small tables.

In the next example, the Stock table is small and thus the costs of reading the table or the index are the same. The first query, doing the full table scan, reads around 20 times more physical space but the cost is the same. When tables are small the processing speed may not be better when using indexes. Additionally when joining tables the Optimizer may very well choose to full scan a small static table rather than read both index and table. The Optimizer may select the full table scan as being quicker. This is often the case with generic static tables containing multiple types since they are typically read more often.

EXPLAIN PLAN SET statement_id='TEST' FOR SELECT * FROM
   stock;

Query                                Cost     Rows    Bytes
----------------------------------   ----     ----    -----
SELECT STATEMENT on                     1      118     9322
  TABLE ACCESS FULL on STOCK            1      118     9322

EXPLAIN PLAN SET statement_id='TEST' FOR SELECT stock_id
   FROM stock;

Query                                Cost     Rows    Bytes
----------------------------------   ----     ----    -----
SELECT STATEMENT on                     1      118      472
INDEX FULL SCAN on XPKSTOCK             1      118      472

So that is a brief look into how to tune simple SELECT statements. Try to use explicit columns and try to read columns in index orders if possible, even to the point of reading indexes and not tables.

6.1.1 A Count of Rows in the Accounts Schema

I want to show a row count of all tables in the Accounts schema I have in my database. If you remember we have already stated that larger tables are more likely to require use of indexes and smaller tables are not. Since the Accounts schema has both large and small tables, SELECT statements and various clauses executed against different tables will very much affect how those different tables should be accessed in the interest of good performance. Current row counts for all tables in the Accounts schema are shown in Figure 6.1.

Figure 6.1: Row Counts of Accounts Schema Tables
Tip

Accounts schema row counts vary throughout this book since the database is continually actively adding rows and occasionally recovered to the initial state shown in Figure 6.1. Relative row counts between tables remain constant.

6.1.2 Filtering with the WHERE Clause

Filtering the results of a SELECT statement with a WHERE clause implies retrieving only a subset of rows from a larger set of rows. The WHERE clause can be used to either include wanted rows, exclude unwanted rows or both.

Once again using the Employees schema, in the following SQL statement we filter rows to include only those rows we want, retrieving only those rows with PROJECTTYPE values starting with the letter "R".

SELECT * FROM projecttype WHERE name LIKE 'R%';

Now we do the opposite and filter out rows we do not want. We get everything with values not starting with the letter "R".

SELECT * FROM projecttype WHERE name NOT LIKE 'R%';

How does the WHERE clause affect the performance of a SELECT statement? If the sequence of expression comparisons in a WHERE clause can match an index it should. The WHERE clause in the SELECT statement above does not match an index and thus the whole table will be read. Since the Employees schema ProjectType table is small, having only 11 rows, this is unimportant. However, in the case of the Accounts schema, where many of the tables have large numbers of rows, avoiding full table scans, and forcing index reading is important. Following is a single WHERE clause comparison condition example SELECT statement. We will once again show the cost of the query using the EXPLAIN PLAN command. This query does an exact match on a very large table by applying an exact value to find a single row. Note the unique index scan and the low cost of the query.

EXPLAIN PLAN SET statement_id='TEST' FOR
      SELECT * FROM stockmovement WHERE stockmovement_id =
         5000;

Query                                Cost     Rows    Bytes
----------------------------------   ----   ------    -----
SELECT STATEMENT on                     3        1       24
  TABLE ACCESS BY INDEX ROWID on
    STOCKMOVEMENT                       3        1       24
  INDEX UNIQUE SCAN on
    XPKSTOCKMOVEMENT                    2        1

Now let's compare the query above with an example which uses another single column index but searches for many more rows than a single row. This example consists of two queries. The first query gives us the WHERE clause literal value for the second query. The result of the first query is displayed here.

SQL> SELECT coa#, COUNT(coa#) "Rows" FROM generalledger
   GROUP BY coa#;

COA#                 Rows
-----      --------------
30001              310086
40003               66284
41000              173511
50001              169717
60001               33142

Now let's look at a query plan for the second query with the WHERE clause filter applied. The second query shown next finds all the rows in one of the groups listed in the result shown above.

EXPLAIN PLAN SET statement_id='TEST' FOR
      SELECT * FROM generalledger WHERE coa# = 40003;

Query                                Cost     Rows      Bytes
----------------------------------   ----   ------    -------
SELECT STATEMENT on                   493   150548    3914248
  TABLE ACCESS FULL on GENERALLEDGER  493   150548    3914248

The query above has an interesting result because the table is fully scanned. This is because the Optimizer considers it more efficient to read the entire table rather than use the index on the COA# column to find specific columns in the table. This is because the WHERE clause will retrieve over 60,000 rows, just shy of 10% of the entire GeneralLedger table. Over 10% is enough to trigger the Optimizer to execute a full table scan.

In comparison to the above query the following two queries read a very small table, the first with a unique index hit, and the second with a full table scan as a result of the range comparison condition (<). In the second query, if the table were much larger possibly the Optimizer would have executed an index range scan and read the index file. However, since the table is small the Optimizer considers reading the entire table as being faster than reading the index to find what could be more than a single row.

EXPLAIN PLAN SET statement_id='TEST' FOR
      SELECT * FROM category WHERE category_id = 1;

Query                                    Cost   Rows    Bytes
----------------------------------       ----   ----    -----
SELECT STATEMENT on                         1      1       12
  TABLE ACCESS BY INDEX ROWID on CATEGORY   1      1       12
    INDEX UNIQUE SCAN on XPKCATEGORY               1

EXPLAIN PLAN SET statement_id='TEST' FOR
      SELECT * FROM category WHERE category_id < 2;

The costs of both index use and the full table scan are the same because the table is small.

Query                                Cost     Rows    Bytes
----------------------------------   ----     ----    -----
SELECT STATEMENT on                     1        1       12
TABLE ACCESS FULL on CATEGORY           1        1       12

So far we have looked at WHERE clauses containing single comparison conditions. In tables where multiple column indexes exist there are other factors to consider. The following two queries produce exactly the same result. Note the unique index scan on the primary key for both queries. As with the ordering of index columns in the SELECT statement, in previous versions of Oracle it is possible that the same result would not have occurred for the second query. This is because in the past the order of table column comparison conditions absolutely had to match the order of columns in an index. In the past the second query shown would probably have resulted in a full table scan. The Optimizer is now more intelligent in Oracle9i Database.

Note

Oracle Database 10 Grid Oracle Database 10g has Optimizer improvements such as less of a need for SQL code statements to be case sensitive.

We had a similar result previously in this chapter using the CHEQUE_ID and SEQ# columns on the CashbookLine table. The same applies to the WHERE clause.

EXPLAIN PLAN SET statement_id='TEST' FOR
      SELECT * FROM ordersline WHERE order_id = 3137 AND
         seq# = 1;

Query                                Cost   Rows    Bytes
----------------------------------   ----   ----    -----
SELECT STATEMENT on                     3      1       17
  TABLE ACCESS BY INDEX ROWID on
      ORDERSLINE                        3      1       17
    INDEX UNIQUE SCAN on XPKORDERSLINE  2      1

EXPLAIN PLAN SET statement_id='TEST' FOR
      SELECT * FROM ordersline WHERE seq# = 1 AND
         order_id= 3137;

Query                                Cost   Rows    Bytes
----------------------------------   ----   ----    -----
SELECT STATEMENT on                     3      1       17
  TABLE ACCESS BY INDEX ROWID on
      ORDERSLINE                        3      1       17
    INDEX UNIQUE SCAN on XPKORDERSLINE  2      1

Let's now try a different variation. The next example query should only use the second column in the composite index on the STOCK_ID and SUPPLIER_ID columns on the StockSource table. What must be done first is to find a StockSource row uniquely identified by both the STOCK_ID and SUPPLIER_ID columns. Let's simply create a unique row. I have not used sequences in the INSERT statements shown because I want to preserve the values of the sequence objects.

Tip

The names of the columns in the Stock table Stock.MIN and Stock.MAX refer to minimum and maximum Stock item values in the Stock table, not the MIN and MAX Oracle SQL functions.

INSERT INTO stock(stock_id, category_id, text, min, max)
      VALUES((SELECT MAX(stock_id)+1 FROM
         stock),1,'text',1,100);

INSERT INTO supplier(supplier_id, name, ticker)
      VALUES((SELECT MAX(supplier_id)+1 FROM supplier)
      ,'name','TICKER');

INSERT INTO stocksource
      VALUES((SELECT MAX(supplier_id) FROM supplier)
      ,(SELECT MAX(stock_id) FROM stock),100.00);

The INSERT statements created a single row in the StockSource table with the primary key composite index uniquely identifying the first column, the second column, and the combination of both. We can find those unique values by finding the maximum values for them.

SELECT COUNT(stock_id), MAX(stock_id)
FROM stocksource
WHERE stock_id = (SELECT MAX(stock_id) FROM stocksource)
GROUP BY stock_id;

COUNT(STOCK_ID)      MAX(STOCK_ID)
---------------    ---------------
              1                119

SELECT COUNT(supplier_id), MAX(supplier_id)
FROM stocksource
WHERE supplier_id = (SELECT MAX(supplier_id) FROM supplier)
GROUP BY supplier_id;

COUNT(SUPPLIER_ID)    MAX(SUPPLIER_ID)
------------------    ----------------
                1                 3875

Now let's attempt that unique index hit on the second column of the composite index in the StockSource table, amongst other combinations.

EXPLAIN PLAN SET statement_id='TEST' FOR
      SELECT * FROM stocksource WHERE supplier_id = 3875;

Something very interesting happens. The foreign key index on the SUPPLIER_ID column is range scanned because the WHERE clause is matched. The composite index is ignored.

Query                                     Cost   Rows    Bytes
---------------------------------------   ----   ----    -----
1.  SELECT STATEMENT on                      2      3       30
2.    TABLE ACCESS BY INDEX ROWID on
        STOCKSOURCE                          2      3       30
3.     INDEX RANGE SCAN on XFK_SS_SUPPLIER   1      3

The following query uses the STOCK_ID column, the first column in the composite index. Once again, even though the STOCK_ID column is the first column in the composite index the Optimizer matches the WHERE clause against the nonunique foreign key index on the STOCK_ID column. Again the result is a range scan.

EXPLAIN PLAN SET statement_id='TEST' FOR
      SELECT * FROM stocksource WHERE stock_id = 119;

Query                                   Cost   Rows    Bytes
-------------------------------------   ----   ----    -----
1.  SELECT STATEMENT on                     8   102     1020
2.    TABLE ACCESS BY INDEX ROWID on
        STOCKSOURCE                         8   102     1020
3.     INDEX RANGE SCAN on XFK_SS_STOCK     1   102

The next query executes a unique index hit on the composite index because the WHERE clause exactly matches the index.

EXPLAIN PLAN SET statement_id='TEST' FOR
      SELECT * FROM stocksource
      WHERE stock_id = 119 AND supplier_id = 3875;

Query                                       Cost    Rows    Bytes
-----------------------------------------   ----   -----    -----
1.  SELECT STATEMENT on                        2       1       10
2.    TABLE ACCESS BY INDEX ROWID on
        STOCKSOURCE                            2       1       10
3.     INDEX UNIQUE SCAN on XPK_STOCKSOURCE    1   12084

Let's clean up and delete the unique rows we created with the INSERT statements above. My script executing queries (see Appendix B) on the PLAN_TABLE contains a COMMIT command and thus ROLLBACK will not work.

DELETE FROM StockSource WHERE supplier_id = 3875 and
   stock_id = 119;
DELETE FROM supplier WHERE supplier_id = 3875;
DELETE FROM stock WHERE stock_id = 119;
COMMIT;

Now let's do something slightly different. The purpose of creating unique stock and supplier items in the StockSource table was to get the best possibility of producing a unique index hit. If we were to select from the StockSource table where more than a single row existed we would once again not get unique index hits. Depending on the number of rows found we could get index range scans or even full table scans.

Firstly, find maximum and minimum counts for stocks duplicated on the StockSource table.

SELECT * FROM(
      SELECT supplier_id, COUNT(supplier_id) AS suppliers
      FROM stocksource GROUP BY supplier_id ORDER BY
         suppliers DESC)
WHERE ROWNUM = 1
UNION
SELECT * FROM(
      SELECT supplier_id, COUNT(supplier_id) AS suppliers
      FROM stocksource GROUP BY supplier_id ORDER BY
         suppliers)
WHERE ROWNUM = 1;

There are nine suppliers with a SUPPLIER_ID column value of 2711 and one with SUPPLIER_ID column value 2.

SUPPLIER_ID          SUPPLIERS
-------------    -------------
            2                1
         2711                9

Both the next two queries perform index range scans. If one of the queries retrieved enough rows, as in the COA# = '40003' previously shown in this chapter, the Optimizer would force a read of the entire table.

EXPLAIN PLAN SET statement_id='TEST' FOR
      SELECT * FROM stocksource WHERE supplier_id = 2;

Query                                     Cost   Rows    Bytes
---------------------------------------   ----   ----    -----
1.  SELECT STATEMENT on                      2      3       30
2.    TABLE ACCESS BY INDEX ROWID on
        STOCKSOURCE                          2      3       30
3.     INDEX RANGE SCAN on XFK_SS_SUPPLIER   1      3

EXPLAIN PLAN SET statement_id='TEST' FOR
      SELECT * FROM stocksource WHERE supplier_id = 2711;

Query                                     Cost   Rows    Bytes
---------------------------------------   ----   ----    -----
1.  SELECT STATEMENT on                      2      3       30
2.    TABLE ACCESS BY INDEX ROWID on
        STOCKSOURCE                          2      3       30
3.     INDEX RANGE SCAN on XFK_SS_SUPPLIER   1      3

So try to always do two things with WHERE clauses. Firstly, try to match comparison condition column sequences with existing index column sequences, although it is not strictly necessary. Secondly, always try to use unique, single-column indexes wherever possible. A single-column unique index is much more likely to produce exact hits. An exact hit is the fastest access method.

6.1.3 Sorting with the ORDER BY Clause

The ORDER BY clause sorts the results of a query. The ORDER BY clause is always applied after all other clauses are applied, such as the WHERE and GROUP BY clauses. Without an ORDER BY clause in an SQL statement, rows will often be retrieved in the physical order in which they were added to the table. Rows are not always appended to the end of a table as space can be reused. Therefore, physical row order is often useless. Additionally the sequence and content of columns in the SELECT statement, WHERE and GROUP BY clauses can also somewhat determine returned sort order to a certain extent.

In the following example we are sorting on the basis of the content of the primary key index. Since the entire table is being read there is no use of the index. Note the sorting applied to rows retrieved from the table as a result of re-sorting applied by the ORDER BY clause.

EXPLAIN PLAN SET statement_id='TEST' FOR
      SELECT customer_id, name FROM customer ORDER BY
         customer_id;

Query                           Cost   Rows    Bytes     Sort
-----------------------------   ----   ----    -----   ------
SELECT STATEMENT on               25   2694    67350
 SORT ORDER BY on                 25   2694    67350   205000
  TABLE ACCESS FULL on CUSTOMER    9   2694    67350

In the next example the name column is removed from the SELECT statement and thus the primary key index is used. Specifying the CUSTOMER_ID column only in the SELECT statement forces use of the index, not the ORDER BY clause. Additionally there is no sorting because the index is already sorted in the required order. In this case the ORDER BY clause is unnecessary since an identical result would be obtained without it.

EXPLAIN PLAN SET statement_id='TEST' FOR
      SELECT customer_id FROM customer ORDER BY
         customer_id;

Query                           Cost   Rows    Bytes     Sort
-----------------------------   ----   ----    -----   ------
SELECT STATEMENT on                6   2694    10776
 INDEX FULL SCAN on XPKCUSTOMER    6   2694    10776

The next example re-sorts the result by name. Again the whole table is read so no index is used. The results are the same as for the query before the previous one. Again there is physical sorting of the rows retrieved from the table.

EXPLAIN PLAN SET statement_id='TEST' FOR
      SELECT customer_id, name FROM customer ORDER BY name;

Query                           Cost   Rows    Bytes     Sort
-----------------------------   ----   ----    -----   ------
SELECT STATEMENT on               25   2694    67350
 SORT ORDER BY on                 25   2694    67350   205000
  TABLE ACCESS FULL on CUSTOMER    9   2694    67350

The ORDER BY clause will re-sort results.

Note

Oracle Database 10 Grid Queries and sorts are now less case sensitive than in previous versions of Oracle Database.

Overriding WHERE with ORDER BY

The following example is interesting because the primary key composite index is used. Note that there is no sorting in the query plan. It is unnecessary to sort since the index scanned is being read in the order required by the ORDER BY clause. In this case the Optimizer ignores the ORDER BY clause. The WHERE clause specifies ORDER_ID only, for which there is a nonunique foreign key index. A nonunique index is appropriate to a range scan using the < range operator as shown in the example. However, the primary key composite index is used to search with, as specified in the ORDER BY clause. Thus the ORDER BY clause effectively overrides the specification of the WHERE clause. The ORDER BY clause is often used to override any existing sorting parameters.

EXPLAIN PLAN SET statement_id='TEST' FOR
      SELECT * FROM ordersline WHERE order_id < 10
      ORDER BY order_id, seq#;

Query                               Cost   Rows    Bytes
---------------------------------   ----   ----    -----
SELECT STATEMENT on                    4      3       51
 TABLE ACCESS BY INDEX ROWID on
   ORDERSLINE                          4      3       51
  INDEX RANGE SCAN on XPKORDERSLINE    3      3

The second example excludes the overriding ORDER BY clause. Note how the index specified in the WHERE clause is utilized for an index range scan. Thus in the absence of the ORDER BY clause in the previous example the Optimizer resorts to the index specified in the WHERE clause.

EXPLAIN PLAN SET statement_id='TEST' FOR
      SELECT * FROM ordersline WHERE order_id < 10;

Query                                    Cost   Rows    Bytes
--------------------------------------   ----   ----    -----
SELECT STATEMENT on                         3      3       51
 TABLE ACCESS BY INDEX ROWID on
   ORDERSLINE                               3      3       51
  INDEX RANGE SCAN on XFK_ORDERLINE_ORDER   2      3

The next example retains the WHERE clause, containing the first column in the primary key index. It also uses an ORDER BY clause containing only the second column in the composite primary key. This query has a higher cost than both the first and second queries shown before. Why? The Optimizer is retrieving based on the WHERE clause and then being overridden by the ORDER BY clause. What is happening is that the ORDER BY is re-sorting the results of the WHERE clause.

EXPLAIN PLAN SET statement_id='TEST' FOR
      SELECT * FROM ordersline WHERE order_id < 10
      ORDER BY seq#;

Query                                      Cost   Rows    Bytes
----------------------------------------   ----   ----    -----
SELECT STATEMENT on                           5      3       51
 SORT ORDER BY on                             5      3       51
  TABLE ACCESS BY INDEX ROWID on ORDERSLINE   3      3       51
   INDEX RANGE SCAN on XFK_ORDERLINE_ORDER    2      3

In general, it is difficult to demonstrate the performance tuning aspects of the ORDER BY clause. This is because re-sorting is executed after everything else has completed. The ORDER BY clause should not be allowed to conflict with the best Optimizer performance choices of previous clauses. An ORDER BY clause can be used as a refinement of previous clauses rather than replacing those previous clauses. The WHERE clause will filter rows and the ORDER BY re-sorts those filtered rows. The ORDER BY clause can sometimes persuade the Optimizer to use a less efficient key.

In some older relational databases it was always inadvisable to apply any sorting in the ORDER BY which was already sorted by the WHERE clause. In Oracle Database this is not the case. The Oracle Database Optimizer is now intelligent enough to often be able to utilize the best index for searching. Leaving columns out of the ORDER BY clause because they are already covered in the WHERE clause is not necessarily a sound approach. Additionally various other SELECT statement clauses execute sorting automatically. The GROUP BY and DISTINCT clauses are two examples that do inherent sorting. Use inherent sorting if possible rather than doubling up with an ORDER BY clause.

So the ORDER BY clause is always executed after the WHERE clause. This does not mean that the Optimizer will choose either the WHERE clause or the ORDER BY clause as the best performing factor. Try not to override the WHERE clause with the ORDER BY clause because the Optimizer may choose a less efficient method of execution based on the ORDER BY clause.

6.1.4 Grouping Result Sets

The GROUP BY clause can perform some inherent sorting. As with the SELECT statement, WHERE clause and ORDER BY clause, matching of GROUP BY clause column sequences with index column sequences is relevant to SQL code performance.

The first example aggregates based on the non-unique foreign key on the ORDER_ID column. The aggregate is executed on the ORDER_ID column into unique values for that ORDER_ID. The foreign key index is the best performing option.

EXPLAIN PLAN SET statement_id='TEST' FOR
      SELECT order_id, COUNT(order_id) FROM ordersline
      GROUP BY order_id;

The foreign key index is already sorted in the required order. The NOSORT content in the SORT GROUP BY NOSORT on clause implies no sorting is required using the GROUP BY clause.

Query                              Cost     Rows    Bytes
--------------------------------   ----   ------  -------
SELECT STATEMENT on                  26   172304   861520
 SORT GROUP BY NOSORT on             26   172304   861520
  INDEX FULL SCAN on
    XFK_ORDERLINE_ORDER              26   540827  2704135

The next example uses both columns in the primary key index and thus the composite index is a better option. However, since the composite index is much larger in both size and rows the cost is much higher.

EXPLAIN PLAN SET statement_id='TEST' FOR
      SELECT order_id, seq#, COUNT(order_id) FROM ordersline
      GROUP BY order_id, seq#;

Query                                 Cost     Rows     Bytes
-----------------------------------   ----   ------   -------
SELECT STATEMENT on                   1217   540827   4326616
 SORT GROUP BY NOSORT on              1217   540827   4326616
  INDEX FULL SCAN on XPKORDERSLINE    1217   540827   4326616

In the next case we reverse the order of the columns in the GROUP BY sequence. As you can see there is no effect on cost since the Optimizer manages to match against the primary key composite index.

EXPLAIN PLAN SET statement_id='TEST' FOR
      SELECT order_id, seq#, COUNT(order_id) FROM ordersline
      GROUP BY seq#, order_id;

Query                                 Cost     Rows     Bytes
-----------------------------------   ----   ------   -------
SELECT STATEMENT on                   1217   540827   4326616
 SORT GROUP BY NOSORT on              1217   540827   4326616
  INDEX FULL SCAN on XPKORDERSLINE    1217   540827   4326616

Sorting with the GROUP BY Clause

This example uses a non-indexed column to aggregate. Thus the whole table is accessed. Note that NOSORT is no longer included in the SORT GROUP BY clause in the query plan. The GROUP BY clause is now performing sorting on the AMOUNT column.

EXPLAIN PLAN SET statement_id='TEST' FOR
      SELECT amount, COUNT(amount) FROM ordersline
      GROUP BY amount;

Query                            Cost     Rows     Bytes      Sort
------------------------------   ----   ------   -------   -------
SELECT STATEMENT on              4832    62371    374226
 SORT GROUP BY on                4832    62371    374226   7283000
  TABLE ACCESS FULL on
    ORDERSLINE                    261   540827   3244962

Let's examine GROUP BY clause sorting a little further. Sometimes it is possible to avoid sorting forced by the ORDER BY clause by ordering column names in the GROUP BY clause. Rows will be sorted based on the contents of the GROUP BY clause.

EXPLAIN PLAN SET statement_id='TEST' FOR
      SELECT amount, COUNT(amount) FROM ordersline
      GROUP BY amount
      ORDER BY amount;

In this case the ORDER BY clause is ignored.

Query                                 Cost     Rows     Bytes
-----------------------------------   ----   ------   -------
1. SELECT STATEMENT on                6722    62371    374226
2.  SORT GROUP BY on                  6722    62371    374226
3.   TABLE ACCESS FULL on ORDERSLINE  1023   540827   3244962

Inherent sorting in the GROUP BY clause can sometimes be used to avoid extra sorting using an ORDER BY clause.

Using DISTINCT

DISTINCT retrieves the first value from a repeating group. When there are multiple repeating groups DISTINCT will retrieve the first row from each group. Therefore, DISTINCT will always require a sort. DISTINCT can operate on a single or multiple columns. The first example executes the sort in order to find the first value in each group. The second example has the DISTINCT clause removed and does not execute a sort. As a result the second example has a much lower cost. DISTINCT will sort regardless.

EXPLAIN PLAN SET statement_id='TEST' FOR
      SELECT DISTINCT(stock_id) FROM stockmovement;

Query                                    Cost     Rows     Bytes
--------------------------------------   ----   ------   -------
SELECT STATEMENT on                       704      118       472
 SORT UNIQUE on                           704      118       472
  INDEX FAST FULL SCAN on XFK_SM_STOCK      4   570175   2280700

EXPLAIN PLAN SET statement_id='TEST' FOR
      SELECT stock_id FROM stockmovement;

Query                                 Cost     Rows     Bytes
-----------------------------------   ----   ------   -------
SELECT STATEMENT on                      4   570175   2280700
 INDEX FAST FULL SCAN on
  XFK_SM_STOCK                           4   570175   2280700

As far as performance tuning is concerned DISTINCT will always require a sort. Sorting slows performance.

The HAVING Clause

Using the COUNT function as shown in the first two examples there is little difference in performance. The slight difference is due to the application of the filter on the HAVING clause, allowing return of fewer rows. The mere act of using the HAVING clause to return fewer rows helps performance.

EXPLAIN PLAN SET statement_id='TEST' FOR
      SELECT customer_id, COUNT(order_id) FROM orders
      GROUP BY customer_id;

Query                                 Cost     Rows     Bytes
-----------------------------------   ----   ------   -------
SELECT STATEMENT on 298 2693 5386
 SORT GROUP BY on 298 2693 5386
  TABLE ACCESS FULL on ORDERS 112 172304 344608

EXPLAIN PLAN SET statement_id='TEST' FOR
      SELECT customer_id, COUNT(order_id) FROM orders
      GROUP BY customer_id
      HAVING customer_id < 10;

Query                            Cost     Rows     Bytes
------------------------------   ----   ------   -------
SELECT STATEMENT on               296       10        20
 FILTER on
  SORT GROUP BY on                296       10        20
   TABLE ACCESS FULL on ORDERS    112   172304    344608

However, the next two examples using the SUM function as opposed to COUNT have a much bigger difference in cost. This is because the COUNT function is faster, especially when counting on indexes or using the COUNT(*) function with the asterisk option. The COUNT function will be demonstrated in detail later in this chapter. There is a lot of processing that the SUM function does which the COUNT function does not.

EXPLAIN PLAN SET statement_id='TEST' FOR
      SELECT customer_id, SUM(order_id) FROM orders
      GROUP BY customer_id;

Query                            Cost     Rows     Bytes      Sort
------------------------------   ----   ------   -------   -------
SELECT STATEMENT on              1383     2693     18851
 SORT GROUP BY on                1383     2693     18851   2827000
  TABLE ACCESS FULL on
    ORDERS                        112   172304   1206128

EXPLAIN PLAN SET statement_id='TEST' FOR
      SELECT customer_id, SUM(order_id) FROM orders
      GROUP BY customer_id
      HAVING customer_id < 10;

Query                            Cost     Rows     Bytes      Sort
------------------------------   ----   ------   -------   -------
SELECT STATEMENT on               366       10        70
 FILTER on
  SORT GROUP BY on                366       10        70
   TABLE ACCESS FULL on ORDERS    112   172304   1206128

The Spreadsheet Clause

Oracle Database 10 Grid The spreadsheet clause extends the HAVING clause and allows display of data into multiple dimensions allowing calculations between rows much like a spreadsheet program can provide. The spreadsheet clause provides additional OLAP type functionality and is more applicable to data warehousing as opposed to Internet OLTP databases. However, using the spreadsheet clause can in some cases possibly reduce the number of tables in mutable joins and remove the need for set operators such as UNION, INTERSECT, and MINUS to merge multiple queries together.

The HAVING clause filter can help performance because it filters, allowing the return and processing of fewer rows. The HAVING clause filtering shown in the query plans above shows that HAVING clause filtering is always executed after the GROUP BY sorting process.

ROLLUP, CUBE, and GROUPING SETS

The ROLLUP, CUBE, and GROUPING SETS clauses can be used to create breaks and subtotals for groups. The GROUPING SETS clause can be used to restrict the results of ROLLUP and CUBE clauses. Before the advent of ROLLUP and CUBE, producing the same types of results would involve extremely complex SQL statements, probably with the use of temporary tables or perhaps use of PL/SQL as well. ROLLUP, CUBE, and GROUPING SETS are more applicable to reporting and data warehouse functionality.

Note

Oracle Database 10 Grid The spreadsheet clause extension to the HAVING clause is similar in function.

The following examples simply show the use of the ROLLUP, CUBE, and GROUPING SETS clauses.

SELECT type, subtype, SUM(balance+ytd)FROM coa
      GROUP BY type, subtype;

SELECT type, subtype, SUM(balance+ytd)FROM coa
      GROUP BY ROLLUP (type, subtype);

SELECT type, subtype, SUM(balance+ytd)FROM coa
      GROUP BY CUBE (type, subtype);

SELECT type, subtype, SUM(balance+ytd)FROM coa
      GROUP BY GROUPING SETS ((type, subtype), (type),
         (subtype));

In general, the GROUP BY clause can perform some sorting if it matches indexing. Filtering aggregate results with the HAVING clause can help to increase performance by filtering aggregated results of the GROUP BY clause.

6.1.5 The FOR UPDATE Clause

The FOR UPDATE clause is a nice feature of SQL since it allows locking of selected rows during a transaction. There are rare circumstances where rows selected should be locked since there are dependent following changes in a single transaction, requiring selected data to remain the same during the course of that transaction.

SELECT …
   FOR UPDATE OF [ [schema.]table.]column [, ¼ ] ]
      [ NOWAIT | WAIT n ]

Note the two WAIT and NOWAIT options in the preceding syntax. When a lock is encountered NOWAIT forces an abort. The WAIT option will force a wait for a number of seconds. The default simply waits until a row is available.

It should be obvious that with respect to tuning and concurrent multiuser capability of applications the FOR UPDATE clause should be avoided if possible. Perhaps the data model could be too granular thus necessitating the need to lock rows in various tables during the course of a transaction across multiple tables. Using the FOR UPDATE clause is not good for the efficiency of SQL code in general due to potential locks and possible resulting waits for and by other concurrently executing transactions.


6.2 Using Functions

The most relevant thing to say about functions is that they should not be used where you expect an SQL statement to use an index. There are function-based indexes of course. A function-based index contains the resulting value of an expression. An index search against that function-based index will search the index for the value of the expression.

Let's take a quick look at a few specific functions.

6.2.1 The COUNT Function

For older versions of Oracle Database the COUNT function has been recommended as performing better when used in different ways. Prior to Oracle9i Database the COUNT(*) function using the asterisk was the fastest form because the asterisk option was specifically tuned to avoid any sorting. Let's take a look at each of four different methods and show that they are all the same using both the EXPLAIN PLAN command and time testing. We will use the GeneralLedger table in the Accounts schema since it has the largest number of rows.

Notice how all the query plans for all the four following COUNT function options are identical. Additionally there is no sorting on anything but the resulting single row produced by the COUNT function, the sort on the aggregate.

Using the asterisk:

EXPLAIN PLAN SET statement_id='TEST' FOR
      SELECT COUNT(*) FROM generalledger;

Query                                 Cost     Rows
-----------------------------------   ----   ------
1. SELECT STATEMENT on                 382        1
2.  SORT AGGREGATE on                             1
3.   INDEX FAST FULL SCAN on
      XPK_GENERALLEDGER                382   752825

Forcing the use of a unique index:

EXPLAIN PLAN SET statement_id='TEST' FOR
      SELECT COUNT(generalledger_id) FROM
         generalledger;

Query                                           Cost     Rows
---------------------------------------------   ----   ------
1. SELECT STATEMENT on                           382        1
2.  SORT AGGREGATE on                                       1
3.   INDEX FAST FULL SCAN on XPK_GENERALLEDGER   382   752825

Using a constant value:

EXPLAIN PLAN SET statement_id='TEST' FOR
      SELECT COUNT(1) FROM generalledger;

Query                                           Cost     Rows
---------------------------------------------   ----   ------
1. SELECT STATEMENT on                           382        1
2.  SORT AGGREGATE on                                       1
3.   INDEX FAST FULL SCAN on XPK_GENERALLEDGER   382   752825

Using a nonindexed column:

EXPLAIN PLAN SET statement_id='TEST' FOR
      SELECT COUNT(dr) FROM generalledger;

Query                                    Cost     Rows
--------------------------------------   ----   ------
1. SELECT STATEMENT on                    382        1
2.  SORT AGGREGATE on                                1
3.   INDEX FAST FULL SCAN on              382   752825
      XPK_GENERALLEDGER

Now with time testing, below I have simply executed the four COUNT function options with SET TIMING set to ON in SQL*Plus. Executing these four SQL statements twice will assure that all data is loaded into memory and that consistent results are obtained.

SQL> SELECT COUNT(*) FROM generalledger;

COUNT(*)
--------
  752741

Elapsed: 00:00:01.01

SQL> SELECT COUNT(generalledger_id) FROM generalledger;

COUNT(GENERALLEDGER_ID)
-----------------------
                 752741

Elapsed: 00:00:01.01

SQL> SELECT COUNT(1) FROM generalledger;

COUNT(1)
--------
  752741

Elapsed: 00:00:01.01

SQL> SELECT COUNT(dr) FROM generalledger;

COUNT(DR)
---------
   752741

Elapsed: 00:00:01.01

As you can see from the time tests above, the COUNT function will perform the same no matter which method is used. In the latest version of Oracle Database different forms of the COUNT function will perform identically. No form of the COUNT function is better tuned than any other. All forms of the COUNT function perform the same; using an asterisk, a constant or a column, regardless of column indexing, the primary key index is always used.

6.2.2 The DECODE Function

DECODE can be used to replace composite SQL statements using a set operator such as UNION. The Accounts Stock table has a QTYONHAND column. This column denotes how many items of a particular stock item are currently in stock. Negative QTYONHAND values indicate that items have been ordered by customers but not yet received from suppliers.

The first example below uses four full reads of the Stock table and concatenates the results together using UNION set operators.

EXPLAIN PLAN SET statement_id='TEST' FOR
SELECT stock_id||' Out of Stock' FROM stock WHERE
   qtyonhand <=0
UNION
SELECT stock_id||' Under Stocked' FROM stock
      WHERE qtyonhand BETWEEN 1 AND min-1
UNION
SELECT stock_id||' Stocked' FROM stock
WHERE qtyonhand BETWEEN min AND max
UNION
SELECT stock_id||' Over Stocked' FROM stock
      WHERE qtyonhand > max;

Query                            Pos    Cost   Rows    Bytes
------------------------------   ---    ----   ----   ------
SELECT STATEMENT on               12      12    123     1543
 SORT UNIQUE on                    1      12    123     1543
  UNION-ALL on                     1
   TABLE ACCESS FULL on STOCK      1       1      4       32
   TABLE ACCESS FULL on STOCK      2       1      1       11
   TABLE ACCESS FULL on STOCK      3       1     28      420
   TABLE ACCESS FULL on STOCK      4       1     90     1080

This second example replaces the UNION set operators and the four full table scan reads with a single full table scan using nested DECODE functions. DECODE can be used to improve performance.

EXPLAIN PLAN SET statement_id='TEST' FOR
      SELECT stock_id||' '||
      DECODE(SIGN(qtyonhand)
            ,-1,'Out of Stock',0,'Out of Stock'
            ,1,DECODE(SIGN(qtyonhand-min)
                   ,-1,'Under Stocked',0,'Stocked'
                   ,1,DECODE(sign(qtyonhand-max)
                        ,-1,'Stocked',0,'Stocked'
                          ,1,'Over Stocked'
                   )
            )
      ) FROM stock;

Query                            Pos    Cost   Rows    Bytes
------------------------------   ---    ----   ----   ------
SELECT STATEMENT on                1       1    118     1770
TABLE ACCESS FULL on STOCK         1       1    118     1770

Using the DECODE function as a replacement for multiple query set operators is good for performance but should only be used in extreme cases such as the UNION clause joined SQL statements shown previously.

6.2.3 Datatype Conversions

Datatype conversions are a problem and will conflict with existing indexes unless function-based indexes are available and can be created. Generally, if a function is executed in a WHERE clause, or anywhere else that can utilize an index, a full table scan is likely. This leads to inefficiency. There is some capability in Oracle SQL for implicit datatype conversion but often use of functions in SQL statements will cause the Optimizer to miss the use of indexes and perform poorly.

The most obvious datatype conversion concerns dates. Date fields in all the databases I have used are stored internally as a Julian number. A Julian number or date is an integer value from a database-specific date measured in seconds. When retrieving a date value in a tool such as SQL*Plus there is usually a default date format. The internal date value is converted to that default format. The conversion is implicit, automatic, and transparent.

SELECT SYSDATE, TO_CHAR(SYSDATE,'J') "Julian" FROM DUAL;

SYSDATE            Julian
------------   ----------
03-MAR-03         2452702

Now for the sake of demonstration I will create an index on the GeneralLedger DTE column.

CREATE INDEX ak_gl_dte ON GENERALLEDGER(DTE);

Now obviously it is difficult to demonstrate an index hit with a key such as this because the date is a datestamp as well as a simple date. A simple date format such as MM/DD/YYYY excludes a timestamp. Simple dates and datestamps (timestamps) are almost impossible to match. Thus I will use SYSDATE in order to avoid a check against a simple formatted date. Both the GeneralLedger DTE column and SYSDATE are timestamps since the date column in the table was created using SYSDATE-generated values. We are only trying to show Optimizer query plans without finding rows.

The first example hits the new index I created and has a very low cost.

EXPLAIN PLAN SET statement_id='TEST' FOR
      SELECT * FROM generalledger WHERE dte = SYSDATE;

Query                                 Cost     Rows     Bytes
-----------------------------------   ----   ------   -------
SELECT STATEMENT on                      2      593     15418
 TABLE ACCESS BY INDEX ROWID on
   GENERALLEDGER                         2      593     15418
  INDEX RANGE SCAN on AK_GL_DTE          1      593

EXPLAIN PLAN SET statement_id='TEST' FOR
      SELECT * FROM generalledger
      WHERE TO_CHAR(dte, 'YYYY/MM/DD') = '2002/08/21';

This second example does not hit the index because the TO_CHAR datatype conversion is completely inconsistent with the datatype of the index. As a result the cost is much higher.

Query                                 Cost     Rows     Bytes
-----------------------------------   ----   ------   -------
SELECT STATEMENT on                    493     7527    195702
 TABLE ACCESS FULL on GENERALLEDGER    493     7527    195702

Another factor to consider with datatype conversions is making sure that datatype conversions are not placed onto columns. Convert literal values not part of the database if possible. In order to demonstrate this I am going to add a zip code column to my Supplier table, create an index on that zip code column and regenerate statistics for the Supplier table. I do not need to add values to the zip code column to prove my point.

ALTER TABLE supplier ADD(zip NUMBER(5));
CREATE INDEX ak_sp_zip ON supplier(zip);
ANALYZE TABLE supplier COMPUTE STATISTICS;

Now we can show two examples. The first uses an index because there is no datatype conversion on the column in the table and the second reads the entire table because the conversion is on the column.

EXPLAIN PLAN SET statement_id='TEST' FOR
      SELECT * FROM supplier WHERE zip = TO_NUMBER('94002');

Query                                   Cost     Rows     Bytes
-------------------------------------   ----   ------   -------
SELECT STATEMENT on                        1        1       142
 TABLE ACCESS BY INDEX ROWID on SUPPLIER   1        1       142
  INDEX RANGE SCAN on AK_SP_ZIP            1        1

EXPLAIN PLAN SET statement_id='TEST' FOR
      SELECT * FROM supplier WHERE TO_CHAR(zip) = '94002';

Query                                 Cost     Rows     Bytes
-----------------------------------   ----   ------   -------
SELECT STATEMENT on                     13        1       142
 TABLE ACCESS FULL on SUPPLIER          13        1       142

Oracle SQL does not generally allow implicit type conversions but there is some capacity for automatic conversion of strings to integers, if a string contains an integer value. Using implicit type conversions is a very bad programming practice and is not recommended. A programmer should never rely on another tool to do their job for them. Explicit coding is less likely to meet with potential errors in the future.

It is better to be precise since the computer will always be precise and do exactly as you tell it to do. Implicit type conversion is included in Oracle SQL for ease of programming. Ease of program coding is a top-down application to database design approach, totally contradictory to database tuning. Using a database from the point of view of how the application can most easily be coded is not favorable to eventual production performance. Do not use implicit type conversions. As can be seen in the following examples implicit type conversions do not appear to make any difference to Optimizer costs.

EXPLAIN PLAN SET statement_id='TEST' FOR
      SELECT * FROM supplier WHERE supplier_id = 3801;

Query                                      Cost     Rows     Bytes
----------------------------------------   ----   ------   -------
1. SELECT STATEMENT on                        2        1       142
2.  TABLE ACCESS BY INDEX ROWID on SUPPLIER   2        1       142
3.   INDEX UNIQUE SCAN on XPK_SUPPLIER        1     3874

EXPLAIN PLAN SET statement_id='TEST' FOR
      SELECT * FROM supplier WHERE supplier_id = '3801';

Query                                      Cost     Rows     Bytes
----------------------------------------   ----   ------   -------
1. SELECT STATEMENT on                        2        1       142
2.  TABLE ACCESS BY INDEX ROWID on SUPPLIER   2        1       142
3.   INDEX UNIQUE SCAN on XPK_SUPPLIER        1     3874

In short, try to avoid using any type of data conversion function in any part of an SQL statement which could potentially match an index, especially if you are trying to assist performance by matching appropriate indexes.

6.2.4 Using Functions in Queries

Now let's expand on the use of functions by examining their use in all of the clauses of a SELECT statement.

Functions in the SELECT Statement

Firstly, let's put a datatype conversion into a SELECT statement, which uses an index. As we can see in the two examples below, use of the index is not affected by the datatype conversion placed into the SELECT statement.

EXPLAIN PLAN SET statement_id='TEST' FOR
      SELECT customer_id FROM customer;

Query                                 Cost     Rows     Bytes
-----------------------------------   ----   ------   -------
SELECT STATEMENT on                      1     2694     10776
 INDEX FAST FULL SCAN on XPKCUSTOMER     1     2694     10776

EXPLAIN PLAN SET statement_id='TEST' FOR
      SELECT TO_CHAR(customer_id) FROM customer;

Query                                 Cost     Rows     Bytes
-----------------------------------   ----   ------   -------
SELECT STATEMENT on                      1     2694     10776
 INDEX FAST FULL SCAN on XPKCUSTOMER     1     2694     10776

Functions in the WHERE Clause

Now let's examine the WHERE clause. In the two examples below the only difference is in the type of index scan utilized. Traditionally the unique index hit produces an exact match and it should be faster. A later chapter will examine the difference between these two types of index reads.

EXPLAIN PLAN SET statement_id='TEST' FOR
      SELECT customer_id FROM customer WHERE
         customer_id = 100;

Query                                 Cost     Rows     Bytes
-----------------------------------   ----   ------   -------
SELECT STATEMENT on                      1        1         4
 INDEX UNIQUE SCAN on XPKCUSTOMER        1        1         4

EXPLAIN PLAN SET statement_id='TEST' FOR
      SELECT customer_id FROM customer
      WHERE TO_CHAR(customer_id) = '100';

Query                                 Cost     Rows     Bytes
-----------------------------------   ----   ------   -------
SELECT STATEMENT on                      1        1         4
 INDEX FAST FULL SCAN on XPKCUSTOMER     1        1         4

Functions in the ORDER BY Clause

The ORDER BY clause can utilize indexing well, as already seen in this chapter, as long as WHERE clause index matching is not compromised. Let's keep it simple. Looking at the following two examples it should suffice to say that it might be a bad idea to include functions in ORDER BY clauses. An index is not used in the second query and consequently the cost is much higher.

EXPLAIN PLAN SET statement_id='TEST' FOR
      SELECT * FROM generalledger ORDER BY coa#;

Query                               Cost     Rows     Bytes      Sort
---------------------------------   ----   ------   -------   -------
SELECT STATEMENT on                  826   752740  19571240
 TABLE ACCESS BY INDEX ROWID on GL   826   752740  19571240
  INDEX FULL SCAN on XFK_GL_COA#      26   752740

EXPLAIN PLAN SET statement_id='TEST' FOR
      SELECT * FROM generalledger ORDER BY TO_CHAR(coa#);

Query                            Cost     Rows     Bytes      Sort
------------------------------   ----   ------   -------   -------
SELECT STATEMENT on             19070   752740  19571240
 SORT ORDER BY on               19070   752740  19571240  60474000
  TABLE ACCESS FULL on
   GENERALLEDGER                  493   752740  19571240

Here is an interesting twist to using the same datatype conversion in the above two examples but with the conversion in the SELECT statement and setting the ORDER BY clause to sort by position rather than using the TO_CHAR(COA#) datatype conversion. The reason why this example is lower in cost than the second example is because the conversion is done on selection and ORDER BY resorting is executed after data retrieval. In other words, in this example the ORDER BY clause does not affect the data access method.

EXPLAIN PLAN SET statement_id='TEST' FOR
      SELECT TO_CHAR(coa#), dte, dr cr FROM generalledger
         ORDER BY 1;

Query                            Cost     Rows     Bytes      Sort
------------------------------   ----   ------   -------   -------
SELECT STATEMENT on             12937   752740  13549320
 SORT ORDER BY on               12937   752740  13549320  42394000
  TABLE ACCESS FULL on
    GENERALLEDGER                 493   752740  13549320

Functions in the GROUP BY Clause

Using functions in GROUP BY clauses will slow performance as shown in the following two examples.

EXPLAIN PLAN SET statement_id='TEST' FOR
      SELECT order_id, COUNT(order_id) FROM ordersline
      GROUP BY order_id;

Query                                 Cost     Rows     Bytes
-----------------------------------   ----   ------   -------
SELECT STATEMENT on                     26   172304    861520
 SORT GROUP BY NOSORT on                26   172304    861520
  INDEX FULL SCAN on
   XFK_ORDERLINE_ORDER                  26   540827   2704135

EXPLAIN PLAN SET statement_id='TEST' FOR
      SELECT TO_CHAR(order_id), COUNT(order_id) FROM
        ordersline
      GROUP BY TO_CHAR(order_id);

Query                            Cost     Rows     Bytes      Sort
------------------------------   ----   ------   -------   -------
SELECT STATEMENT on              3708   172304    861520
 SORT GROUP BY on                3708   172304    861520   8610000
  INDEX FAST FULL SCAN on
    XFK_ORDERLINE_ORDER             4   540827   2704135

When using functions in SQL statements it is best to keep the functions away from any columns involving index matching.


6.3 Pseudocolumns

There are some ways in which pseudocolumns can be used to increase performance.

6.3.1 Sequences

A sequence is often used to create unique integer identifiers as primary keys for tables. A sequence is a distinct database object and is accessed as sequence.NEXTVAL and sequence.CURRVAL. Using the Accounts schema Supplier table we can show how a sequence is an efficient method in this case.

EXPLAIN PLAN SET statement_id='TEST' FOR
      INSERT INTO supplier (supplier_id, name, ticker)
      VALUES(supplier_seq.NEXTVAL,'A new supplier', 'TICK');

Query                            Cost     Rows     Bytes
------------------------------   ----   ------   -------
INSERT STATEMENT on                 1       11       176
 SEQUENCE on SUPPLIER_SEQ

EXPLAIN PLAN SET statement_id='TEST' FOR
      INSERT INTO supplier (supplier_id, name, ticker)
      VALUES((SELECT MAX(supplier_id)+1
      FROM supplier), 'A new supplier', 'TICK');

Query                            Cost     Rows     Bytes
------------------------------   ----   ------   -------
INSERT STATEMENT on                 1       11       176

The query plan above is the same. There is a problem with it. Notice that a subquery is used to find the next SUPPLIER_ID value. This subquery is not evident in the query plan. Let's do a query plan for the subquery as well.

EXPLAIN PLAN SET statement_id='TEST' FOR
      SELECT MAX(supplier_id)+1 FROM supplier;

Query                            Cost     Rows     Bytes
------------------------------   ----   ------   -------
1. SELECT STATEMENT on              2        1         3
2.  SORT AGGREGATE on 1 3
3.   INDEX FULL SCAN (MIN/MAX)
      on XPK_SUPPLIER               2     3874     11622

We can see that the subquery will cause extra work. Since the query plan seems to have difficulty with subqueries it is difficult to tell the exact cost of using the subquery. Use sequences for unique integer identifiers; they are centralized, more controllable, more easily maintained, and perform better than other methods of counting.

6.3.2 ROWID Pointers

A ROWID is a logically unique database pointer to a row in a table. When a row is found using an index the index is searched. After the row is found in the index the ROWID is extracted from the index and used to find the exact logical location of the row in its respective table. Accessing rows using the ROWID pseudocolumn is probably the fastest row access method in Oracle Database since it is a direct pointer to a unique address. The downside about ROWID pointers is that they do not necessarily point at the same rows in perpetuity because they are relative to datafile, tablespace, block, and row. These values can change. Never store a ROWID in a table column as a pointer to other tables or rows if data or structure will be changing in the database. If ROWID pointers can be used for data access they can be blindingly fast but are not recommended by Oracle Corporation.

6.3.3 ROWNUM

A ROWNUM is a row number or a sequential counter representing the order in which a row is returned from a query. ROWNUM can be used to restrict the number of rows returned. There are numerous interesting ways in which ROWNUM can be used. For instance, the following example allows creation of a table from another, including all constraints but excluding any rows. This is a useful and fast method of making an empty copy of a very large table.

CREATE TABLE tmp AS SELECT * FROM generalledger WHERE
   ROWNUM < 1;

One point to note is as in the following example. A ROWNUM restriction is applied in the WHERE clause. Since the ORDER BY clause occurs after the WHERE clause the ROWNUM restriction is not applied to the sorted output. The solution to this problem is the second example.

SELECT * FROM customer WHERE ROWNUM < 25 ORDER BY
   name;

SELECT * FROM (SELECT * FROM customer ORDER BY name) WHERE
   ROWNUM < 25;


6.4 Comparison Conditions

Different comparison conditions can have sometimes vastly different effects on the performance of SQL statements. Let's examine each in turn with various options and recommendations for potential improvement. The comparison conditions are listed here.

  • Equi, anti, and range

    • expr { [!]= | > | < | <= | >= } expr

    • expr [ NOT ] BETWEEN expr AND expr

  • LIKE pattern matching

    • expr [ NOT ] LIKE expr

  • Set membership

    • expr [ NOT ] IN expr

    • expr [ NOT ] EXISTS expr

      Note

      Oracle Database 10 Grid IN is now called an IN rather than a set membership condition in order to limit confusion with object collection MEMBER conditions.

    • Groups

    • expr [ = | != | > | < | >= | <= ] [ ANY | SOME | ALL ] expr

6.4.1 Equi, Anti, and Range

Using an equals sign (equi) is the fastest comparison condition if a unique index exists. Any type of anti comparison such as != or NOT is looking for what is not in a table and thus must read the entire table; sometimes full index scans can be used. Range comparisons scan indexes for ranges of rows. Let's look at some examples.

This example does a unique index hit; using the equals sign an exact hit single row is found.

EXPLAIN PLAN SET statement_id='TEST' FOR
      SELECT * FROM generalledger WHERE generalledger_id =
         100;

Query                                  Cost     Rows     Bytes
------------------------------------   ----   ------   -------
SELECT STATEMENT on                       3        1        26
 TABLE ACCESS BY INDEX ROWID on
   GENERALLEDGER                          3        1        26
  INDEX UNIQUE SCAN on XPKGENERALLEDGER   2        1

The anti (!=) comparison finds everything but the single row specified and thus must read the entire table.

EXPLAIN PLAN SET statement_id='TEST' FOR
      SELECT * FROM generalledger WHERE generalledger_id !=
         100;

Query                             Pos     Cost     Rows      Bytes
------------------------------   ----   ------   -------   -------
SELECT STATEMENT on                493     493    752739  19571214
TABLE ACCESS FULL on GENERAL         1     493    752739  19571214

In the next case using the range (<) comparison searches a range of index values rather than a single unique index value.

EXPLAIN PLAN SET statement_id='TEST' FOR
      SELECT * FROM generalledger WHERE generalledger_id < 10;

Query                                 Cost     Rows     Bytes
-----------------------------------   ----   ------   -------
SELECT STATEMENT on                      4        1        26
 TABLE ACCESS BY INDEX ROWID on
   GENERALLEDGE                          4        1        26
  INDEX RANGE SCAN on XPKGENERALLEDGER   3        1

In the next example the whole table is read rather than using an index range scan because most of the table will be read and thus the Optimizer considers reading the table as being faster.

EXPLAIN PLAN SET statement_id='TEST' FOR
      SELECT * FROM generalledger WHERE generalledger_id >=
         100;

Query                             Pos   Cost     Rows      Bytes
------------------------------   ----   ----   -------   -------
SELECT STATEMENT on               493    493    752740  19571240
 TABLE ACCESS FULL on GENERAL       1    493    752740  19571240

Here the BETWEEN comparison causes a range scan on an index because the range of rows is small enough to not warrant a full table scan.

EXPLAIN PLAN SET statement_id='TEST' FOR
      SELECT * FROM generalledger
      WHERE generalledger_id BETWEEN 100 AND 200;

Query                                  Cost     Rows     Bytes
------------------------------------   ----   ------   -------
SELECT STATEMENT on                       4        1        26
 TABLE ACCESS BY INDEX ROWID on
   GENERALLEDGE                           4        1        26
  INDEX RANGE SCAN on XPKGENERALLEDGER    3        1

6.4.2 LIKE Pattern Matching

The approach in the query plan used by the Optimizer will depend on how many rows are retrieved and how the pattern match is constructed.

This query finds one row.

EXPLAIN PLAN SET statement_id='TEST' FOR
      SELECT * FROM supplier WHERE name like '24/7 Real
         Media, Inc.';

Query                                  Cost     Rows     Bytes
------------------------------------   ----   ------   -------
SELECT STATEMENT on                       2        1       142
 TABLE ACCESS BY INDEX ROWID on SUPPLIER  2        1       142
  INDEX UNIQUE SCAN on AK_SUPPLIER_NAME   1        1

This query also retrieves a single row but there is a wildcard pattern match and thus a full table scan is the result.

EXPLAIN PLAN SET statement_id='TEST' FOR
      SELECT * FROM supplier WHERE name LIKE '21st%';

Query                                 Cost     Rows     Bytes
-----------------------------------   ----   ------   -------
SELECT STATEMENT on                     13      491     69722
 TABLE ACCESS FULL on SUPPLIER          13      491     69722

The next query finds almost 3,000 rows and thus a full scan of the table results regardless of the exactness of the pattern match.

Tip

A pattern match using a % full wildcard pattern matching character anywhere in the pattern matching string will usually produce a full table scan.

SQL> SELECT COUNT(*) FROM supplier WHERE name LIKE '%a%';

COUNT(*)
--------
    2926

EXPLAIN PLAN SET statement_id='TEST' FOR
      SELECT * FROM supplier WHERE name LIKE '%a%';

Query                                 Cost     Rows     Bytes
-----------------------------------   ----   ------   -------
SELECT STATEMENT on                     13      194     27548
 TABLE ACCESS FULL on SUPPLIER          13      194     27548

In general, since LIKE will match patterns which are in no way related to indexes, LIKE will usually read an entire table.

6.4.3 Set Membership

IN should be used to test against literal values and EXISTS is often used to create a correlation between a calling query and a subquery. IN is best used as a pre-constructed set of literal values. IN will cause a subquery to be executed in its entirety before passing the result back to the calling query. EXISTS will stop once a result is found.

EXPLAIN PLAN SET statement_id='TEST' FOR
      SELECT * FROM coa WHERE type IN ('A','L','I','E');

Query                        Cost     Rows     Bytes
--------------------------   ----   ------   -------
SELECT STATEMENT on             1       38       950
 TABLE ACCESS FULL on COA       1       38       950

There are two advantages to using EXISTS over using IN. The first advantage is the ability to pass values from a calling query to a subquery, never the other way around, creating a correlated query. The correlation allows EXISTS the use of indexes between calling query and subquery, particularly in the subquery. The second advantage of EXISTS is, unlike IN, which completes a subquery regardless, EXISTS will halt searching when a value is found. Thus the subquery can be partially executed, reading fewer rows.

EXPLAIN PLAN SET statement_id='TEST' FOR
      SELECT * FROM coa WHERE EXISTS
            (SELECT type FROM type WHERE type = coa.type);

Query                                 Cost     Rows     Bytes
-----------------------------------   ----   ------   -------
SELECT STATEMENT on                      1       55      1485
 NESTED LOOPS SEMI on                    1       55      1485
  TABLE ACCESS FULL on COA               1       55      1375
  INDEX UNIQUE SCAN on XPKTYPE                    6        12

Now let's compare the use of IN versus the use of EXISTS. The next two examples both use indexes and have the same result. The reason why IN is the same cost as EXISTS is because the query contained within the IN subquery matches an index based on the single column it selects.

EXPLAIN PLAN SET statement_id='TEST' FOR
      SELECT stock_id FROM stock s WHERE EXISTS
      (SELECT stock_id FROM stockmovement WHERE stock_id =
         s.stock_id);

Query                                 Cost     Rows     Bytes
-----------------------------------   ----   ------   -------
SELECT STATEMENT on 119 118 944
 NESTED LOOPS SEMI on 119 118 944
  INDEX FULL SCAN on XPKSTOCK 1 118 472
  INDEX RANGE SCAN on XFK_SM_STOCK 1 570175 2280700

EXPLAIN PLAN SET statement_id='TEST' FOR
      SELECT stock_id FROM stock WHERE stock_id IN
      (SELECT stock_id FROM stockmovement);

Query                                 Cost     Rows     Bytes
-----------------------------------   ----   ------   -------
NESTED LOOPS SEMI on                   119      118       944
 SELECT STATEMENT on                   119      118       944
  INDEX FULL SCAN on XPKSTOCK            1      118       472
  INDEX RANGE SCAN on XFK_SM_STOCK       1   570175   2280700

Now let's do some different queries to show a very distinct difference between IN and EXISTS. Note how the first example is much lower in cost than the second. This is because the second option cannot match indexes and executes two full table scans.

EXPLAIN PLAN SET statement_id='TEST' FOR
      SELECT * FROM stockmovement sm WHERE EXISTS
      (SELECT * FROM stockmovement
            WHERE stockmovement_id = sm.stockmovement_id);

Query                            Cost     Rows     Bytes      Sort
------------------------------   ----   ------   -------   -------
SELECT STATEMENT on              8593   570175  16535075
 MERGE JOIN SEMI on              8593   570175  16535075
  TABLE ACCESS BY INDEX
     ROWID on SM                 3401   570175  13684200
   INDEX FULL SCAN on
     XPKSTOCKMOVEMENT            1071   570175
   SORT UNIQUE on                5192   570175   2850875  13755000
     INDEX FAST FULL SCAN on
     XPKSTMOVE                    163   570175   2850875

EXPLAIN PLAN SET statement_id='TEST' FOR
      SELECT * FROM stockmovement sm WHERE qty IN
         (SELECT qty FROM stockmovement);

Query                            Cost     Rows     Bytes      Sort
------------------------------   ----   ------   -------   -------
SELECT STATEMENT on             16353   570175  15964900
 MERGE JOIN SEMI on             16353   570175  15964900
  SORT JOIN on                  11979   570175  13684200  45802000
   TABLE ACCESS FULL on
    STOCKMOVEMENT                 355   570175  13684200
  SORT UNIQUE on                 4374   570175   2280700  13755000
   TABLE ACCESS FULL on
    STOCKMOVEMENT                 355   570175   2280700

Now let's go yet another step further and restrict the calling query to a single row result. What this will do is ensure that EXISTS has the best possible chance of passing a single row identifier into the subquery, thus ensuring a unique index hit in the subquery. The StockMovement table has been joined to itself to facilitate the demonstration of the difference between using EXISTS and IN. Note how the IN subquery executes a full table scan and the EXISTS subquery does not.

EXPLAIN PLAN SET statement_id='TEST' FOR
      SELECT * FROM stockmovement sm
      WHERE EXISTS(
            SELECT qty FROM stockmovement
            WHERE stockmovement_id = sm.stockmovement_id)
      AND stockmovement_id = 10;

Query                                 Cost     Rows     Bytes
-----------------------------------   ----   ------   -------
1. SELECT STATEMENT on                   2        1        29
2.  NESTED LOOPS SEMI on                 2        1        29
3.   TABLE ACCESS BY INDEX ROWID on
       STOCKMOVEMENT                     2        1        24
4.   INDEX UNIQUE SCAN on
       XPK_STOCKMOVEMENT                 1   570175
3.   INDEX UNIQUE SCAN on
       XPK_STOCKMOVEMENT                 1        5

EXPLAIN PLAN SET statement_id='TEST' FOR
      SELECT * FROM stockmovement sm
      WHERE qty IN (SELECT qty FROM
         stockmovement)
      AND stockmovement_id = 10;

Query                                 Cost     Rows     Bytes
-----------------------------------   ----   ------   -------
1. SELECT STATEMENT on                 563        1        28
2.  NESTED LOOPS SEMI on               563        1        28
3.   TABLE ACCESS BY INDEX ROWID on
      STOCKMOVEMENT                      2        1        24
4.   INDEX UNIQUE SCAN on
      XPK_STOCKMOVEMENT                  1   570175
3.  TABLE ACCESS FULL on
      STOCKMOVEMENT                    561   570175   2280700

The benefit of using EXISTS rather than IN for a subquery comparison is that EXISTS can potentially find much fewer rows than IN. IN is best used with literal values and EXISTS is best used as applying a fast access correlation between a calling and a subquery.

6.4.4 Groups

ANY, SOME, and ALL comparisons are generally not very conducive to SQL tuning. In some respects they are best not used.


6.5 Joins

A join is a combination of rows extracted from two or more tables. Joins can be very specific, for instance an intersection between two tables, or they can be less specific such as an outer join. An outer join is a join returning an intersection plus rows from either or both tables, not in the other table.

This discussion on tuning joins is divided into three sections: join syntax formats, efficient joins, and inefficient joins. Since this book is about tuning it seems sensible to divide joins between efficient joins and inefficient joins.

Firstly, let's take a look at the two different available join syntax formats in Oracle SQL.

6.5.1 Join Formats

There are two different syntax formats available for SQL join queries. The first is Oracle Corporation's proprietary format and the second is the ANSI standard format. Let's test the two formats to see if either format can be tuned to the best performance.

The Oracle SQL proprietary format places join specifications into the WHERE clause of an SQL query. The only syntactical addition to the standard SELECT statement syntax is the use of the (+) or outer join operator. We will deal with tuning outer joins later in this chapter. Following is an example of an Oracle SQL proprietary join formatted query with its query plan, using the Employees schema. All tables are fully scanned because there is joining but no filtering. The Optimizer forces full table reads on all tables because it is the fastest access method to read all the data.

EXPLAIN PLAN SET statement_id='TEST' FOR
      SELECT di.name, de.name, prj.name
      FROM division di, department de, project prj
      WHERE di.division_id = de.division_id
      AND de.department_id = prj.department_id;

Query                                 Cost     Rows     Bytes
-----------------------------------   ----   ------   -------
SELECT STATEMENT on                     23    10000    640000
 HASH JOIN on                           23    10000    640000
  HASH JOIN on                           3      100      3600
   TABLE ACCESS FULL on DIVISION         1       10       170
   TABLE ACCESS FULL on DEPARTMENT       1      100      1900
  TABLE ACCESS FULL on PROJECT          13    10000    280000

The next example shows the same query except using the ANSI standard join format. Notice how the query plan is identical.

EXPLAIN PLAN SET statement_id='TEST' FOR
      SELECT di.name, de.name, prj.name
      FROM division di JOIN department de
         USING(division_id)
            JOIN project prj USING (department_id);

Query                                 Cost     Rows     Bytes
-----------------------------------   ----   ------   -------
SELECT STATEMENT on                     23    10000    640000
 HASH JOIN on                           23    10000    640000
  HASH JOIN on                           3      100      3600
   TABLE ACCESS FULL on DIVISION         1       10       170
   TABLE ACCESS FULL on DEPARTMENT       1      100      1900
  TABLE ACCESS FULL on PROJECT          13    10000    280000

What is the objective of showing the two queries above, including their query plan details? The task of this book is performance tuning. Is either of the two of Oracle SQL proprietary or ANSI join formats inherently faster? Let's try to prove it either way. Once again the Oracle SQL proprietary format is shown below but with a filter added, finding only a single row in the join.

EXPLAIN PLAN SET statement_id='TEST' FOR
      SELECT di.name, de.name, prj.name
      FROM division di, department de, project prj
      WHERE di.division_id = 5
      AND di.division_id = de.division_id
      AND de.department_id = prj.department_id;

Query                                      Cost     Rows     Bytes
----------------------------------------   ----   ------   -------
SELECT STATEMENT on                           4      143      9152
 TABLE ACCESS BY INDEX ROWID on PROJECT       2    10000    280000
  NESTED LOOPS on                             4      143      9152
  NESTED LOOPS on                             2        1        36
   TABLE ACCESS BY INDEX ROWID on DIVISION    1        1        17
    INDEX UNIQUE SCAN on XPKDIVISION                   1
   TABLE ACCESS FULL on DEPARTMENT            1       10       190
  INDEX RANGE SCAN on XFKPROJECT_DEPARTMENT   1    10000

Next is the ANSI standard equivalent of the previous join, including the filter. Two of the most important aspects of tuning SQL join queries are the ability to apply filtering prior to joining tables and specifying the table with the largest filter applied as being the first table in the FROM clause, especially for very large tables. The question is this: Does the ANSI format allow for tuning of joins down to these levels of detail? Is the ANSI format a faster and more tunable option?

EXPLAIN PLAN SET statement_id='TEST' FOR
      SELECT di.name, de.name, prj.name
      FROM division di JOIN department de
            ON(di.division_id = de.division_id)
            JOIN project prj ON(de.department_id =
               prj.department_id)
      WHERE di.division_id = 5;

In the previous join query filtering is visibly applied after the specification of the join. Also note that with the addition of filtering the ON clause rather than the USING clause is required. In the following query plan note that the Optimizer has not changed its plan of execution between the Oracle SQL proprietary and ANSI join formats. There is no difference in performance between Oracle SQL proprietary and ANSI standard join formats.

Query                                    Cost     Rows     Bytes
--------------------------------------   ----   ------   -------
SELECT STATEMENT on                         4      143      9152
TABLE ACCESS BY INDEX ROWID on PROJECT      2    10000    280000
NESTED LOOPS on                             4      143      9152
NESTED LOOPS on                             2        1        36
TABLE ACCESS BY INDEX ROWID on DIVISION     1        1        17
INDEX UNIQUE SCAN on XPKDIVISION            1
TABLE ACCESS FULL on DEPARTMENT             1       10       190
INDEX RANGE SCAN on XFKPROJECT_DEPARTMENT   1    10000

A more visibly tunable join could be demonstrated by retrieving a single row from the largest rather than the smallest table. Here is the Oracle SQL proprietary format.

EXPLAIN PLAN SET statement_id='TEST' FOR
      SELECT di.name, de.name, prj.name
      FROM project prj, department de, division di
      WHERE prj.project_id = 50
      AND de.department_id = prj.department_id
      AND di.division_id = de.division_id;

Notice in the following query plan that the cost is the same but the number of rows and bytes read are substantially reduced; only a single row is retrieved. Since the Project table is being reduced in size more than any other table it appears first in the FROM clause. The same applies to the Department table being larger than the Division table.

Query                                    Cost     Rows     Bytes
--------------------------------------   ----   ------   -------
SELECT STATEMENT on                         4        1        67
 NESTED LOOPS on                            4        1        67
  NESTED LOOPS on                           3        1        50
   TABLE ACCESS BY INDEX ROWID on PROJECT   2        1        31
    INDEX UNIQUE SCAN on XPKPROJECT         1        1
   TABLE ACCESS BY INDEX ROWID on
    DEPARTMENT                              1      100      1900
   INDEX UNIQUE SCAN on XPKDEPARTMENT 100
 TABLE ACCESS BY INDEX ROWID on DIVISION    1       10       170
  INDEX UNIQUE SCAN on XPKDIVISION                  10

Now let's do the same query but with the ANSI join format. From the following query plan we can once again see that use of either the Oracle SQL proprietary or ANSI join format does not appear to make any difference to performance and capacity for tuning.

EXPLAIN PLAN SET statement_id='TEST' FOR
      SELECT di.name, de.name, prj.name
      FROM project prj JOIN department de
            ON(prj.department_id = de.department_id)
            JOIN division di ON(de.division_id =
               di.division_id)
      WHERE prj.project_id = 50;

Query                                    Cost     Rows     Bytes
--------------------------------------   ----   ------   -------
SELECT STATEMENT on                         4        1        67
 NESTED LOOPS on                            4        1        67
  NESTED LOOPS on                           3        1        50
   TABLE ACCESS BY INDEX ROWID on PROJECT   2        1        31
    INDEX UNIQUE SCAN on XPKPROJECT         1        1
   TABLE ACCESS BY INDEX ROWID on
    DEPARTMENT                              1      100      1900
    INDEX UNIQUE SCAN on XPKDEPARTMENT             100
  TABLE ACCESS BY INDEX ROWID on DIVISION   1       10       170
   INDEX UNIQUE SCAN on XPKDIVISION                 10

Let's take this further and do some time testing. We will use the Accounts schema since the Employees schema does not have much data. We want to retrieve more rows to give a better chance of getting a time difference, thus we will not filter on the largest table first. As can be seen from the following results the timing is identical. Perhaps changing the join orders could make subtle differences but there is no reason why the ANSI join format should be considered less tunable.

SQL> SELECT COUNT(*) FROM (
  2 SELECT t.text, st.text, coa.text, gl.dr, gl.cr
  3 FROM type t , subtype st, coa, generalledger gl
  4 WHERE t.type = 'A'
  5 AND coa.type = t. type
  6 AND coa.subtype = st.subtype
  7 AND gl.coa# = coa.coa#);

   COUNT(*)
-----------
     239848

Elapsed: 00:00:04.06

SQL> SELECT COUNT(*) FROM (
  2 SELECT t.text, st.text, coa.text, gl.dr, gl.cr
  3 FROM type t JOIN coa ON(t.type = coa.type)
  4 JOIN subtype st ON(st.subtype = coa.subtype)
  5 JOIN generalledger gl ON(gl.coa# = coa.coa#)
  6 WHERE t.type = 'A');

   COUNT(*)
-----------
     239848

Elapsed: 00:00:04.06

6.5.2 Efficient Joins

What is an efficient join? An efficient join is a join SQL query which can be tuned to an acceptable level of performance. Certain types of join queries are inherently easily tuned and thus can give good performance. In general, a join is efficient when it can use indexes on large tables or is reading only very small tables. Moreover, any type of join will be inefficient if coded improperly.

Intersections

An inner or natural join is an intersection between two tables. In Set parlance an intersection contains all elements occurring in both of the sets, or common to both sets. An intersection is efficient when index columns are matched together in join clauses. Obviously intersection matching not using indexed columns will be inefficient. In that case you may want to create alternate indexes. On the other hand, when a table is very small the Optimizer may conclude that reading the whole table is faster than reading an associated index plus the table. How the Optimizer makes a decision such as this will be discussed in later chapters since this subject matter delves into indexing and physical file block structure in Oracle Database datafiles.

In the example below both of the Type and COA tables are so small that the Optimizer does not bother with the indexes and simply reads both of the tables fully.

EXPLAIN PLAN SET statement_id='TEST' FOR
      SELECT t.text, coa.text FROM type t JOIN coa
         USING(type);

Query                                    Cost     Rows     Bytes
--------------------------------------   ----   ------   -------
SELECT STATEMENT on                         3       55      1430
 HASH JOIN on                               3       55      1430
  TABLE ACCESS FULL on TYPE                 1        6        54
   TABLE ACCESS FULL on COA                 1       55       935

With the next example the Optimizer has done something a little odd by using a unique index on the Subtype table. The Subtype table has only four rows and is extremely small.

EXPLAIN PLAN SET statement_id='TEST' FOR
      SELECT t.text, coa.text FROM type t JOIN coa
         USING(type)
            JOIN subtype st USING(subtype);

Query                                    Cost     Rows     Bytes
--------------------------------------   ----   ------   -------
SELECT STATEMENT on                         3       55      1650
 NESTED LOOPS on                            3       55      1650
  HASH JOIN on                              3       55      1540
   TABLE ACCESS FULL on TYPE                1        6        54
  TABLE ACCESS FULL on COA                  1       55      1045
   INDEX UNIQUE SCAN on XPKSUBTYPE          4        8

Once again in the following example the Optimizer has chosen to read the index for the very small Subtype table. However, the GeneralLedger table has its index read because it is very large and the Optimizer considers that more efficient. The reason for this is that the GeneralLedger table does have an index on the COA# column and thus the index is range scanned.

EXPLAIN PLAN SET statement_id='TEST' FOR
      SELECT t.text, coa.text
      FROM type t JOIN coa USING(type)
            JOIN subtype st USING(subtype)
                  JOIN generalledger gl ON(gl.coa# =
                     coa.coa#);

Query                                    Cost     Rows     Bytes
--------------------------------------   ----   ------   -------
SELECT STATEMENT on                        58   752740  31615080
 NESTED LOOPS on                           58   752740  31615080
  NESTED LOOPS on                           3       55      1980
   HASH JOIN on                             3       55      1870
    TABLE ACCESS FULL on TYPE               1        6        54
    TABLE ACCESS FULL on COA                1       55      1375
   INDEX UNIQUE SCAN on XPKSUBTYPE          4        8
  INDEX RANGE SCAN on XFK_GL_COA#           1   752740   4516440

The most efficient type of inner join will generally be one retrieving very specific rows such as that in the next example. Most SQL is more efficient when retrieving very specific, small numbers of rows.

EXPLAIN PLAN SET statement_id='TEST' FOR
      SELECT t.text, st.text, coa.text, gl.dr, gl.cr
      FROM generalledger gl JOIN coa ON(gl.coa# = coa.coa#)
            JOIN type t ON(t.type = coa.type)
                  JOIN subtype st ON(st.subtype =
                     coa.subtype)
      WHERE gl.generalledger_id = 100;

Note how all tables in the query plan are accessed using unique index hits.

Query                                  Pos     Cost     Rows      Bytes
-----------------------------------   ----   ------   -------   -------
SELECT STATEMENT on                      6        6         1        64
 NESTED LOOPS on                         1        6         1        64
  NESTED LOOPS on                        1        5         1        55
   NESTED LOOPS on                       1        4         1        45
    TABLE ACCESS BY INDEX ROWID on GENE  1        3         1        20
     INDEX UNIQUE SCAN on XPKGENERALLED  1        2         1
    TABLE ACCESS BY INDEX ROWID on COA   2        1        55      1375
     INDEX UNIQUE SCAN on XPKCOA         1                 55
   TABLE ACCESS BY INDEX ROWID on SUBTY  2        1         4        40
    INDEX UNIQUE SCAN on XPKSUBTYPE      1       4
  TABLE ACCESS BY INDEX ROWID on TYPE    2        1         6        54
   INDEX UNIQUE SCAN on XPKTYPE          1        6

Self Joins

A self join joins a table to itself. Sometimes self-joining tables can be handled with hierarchical queries. Otherwise a self join is applied to a table containing columns within each row which link to each other. The Employee table in the Employees schema is such a table. Since both the MANAGER_ID and EMPLOYEE_ID columns are indexed it would be fairly efficient to join the tables using those two columns.

EXPLAIN PLAN SET statement_id='TEST' FOR
      SELECT manager.name, employee.name
      FROM employee manager, employee employee
      WHERE employee.manager_id = manager.employee_id;

In the query plan the Employee table is fully scanned twice because all the data is read and the Optimizer considers this faster because the Employee table is small.

Query                                 Cost     Rows     Bytes
-----------------------------------   ----   ------   -------
SELECT STATEMENT on                      3      110      2970
 HASH JOIN on                            3      110      2970
  TABLE ACCESS FULL on EMPLOYEE          1      111      1554
   TABLE ACCESS FULL on EMPLOYEE         1      111      1443

Equi-Joins and Range Joins

An equi-join uses the equals sign (=) and a range join uses range operators (<, >, <=, >=) and the BETWEEN operator. In general, the = operator will execute an exact row hit on an index and thus use unique index hits. The range operators will usually require the Optimizer to execute index range scans. BTree (binary tree) indexes, the most commonly used indexes in Oracle Database, are highly amenable to range scans. A BTree index is little like a limited depth tree and is optimized for both unique hits and range scans.

Going back into the Accounts schema, this first query uses two unique index hits. The filter helps that happen.

EXPLAIN PLAN SET statement_id='TEST' FOR
      SELECT coa.*, gl.*
      FROM generalledger gl JOIN coa ON(gl.coa# = coa.coa#)
      WHERE generalledger_id = 10;

Query                                   Cost     Rows     Bytes
-------------------------------------   ----   ------   -------
SELECT STATEMENT on                        4        1        51
 NESTED LOOPS on                           4        1        51
  TABLE ACCESS BY INDEX ROWID on
    GENERALLEDG                            3        1        26
   INDEX UNIQUE SCAN on XPKGENERALLEDGER   2        1
  TABLE ACCESS BY INDEX ROWID on COA       1       55      1375
   INDEX UNIQUE SCAN on XPKCOA                     55

This second query uses a range index scan on the GeneralLedger table as a result of the range operator in the filter. Do you notice that the join clause inside the ON clause is where the range join operator is placed? Well there isn't really much point in joining ON (gl.coa# >= coa.coa#). I do not think I have ever seen an SQL join joining using a range operator. The result would be a very unusual type of outer join perhaps. Thus there is no need for a query plan.

EXPLAIN PLAN SET statement_id='TEST' FOR
      SELECT coa.*, gl.*
      FROM generalledger gl JOIN coa ON(gl.coa# >= coa.coa#)
      WHERE generalledger_id = 10;
Tip

A Cartesian product is generally useless in a relational database, so is a range join.

6.5.3 Inefficient Joins

What is an inefficient join? An inefficient join is an SQL query joining tables which is difficult to tune or cannot be tuned to an acceptable level of performance. Certain types of join queries are inherently both poor performers and difficult if not impossible to tune. Inefficient joins are best avoided.

Cartesian Products

The ANSI join format calls a Cartesian product a Cross Join. A Cross Join is only tunable as far as columns selected match indexes such that rows are retrieved from indexes and not tables.

The following second query has a lower cost than the first because the selected columns match indexes on both tables. I have left the Rows and Bytes columns in the query plans as overflowed numbers replaced with a string of # characters. This is done to stress the pointlessness of using a Cartesian product in a relational database.

EXPLAIN PLAN SET statement_id='TEST' FOR
      SELECT * FROM coa, generalledger;

Query                                 Cost     Rows       Bytes
-----------------------------------   ----   ------     -------
SELECT STATEMENT on                  27116  #######   #########
 MERGE JOIN CARTESIAN on             27116  #######   #########
  TABLE ACCESS FULL on COA               1       55        1375
   BUFFER SORT on                    27115   752740    19571240
    TABLE ACCESS FULL on
     GENERALLEDGER                     493   752740    19571240

EXPLAIN PLAN SET statement_id='TEST' FOR
      SELECT coa.coa#, gl.generalledger_id FROM coa,
         generalledger gl;

Query                                 Cost     Rows       Bytes
-----------------------------------   ----   ------     -------
SELECT STATEMENT on                  11936  #######   #########
 MERGE JOIN CARTESIAN on             11936  #######   #########
  INDEX FULL SCAN on XPKCOA              1       55         330
  BUFFER SORT on                     11935   752740     4516440
   INDEX FAST FULL SCAN on
    XPKGENERALLEDGER                   217   752740     4516440

Outer Joins

Tuning an outer join requires the same approach to tuning as with an inner join. The only point to note is that if applications require a large quantity of outer joins there is probably potential for data model tuning. The data model could be too granular. Outer joins are probably more applicable to reporting and data warehouse type applications.

An outer join is not always inefficient. The performance and to a certain extent the indication of a need for data model tuning depends on the ratio of rows retrieved from the intersection to rows retrieved outside the intersection. The more rows retrieved from the intersection the better.

My question is this: Why are outer joins needed? Examine the data model first.

Anti-Joins

An anti-join is always a problem. An anti-join simply does the opposite of a requirement. The result is that the Optimizer must search for everything not meeting a condition. An anti-join will generally always produce a full table scan as seen in the first example following. The second example uses one index because indexed columns are being retrieved from one of the tables. Again the Rows and Bytes columns are left as overflowing showing the possibly folly of using anti-joins.

EXPLAIN PLAN SET statement_id='TEST' FOR
      SELECT t.text, coa# FROM type t, coa WHERE
         t.type != coa.type;

Query                                 Cost     Rows     Bytes
-----------------------------------   ----   ------   -------
SELECT STATEMENT on                      7      275      4675
 NESTED LOOPS on                         7      275      4675
  TABLE ACCESS FULL on TYPE              1        6        54
   TABLE ACCESS FULL on COA              1       55       440

EXPLAIN PLAN SET statement_id='TEST' FOR
      SELECT coa.coa#, gl.generalledger_id FROM coa,
         generalledger gl
      WHERE coa.coa# != gl.coa#;

Query                       Pos     Cost     Rows        Bytes
------------------------   ----   ------   -------     -------
SELECT STATEMENT on       27116    27116   #######   #########
 NESTED LOOPS on              1    27116   #######   #########
  INDEX FULL SCAN on XPKCOA   1        1        55         330
   TABLE ACCESS FULL on
     GENERALLEDGER            2      493    752740     9032880

Mutable and Complex Joins

A mutable join is a join of more than two tables. A complex join is a mutable join with added filtering. We have already examined a complex mutable join in the section on intersection joins and various other parts of this chapter.

6.5.4 How to Tune a Join

So how can a join be tuned? There are a number of factors to consider.

  • Use equality first.

  • Use range operators only where equality does not apply.

  • Avoid use of negatives in the form of != or NOT.

  • Avoid LIKE pattern matching.

  • Try to retrieve specific rows and in small numbers.

  • Filter from large tables first to reduce rows joined. Retrieve tables in order from the most highly filtered table downwards; preferably the largest table has the most filtering applied.

    Tip

    The most highly filtered table is the table having the smallest percentage of its rows retrieved, preferably the largest table.

  • Use indexes wherever possible except for very small tables.

  • Let the Optimizer do its job.


6.6 Using Subqueries for Efficiency

Tuning subqueries is a highly complex topic. Quite often subqueries can be used to partially replace subset parts of very large mutable joins, with possible enormous performance improvements.

6.6.1 Correlated versus Noncorrelated Subqueries

A correlated subquery allows a correlation between a calling query and a subquery. A value for each row in the calling query is passed into the subquery to be used as a constraint by the subquery. A noncorrelated or regular subquery does not contain a correlation between calling query and subquery and thus the subquery is executed in its entirety, independently of the calling query, for each row in the calling query. Tuning correlated subqueries is easier because values in subqueries can be precisely searched for in relation to each row of the calling query.

A correlated subquery will access a specified row or set of rows for each row in the calling query. Depending on circumstances a correlated subquery is not always faster than a noncorrelated subquery. Use of indexes or small tables inside a subquery, even for noncorrelated subqueries, does not necessarily make a subquery perform poorly.

6.6.2 IN versus EXISTS

We have already seen substantial use of IN and EXISTS in the section on comparison conditions. We know already that IN is best used for small tables or lists of literal values. EXISTS is best used to code queries in a correlated fashion, establishing a link between a calling query and a subquery. To reiterate it is important to remember that using EXISTS is not always faster than using IN.

6.6.3 Nested Subqueries

Subqueries can be nested where a subquery can call another subquery. The following example using the Employees schema shows a query calling a subquery, which in turn calls another subquery.

EXPLAIN PLAN SET statement_id='TEST' FOR
SELECT * FROM division WHERE division_id IN
      (SELECT division_id FROM department WHERE
         department_id IN
               (SELECT department_id FROM project));

Notice in the query plan how the largest table is scanned using an INDEX FAST FULL SCAN. The Optimizer is intelligent enough to analyze this nested query and discern that the Project table is much larger than both of the other two tables. The other two tables are so small that the only viable option is a full table scan.

Query                                 Cost     Rows     Bytes
-----------------------------------   ----   ------   -------
SELECT STATEMENT on                     14       10       590
 HASH JOIN SEMI on                      14       10       590
  TABLE ACCESS FULL on DIVISION          1       10       460
  VIEW on VW_NSO_1                       8    10000    130000
   HASH JOIN on                          8    10000     60000
    TABLE ACCESS FULL on DEPARTMENT      1      100       400
    INDEX FAST FULL SCAN on
      XFKPROJECT_DEPT                    4    10000     20000

Nested subqueries can be difficult to tune but can often be a viable and sometimes highly effective tool for the tuning of mutable complex joins, with three and sometimes many more tables in a single join. There is a point when there are so many tables in a join that the Optimizer can become less effective.

6.6.4 Replacing Joins with Subqueries

For very large complex mutable joins it is often possible to replace joins or parts of joins with subqueries. Very large joins can benefit the most because they are difficult to decipher and tune. Some very large joins are even beyond the intelligence of the Optimizer to assess in the best possible way. Two ways in which subqueries can replace joins in complex mutable joins are as follows:

  • A table in the join not returning a column in the primary calling query can be removed from the join and checked using a subquery.

  • FROM clauses can contain nested subqueries to break up joins much in the way that PL/SQL would use nested looping cursors.

Certain aspects of SQL coding placed in subqueries can cause problems:

  • An ORDER BY clause is always applied to a final result and should not be included in subqueries if possible.

  • DISTINCT will always cause a sort and is not always necessary. Perhaps a parent table could be used where a unique value is present.

  • When testing against subqueries retrieve, filter, and aggregate on indexes not tables. Indexes usually offer better performance.

  • Do not be too concerned about full table scans on very small static tables.

    Tip

    Instances where joins can be replaced with subqueries often involve databases with heavy outer join requirements. Excessive use of SQL outer joins is possibly indicative of an over-granular data model structure. However, it could also indicate orphaned child table rows or the opposite: redundant static data. Cleaning out redundant or orphaned rows can sometimes help performance immensely by negating the need for outer joins.

Remove Tables Without Returned Columns Using EXISTS

Going back to the Accounts schema once again look at the following complex mutable join. We are joining four tables and selecting a column from only one of the tables. EXISTS comparisons can be placed into the WHERE clause to force index access, removing three tables from the join.

EXPLAIN PLAN SET statement_id='TEST' FOR
      SELECT c.name
      FROM customer c JOIN orders o USING(customer_id)
            JOIN ordersline ol USING(order_id)
                  JOIN transactions t USING(customer_id)
                        JOIN transactionsline tl
                           USING(transaction_id)
      WHERE c.balance > 0;

Its query plan is a little scary. There are three full table scans and two full index scans. The objective is to remove full table scans and change as many index scans as possible into unique index scans.

Query                                 Cost     Rows       Bytes
---------------------------------   ------   ------   ---------
SELECT STATEMENT on                 ######  #######   #########
 MERGE JOIN on                      ######  #######   #########
  SORT JOIN on                      ######  #######   #########
   MERGE JOIN on                      4988  #######   #########
    SORT JOIN on                      3209   100136     3805168
     HASH JOIN on                      762   100136     3805168
      TABLE ACCESS FULL on
        CUSTOMER                         9     2690      69940
      MERGE JOIN on                    316   100237    1202844
       INDEX FULL SCAN on
        XFK_ORDERLINE_ORDER             26   540827    2704135
       SORT JOIN on                    290    31935     223545
        TABLE ACCESS FULL on
         ORDERS                        112    31935     223545
   SORT JOIN on                       1780   188185    1317295
    TABLE ACCESS FULL on
      TRANSACTIONS                     187   188185    1317295
SORT JOIN on                          5033   570175    2850875
 INDEX FAST FULL SCAN on
   XFK_TRANSLINE_TRANS                   4   570175    2850875

Only the Customer.NAME column is selected. This query is an extreme case but we can actually remove every table from the join except the Customer table. Let's show this in two stages. Firstly, I will remove the transaction tables from the join.

EXPLAIN PLAN SET statement_id='TEST' FOR
      SELECT c.name
      FROM customer c JOIN orders o ON(c.customer_id =
         o.customer_id)
            JOIN ordersline ol USING(order_id)
      WHERE c.balance > 0
      AND EXISTS(
            SELECT t.transaction_id FROM transactions t
            WHERE t.customer_id = c.customer_id
            AND EXISTS(
                  SELECT transaction_id FROM
                     transactionsline
                  WHERE transaction_id = t.transaction_id
            )
      );

We have now reduced the full table scans to two, have a single full index scan and most importantly index range scans on both of the transaction tables.

Query                                 Cost     Rows       Bytes
---------------------------------   ------   ------   ---------
SELECT STATEMENT on                    359     5007      190266
 FILTER on
  HASH JOIN on                         359     5007      190266
   TABLE ACCESS FULL on CUSTOMER         9     135        3510
   MERGE JOIN on                       316   100237     1202844
    INDEX FULL SCAN on
      XFK_ORDERLINE_ORDER               26   540827     2704135
    SORT JOIN on                       290    31935      223545
     TABLE ACCESS FULL on ORDERS       112    31935      223545
  NESTED LOOPS on                       72      212        2544
   TABLE ACCESS BY INDEX ROWID on
     TRANSACTIONS                        2        70        490
    INDEX RANGE SCAN on
     XFX_TRANS_CUSTOMER                  1        70
  INDEX RANGE SCAN on
    XFK_TRANSLINE_TRANS                  1     570175 2850875

Now let's get completely ridiculous and remove every table from the join but the Customer table.

EXPLAIN PLAN SET statement_id='TEST' FOR
      SELECT c.name FROM customer c
      WHERE c.balance > 0
      AND EXISTS(
            SELECT o.order_id FROM orders o
            WHERE o.customer_id = c.customer_id
            AND EXISTS(
                  SELECT order_id FROM ordersline
                  WHERE order_id = o.order_id
            )
      )
      AND EXISTS(
            SELECT t.transaction_id FROM transactions t
            WHERE t.customer_id = c.customer_id
            AND EXISTS(
                  SELECT transaction_id FROM
                     transactionsline
                  WHERE transaction_id = t.transaction_id
            )
      );

This is about the best that can be done with this query, now no longer a join. This final result has full table access on the Customer table only, along with four index range scans. We could possibly improve the query further by decreasing the number of Customer rows retrieved using filtering.

Query                                 Cost     Rows       Bytes
---------------------------------   ------   ------   ---------
SELECT STATEMENT on                      9        7         182
 FILTER on
  TABLE ACCESS FULL on CUSTOMER          9        7         182
  NESTED LOOPS on                       66      201        2412
   TABLE ACCESS BY INDEX ROWID on
     ORDERS                              2       64         448
    INDEX RANGE SCAN on
      XFK_ORDERS_CUSTOMER                1       64
   INDEX RANGE SCAN on
     XFK_ORDERLINE_ORDER                 1   540827     2704135
  NESTED LOOPS on                       72      212        2544
   TABLE ACCESS BY INDEX ROWID on
     TRANSACTIONS                        2       70         490
    INDEX RANGE SCAN on
      XFX_TRANS_CUSTOMER                 1       70
   INDEX RANGE SCAN on
     XFK_TRANSLINE_TRANS                 1   570175     2850875

FROM Clause Subquery Nesting

Now what we want to do is to retrieve columns from different tables. Columns cannot be retrieved from an EXISTS comparison in the WHERE clause. We have to use another method. Nested subqueries in the FROM clause allow retrieval of columns.

In this example I am adding extra filtering to the TransactionsLine table; at over 500,000 rows it is the largest table in the query. Since the TransactionsLine table is larger than the Customer table it is filtered first.

EXPLAIN PLAN SET statement_id='TEST' FOR
      SELECT c.name, tl.amount FROM customer c
         JOIN orders o USING(customer_id)
            JOIN ordersline ol USING(order_id)
               JOIN transactions t USING(customer_id)
                  JOIN transactionsline tl USING(transaction_id)
      WHERE tl.amount > 3170
      AND c.balance > 0;

We start with three full table scans, one index range scan and a unique index hit.

Query                                 Cost     Rows       Bytes
---------------------------------   ------   ------   ---------
SELECT STATEMENT on                   1860      605       33880
 NESTED LOOPS on                      1860      605       33880
  HASH JOIN on                        1667      193        9843
   TABLE ACCESS FULL on ORDERS         112    31935      223545
   MERGE JOIN CARTESIAN on             436    43804     1927376
    NESTED LOOPS on                    292       16         288
     TABLE ACCESS FULL on
       TRANSACTIONSLINE                276       16         176
     TABLE ACCESS BY INDEX ROWID on
       TRANSACTIO                        1   188185     1317295
      INDEX UNIQUE SCAN on
        XPKTRANSACTIONS                      188185
    BUFFER SORT on                     435     2690       69940
     TABLE ACCESS FULL on CUSTOMER       9     2690       69940
  INDEX RANGE SCAN on
    XFK_ORDERLINE_ORDER                  1   540827     2704135

Firstly, some appropriate simple tuning can be done. Since the TransactionsLine table is the largest table with the smallest relative filtered result, it should be selected from first.

Tip

The first table to be processed should be the largest table with the largest relative row reduction filter. In other words, the biggest table with the lowest number of rows retrieved from it. This applies to both the FROM clause and the WHERE clause. Always reduce rows to be joined first.

EXPLAIN PLAN SET statement_id='TEST' FOR
      SELECT c.name, tl.amount FROM transactionsline tl
         JOIN transactions t USING(transaction_id)
            JOIN customer c USING(customer_id)
               JOIN orders o USING(customer_id)
                  JOIN ordersline ol ON(ol.order_id = o.order_id)
      WHERE tl.amount > 3170
      AND c.balance > 0;

Appropriate simple tuning yields one full table scan, two index range scans, and two unique index hits.

Query                                 Cost     Rows       Bytes
---------------------------------   ------   ------   ---------
SELECT STATEMENT on                   1381     3267      182952
 NESTED LOOPS on                      1381     3267      182952
  NESTED LOOPS on                      340     1041       53091
   NESTED LOOPS on                     308       16         704
    NESTED LOOPS on                    292       16         288
     TABLE ACCESS FULL on
       TRANSACTIONSLINE                276       16         176
     TABLE ACCESS BY INDEX ROWID
       on TRANSACTIO                     1    33142      231994
      INDEX UNIQUE SCAN on
        XPKTRANSACTIONS  33142
    TABLE ACCESS BY INDEX ROWID on
      CUSTOMER                           1     2690       69940
     INDEX UNIQUE SCAN on
       XPKCUSTOMER  2690
   TABLE ACCESS BY INDEX ROWID on
     ORDERS                              2   172304     1206128
    INDEX RANGE SCAN on
      XFK_ORDERS_CUSTOMER                1   172304
  INDEX RANGE SCAN on
    XFK_ORDERLINE_ORDER                  1   540827     2704135

Now let's use the FROM clause to create nested subqueries. The trick is to put the largest table with the most severe filter at the deepest nested level, forcing it to execute first. Thus we start with the TransactionsLine table.

EXPLAIN PLAN SET statement_id='TEST' FOR
      SELECT c.name, b.amount
      FROM customer c,
      (
            SELECT t.customer_id, a.amount
            FROM transactions t,(
                  SELECT transaction_id, amount FROM
                     transactionsline
                  WHERE amount > 3170
            ) a
            WHERE t.transaction_id = a.transaction_id
      ) b, orders o, ordersline ol
      WHERE c.balance > 0
      AND c.customer_id = b.customer_id
      AND o.customer_id = c.customer_id
      AND ol.order_id = o.order_id;

The cost is reduced further with the same combination of scans because fewer rows are being joined.

Query                                 Cost     Rows       Bytes
---------------------------------   ------   ------   ---------
SELECT STATEMENT on                    533      605       33880
 NESTED LOOPS on                       533      605       33880
  NESTED LOOPS on                      340      193        9843
   NESTED LOOPS on                     308       16         704
    NESTED LOOPS on                    292       16         288
     TABLE ACCESS FULL on
       TRANSACTIONSLINE                276       16         176
     TABLE ACCESS BY INDEX ROWID
       on TRANSACTIO                     1    33142      231994
      INDEX UNIQUE SCAN on
        XPKTRANSACTIONS                       33142
    TABLE ACCESS BY INDEX ROWID
      on CUSTOMER                        1     2690       69940
     INDEX UNIQUE SCAN on
       XPKCUSTOMER                             2690
   TABLE ACCESS BY INDEX ROWID on
     ORDERS                              2    31935      223545
    INDEX RANGE SCAN on
      XFK_ORDERS_CUSTOMER                1    31935
  INDEX RANGE SCAN on
    XFK_ORDERLINE_ORDER                  1   540827     2704135

Now let's combine WHERE clause comparison subqueries and FROM clause embedded subqueries.

EXPLAIN PLAN SET statement_id='TEST' FOR
      SELECT c.name, b.amount
      FROM customer c,
      (
            SELECT t.customer_id, a.amount
            FROM transactions t,(
                  SELECT transaction_id, amount FROM
                     transactionsline
                  WHERE amount > 3170
            ) a
            WHERE t.transaction_id = a.transaction_id
      ) b
      WHERE c.balance > 0
      AND EXISTS(
            SELECT o.order_id FROM orders o
            WHERE o.customer_id = c.customer_id
            AND EXISTS(
                  SELECT order_id FROM ordersline
                  WHERE order_id = o.order_id
            )
      );

Using EXISTS makes the query just that little bit faster with lower cost since the number of tables joined is reduced.

Query                                 Cost     Rows       Bytes
---------------------------------   ------   ------   ---------
SELECT STATEMENT on                    420     2190       91980
 FILTER on
  NESTED LOOPS on                      420     2190       91980
   MERGE JOIN CARTESIAN on             420     2190       81030
    TABLE ACCESS FULL on
      TRANSACTIONSLINE                 276       16         176
    BUFFER SORT on 144 135 3510
     TABLE ACCESS FULL on CUSTOMER       9      135        3510
  INDEX UNIQUE SCAN on
    XPKTRANSACTIONS                          188185      940925
 NESTED LOOPS on                        66      201        2412
  TABLE ACCESS BY INDEX ROWID on
    ORDERS                               2       64         448
   INDEX RANGE SCAN on
     XFK_ORDERS_CUSTOMER                 1       64
  INDEX RANGE SCAN on
    XFK_ORDERLINE_ORDER                  1   540827     2704135


6.7 Using Synonyms

A synonym is, as its name implies, another name for a known object. Synonyms are typically used to reference tables between schemas. Public synonyms make tables contained within schemas available to all schemas. Apart from the obvious security issues there can be potential performance problems when over-using synonyms in highly concurrent environments. It may not necessarily be effective to divide functionality between different schemas only to allow users global or semi-global access to all of the underlying schemas. Simplicity in development using objects like synonyms often leads to complexity and performance problems in production. Additionally too many metadata objects can cause problems with the shared pool.


6.8 Using Views

Views are application-and security-friendly. Views can also be used to reduce complexity, particularly in development. In general, views are not conducive to good performance. A view is a logical overlay on top of one or more tables. A view is created using an SQL statement. A view does not contain data itself. The biggest problem with a view is that whenever it is queried its defining SQL statement is re-executed. It is common in applications for a developer to query a view and add additional filtering. The potential results are views containing large queries where programmers will then execute small row number retrievals from the view. Thus two queries are executed, commonly with the view query selecting all the rows in the underlying table or join.

Let's try to prove that views are inherently slower than direct table queries. Firstly, I create a view on my largest Accounts schema table.

CREATE VIEW glv AS SELECT * FROM generalledger;

Now let's do some query plans. I have four queries and query plans listed. The first two retrieve a large number of rows from the view and then the table. It is apparent that the query plans are identical in cost.

Selecting from the view:

EXPLAIN PLAN SET statement_id='TEST' FOR
      SELECT * FROM glv WHERE coa# = '40003';

Query                                 Cost     Rows       Bytes
---------------------------------   ------   ------   ---------
SELECT STATEMENT on                    165   150548     3914248
 TABLE ACCESS BY INDEX ROWID on
   GENERALLEDGE                        165   150548     3914248
  INDEX RANGE SCAN on XFK_GL_COA#        5   150548

Selecting from the table:

EXPLAIN PLAN SET statement_id='TEST' FOR
      SELECT * FROM generalledger WHERE coa# = '40003';

Query                                 Cost     Rows       Bytes
---------------------------------   ------   ------   ---------
SELECT STATEMENT on                    165   150548     3914248
 TABLE ACCESS BY INDEX ROWID on
   GENERALLEDGE                        165   150548     3914248
  INDEX RANGE SCAN on XFK_GL_COA#        5   150548

Now let's filter and return much fewer rows. Once again the query plans are the same.

Selecting from the view:

EXPLAIN PLAN SET statement_id='TEST' FOR
      SELECT * FROM glv WHERE generalledger_id = 500000;

Query                                 Cost     Rows       Bytes
---------------------------------   ------   ------   ---------
SELECT STATEMENT on                      3        1          26
 TABLE ACCESS BY INDEX ROWID on
   GENERALLEDGE                          3        1          26
  INDEX UNIQUE SCAN on XPKGENERALLEDGER  2        1

Selecting from the table:

EXPLAIN PLAN SET statement_id='TEST' FOR
      SELECT * FROM generalledger WHERE generalledger_id =
         500000;

Query                                 Cost     Rows       Bytes
---------------------------------   ------   ------   ---------
SELECT STATEMENT on                      3        1          26
 TABLE ACCESS BY INDEX ROWID on
   GENERALLEDGE                          3        1          26
  INDEX UNIQUE SCAN on XPKGENERALLEDGER  2        1

So let's now try some time tests. The COUNT function is used as a wrapper and each query is executed twice to ensure there is no conflict between reading from disk and memory.

SELECT COUNT(*) FROM(SELECT * FROM glv WHERE coa# = '40003');
SELECT COUNT(*) FROM(SELECT * FROM generalledger WHERE
   coa# = '40003');
SELECT COUNT(*) FROM(SELECT * FROM glv
      WHERE generalledger_id = 500000);
SELECT COUNT(*) FROM(SELECT * FROM generalledger
      WHERE generalledger_id = 500000);

In the first two instances retrieving from the view is faster than reading from the table.

SQL> SELECT COUNT(*) FROM(SELECT * FROM glv WHERE
   coa# = '40003');

  COUNT(*)
----------
     66287

Elapsed: 00:00:04.04

SQL> SELECT COUNT(*) FROM(SELECT * FROM generalledger WHERE
   coa# = '40003');

   COUNT(*)
-----------
      66287

Elapsed: 00:00:04.09

SQL> SELECT COUNT(*) FROM(SELECT * FROM glv WHERE
   generalledger_id = 500000);

   COUNT(*)
-----------
          1

Elapsed: 00:00:00.00

SQL> SELECT COUNT(*) FROM(SELECT * FROM generalledger WHERE
   generalledger_id = 500000);

   COUNT(*)
-----------
          1

Elapsed: 00:00:00.00

For a single table and a view on that table there is no difference in query plan or execution time.

Now let's go and re-create our view and re-create it with a join rather than just a single table. This code drops and re-creates the view I created previously.

DROP VIEW glv;
CREATE VIEW glv AS
      SELECT gl.generalledger_id, coa.coa#, t.text AS type,
            st.text AS subtype, coa.text as coa, gl.dr,
               gl.cr, gl.dte
      FROM type t JOIN coa USING(type)
            JOIN subtype st USING(subtype)
                  JOIN generalledger gl ON(gl.coa# =
                     coa.coa#);

When retrieving a large percentage of rows in the following two queries the cost in the query plan is much better when retrieving using the tables join rather than the view.

Selecting from the view:

EXPLAIN PLAN SET statement_id='TEST' FOR
      SELECT * FROM glv WHERE coa# = '40003';

Query                                 Cost     Rows     Bytes
-----------------------------------   ----   ------   -------
SELECT STATEMENT on                    168    30110   2107700
 NESTED LOOPS on                       168    30110   2107700
  NESTED LOOPS on                        3        1        44
   NESTED LOOPS on                       2        1        34
    TABLE ACCESS BY INDEX ROWID on COA   1        1        25
     INDEX UNIQUE SCAN on XPKCOA                  1
    TABLE ACCESS BY INDEX ROWID on
      TYPE                               1        6        54
     INDEX UNIQUE SCAN on XPKTYPE                 6
   TABLE ACCESS BY INDEX ROWID on
     SUBTYPE                             1        4        40
    INDEX UNIQUE SCAN on XPKSUBTYPE               4
  TABLE ACCESS BY INDEX ROWID on
    GENERALLEDG                        165   150548   3914248
   INDEX RANGE SCAN on XFK_GL_COA#       5   150548

Selecting from the table join:

EXPLAIN PLAN SET statement_id='TEST' FOR
      SELECT gl.generalledger_id, coa.coa#, t.text AS type,
            st.text AS subtype, coa.text as coa, gl.dr,
               gl.cr, gl.dte
      FROM type t JOIN coa USING(type)
            JOIN subtype st USING(subtype)
                  JOIN generalledger gl ON(gl.coa# = coa.coa#)
      WHERE gl.coa# = '40003';

Query                                 Cost     Rows     Bytes
-----------------------------------   ----   ------   -------
SELECT STATEMENT on                      5    30110   2107700
 NESTED LOOPS on                         5    30110   2107700
  NESTED LOOPS on                        3        1        44
   NESTED LOOPS on                       2        1        34
    TABLE ACCESS BY INDEX ROWID
      on COA                             1        1        25
     INDEX UNIQUE SCAN on XPKCOA                  1
    TABLE ACCESS BY INDEX ROWID on
      TYPE                               1        6        54
     INDEX UNIQUE SCAN on XPKTYPE                 6
   TABLE ACCESS BY INDEX ROWID on
     SUBTYPE                             1        4        40
    INDEX UNIQUE SCAN on XPKSUBTYPE               4
  TABLE ACCESS BY INDEX ROWID on
    GENERALLEDG                          2   150548   3914248
   INDEX RANGE SCAN on XFK_GL_COA#       1   150548

Let's try some timing tests. The first timing test retrieves a large set of rows from the view.

SQL> SELECT COUNT(*) FROM(SELECT * FROM glv WHERE coa# =
   '40003');

   COUNT(*)
-----------
      66287

Elapsed: 00:00:04.02

The second timing test retrieves the same large set of rows from the table join and is obviously much faster.

SQL> SELECT COUNT(*) FROM(
  2 SELECT gl.generalledger_id, coa.coa#, t.text AS type,
      st.text AS subtype, coa.text as coa, gl.dr, gl.cr,
      gl.dte
  3 FROM type t JOIN coa USING(type)
  4 JOIN subtype st USING(subtype)
  5 JOIN generalledger gl ON(gl.coa# = coa.coa#)
  6 WHERE gl.coa# = '40003');

   COUNT(*)
-----------
      66287

Elapsed: 00:00:00.07

Comparing times to retrieve a single row, there is no difference between the view and the retrieval from the join. In a highly active concurrent environment this would probably not be the case.

SQL> SELECT COUNT(*) FROM(SELECT * FROM glv WHERE
   generalledger_id = 500000);

   COUNT(*)
-----------
          1

Elapsed: 00:00:00.00

SQL> SELECT COUNT(*) FROM (SELECT gl.generalledger_id,
  coa.coa#, t.text
AS type, st.text AS subtype, coa.text as coa, gl.dr, gl.cr,
gl.dte
  2 FROM generalledger gl JOIN coa ON(gl.coa# = coa.coa#)
  3 JOIN type t USING(type)
  4 JOIN subtype st USING(subtype)
  5 WHERE generalledger_id = 500000);

   COUNT(*)
-----------
          1

Elapsed: 00:00:00.00

Views can now have constraints, including primary and foreign key constraints. These may help performance of data retrieval from views. However, assuming that views are created for coding development simplicity and not security, adding complexity to a view would negate the simplification issue.

The exception to views re-executing every time they are queried is a Materialized View. A Materialized View is a separate database object in Oracle Database and stores the results of a query. Thus when a Materialized View is queried, data is extracted from the view and not the underlying objects in the query. Materialized Views are read only and intended for use in data warehousing and replication.

Views are not performance friendly! For the sake of performance do not use views. Some applications are built with multiple layers of views. This type of application design is often application convenient and can produce disappointing results with respect to database performance. There is simply too much metadata in the shared pool. A brute force method of resolving selection of rows from multiple layered sets of views is to use a form of the FROM clause in the SELECT statement with the ONLY clause included as shown in the following syntax. The ONLY clause will not retrieve rows from subset views.

SELECT … FROM ONLY (query) …


6.9 Temporary Tables

In years past in traditional relational databases temporary tables were created as shell structures to contain temporarily generated data, usually for the purposes of reporting. Oracle Database has the ability to create tables containing temporary rows. Rows created in temporary table objects are created locally to a session, somewhat managed by the database. It is more efficient to use Oracle Database temporary table objects in some circumstances. Creating traditional temporary tables, filling them, emptying them, and storing or deleting their rows is unlikely to perform better than Oracle Database temporary table objects.


6.10 Resorting to PL/SQL

PL/SQL is an acronym for Programming Language for SQL. From a purely programming perspective PL/SQL is really SQL with programming logic wrapper controls, amongst numerous other bells and whistles. PL/SQL is a very primitive programming language at best. Beware of writing entire applications using PL/SQL. SQL is designed to retrieve sets of data from larger sets and is not procedural, sequential, or object-like in nature. Programming languages are required to be one of those or a combination thereof. PL/SQL has its place as a relational database access language and not as a programming language. Additionally PL/SQL is interpretive which means it is slow!

Note

Oracle Database 10 Grid PL/SQL objects are now stored in compiled form in binary object variables or BLOB objects. This helps PL/SQL procedure execution performance.

Does PL/SQL allow for faster performance? The short answer is no. The long answer is as follows. PL/SQL allows much greater control over SQL coding than simple SQL does. However, modern Oracle SQL is much more sophisticated than it was years ago and some of the aspects of PL/SQL used in the past allowing better performance in PL/SQL than SQL are effectively redundant, particularly with retrieval of data.

There are a number of reasons for resorting to PL/SQL:

  • PL/SQL will not provide better performance but will allow a breakdown of complexity. Breaking down complexity can allow easier tuning of SQL statements through better control of cursors.

  • An obvious benefit of PL/SQL is the ability to build stored procedures and triggers. Triggers should be used sparingly and avoided for implementation of Referential Integrity; constraints are much faster. In the case of stored procedures the obvious benefits are centralized control and potential performance increases because stored procedure code is executed on the server. Execution on a database server reduces network traffic.

  • There are some situations where it is impossible to code SQL code using SQL alone and thus PL/SQL has to be resorted to. As Oracle Corporation has developed SQL this has become less frequent.

  • Some exceedingly complex SQL code can benefit from the use of PL/SQL instead of SQL. Some SQL code can become so complex that it is impossible to tune or even code in SQL and hence PL/SQL becomes the faster option. Using the DECODE function is similar to control structures such as IF and CASE statements. PL/SQL allows all the appropriate control structures.

  • PL/SQL packages can be cached into memory to avoid re-parsing.

PL/SQL can provide better program coding control of cursors plus execution and maintenance from a central, single point. Perhaps the most significant benefit of resorting to PL/SQL is a potential reduction in complexity. Once again, as with views, reducing complexity is not necessarily conducive to performance tuning. In fact simplicity, as with over-Normalization can often hurt performance in a relational database. Thus further discussion of tuning SQL retrieval code using PL/SQL is largely irrelevant.

Tuning PL/SQL is a programming task and has little to do with coding well-performing SQL code. The obvious programming points to note are as follows:

  • Do not process more in a loop or a cursor than is necessary. Break out of loops when no more processing is required.

  • Large IF statements should be replaced with CASE statements or appropriate breaks should be used. Traditionally CASE statements are faster than IF statements.

  • Recursion using embedded procedures creates very elegant code but in PL/SQL can cause performance problems, especially where transactions are not completed. PL/SQL is not executed as compiled code in the sense that compiled C code is. PL/SQL compilation involves syntax parsing and interpretation; no binary copy is created. When PL/SQL code is executed it is interpreted and not executed from a compiled binary form.

    Note

    Oracle Database 10 Grid PL/SQL objects are now stored in compiled form in binary object variables or BLOB objects. This helps PL/SQL procedure execution performance.

  • The WHERE CURRENT OF clause can refer to a cursor ROWID, giving direct pointer access to a cursor row in memory. The RETURNING INTO clause can help with reducing the need for subsequent SELECT statements further on in a procedure by returning values into variables.

  • Explicit cursors can be faster than implicit cursors but are more difficult to code.

  • PL/SQL has three parameter formats for passing variables in and out of procedures. IN and OUT pass values in and out of procedures, respectively. IN OUT passes a pointer both in and out of a procedure. Use only what is needed.

  • As with SQL code embedded in applications, PL/SQL procedures of any kind can use bind variables. Using bind variables can have a profound effect on increasing performance by lowering parsing requirements in the library cache.

6.10.1 Tuning DML in PL/SQL

There are two interesting points to note in relation to performance of DML statements in PL/SQL. Other than what has already been covered in this chapter there is little else which can be done to tune DML (INSERT, UPDATE, DELETE, MERGE) or SELECT statements in PL/SQL.

The RETURNING INTO Clause

In the following example the RETURNING clause prevents the coder having to code an extra SELECT statement to find the identifier value required for the second INSERT statement. The same applies similarly to the second INSERT statement returning a second value for the DBMS_OUTPUT.PUT_LINE procedure.

DECLARE
      division_id division.division_id%TYPE;
      division_name division.name%TYPE;
      department_name department.name%TYPE;
BEGIN
      INSERT INTO division(division_id, name)
      VALUES(division_seq.NEXTVAL, 'A New Division')
      RETURNING division_id, name INTO division_id,
         division_name;

      INSERT INTO department(department_id, division_id, name)
      VALUES(department_seq.NEXTVAL, division_id, 'A New
         Department')
      RETURNING name INTO department_name;

      DBMS_OUTPUT.PUT_LINE('Added : '||division_name
            ||', '||department_name);
END;
/
Note

Oracle Database 10 Grid The RETURNING clause can be used to return collections.

6.10.2 When to Resort to PL/SQL and Cursors

In general, SQL in Oracle Database is now powerful enough to deal with almost any requirement. Some occasions do call for substituting SQL code nested loop type queries with PL/SQL. PL/SQL provides better programming control and allows much easier management of highly complex SQL code. It is sometimes the case that better performance will be gained using PL/SQL to control nested SQL statements.

PL/SQL can sometimes be faster when compiled, pinned, and executed on the database server, depending on the application. The goal of stored procedures is to minimize on network traffic. With the speed of modern network connections available this is not necessarily an issue anymore.

So what do I mean by resorting to PL/SQL and cursors? After all I did not just state "Resorting to PL/SQL". So what am I talking about? Look at this example.

EXPLAIN PLAN SET statement_id='TEST' FOR
      SELECT * FROM coa NATURAL JOIN generalledger;

This query plan shows the COA table in the outer loop and the GeneralLedger table in the inner loop.

Query                                 Cost     Rows     Bytes
-----------------------------------   ----   ------   -------
1. SELECT STATEMENT on                1642  1068929  50239663
2.  HASH JOIN on                      1642  1068929  50239663
3.   TABLE ACCESS FULL on COA            2       55      1320
3.    TABLE ACCESS FULL on
        GENERALLEDGER                 1128  1068929  24585367

To convert the SQL statement shown previously to PL/SQL I would need PL/SQL code something similar to that shown here.

DECLARE
   CURSOR cCOA IS SELECT * FROM coa;
   TYPE tGL IS REF CURSOR RETURN generalledger%ROWTYPE;
   cGLs tGL;
   rGL generalledger%ROWTYPE;
BEGIN
   FOR rCOA IN cCOA LOOP
      OPEN cGLs FOR SELECT * FROM generalledger WHERE
         coa# = rCOA.coa#;
      LOOP
         FETCH cGLs INTO rGL;
         EXIT WHEN cGLs%NOTFOUND;
         DBMS_OUTPUT.PUT_LINE(
            rCOA.coa#||' '||
            TO_CHAR(rGL.dte)||' '||
            TO_CHAR(rGL.dr)||' '||
            TO_CHAR(rGL.cr));
      END LOOP;
      CLOSE cGLs;
   END LOOP;
EXCEPTION WHEN OTHERS THEN CLOSE cGLs;
END;
/

In general, PL/SQL should only replace SQL when coding simply cannot be achieved in SQL, it is too complex and convoluted for SQL, or centralization on the database server is required.

6.10.3 Java or PL/SQL?

Java can be used to construct stored procedures in much the same way that PL/SQL can. When to use Java? That is a question with a very simple answer. Use Java when not accessing the database and writing code which is computationally heavy. What does this mean? When answering a question such as this I always prefer to go back into the roots of a programming language or a database. In other words, what were PL/SQL and Java originally built for? What is their purpose? Let's start by looking at PL/SQL.

PL/SQL is effectively a primitive programming language and is purely an extension of SQL. SQL was originally built purely for the purpose of accessing data in a relational database. Therefore, it follows that PL/SQL is of the same ilk as SQL. PL/SQL was originally devised to create stored procedures in Oracle Database. Stored procedures were devised for relational databases in general, not just Oracle Database, to allow for coding of self-contained blocks of transaction-based SQL code. Those blocks of code are executed on the database server thus minimizing on network traffic. PL/SQL is much richer than stored procedure languages used in other relational databases. PL/SQL can be used to code complexity. This takes us to Java.

Why use Java? Java is an object-oriented programming language built for coding of highly complex front-end and back-end applications. If you know anything about objects at all you will understand that objects in programming are superb at handling complexity. It is in the very nature of objects to handle complexity by breaking everything down into its most simplistic parts. Java can be used to handle highly complex coding sequences and can be used to create Oracle Database stored procedures, much in the same way that PL/SQL can. Java is much more powerful than PL/SQL in doing lots of computations. Java is better than PL/SQL at anything that does not involve accessing the database, especially complex code. Coding requirements when accessing a database are trivial in relation to the complex routines and functions required by applications-level coding.

There is one small but fairly common problem with using Java. Java is object oriented. Oracle Database is relational. Object and relational methodologies do not mix well. Many Java applications, due to their object nature, break things into very small, easily manageable pieces. Object-oriented design is all about easy management of complexity. In relational terms management of complexity by severe break-down and object "black boxing" is an incredible level of Normalization. Too much Normalization leads to too much granularity and usually very slow performance. What commonly happens with Java applications is one or all of a number of things. These are some of the possibilities.

  • Pre-loading of large data sets.

  • Separation of parts of SQL statements into separate commands sent to the database. For instance, a SELECT statement could be submitted and then filtering would be performed in the Java application itself. This leads to lots of full table scans and a plethora of other performance problems.

  • Sometimes object design is imposed onto the database to such an extent as to continually connect to and disconnect from the database for every SQL code execution. This is very heavy on database resource consumption.

So what's the answer to the nagging question of using Java or PL/SQL? The answer is to use both, if your skills set permits it. Java can be used to handle any kind of complex code not accessing the database. PL/SQL should be used to access the database. If you cannot or do not wish to use a mix and prefer Java then be aware that a relational database is not able to manage the complete and total "black box" breakdown into easily definable and understandable individual objects. Do not attempt to impose an object structure on a relational or furthermore even an object-relational database: they are not designed for that level of granularity.


6.11 Replacing DELETE with TRUNCATE

The DDL TRUNCATE command is faster than using the DELETE command when deleting all the rows from a table. The TRUNCATE command automatically commits and does not create any log entries or rollback. The obvious problem with TRUNCATE is that an error cannot be undone.


6.12 Object and Relational Conflicts

Relational and object data model structures are completely different to each other. There is great potential for conflict when combining these two methodologies into what is called an object-relational database. These conflicts can hamper performance in a relational database such as Oracle Database.

6.12.1 Large Binary Objects in a Relational Database

The biggest benefit to the combination of objects and relations is that of including large binary objects into a relational structure, such as multimedia objects. However, there is a twist. In both relational and object databases the most efficient storage method for multimedia objects is to use a BFILENAME pointer. A BFILENAME pointer does not store the object itself in the database but contains a path and file name for the object. The object is stored externally to the database in the file system. Storing large binary objects in any database is inefficient because those binary objects can span multiple blocks, both Oracle Database blocks and operating system blocks. Even with Oracle9i Database multiple block sizes and specific storage structures for LOB datatypes large multimedia objects will span more than one block. Storing data into a database which spans more than a single block is effectively row chaining where a row is "chained" from one block to another. Chaining is not really an issue if the blocks are physically next to each other. Also the DB_FILE_MULTIBLOCK_READ_COUNT parameter can help to counteract this. The fact is contiguous, defragmented block storage is extremely unlikely in any database environment. It is usually best to store large singular binary objects in the file system.

Note

Oracle Database 10 Grid BIGFILE tablespaces can possibly help to alleviate these issues somewhat.

6.12.2 Object-Relational Collections

Including TABLE and VARRAY collections inside relational tables and within PL/SQL is generally a very bad idea. Collections are an object methodological approach to object data abstraction and encapsulation. Object methodologies are completely opposed to those of relational databases. An object structure is spherical allowing access from any point to any point within an entire database. A relational database is more two-dimensional in nature and requires that access to data be passed through or down semi-hierarchical structures or into subset parts of a data model. Since PL/SQL is merely an extension of SQL, and SQL is a purely relational database access language, any type of object coding is best avoided using PL/SQL. If you want to write object code use Java. Java is built for object structures.

The same approach applies to including object structures in tables. There are various instances in which collection substructures can be utilized in relational database tables. 1st Normal Form master detail and 2nd Normal Form foreign key static table inclusion relationships are possibilities. However, storage structures remain applicable to a relational methodology and are still ultimately stored in rows (tuples). Tables and rows are two-dimensional. Object structures are multi-dimensional. The two do not fit together. If you must contain collections in tables in Oracle Database there are various object collection types that could be used. Associative arrays are the most efficient.

  • Nested Table. A nested table is a dynamic array. A dynamic array is a pointer. A pointer does not require a specific size limit and thus the use of the term dynamic. Dynamic implies that it can have any number of rows or entries in its array.

  • VARRAY. A VARRAY is a fixed-length array. A fixed-length array is a reserved chunk of memory saved for multiple array rows. Unlike a dynamic array a fixed-length array has space in memory reserved for the whole array structure.

  • Associative Array. An associative array is an indexed dynamic array and can potentially be accessed faster than a nested table because it can be accessed using an index.

Problems with objects in relational models are more often than not a misunderstanding of object-modeling techniques. It is common knowledge that the approach required for object data modeling is completely different to that of relational modeling. The same is very likely to apply to object-relational modeling using object types such as nested table, VARRAY or associative array collections in Oracle Database tables. Using these collection data types requires object and relational modeling skills, not only relational modeling skills.

That ends this chapter on examining the basics of how to create efficient SQL. In the next chapter we will look at indexing and start to delve a little deeper into Oracle Database specifics.


Chapter 7: Common Sense Indexing

Overview

We have seen in the previous chapter how important it is to match various SQL code clauses with indexes properly. This chapter will examine the internal structure of various different types of indexes. Different types of indexes apply in different circumstances and for different tasks. Where different index structures are applicable to different tasks, indexes will often perform very poorly when used inappropriately. Thus the title of this chapter being "Common Sense Indexing" implies utilizing the appropriate index type under the correct circumstances.

Let's start with looking at what should be indexed and how.


7.1 What and How to Index

When considering indexes never be afraid of not indexing. On the contrary, do not always assume that an existing index should exist, simply because it does exist.

When considering use of unusual indexes such as clusters, bitmaps, and hashing be aware of their applications. The only index type amenable to data changes is a BTree index. All other index types are effectively read-only type indexes and are preferably never used otherwise. This is not always the case in some other relational databases but it is more often than not the case for Oracle Database. In most relational databases clusters, bitmaps, and hash indexes do not manage overflow well, if at all. Overflow in an index results in a bounce from the index block to an area outside of the original index structure, often negating the point of using the index in the first place.

Database administrators should always keep a watchful eye on indexing in a database. There is never really available time but when an application is released it is always best to reexamine all indexing. Quite often developers will create many indexes, sometimes each creating their own sets of indexes. The result can be overindexing. Too many indexes on a table will create a performance problem.

Tip

Executing a DML command on a table will execute the same command on all of its indexes. For instance inserting a single row into a table with four indexes comprises five changes to the database.

Be especially vigilant for the use of bitmap and function-based indexes. Bitmap indexes are generally only effective in read-only situations. Function-based indexes are nice but I have never seen the Optimizer use one effectively in a commercial environment. However, this is often because configuration parameters are inappropriately set. The parameters QUERY_REWRITE_ENABLED and QUERY_REWRITE_INTEGRITY must be set to TRUE and TRUSTED respectively to allow use of function-based indexes. Developers and database administrators tend to like function-based indexes and bitmaps, especially when they have just read about them in the manual. Be alert for these unusual indexes because both bitmap and function-based indexes are largely ineffective and can be exceedingly detrimental to performance. One particular case I remember where many bitmap indexes were used. The database was in excess of 200 Gb and had a fairly large combination online and reporting production database. When one of the bitmaps was changed to a BTree index, a process taking 8 h to run completed in less than 3 min. This case involved heavy DML activity on the table on which the bitmap index was created.

7.1.1 When Not to Use Indexes

There are some circumstances where indexes can be detrimental to performance and sometimes those indexes should not exist. The Optimizer will occasionally ignore indexes and consider reading the entire table a faster option.

  • A table with a small number of columns may not benefit from an index if a large percentage of its rows are always retrieved from it.

  • Small static data tables may be so small that the Optimizer will simply read those tables as a full table scan and ignore indexes altogether. In some cases the Optimizer may use an index on a small table where it should not since a full table scan would be faster. An exception to this rule will be in mutable joins where unique index hits are often used, even on small tables. If full table scans are faster than index reads in those mutable joins you might want to remove indexes from the small tables altogether or override with hints. Examine your small static data tables. Do they really need indexing?

    Tip

    A hint is a comment added to an SQL command allowing overriding of the Optimizer.

  • Sometimes tables created for reporting or data warehousing may already be in the correct physical order, commonly if those tables are only ever appended to using mass insertions. Creating an index may be pointless in this type of situation since the data may already be in the required physical order.

  • Indexes should usually be created on a small percentage of the columns in a table. Large composite indexes may be relatively large compared with the table. The relative size between index and table is important. The larger the ratio of index to table physical size then the less helpful the index will be in terms of decreasing physical space to be read. Also many columns not in the index may contain NULL values. It may be faster to read the entire table. Why create indexes?

  • NULL values are generally not included in indexes. Do not index columns containing many NULL values unless there is a specific use for it such as filtering the column as NOT NULL, thus only scanning the index and the rows without NULL values.

7.1.2 Utilizing Referential Integrity Indexes

Referential Integrity uses primary and foreign keys to validate relationships between rows in related tables. Oracle Database does not automatically create indexes on foreign keys and it is advisable to do so manually.

Data models of different forms can have either single-column unique integer identifiers at one extreme or large composite-column indexes at the other extreme. It is quite often possible that application SQL code can make good use of Referential Integrity indexes. This is generally the case when the data model matches the functionality of the application well. This is especially true in two cases.

  • Java object applications are often compatible with data model structure containing single-column unique integer identifiers, where the integers match Java object identifiers. This type of data model is often highly normalized and usually in great danger of being over-normalized. Alternate indexing is not common in Java top-down designed data models but can be used to validate non-identifying unique values such as names.

  • More traditional key-structured data models containing multiple-column composite keys and indexes can also match application functionality well but not as often as Java applications object identifier structures. Typically these types of data models have difficulty avoiding extensive alternate indexing. Since composite indexes contain multiple columns alternate keys are not only additional in nature but are also used to apply different orders to primary and foreign keys, or even comprise additional, perhaps reordered subsets of those keys. The Oracle Database Optimizer is over time becoming more capable of matching SQL code with dissimilar index orders but there will always be limitations.

Any kind of reporting or data warehousing application requirements will cause requirements for alternate indexing. Any online application with any type of reporting will generally be inclined in this direction as well.

If Referential Integrity indexing can be used for general SQL code tuning then do so as much as possible. Ultimately fewer indexes will be required and thus applications will perform better in general.

Alternate and Secondary Indexing

Alternate indexing is often referred to as secondary indexing. Alternate indexing includes any indexes created against tables in a data model which are not part of Referential Integrity constraints. Quite often the need for alternate indexing is a mismatch between the data model and functionality required by applications. Sometimes unique indexes are required on specific columns in data models other than unique integer identifier columns in every table. The application could deal with this particular issue but it may be best to place unique constraints in the database for the same reasons that Referential Integrity is not placed in the application or in triggers. Once again excessive alternate indexing could indicate data model problems or simply a mismatch between data model and application requirements.


7.2 Types of Indexes

There are a number of general types of indexes.

  • No Index. A table without an index is referred to as heap structured. It is quite literally in a heap. If this method of table access is efficient, which it can be, then do not create indexes for that table. Sometimes even Referential Integrity indexes can be detrimental to performance.

  • Static Data Index. This is an index placed on a small static data table. Sometimes these tables do not require indexes for anything other than Referential Integrity because the Optimizer may always choose a full table scan for better performance. In some cases Referential Integrity indexes can be removed for better performance, especially where the static table contains no foreign key entries. Static implies no changes, ever! If data in a table never changes why include added processing checking Referential Integrity?

  • Dynamic Data Index. These indexes expect changes to data and are thus subject to overflow. They may require frequent rebuilding. Never ever use anything but BTree type indexing for dynamic data. Clustering, hashing, and bitmaps are only appropriate for read-only tables with the exception of index-organized tables in rare circumstances. Index-organized tables effectively sort index and data values for an entire table into a BTree and can perform well in OLTP type databases.

  • Read-Only Reporting Index. Read-only tables are much more flexible with respect to indexing. Oracle Database allows numerous types of indexes for read-only tables where each index type has a specific application.

  • Unique and Non-unique Indexes. A unique index is an index allowing only a single value in a table. Be careful creating unique indexes because every insertion or update to a uniquely indexed column or columns requires a scan of at least the entire index space.

  • Single-Column and Composite Indexes. Single-column indexes are more efficient than composite multiple-column indexes.

Indexes can be created on columns of most datatypes and any combination thereof. Just remember this one thing. By far the most efficient and easy to manage type of index is an index created on a unique integer identifier. In Oracle Database these indexes are best incremented using Oracle Database sequence generators.


7.3 Types of Indexes in Oracle Database

These are the types of indexes available in Oracle Database.

  • Efficient as read-write indexes.

    • BTree (binary tree).

    • Function based.

    • Reverse key.

  • Efficient as read-only indexes.

    • Bitmap.

    • Bitmap join.

    • Index-organized table, sometimes useful as read-write indexes.

    • Cluster and hash cluster.

  • Domain indexes have a very specific application in an Oracle Database optional add-on and are beyond the scope of this book.

There are also certain attributes of some index types relevant to tuning.

  • Ascending or descending.

  • Uniqueness.

  • Composites.

  • Compression.

  • Reverse key indexes.

  • Unsorted indexes using the NOSORT option.

  • Null values.

Let's take a very brief look at index syntax. Then we will proceed to describe each index structure, how they should be used and how they can be tuned without delving too much into physical tuning.

7.3.1 The Syntax of Oracle Database Indexes

Following is a syntax diagram for the CREATE INDEX and ALTER INDEX commands. Since we are in the SQL code tuning section of this book any physical attributes are reserved for possible discussion in Part III on physical and configuration tuning. Parts of the syntax diagram are highlighted to denote parts of Oracle Database index object maintenance relevant to SQL code tuning. Some parts are physical tuning.

CREATE [ UNIQUE | BITMAP ] INDEX [ schema.] index ON

{
   [ schema.]table [ alias ] ( column [, column
      … ] | expression )
   {
         [[ NO ] COMPRESS [ n ] ] [ NOSORT | REVERSE ]
         [ ONLINE ] [ COMPUTE STATISTICS ] [ TABLESPACE
            tablespace ]
         [[ NO ] LOGGING ]
         [ physical properties ]
         [ partitioning properties ]
         [ parallel properties ]
   }
}
| CLUSTER [ schema.]cluster { cluster properties }
| Bitmap join index clause;
ALTER [ UNIQUE | BITMAP ] INDEX [ schema.]index
[
      [ ENABLE | DISABLE ] [ UNUSABLE ] [ COALESCE ]
      [ RENAME TO index ] [[ NO ] LOGGING ] [[ NO ]
         MONITORING USAGE ]
      [ REBUILD [[ NO ] COMPRESS [ n ] ] [[ NO ] REVERSE ]
            [ ONLINE ] [ COMPUTE STATISTICS ] [ TABLESPACE
               tablespace ]
            [[ NO ] LOGGING ]
      ]
      [ physical properties ]
      [ partitioning properties ]
      [ parallel properties ]
      [ deallocate unused clause ]
      [ allocate extent clause ]
]
| CLUSTER [ schema.]cluster { cluster properties };

7.3.2 Oracle Database BTree Indexes

Figure 7.1 shows the internal structure of a BTree index. Note that there are only three layers. Each separate boxed numbered block node in Figure 7.1 represents a block in an Oracle database. A block is the smallest physical unit in an Oracle database. Every read of a datafile will read a single block or more blocks. A unique index scan reading a single row will read the index datafile up to three times and the table once. Three blocks will be read: the root node index block, the second layer index branch block, the index leaf block, and finally the block in the table containing the row using the ROWID pointer stored in the index. Indexes for very small tables may be contained in a single block. All these block references are accessed using ROWID pointers. ROWID pointers are very fast. Unique index hits are the fastest way of accessing data in a database.

Click To expand
Figure 7.1: An Oracle Database BTree Index

A BTree index does not overflow like other types of indexes. If a block is filled a new block is created and branch nodes are changed or added as required. The parent branch and root node contents could even be adjusted to prevent skewing. Only a change to an indexed table column will change a BTree index. BTree indexes are the best index for obtaining exact row hits, range scans, and changes to the database. The only exception to the overflow rule for a BTree index is where duplicate index values not fitting into a single block can extend into new blocks. This type of overflow is not quite the same as overflow in other index types since other index types do not update index structure "on the fly" as effectively as BTree indexes, if at all.

The following query retrieves index statistics from the USER_INDEXES metadata view, sorted in branch-level order for all the Accounts schema tables. The lower the branch depth level of the BTree the better since less searching through branches is required. In general, any index will become less efficient as it becomes larger and contains more duplicates.

  • Branch level 0 indexes are all small static data tables.

  • Branch level 1 indexes are dynamic tables where foreign or alternate key indexes are usually on unique integer identifier column values.

  • Branch level 2 indexes are very large tables or low cardinality, nonunique foreign key indexes.

    SELECT table_name,index_name,num_rows "Rows"
      ,blevel "Branch",leaf_blocks "LeafBlocks"
FROM user_indexes WHERE index_type = 'NORMAL' AND num_rows > 0
ORDER BY blevel, table_name, index_name;

                                                         Leaf
TABLE_NAME      INDEX_NAME              Rows  Branch  Blocks
-------------   -------------------     ----  ------  ------
CATEGORY        AK_CATEGORY_TEXT          13       0       1
CATEGORY        XPKCATEGORY               13       0       1
COA             XFK_COA_SUBTYPE           55       0       1
COA             XFK_COA_TYPE              55       0       1
COA             XPKCOA                    55       0       1
POSTING         XFK_POSTING_CRCOA#         8       0       1
POSTING         XFK_POSTING_DRCOA#         8       0       1
POSTING         XPKPOSTING                 8       0       1
STOCK           XFK_STOCK_CATEGORY       118       0       1
STOCK           XPKSTOCK                 118       0       1
SUBTYPE         XPKSUBTYPE                 4       0       1
TYPE            XPKTYPE                    6       0       1
CASHBOOK        XPKCASHBOOK           188185       1     352
CASHBOOKLINE    XFK_CBL_CHEQUE        188185       1     418
CASHBOOKLINE    XFK_CBL_TRANS         188185       1     418
CASHBOOKLINE    XPKCASHBOOKLINE       188185       1     423
CUSTOMER        AK_CUSTOMER_NAME        2694       1      17
CUSTOMER        AK_CUSTOMER_TICKER      2694       1      12
CUSTOMER        XPKCUSTOMER             2694       1       5
ORDERS          XFK_ORDERS_CUSTOMER    31935       1      67
ORDERS          XFK_ORDERS_CUSTOMER   140369       1     293
ORDERS          XFK_ORDERS_TYPE       172304       1     313
ORDERS          XPKORDERS             172304       1     323
STOCK           AK_TEXT                  118       1       2
STOCKSOURCE     XFK_STOCKSOURCE_STOCK  12083       1      24
STOCKSOURCE     XFK_STOCKSOURCE_       12083       1      26
                   SUPPLIER
STOCKSOURCE     XPKSUPPLIER_STOCK      12083       1      46
SUPPLIER        AK_SUPPLIER_NAME        3874       1      27
SUPPLIER        AK_SUPPLIER_TICKER      3874       1      17
SUPPLIER        XPKSUPPLIER             3874       1       7
TRANSACTIONS    XFX_TRANS_CUSTOMER     33142       1      70
TRANSACTIONS    XFX_TRANS_ORDER       173511       1     386
TRANSACTIONS    XFX_TRANS_SUPPLIER    155043       1     324
TRANSACTIONS    XFX_TRANS_TYPE        188185       1     341
TRANSACTIONS    XPKTRANSACTIONS       188185       1     378
CASHBOOK        XFK_CB_CRCOA#         188185       2     446
CASHBOOK        XFK_CB_DRCOA#         188185       2     446
GENERALLEDGER   AK_GENERALLEDGER_1    752744       2    4014
GENERALLEDGER   AK_GL_DTE             752744       2    1992
GENERALLEDGER   XFK_GL_COA#           752744       2    1784
GENERALLEDGER   XPKGENERALLEDGER      752744       2    1425
ORDERSLINE      XFK_ORDERLINE_ORDER   540827       2    1201
ORDERSLINE      XFK_ORDERLINE_SM      540827       2    1204
ORDERSLINE      XPKORDERSLINE         540827       2    1215
STOCKMOVEMENT   XFK_SM_STOCK          570175       2    1124
STOCKMOVEMENT   XPKSTOCKMOVEMENT      570175       2    1069
TRANSACTIONS    XFX_TRANS_CRCOA#      188185       2     446
TRANSACTIONS    XFX_TRANS_DRCOA#      188185       2     446
TRANSACTIONS-   XFK_TRANSLINE_SM      570175       2    1270
   LINE
TRANSACTIONS-   FK_TRANSLINE_TRANS    570175       2    1267
   LINE
TRANSACTIONS-   XPKTRANSACTIONSLINE   570175       2    1336
   LINE

That's enough about BTree indexes for now. We will cover some nonphysical tuning aspects shortly. Let's take a look at the other Oracle Database index types.

7.3.3 Read-Only Indexing

Read-only indexes are generally applicable to reporting and data warehouse databases. This book is not intended as a data warehouse book. However, these types of indexes can be used in online and client-server databases with one restriction: the more table changes are made the more often these types of indexes must be rebuilt. There are a number of big issues with respect to non-BTree indexes.

  • They can be extremely fast in proper circumstances. Read-only indexes are designed for exactly that and are built to perform well when reading data.

  • Two issues are associated with read-only indexing when making changes to tables.

    • When changed, these indexes will generally overflow and rapidly become less efficient. Future read access requiring overflowed rows will search outside the original index structure. The point of an index is searching of preordered physical space.

    • The more changes made, the slower future changes will be.

Let's take a brief look at the different types of read-only indexes.

Bitmap Indexes

A bitmap index stores a value of 0 or 1 for a ROWID. The ROWID points to a row in a table. In Figure 7.2 a bitmap index is created on a column containing codes for states.

Click To expand
Figure 7.2: A Bitmap Index

There are some things to be remembered when considering using bitmap indexes.

  • In general bitmaps are effective for single-column indexes with low cardinality. Low cardinality implies very few different values. However, bitmap indexes can perform relatively well with thousands of different values. The size of a bitmap index will be smaller than that of a BTree and thus search speed is increased by searching through less physical space.

  • Bitmap indexes are much faster than BTree indexes in read-only environments.

  • Probably the most significant performance issue with bitmap indexes is that unlike BTree indexes bitmaps will lock at the block level rather than the row level. Since a bitmap index will have more index entries per block due to less space used, it becomes even more likely that locking will cause performance problems with UPDATE and DELETE commands. UPDATE and DELETE commands require exclusive locks.

  • Bitmap indexes should not be used to create composite-column indexes. Composite values are likely to have many different values (high cardinality). If composite-column indexes are required simply create multiple single-column bitmap indexes on each column. SQL WHERE clause filtering in Oracle Database can match multiple single-column bitmap indexes in the same query, in any order.

  • Bitmap indexes may be subject to overflow. Quite often bitmaps are used in large data warehouse databases where changes are not made to tables other than complete reloads.

    Tip

    Traditionally any index type other than a BTree index involves overflow. Overflow involves links between blocks where values cannot fit into previously reserved index space. Overflow is detrimental to performance.

Like BTree indexes bitmap indexes should probably be regularly rebuilt. However, for large databases regular bitmap index rebuilds can consume more time than is acceptable. Therefore bitmap indexes are most appropriate for large, low cardinality, read-only, static or transactional data tables.

Let's do some experimenting and show that bitmap indexes are faster than BTree indexes.

Are Bitmap Indexes Faster than BTree Indexes?

The Transactions table contains a column called type having two possible values representing a customer or a supplier-based transaction.

SQL> SELECT type "Type", COUNT(type) FROM transactions
  GROUP BY type;

T  COUNT(TYPE)
-  -----------
P       175641
S        61994

I am going to create two temporary copies of the Transactions table. One will have a BTree index on the type column and the other a bitmap index.

CREATE TABLE btree_table AS SELECT * FROM transactions;
CREATE INDEX btree_index ON btree_table(type) COMPUTE
   STATISTICS;
CREATE TABLE bitmap_table AS SELECT * FROM transactions;
CREATE BITMAP INDEX bitmap_index ON bitmap_table(type)
      COMPUTE STATISTICS;

Since we will be using the EXPLAIN PLAN command to demonstrate we need to generate statistics for the new objects.

ANALYZE TABLE btree_table COMPUTE STATISTICS;
ANALYZE TABLE bitmap_table COMPUTE STATISTICS;

Looking at the query plans we can see that accessing the bitmap index is faster than using the BTree index. I used hints in order to force usage of the required indexes. The type = 'P' comparison accesses just shy of 75% of the table. The Optimizer will do a full table scan without the hints.

EXPLAIN PLAN SET statement_id='TEST' FOR
      SELECT /*+ INDEX(btree_table, btree_index) */ *
      FROM btree_table WHERE type = 'P';

Query                                 Cost     Rows     Bytes
-----------------------------------   ----   ------   -------
SELECT STATEMENT on                   1347    94093   3387348
 TABLE ACCESS BY INDEX ROWID on
   BTREE_TABLE                        1347    94093   3387348
  INDEX RANGE SCAN on BTREE_INDEX      171    94093

EXPLAIN PLAN SET statement_id='TEST' FOR
      SELECT /*+ INDEX(bitmap_table, bitmap_index) */ *
      FROM bitmap_table WHERE type = 'P';

Query                                 Cost     Rows     Bytes
-----------------------------------   ----   ------   -------
SELECT STATEMENT on                    806    94093   3387348
 TABLE ACCESS BY INDEX ROWID on        806    94093   3387348
  BITMAP_TABLE
  BITMAP CONVERSION TO ROWIDS on
   BITMAP INDEX SINGLE VALUE on
    BITMAP_INDEX

Timing the SQL statements we can see again that accessing the bitmap is much faster. A different-sized table with a different selection may have had a more significant difference.

SQL> SELECT COUNT(*) FROM (
  2 SELECT /*+ INDEX(btree_table, btree_index) */ *
  3 FROM btree_table WHERE type = 'P');

   COUNT(*)
-----------
      33142

Elapsed: 00:00:01.01

SQL> SELECT COUNT(*) FROM (
  2 SELECT /*+ INDEX(bitmap_table, bitmap_index) */ *
  3 FROM bitmap_table WHERE type = 'P');

   COUNT(*)
-----------
      33142

Elapsed: 00:00:00.00

Bitmap indexes can be significantly faster than BTree indexes. Costs in the two previous query plans decreased for the bitmap index from 1347 to 806, a 40% decrease in cost.

Tip

During the period in which these query plans were generated no

DML type changes were made to the two new tables.

Let's clean up by dropping the tables. Dropping the tables will drop the attached indexes.

DROP TABLE btree_table;
DROP TABLE bitmap_table;

Bitmap Index Locking

As already stated there is one particular case I remember where many bitmap indexes were used. The database was in excess of 200 Gb and had a fairly large combination online and reporting production database. When one of the bitmaps was changed to a BTree index, a process taking 8 h to run completed in less than 3 min. Two possible causes would have been bitmap index block level locks as a result of heavy DML activity or, secondly, perhaps uncontrollable bitmap index overflow.

Using Composite Bitmap Indexes

Let's do query plans and some speed checking with composite-column indexing.

CREATE TABLE btree_table AS SELECT * FROM stockmovement;
CREATE INDEX btree_index ON btree_table
   (stock_id,qty,price,dte)
      COMPUTE STATISTICS;
CREATE TABLE bitmap_table AS SELECT * FROM stockmovement;
CREATE BITMAP INDEX bitmap_index ON bitmap_table
      (stock_id,qty,price,dte) COMPUTE STATISTICS;

Since we will be using the EXPLAIN PLAN command to demonstrate we need to generate statistics for the new objects.

ANALYZE TABLE btree_table COMPUTE STATISTICS;
ANALYZE TABLE bitmap_table COMPUTE STATISTICS;

Start with query plans using the EXPLAIN PLAN command.

EXPLAIN PLAN SET statement_id='TEST' FOR
      SELECT /*+ INDEX(btree_table, btree_index) */ *
      FROM btree_table
      WHERE stock_id IN (SELECT stock_id FROM stock)
      AND qty > 0
      AND price > 0
      AND dte < SYSDATE;

Query                                 Cost     Rows     Bytes
-----------------------------------   ----   ------   -------
SELECT STATEMENT on                  10031    16227    105675
 TABLE ACCESS BY INDEX ROWID on
   BTREE_TABLE                          85      138      3036
  NESTED LOOPS on                    10031    16227    105675
   INDEX FULL SCAN on XPK_STOCK          1      118       354
   INDEX RANGE SCAN on BTREE_INDEX      20      138

EXPLAIN PLAN SET statement_id='TEST' FOR
      SELECT /*+ INDEX_COMBINE(bitmap_table, bitmap_index) */ *
      FROM bitmap_table
      WHERE stock_id IN (SELECT stock_id FROM stock)
      AND qty > 0
      AND price > 0
      AND dte < SYSDATE;

Query                                 Cost     Rows     Bytes
-----------------------------------   ----   ------   -------
SELECT STATEMENT on                   7157    16227    405675
 NESTED LOOPS on                      7157    16227    405675
  TABLE ACCESS BY INDEX ROWID on      7156    16227    356994
   BITMAP CONVERSION TO ROWIDS on
    BITMAP INDEX FULL SCAN on
     BITMAP_INDEX
 INDEX UNIQUE SCAN on XPK_STOCK                   1         3

Now some speed testing.

SQL> SELECT COUNT(*) FROM(
  2 SELECT /*+ INDEX(btree_table, btree_index) */ *
  3 FROM btree_table
  4 WHERE stock_id IN (SELECT stock_id FROM stock)
  5 AND qty > 0
  6 AND price > 0
  7 AND dte < SYSDATE);

   COUNT(*)
-----------
     115590

Elapsed: 00:00:00.09

SQL> SELECT COUNT(*) FROM(
  2 SELECT /*+ INDEX(bitmap_table, bitmap_index) */ *
  3 FROM bitmap_table
  4 WHERE stock_id IN (SELECT stock_id FROM stock)
  5 AND qty > 0
  6 AND price > 0
  7 AND dte < SYSDATE);

   COUNT(*)
-----------
     115590

Elapsed: 00:00:00.02

We can see that even a composite bitmap index is faster than a composite BTree index. However, the cost decrease is 10,031 to 7,157, only a 30% decrease in cost. It could probably be concluded that composite bitmap indexes do not perform as well as single-column bitmap indexes.

Tip

During the period in which these query plans were generated no DML type changes were made to the two new tables.

Now let's break the bitmap index into separate single-column indexes, run the query plan and timing tests again and see if we get a performance improvement over the composite bitmap index. Drop the previous version of the bitmap index and create four separate bitmap indexes.

DROP INDEX bitmap_index;
CREATE BITMAP INDEX bitmap_index _stock _id ON bitmap table
   (stock id);
CREATE BITMAP INDEX bitmap_index_qty ON bitmap_table(qty);
CREATE BITMAP INDEX bitmap_index_price ON bitmap_table(price);
CREATE BITMAP INDEX bitmap_index_dte ON bitmap_table(dte);

Note that the hint has changed. The INDEX_COMBINE hint attempts to force usage of multiple bitmap indexes.

EXPLAIN PLAN SET statement_id='TEST' FOR
      SELECT /*+ INDEX_COMBINE(bitmap_table) */ * FROM
         bitmap_table
      WHERE stock_id IN (SELECT stock_id FROM stock)
      AND qty > 0
      AND price > 0
      AND dte < SYSDATE;

Query                                 Cost     Rows     Bytes
-----------------------------------   ----   ------   -------
SELECT STATEMENT on                   1004    16226    405650
 NESTED LOOPS on                      1004    16226    405650
  TABLE ACCESS BY INDEX ROWID on
    BITMAP_TABLE                      1004    16226    356972
   BITMAP CONVERSION TO ROWIDS on
    BITMAP AND on
     BITMAP MERGE on
      BITMAP INDEX RANGE SCAN on
       BITMAP_INDEX_DTE
     BITMAP MERGE on
      BITMAP INDEX RANGE SCAN on
       BITMAP_INDEX_QTY
     BITMAP MERGE on
      BITMAP INDEX RANGE SCAN on
       BITMAP_INDEX_PRICE
 INDEX UNIQUE SCAN on XPK_STOCK                   1         3

The cost difference between the composite bitmap index and the single-column bitmap indexes has increased significantly. Using single-column bitmap indexes is much faster than using composite bitmap indexes.

Tip

During the period in which these query plans were generated no DML type changes were made to the two new tables.

Now let's clean up.

DROP TABLE btree_table;
DROP TABLE bitmap_table;

Do Bitmap Indexes Overflow?

In the previous examples creating the indexes after creating and populating the tables built the indexes in the most efficient way. Now I am going to create a bitmap index in a slightly less appropriate place on a column with lower cardinality. The purpose here is to demonstrate the potential deterioration of bitmap indexes due to overflow. Following is a count of the rows of repeated values of the GeneralLedger.COA# column. The total row count is currently 1,045,681 rows.

SQL> SELECT coa#, COUNT(coa#) FROM generalledger GROUP BY
   coa#;

COA#    COUNT(COA#)
-----   -----------
30001        377393
40003        145447
41000        246740
50001        203375
50028             1
60001         72725

Firstly, we need to do a query plan for the current existing BTree index on the COA# column foreign key BTree index.

EXPLAIN PLAN SET statement_id='TEST' FOR
      SELECT /*+ INDEX(generalledger, xfk_coa#) */ *
      FROM generalledger WHERE coa# = '60001';

Query                                 Cost     Rows     Bytes
-----------------------------------   ----   ------   -------
SELECT STATEMENT on                   3441   177234   4076382
 TABLE ACCESS BY INDEX ROWID on
   GENERALLEDGER                      3441   177234   4076382
  INDEX RANGE SCAN on XFK_COA#         422   177234

Now I will change the foreign key index to a bitmap, generate statistics and run the EXPLAIN PLAN command again. Clearly the bitmap is faster than the BTree index.

DROP INDEX xfk_coa#;
CREATE BITMAP INDEX xfk_coa#_bitmap ON generalledger(coa#)
      COMPUTE STATISTICS;
ANALYZE TABLE generalledger COMPUTE STATISTICS;

EXPLAIN PLAN SET statement_id='TEST' FOR
      SELECT /*+ INDEX(generalledger, xfk_coa#_bitmap) */ *
      FROM generalledger WHERE coa# = '60001';

Query                                 Cost     Rows     Bytes
-----------------------------------   ----   ------   -------
SELECT STATEMENT on                   1707   177234   4076382
 TABLE ACCESS BY INDEX ROWID on       1707   177234   4076382
   GENERALLEDGER
  BITMAP CONVERSION TO ROWIDS on
   BITMAP INDEX SINGLE VALUE on
     XFK_COA#_BITMAP

Notice how the cost of using the bitmap index is much lower than that of using the BTree index.

In the next step I restart my database and begin processing to execute a large amount of concurrent activity, letting it run for a while. The objective is to find any deterioration in the bitmap index due to excessive DML activity. The row count for the GeneralLedger table is now 10,000 rows more than previously.

ANALYZE TABLE generalledger COMPUTE STATISTICS;
ANALYZE INDEX xfk_coa#_bitmap COMPUTE STATISTICS;

EXPLAIN PLAN SET statement_id='TEST' FOR
      SELECT /*+ INDEX(generalledger, xfk_coa#_bitmap) */ *
      FROM generalledger WHERE coa# = '60001';

Query                                 Cost     Rows     Bytes
-----------------------------------   ----   ------   -------
SELECT STATEMENT on                   1729   178155   4097565
 TABLE ACCESS BY INDEX ROWID on       1729   178155   4097565
   GENERALLEDGER
  BITMAP CONVERSION TO ROWIDS on
   BITMAP INDEX SINGLE VALUE on
     XFK_COA#_BITMAP

We can see from the two previous query plans that the cost of using the bitmap index has decreased by over 1% after adding around 10,000 rows to the GeneralLedger table. Now I will run the same highly concurrent DML commands and add some more rows to the GeneralLedger table, only this time we will change the bitmap index back to a BTree index and compare the relative query cost deterioration.

DROP INDEX xfk_coa#_bitmap;
CREATE INDEX xfk_coa# ON generalledger(coa#);

Once again I have executed my concurrent processing and added another 10,000 rows to the GeneralLedger table. This time the BTree index is back in place and there is no longer a bitmap index. When processing is complete we analyze statistics and run the EXPLAIN PLAN command again.

ANALYZE TABLE generalledger COMPUTE STATISTICS;
ANALYZE INDEX xfk_coa# COMPUTE STATISTICS;

EXPLAIN PLAN SET statement_id='TEST' FOR
      SELECT /*+ INDEX(generalledger, xfk_coa#) */ *
      FROM generalledger WHERE coa# = '60001';

Query                                 Cost     Rows     Bytes
-----------------------------------   ----   ------   -------
SELECT STATEMENT on                   3466   178155   4097565
 TABLE ACCESS BY INDEX ROWID on
   GENERALLEDGER                      3466   178155   4097565
  INDEX RANGE SCAN on XFK_COA#         425   178155

The increase in cost is slightly below 1% for the BTree index. This is unexpected. Where INSERT statements are concerned bitmap indexes are not subject to much overflow on this scale. At a much larger scale the situation may be very different.

Bitmap Join Indexes

A bitmap join index is simply a bitmap containing ROWID pointers of columns or keys used in the join from the tables joined together. The index is created using an SQL join query statement.

Clusters

A cluster is quite literally a cluster or persistent "joining together" of data from one or more sources. These multiple sources are tables and indexes. In other words, a cluster places data and index space rows together into the same object. Obviously clusters can be arranged such that they are very fast performers for read-only data. Any type of DML activity on a cluster will overflow. Rows read from overflow will be extremely heavy on performance. Clusters are intended for data warehouses.

A standard cluster stores index columns for multiple tables and some or all nonindexed columns. A cluster simply organizes parts of tables into a combination index and data space sorted structure. Datatypes must be consistent across tables.

Hash Clusters

A hash cluster is simply a cluster indexed using a hashing algorithm. Hash clusters are more efficient than standard clusters and are even more appropriate to read-only type data. In older relational databases hash indexes were often used against integer values for better data access speed. If data was changed the hash index had to be rebuilt.

Sorted Hash Clusters

Oracle Database 10 Grid A hash cluster essentially breaks up data into groups of hash values. Hash values are derived from a cluster key value forcing common rows to be stored in the same physical location. A sorted hash cluster has an additional performance benefit for queries, accessing rows in the order in which the hash cluster is ordered: thus the term sorted hash cluster.

Index-Organized Tables

A cluster is a join of more than one table. An index-organized table is a single table, physically organized in the order of a BTree index. In other words, the data space columns are added into the index structure leaf blocks of a binary tree. An index-organized table is effectively a BTree table where the entire table is the index. Block space is occupied by both data and index values. Much like clusters the sorted structure of an index-organized table is not usually maintained efficiently by DML activity. Any changes to any of the columns in the table will affect the index directly because the table and the index are one and the same thing. It is likely that any DML activity will cause serious performance problems due to overflow and skewing. On the contrary, there are reports of good performance gains using index-organized tables in OLTP type databases. However, I am currently unaware of exactly how these tables are used in relation to DML activity.


7.4 Tuning BTree Indexes

This is not a data warehouse tuning book. Tuning data warehouses is another subject. Any read-only indexes should generally not be used in OLTP or client-server databases. Therefore, this book will concentrate on potential tuning issues with BTree indexes. Some tuning issues have already been covered in this chapter. These issues have dealt with the philosophical side of how and what to index, some details about BTree indexing and use of read-only indexes.

What can be done with BTree indexes to make them work faster?

7.4.1 Overflow and Rebuilding

Oracle Database BTree indexes do not overflow but they can become skewed. Skewing can happen with a binary tree for instance when a large percentage of index values belong to a single branch. Take a look at the BTree representation in Figure 7.3. Figure 7.3 is a picture of what the Accounts schema GeneralLedger.COA# column foreign key index might look like physically. The set of leaf blocks for branch 1 has over 300,000 rows, about 40% of the total rows. Branch 4 has over 30,000 rows, about 5% of the total rows. That is not proportionate. In other words when searching for COA# = '30001' 40% of the total physical space is searched, when searching for COA# = '60001' only 5% of the total physical space is searched. This is a potentially skewed binary tree because it is heavier in some parts.

Click To expand
Figure 7.3: A Skewed BTree Index

The example in Figure 7.3 is a poor example and is more due to the nature of the GeneralLedger.COA# column data. The reason is because most skewing of binary trees occurs when data is changed such that one particular area of a table is added to rather than another. The result is a binary tree heavily loaded in one or more specific places. The temptation in a case such as this is to use a bitmap index. Don't! The GeneralLedger table is a heavily DML active table. Over a long period of time a bitmap index could deteriorate to the point of uselessness.

The answer to skewed binary indexes is the ALTER INDEX command using the REBUILD ONLINE options. Rebuilding an index will rebalance a BTree index and help to spread everything out more evenly, particularly with respect to overflow. The ONLINE option will prevent disruption of current DML activity.

Lost Index Space

Let's dive just a little into some physical index tuning. When setting block storage management parameters for tables and indexes there are numerous modifiable parameters. Two of these parameters are PCTUSED and PCTFREE. For tables there are two parameters and for indexes there is one. PCTUSED allows reuse of a block when the occupied space in a block falls below the percentage specified, as a result of DML DELETE commands. The PCTFREE parameter allows allocation of free space to a block only usable by future DML UPDATE commands, including expanding column values. PCTFREE helps to avoid row chaining. PCTUSED and PCTFREE will be covered in detail in physical tuning. The issue with indexing is that an index cannot have a PCTUSED parameter. What does this tell us about indexes? Deleted index space is not reclaimed unless an index is rebuilt using the ALTER INDEX index-name REBUILD; command.

Tip

PCTUSED and PCTFREE settings cannot be changed for already existing blocks.
Note

Oracle Database 10 Grid PCTUSED parameter values are ignored in locally managed table-spaces using automated segment space management. Automated segment space management will be covered in Part III. Automated segment space management is an accepted practice in Oracle9i Database and recommended in Oracle Database 10g.

So large numbers of row insertions and deletions can over time denigrate the quality of BTree indexes. It is advisable to regularly rebuild Oracle Database BTree indexes for this reason, if possible. The REBUILD ONLINE option will prevent interference with DML activity during an index rebuild.

It is possible to use the ALTER INDEX command with the COALESCE and DEALLOCATE UNUSED options. These options will not help very much and might make things worse. COALESCE attempts to amalgamate empty extents and DEALLOCATE UNUSED tries to reclaim space above the high water mark, both rare possibilities. On a consulting job in the past I executed a COALESCE command on some highly deteriorated BTree indexes only to find the datafile containing those indexes about 25% larger after completion. Use index rebuilds. Once again indexes can be rebuilt online as well without disrupting DML activity.

7.4.2 Reverse Key Indexes

Reverse key indexes are useful for high cardinality (unique values) in highly concurrent multi-user environments. Where index values are created in a sequential sequence there can be problems with contention for index blocks when multiple DML statements are accessing the same block. The sequential numbers will all be added into the same index block. This kind of problem often occurs with INSERT statements using incremental Oracle Database sequences as primary keys. All the INSERT commands take a sequence number close to every other recently generated sequence. Thus all those sequences will be placed into the same block of the index causing contention for the block and causing serious availability problems. This is known as "hot blocking".

An answer to this problem is using reverse key indexes. What is a reverse key index? A reverse key index simply stores the bytes in the indexed column value, into the index, with the string of the bytes in reverse order as shown in Figure 7.4.

Click To expand
Figure 7.4: A Reverse Key Index

How do reverse key indexes resolve high concurrency contention for index blocks on sequential key values? The act of reversing the bytes in the keys serves to place those index values into separate blocks, thus avoiding contention for a single index block. The down-side to use of reverse key indexes is that no efficient BTree tree traversals can be performed and thus no index range scans. The index values are randomly scattered across the blocks of the index. The order of the index is maintained and thus the index remains usable in the required order.

In the Accounts schema the GeneralLedger table is only ever inserted into and the primary key is a sequence number. This primary key is a candidate for a reverse key index. Since the GeneralLedger table is large and is accessed with DML commands other than INSERT it is probably unwise to make this table's primary key index a reverse key index.

Two things warrant mentioning. Firstly, attempting to detect index block contention is physical tuning and is not appropriate for this chapter. Secondly, reverse index keys generally only apply in high concurrency clustered Oracle Database RAC (Parallel Server) environments. Reverse key indexes allow for efficient heavy insertion activity but can be problematic for any sequentially ordered index matching or filtering activity unless searching for an exact row (no index range scans).

7.4.3 Compressed Composite Indexes

Compression of BTree indexes applies to composite-column indexes only. A composite index sometimes represents a many-to-many join resolution join entity or a master detail relationship between a parent and child table; the composite index is placed on the child table.

Compressing the index allows removal of repeating index column values from leaf blocks to a prefix. Figure 7.5 shows an example of what index compression does to an index; duplications are removed. A compressed index maintains a prefix key value pointing to ROWID pointers, which point to the table rows. A compressed index saves a little space. For really large indexes it could potentially save a lot of space.

Click To expand
Figure 7.5: A Compressed Composite-Column Index

Index compression can have the following effects:

  • Decrease the physical size of the index allowing faster searching through less physical disk space.

  • Searching composite indexes may be problematic when not ordering columns in indexes according to the exact order of the composite columns.

  • DML activity may be a little slower for composite indexes due to the possible need to reconstruct the split structure between compressed and uncompressed columns.

  • In general searches using composite indexes cannot perform unique index hits and default to range scans. It is possible that exact hits could be obtained on the first prefix column of a compressed index since it is physically unique to the index.

  • In highly concurrent environments compressed composite indexes and the addition of prefix entries could potentially lock more rows in indexes. Compressed indexes may perform less effectively under conditions of high concurrency.

Obviously a single-column index cannot be compressed. There would not be any point since there is nothing to compress. When creating a compressed index one can compress up to the number of columns in the composite index less one. Let's use the TransactionsLine table for our first example.

EXPLAIN PLAN SET statement_id='TEST' FOR
      SELECT transaction_id, seq# FROM transactionsline;

Query                                 Cost     Rows     Bytes
-----------------------------------   ----   ------   -------
SELECT STATEMENT on                    283   739772   3698860
 INDEX FAST FULL SCAN on
   XPK_TRANSACTIONSLINE                283   739772   3698860

The following ALTER INDEX DDL command will compress the two-column primary key index in the Accounts schema TransactionsLine table.

ALTER INDEX xpk_transactionsline REBUILD ONLINE COMPUTE
      STATISTICS COMPRESS 1;

Now let's run the same query once again.

EXPLAIN PLAN SET statement_id='TEST' FOR
      SELECT transaction_id, seq# FROM transactionsline;

Query                                 Cost     Rows     Bytes
-----------------------------------   ----   ------   -------
SELECT STATEMENT on                    254   739772   3698860
 INDEX FAST FULL SCAN on
   XPK_TRANSACTIONSLINE                254   739772   3698860

In the previous TransactionsLine table examples there is a small decrease in cost on compressing the index. The cost has been reduced from 283 down to 254, a 10% cost decrease.

Let's now use a different table much more applicable to composite indexing. The following query shows that there are in excess of thousands of rows for every COA# value. This is not the case with the TransactionsLine table master detail relationship between the TRANSACTION_ID and SEQ# composite primary key columns.

SQL> SELECT coa#, COUNT(coa#) FROM generalledger GROUP BY
   coa#;

COA#    COUNT(COA#)
------  -----------
30001        377393
40003        145447
41000        246740
50001        203375
50028             1
60001         72725

Create a new index.

CREATE INDEX xak_composite ON generalledger(coa#,
   generalledger_id)
         ONLINE COMPUTE STATISTICS;

Find the cost of the use of the index uncompressed.

EXPLAIN PLAN SET statement_id='TEST' FOR
      SELECT coa#, generalledger_id FROM generalledger;

Query                                 Cost     Rows     Bytes
-----------------------------------   ----   ------   -------
SELECT STATEMENT on                    432   912085   9120850
 INDEX FAST FULL SCAN on
   XAK_COMPOSITE                       432   912085   9120850

Now using the COMPRESS clause we will remove duplicates of the COA# column from the index.

ALTER INDEX xak_composite REBUILD ONLINE COMPUTE STATISTICS
   COMPRESS 1;

Now use the EXPLAIN PLAN command again.

EXPLAIN PLAN SET statement_id='TEST' FOR
      SELECT coa#, generalledger_id FROM generalledger;

Looking at the following result, the cost is lower. The difference is now 432 down to 316, a 25% decrease in cost.

Query                                  Cost     Rows     Bytes
-----------------------------------    ----   ------   -------
SELECT STATEMENT on                     316   912085   9120850
 INDEX FAST FULL SCAN on XAK_COMPOSITE  316   912085   9120850

The TransactionsLine table examples showed a 10% decrease in cost and the GeneralLedger table examples showed a decrease in cost of much more at 25%. The more duplicates removed by compression the better the increase in performance.

On a more detailed level we can conclude that the relationship between the TRANSACTION_ID and SEQ# columns should be in the range of thousands. This is not the case. The TransactionsLine table is probably inappropriate for a composite index. In a properly structured data model composite indexes would probably be inappropriate for most many-to-many join resolution entities and master detail relationships unless a traditional composite parent-child data model structure was used. Why? There are generally not enough repetitions of the prefix (the compressed columns) of the compressed index to warrant its use. The GeneralLedger.COA# column values repeat enough to show a marked performance increase.

Compressed Indexes and DML Activity

Do compressed indexes affect the performance of DML activity? A DML command executed using a compressed index may incur a small amount of overhead as opposed to an uncompressed index. Suffix index values are not stored directly with prefix values and further interpretation is required. Here I am going to attempt to compare DML performance between compressed and uncompressed indexes.

The rows for GeneralLedger.COA# = '60001' table entries account for over 5% of the table. The chances are the Optimizer will often choose a full table scan rather than an index read. I am going to force the Optimizer to use the composite index with a hint to avoid full table scans because we are trying to compare indexes. We will start with the index uncompressed. I am using an UPDATE and a DELETE statement. INSERT is irrelevant in this case.

ALTER INDEX xak_composite REBUILD ONLINE COMPUTE STATISTICS
   NOCOMPRESS;

EXPLAIN PLAN SET statement_id='TEST' FOR
      UPDATE /*+ INDEX(generalledger, xak_composite) */
            generalledger SET dr=0, cr=0 WHERE coa# =
               '60001';

The cost of the UPDATE query is high. The Optimizer probably would have selected a full table scan without the hint.

Query                                 Cost     Rows     Bytes
-----------------------------------   ----   ------   -------
UPDATE STATEMENT on                   3016   156159   1717749
 UPDATE on GENERALLEDGER
  INDEX RANGE SCAN on XAK_COMPOSITE    488   156159   1717749

EXPLAIN PLAN SET statement_id='TEST' FOR
      DELETE FROM /*+ INDEX(generalledger, xak_composite) */
            generalledger WHERE coa# = '60001';

Query                                 Cost     Rows     Bytes
-----------------------------------   ----   ------   -------
DELETE STATEMENT on                    488   156159   1561590
 DELETE on GENERALLEDGER
  INDEX RANGE SCAN on XAK_COMPOSITE    488   156159   1561590

Now let's compress the index.

ALTER INDEX xak_composite REBUILD ONLINE COMPUTE STATISTICS
   COMPRESS 1;

EXPLAIN PLAN SET statement_id='TEST' FOR
      UPDATE /*+ INDEX(generalledger, xak_composite) */
            generalledger SET dr=0, cr=0 WHERE coa# =
               '60001';

Query                                 Cost     Rows     Bytes
-----------------------------------   ----   ------   -------
UPDATE STATEMENT on                   2898   156622   1722842
 UPDATE on GENERALLEDGER
  INDEX RANGE SCAN on XAK_COMPOSITE    359   156622   1722842

EXPLAIN PLAN SET statement_id='TEST' FOR
      DELETE FROM /*+ INDEX(generalledger, xak_composite) */
            generalledger WHERE coa# = '60001';

Query                                 Cost     Rows     Bytes
-----------------------------------   ----   ------   -------
DELETE STATEMENT on                    359   156622   1566220
 DELETE on GENERALLEDGER
  INDEX RANGE SCAN on XAK_COMPOSITE    359   156622   1566220

We can see from the four preceding query plans that DML activity is actually speeded up with the compressed index, even in an active database. Commercial databases with thousands of transactions per second may behave differently.

Let's clean up by dropping extra indexes and uncompressing those still compressed for both the TransactionsLine and GeneralLedger tables. It is important to clean up after yourself.

ALTER INDEX xpk_transactionsline REBUILD ONLINE COMPUTE
   STATISTICS NOCOMPRESS;
DROP INDEX ak_composite;

7.4.4 Function-Based Indexes

A regular BTree index contains column values of indexed columns in index leaf blocks, which link to tables using ROWID pointers. A function-based index contains the result of an expression on a column or column values, placing the expression result into the index leaf blocks. How is this relevant to tuning? A regular BTree index on a column will be completely ignored by the Optimizer when a column is part of an expression in an SQL statement. Figure 7.6 shows an example of a function-based index.

Click To expand
Figure 7.6: A Function-Based Index

Some specific settings are required in Oracle Database to allow use of function-based indexes.

  • The cost-based Optimizer is required.

  • The user must have:

    • The QUERY_REWRITE system privilege.

    • Execute privileges on any user-defined functions.

  • Oracle Database configuration parameters must be set as follows:

    • QUERY_REWRITE_ENABLED = TRUE.

    • QUERY REWRITE_INTEGRITY = TRUSTED.

Let's go through some examples of the uses of function-based indexing.

The following example simply shows a full table scan based on a WHERE clause filter on an expression of two columns.

Individual indexes on the QTY and PRICE columns would make no difference to query plan cost. I am creating these indexes to prove the point.

CREATE INDEX xak_sm_qty ON stockmovement(qty)
      ONLINE COMPUTE STATISTICS;
CREATE INDEX xak_sm_price ON stockmovement(price)
      ONLINE COMPUTE STATISTICS;

EXPLAIN PLAN SET statement_id='TEST' FOR
      SELECT * FROM stockmovement WHERE qty*price = 0;

Query                                 Cost     Rows     Bytes
-----------------------------------   ----   ------   -------
SELECT STATEMENT on                    404     6456    135576
 TABLE ACCESS FULL on STOCKMOVEMENT    404     6456    135576

Let's drop those two indexes.

DROP INDEX xak_sm_qty;
DROP INDEX xak_sm_price;

Now let's go ahead and create a function-based index for the expression QTY*PRICE. My database is highly active. Once again I use the ONLINE option of the CREATE INDEX command allowing DML activity against the index during the index creation process. The ONLINE option creates the index elsewhere and copies it to the index file after creation, including any changes during the copying process. Once again the EXPLAIN PLAN command is useless without current statistics generation.

CREATE INDEX xakfb_sm_amount ON stockmovement(qty*price)
      ONLINE COMPUTE STATISTICS;

EXPLAIN PLAN SET statement_id='TEST' FOR
      SELECT * FROM stockmovement WHERE qty*price = 0;

Notice in the query plan how the newly created function-based index is being range scanned. If the order of the columns in the expression was switched to PRICE*QTY the index would still be used since the function-based index contains the result of the expression. The next example will reverse the order of the QTY and PRICE columns in the expression.

Query                                 Cost     Rows     Bytes
-----------------------------------   ----   ------   -------
SELECT STATEMENT on                      5     6456    135576
 TABLE ACCESS BY INDEX ROWID on          5     6456    135576
   STOCKMOVEMENT
  INDEX RANGE SCAN on XAKFB_SM_AMOUNT    3     2582

In the following example I have placed the expression into the SELECT statement alone, allowing a full index scan as opposed to a full table scan. Also the order of the QTY and PRICE columns is the reverse of the previous example.

EXPLAIN PLAN SET statement_id='TEST' FOR
      SELECT price*qty FROM stockmovement;

Query                                    Cost     Rows     Bytes
--------------------------------------   ----   ------   -------
SELECT STATEMENT on                       382   645604   4519228
 INDEX FAST FULL SCAN on XAKFB_SM_AMOUNT  382   645604   4519228

We can also use user-defined functions to create function-based indexes.

CREATE OR REPLACE FUNCTION getAmount(
      pqty IN INTEGER DEFAULT 0
      ,pprice IN FLOAT DEFAULT 0) RETURN NUMBER
         DETERMINISTIC IS
BEGIN
      RETURN pqty*pprice;
END;
/
ALTER FUNCTION getAmount COMPILE;

Now let's drop and recreate the index again to use the user-defined function.

DROP INDEX xakfb_sm_amount;
CREATE INDEX xakfb_sm_amount ON stockmovement(getAmount
   (qty, price))
      ONLINE COMPUTE STATISTICS;

The following query plan should show a fast full index scan of the index created using the user-defined function.

EXPLAIN PLAN SET statement_id='TEST' FOR
      SELECT getAmount(qty, price) FROM stockmovement;

Query                                   Cost     Rows     Bytes
-------------------------------------   ----   ------   -------
1. SELECT STATEMENT on                   561   570298   2851490
2.  TABLE ACCESS FULL on STOCKMOVEMENT   561   570298   2851490

We are supposed to be able to use a function in a function-based index. The QTY*PRICE index and function will not be used again so they will be dropped. Leaving unused indexes in the database is bad for performance. Whenever a table is updated all its associated indexes are updated as well.

DROP INDEX xakfb_sm_amount;
DROP FUNCTION getAmount;

Function-based indexes can be useful. The biggest danger with function-based indexes is that developers could create a lot of them. It may be best to only create function-based indexes at somewhere between the development and production release cycles. This is especially the case if database administrators are completely uninvolved in the development process, which is often the case.

7.4.5 NULLs and Indexes

What do we need to know about NULL values and indexes? NULL values are not included in indexes except for bitmap indexes. Therefore when using an index against a table where NULL values exist, not only can the index be read ignoring table columns, but also only non-NULL valued rows can be read. Thus reading the index will not only read fewer columns but could potentially read fewer rows as well.

The Transactions table in the Accounts schema is a good example. A type = 'S' transaction will have a NULL SUPPLIER_ID and a TYPE = 'P' transaction will have a NULL CUSTOMER_ID. These are the counts for different row types in the Transactions table.

SELECT t.counter "Transactions", s.counter "Sales"
 ,p.counter "Purchases", type_S.counter "Type=S"
 ,type_P.counter "Type=P"
FROM (SELECT COUNT(*) as counter FROM transactions) t
 ,(SELECT COUNT(*) as counter FROM transactions
   WHERE customer_id IS NOT NULL) s
 ,(SELECT COUNT(*) as counter FROM transactions
   WHERE supplier_id IS NOT NULL) p
 ,(SELECT COUNT(*) as counter FROM transactions
   WHERE type = 'S') type_S
 ,(SELECT COUNT(*) as counter FROM transactions
   WHERE type = 'P') type_P;
Transactions   Sales   Purchases   Type=S   Type=P
------------   -----   ---------   ------   ------
      188199   33158      155041    33158   155041

The CUSTOMER_ID and SUPPLIER_ID columns already have foreign key indexes. I do not have to create any new indexes. Let's use the CUSTOMER_ID column as an example. Note from the previous query result that the Transactions table has a total of 188,199 rows. Now we can force reading the index and demonstrate the resulting much lower number of rows accessed by reading only the non-NULL valued index rows.

EXPLAIN PLAN SET statement_id='TEST' FOR
      SELECT customer_id FROM transactions
      WHERE customer_id IS NOT NULL;

We now appear to have read 33,142 rows. The number of rows accessed is much fewer than the total number of rows in the table as expected. However, and quite confusingly so, there are 33,158 non-NULL valued CUSTOMER_ID rows and the fast full index scan reads slightly fewer rows. Why I do not know.

Query                                      Cost     Rows     Bytes
----------------------------------------   ----   ------   -------
1. SELECT STATEMENT on                       16    33142     66284
2.  INDEX FAST FULL SCAN on XFK_T_CUSTOMER   16    33142     66284

One point should be noted. If we want to read the NULL valued CUSTOMER_ID rows from the Transactions table we could change the CUSTOMER_ID column to NOT NULL and add a DEFAULT setting. We could subsequently search the index for the DEFAULT value and get the same result as searching for IS NULL on the table. This would ensure that we never get NULL values in the CUSTOMER_ID column and that a CUSTOMER_ID index can always be used instead of a full table scan. However, why would one ever want to read NULL values? In this case one could simply SELECT using the SUPPLIER_ID column as NOT NULL and get the same result, again reading fewer index entries. If many SQL code statements are accessing tables with IS NULL comparisons, you might want to revisit your data model. IS NULL is a form of a negative comparison condition because you are attempting to find something which is not there. Think about why you are doing something like this. This type of SQL code is consistent with an approach to high usage of outer joins in SQL code. High numbers of outer joins are sometimes indicative of poor data Referential Integrity or perhaps data model structure not providing for application requirements. The same potential indicators are apparent when searching using IS NULL conditions. Once again why would you want to be searching for something which is simply not there?

That covers NULL values and indexing.

So how can BTree indexes be tuned for better performance?

  • Use as few columns as possible.

  • Only index integers if possible, preferably unique integer identifier values.

  • Variations on BTree indexes are reversing index strings, compression and function-based indexes. Each of these variations will generally have very specific applications. They should be used sparingly and carefully.

  • NULL values are not indexed.


7.5 Summarizing Indexes

This concludes the discussion of what could be termed "Common Sense Indexing". Handling indexing in Oracle Database is somewhat based on indexing in any relational database. Indexing in a database should be as close as possible to indexing provided by the data model, particularly Referential Integrity indexing. If you have too much alternate and secondary indexing you may want to consider revisiting your data model, if this is possible. If your data model does not provide the basic structural functionality for attached applications then application requirements may require significant alteration to your data model. There is much to be said for building the data model with application functionality in mind. On the other hand, if your applications require large numbers of joins and additions to the data model it could be that performance may benefit substantially from creation of a separate reporting or data warehouse database. Highly active changeable concurrent environments such as OLTP databases and applications have very different requirements to that of any type of reporting or data warehouse functionality. The two simply do not mix very well. Mixing large quantities of small transactions in an OLTP database with large throughput of reporting and data warehouse activity can cause severe performance issues.

In the next chapter we will revert back to the study of how to perform SQL code tuning. So far in this book we have not dug too far into the internal specifics of tuning SQL code in Oracle Database. You should now have a basic knowledge of how to tune SQL code and build proper indexing in general. The next chapter will start to examine the explicit details of how to tune SQL code in Oracle Database.


Chapter 8: Making SQL Efficient in Oracle Database

Overview

So far in Part II, SQL Code Tuning, we have covered what SQL is, the basics of efficient SQL and common sense indexing. The first three chapters are not necessarily directly Oracle related and could apply to SQL in any relational database. Now we need to look at the specifics of the Oracle database with respect to SQL code tuning and the Oracle SQL code Optimizer.

The last two chapters made use of the EXPLAIN PLAN command extensively for demonstration purposes. All the types of data access shown in query plans will be described in detail in this chapter.

In this chapter we will try to understand how Oracle Database executes SQL code and how that SQL code can be tuned from a purely Oracle Database perspective. There will be some crossover in this chapter with configuration tuning. Numerous Oracle Database configuration parameters affect the way that SQL code is executed. The essence of this chapter is to discover what exactly Oracle Database does when an SQL statement is submitted to the Oracle Database engine. There are specific steps:

  • Parsing. Parsing is the verification of SQL code syntax and interpretive compilation into a form understandable by the Oracle Database engine.

  • Optimization. The Optimizer attempts to automatically pick the fastest method of execution for an SQL code statement.

  • Execution. SQL code statements are executed in various forms and by various routes, decided by the Optimizer.

  • Return of results. Results are passed back to the calling process.

Let's start with some brief descriptions.


8.1 What is the Parser?

What happens when an SQL statement is sent to the database?

A portion of memory is allocated to store the results of an SQL statement. This chunk of memory is called a cursor. Amongst other things a cursor contains a pointer to the row in an index or table currently being fetched from the database. Unless a cursor is declared explicitly in a tool such as PL/SQL then a cursor is implicitly created for every SQL SELECT statement submitted to the Oracle Database. When an SQL statement has finished executing, the cursor can either be left in memory as a reusable, already allocated chunk of memory or it can be removed from memory altogether.

The SQL statement is parsed. What is parsing? The meaning of the word parsing is that of syntactical and perhaps grammatical or semantic analysis of a sentence. In programming terms parsing is a syntax check on a line of program code and between related lines. SQL statements are lines of program code. So the parser is effectively a compiler of SQL statements, much like a C compiler is a compiler or syntax analyzer of C programming language commands.

The Oracle SQL parser does a few other things. There is a portion of memory or a buffer in the Oracle Database configuration called the shared pool. The shared pool is used to store parsed execution plans for previously executed SQL statements. These previously parsed execution plans can be used to re-execute the SQL statement if exactly, and I mean exactly, the same string is executed once again.

Note

Oracle Database 10 Grid Oracle Database 10g has Optimizer improvements such as less of a need for SQL code statements to be case sensitive.

Thus the Optimizer does not have to do its work all over again. The result is that if the parser finds an existing SQL code statement in the shared pool it does not have to call the Optimizer, and recalculate the execution plan for the SQL statement.

Parses can be split into two categories:

  • Hard parse. No match is found in the shared pool for an SQL statement never before submitted to the database engine.

  • Soft parse. A match is found for an SQL statement perhaps submitted to the database engine by a different session.

Hard parses are more expensive in system resources than soft parses but reparsing is best avoided altogether if possible.


8.2 What is the Optimizer?

The purpose of the Optimizer is to optimize the performance of SQL statements. The Optimizer decides the best way to execute an SQL code statement in the fastest way possible, based on information available to it. The Optimizer can use various methods of assessing the best path of performance for an SQL code statement. These methods include:

  • Rule-based approach.

    • Outlines.

  • Cost-based approach.

    • Statistics.

    • Hints.

    • Histograms.

    Note

    Oracle Database 10 Grid Rule-based Optimization is desupported.

Indexes against tables may or may not be used as the better option to execute an SQL statement. Sometimes the Optimizer will decide that reading an entire table is more efficient than reading a small part of an index, to access a ROWID pointer accessing a small number of table rows.

Through the years various versions of the Oracle SQL code Optimizer have become more and more intelligent. In Oracle9i Database it can be relied on fairly heavily to produce the best performance for SQL code.

Note

Oracle Database 10 Grid Oracle Database 10g has Optimizer improvements such as less of a need for SQL code statements to be case sensitive.

In general the capabilities of the Optimizer will deteriorate as the complexity of SQL code increases and particularly the quality of SQL code decreases. The Optimizer should not be affected by database size unless SQL code or data model are poorly tuned and constructed. Database size merely exacerbates problems caused by poor design. Unfortunately these circumstances happen often. In OLTP systems SQL code can sometimes become so complex and convoluted that the Optimizer simply cannot cope and executes full table scans on all tables. In data warehouses tables are so large sometimes that special attention is needed because scale and poor coding are simply way beyond the capabilities of the Optimizer.

The rule-based approach is out of date and will probably be deprecated in a future version of Oracle Database.

Note

Oracle Database 10 Grid Rule-based Optimization is desupported.

Statistics are very important to the Optimizer and the cost-based approach. The more up to date those statistics, the more accurately the Optimizer can predict the best way to execute an SQL statement. In very large or very active databases continual update to statistics is impractical, likely to cause contention and is time consuming. Thus the more out of date statistics are, the less accurate the Optimizer will be.

8.2.1 What are Statistics?

Statistics are a computation or estimation of the exact size and placement of data in tables and indexes. Statistics can be used by the Optimizer to make a better assessment of actual data and thus produce a better query plan, more accurately matching the data as it actually is in the database.

Statistics are used by cost-based optimization and not rule-based optimization.


8.3 How Best to Access Data in the Database

There are different ways of accessing data. Some are better than others depending on circumstances. Accessing a table through an index is usually but not always faster than accessing a table directly. Accessing a table directly will always result in a full table scan. A full table scan reads an entire table reading all the blocks in the table. This is no problem if the table is small. If a table is large reading the entire table can be a serious performance issue. There are four basic methods of accessing data:

  • Full Table Scan. Reading an entire table.

  • Full Index Scan. Reading an entire index and accessing table rows using ROWID pointers found in the index.

  • Index Range Scan. Traverses a subset of an index to search for a single value or a group of values across a range of index values. Table rows are accessed using ROWID pointers found in the index.

  • Index Unique Scan or Hit. An exact hit performs a minimal number of block reads on an index, usually three at the most, rapidly accessing a single row in the index. A ROWID pointer in the index is used to access a single row in a table.

Index unique scans are the fastest way to find data in an Oracle database. An index unique scan is always used when finding a single, uniquely identified row, using either the primary key or a unique alternate key. Range scans are a little slower since they potentially read more blocks in the index and pass more ROWID pointers to the table. A full index scan is faster than a full table scan because an index contains fewer columns than the table and thus occupies less physical space. Also an index, being fewer columns in size, is often a compressed form of a table and thus more index rows fit into a block than table blocks. Thus more indexes can be scanned in a single block read.


8.4 Rule-versus Cost-Based Optimization

Rule-based optimization uses a set of predetermined rules to find the fastest access path to data in the database. Cost-based optimization uses statistical measurements of the data in the database and calculates the fastest access path. Cost-based optimization uses a realistic picture of data and thus is potentially much more efficient than rule-based optimization, assuming statistics are maintained adequately.

Note

Oracle Database 10 Grid Rule-based Optimization is desupported.

8.4.1 Setting the Optimization Mode

Optimization can be set on a database-wide level in the Oracle Database configuration parameter called OPTIMIZER_MODE. The default value is CHOOSE. CHOOSE will allow use of cost-based optimization if statistics exist, otherwise rule-based optimization will be used.

The OPTIMIZER_MODE parameter can be changed online using the ALTER SESSION command for the current session only. If you wish to use rule-based optimization for a specific SQL statement perhaps using the RULE hint is the better option. All OPTIMIZER_MODE settings are covered by hints. A hint can be placed in an individual SQL statement. Hints will be covered later in this chapter. OPTIMIZER_MODE settings are as follows:

Tip

Setting OPTIMIZER_MODE = CHOOSE is by far the best all-round option. Oracle Database 10 Grid ALL_ROWS is the default value for the OPTIMIZER_MODE parameter in Oracle Database 10g.

  • OLTP Small Transactional Databases.

    • FIRST_ROWS_n. Cost-based optimizing even without statistics for fastest performance of the first 1, 10, 100, or 1,000 rows retrieved. In other words, full index and table scans could suffer immensely but short transactions retrieving small numbers of rows should perform better.

    • FIRST_ROWS. Fast delivery of the first few rows using a mixed rule and cost-based approach. FIRST_ROWS_n is more precise if your database needs this level of precision.

    Tip

    FIRST_ROWS_1000 might always be the best option of these. Any full index or table scanning over 1,000 rows could suffer using either of these options. There are hints applying behavior similar to FIRST_ROWS settings.

  • CHOOSE. Rule-based is selected when no statistics are present and cost-based when some statistics are available, regardless of the reliability of those statistics. CHOOSE is optimized for full index and table scans but is the most common choice for all types of databases.

    Note

    Oracle Database 10 Grid CHOOSE is desupported.

  • Data Warehouse or Reporting Databases.

    • CHOOSE. Rule-based is selected when no statistics are present and cost-based when some statistics are available, regardless of the reliability of those statistics. CHOOSE is optimized for full index and table scans.

    • ALL_ROWS. The opposite of FIRST_ROWS optimization using a cost-based approach even without statistics. Optimized for full index and table scans.

    Note

    Oracle Database 10 Grid ALL_ROWS is the default value for the OPTIMIZER_MODE parameter.

  • Preferably Never.

    • RULE. Rule-based optimization ignores any existing statistics presenting an unrealistic picture of data. Rule-based optimization can be used for backwards compatibility with previous Oracle Database version tuned outlines. Otherwise avoid it!

    Note

    Oracle Database 10 Grid CHOOSE and RULE are desupported since rule-based Optimization is desupported. Functionality still exists but will be deprecated in a future release of Oracle Database.

8.4.2 What is Rule-Based Optimization?

Rule-based optimization uses a set of rules allowing a best guess at how SQL code can be optimally executed. Rule-based optimization will eventually be removed from Oracle Database. Many advanced SQL code tuning features and useful bells and whistles introduced in recent Oracle Database versions are only available with cost-based optimization. Any SQL code statement can be persuaded to use a rule-based approach by inclusion of the RULE hint, regardless of the OPTIMIZER_MODE setting in the configuration parameter file.

Note

Oracle Database 10 Grid Rule-based Optimization and the RULE hint are desupported. Applications currently using rule-based optimization should be updated.

Outlines

Outlines can be used by a rule-based optimization approach in order to store previously tuned execution plans for SQL code. For cost-based optimization an execution plan is the sequence of steps written to the PLAN_TABLE when the EXPLAIN PLAN command is used. We have seen extensive use of the EXPLAIN PLAN command in recent chapters. Outlines are stored in a specific schema in tables containing what effectively are previously tuned hint additions. These hint additions are used to override SQL code statements to tune performance of SQL code statements during execution. Like rule-based optimization outlines will probably be removed in a future version of Oracle Database; possibly to be replaced with use of hints in SQL code.

Hints and Rule-Based Optimization

Hints can be used in a rule-based optimized database to produce similar results to that of cost-based optimization. Perhaps you are unable to regularly generate accurate statistics or statistics rapidly become out of date. Statistics can rapidly become out of date in large active databases and can be expensive in resources to constantly maintain.

Tip

If statistics cannot be regularly maintained improve SQL code performance using hints.

The OPTIMIZER_MODE in my database is set to CHOOSE and I have statistics. Thus in order to show hints functioning I can temporarily use rule-based optimization or remove statistics, or both. I do both.

ALTER SESSION SET OPTIMIZER_MODE=RULE;

ANALYZE TABLE generalledger DELETE STATISTICS;
ANALYZE INDEX xpk_generalledger DELETE STATISTICS;

This first query plan uses a full table scan under rule-based optimization; note there are no cost, row or bytes values shown.

EXPLAIN PLAN SET statement_id='TEST' FOR
      SELECT gl.generalledger_id FROM generalledger gl;

Query                                    Cost     Rows     Bytes
--------------------------------------   ----   ------   -------
1. SELECT STATEMENT on
2.  TABLE ACCESS FULL on GENERALLEDGER

The INDEX_FFS hint is used to override the rule-based Optimizer and change full access on the GeneralLedger table to an index fast full scan on the primary key index of the GeneralLedger table.

EXPLAIN PLAN SET statement_id='TEST' FOR
      SELECT /*+ INDEX_FFS(gl XPK_GENERALLEDGER) */
         gl.generalledger_id
      FROM generalledger gl;

Query                                 Cost     Rows     Bytes
-----------------------------------   ----   ------   -------
1. SELECT STATEMENT on                8        2000     26000
2.  INDEX FAST FULL SCAN on
      XPK_GENERALLEDGER               8        2000     26000

I wanted to show that under rule-based optimization hints can be used to override a query plan chosen by the rule-based Optimizer, regardless of the presence of statistics. Let's do some cleanup.

ALTER SESSION SET OPTIMIZER_MODE=CHOOSE;

ANALYZE TABLE generalledger COMPUTE STATISTICS;
ANALYZE INDEX xpk_generalledger COMPUTE STATISTICS;

The problem with using hints to override the rule-based Optimizer is as follows. If relative sizes of objects in a database change over time any SQL code tuned with hints and executed under the rule-based Optimizer could eventually become ugly with respect to performance.

Note

Oracle Database 10 Grid Rule-based Optimization is desupported.

8.4.3 What is Cost-Based Optimization?

As already stated in the introduction to this section, cost-based optimization uses statistical measurements of the data as it is in the database. These statistics are used to calculate the fastest access path to data. Cost-based optimization uses a realistic picture of data and thus is potentially much more efficient than rule-based optimization. Statistics must exist on at least one object accessed in an SQL code statement for cost-based optimization to function. Keeping statistics up to date is absolutely critical.

There are numerous Oracle Database configuration parameters which can be used to influence the way the Optimizer behaves with the cost-based approach.

Configuration Parameters and Cost-Based Optimization

Some configuration parameters will affect cost-based optimization. Most of these parameters should simply not normally "be messed with" and should be altered as a last resort. If you have a small database and are changing these parameters frequently you may be avoiding essential SQL code tuning. SQL code tuning is a lot easier than altering configuration parameters. Altering configuration parameters can sometimes have some very unexpected results. These parameters are only listed in this chapter. More explicit explanations will be included in the Part III.

  • DB_FILE_MULTIBLOCK_READ_COUNT.

  • CURSOR_SHARING.

  • SORT_AREA_SIZE.

  • QUERY_REWRITE_ENABLED.

  • HASH_JOIN_ENABLED.

  • HASH_AREA_SIZE.

  • BITMAP_MERGE_AREA_SIZE.

  • OPTIMIZER_INDEX_CACHING.

  • OPTIMIZER_INDEX_COST_ADJ.

  • OPTIMIZER_MAX_PERMUTATIONS.

  • OPTIMIZER_DYNAMIC SAMPLING.

    Note

    Oracle Database 10 Grid Some parameters may have changed or been removed. Further discussion will be in Part III.

The Importance of Statistics and Realistic Statistics

The SQL code Optimizer utilizes statistics to compile the most efficient methods of executing SQL statements. Statistics are measurements of the data itself, such as how large a table is and how useful an index is. When an SQL statement accesses a table both the table and index states are important. States of database objects such as tables and indexes are contained within statistics. If statistics are out of date the Optimizer is not functioning realistically. Out-of-date statistics would have the same effect on all types of databases. It is very important to duplicate statistics from production to tuning environments, either by copying onto or executing statistics gathering on a tuning database, consistent with the production database.

Making a copy of a production database to a tuning database is not an issue when the production database is small. When the production database is large continuous copying to a tuning database could be very time consuming. Be aware that using the database import utility for even single schema imports on even a small database can take a lot longer than the production database export. The DBMS_STATS package can also be used to copy only statistics between databases, allowing for more realistic query plan testing.

Dynamic Sampling

For statistics to be realistic and effective they must be frequently generated. In an OLTP database data is changing all the time.

Statistics can rapidly become redundant. Cost-based optimization against redundant statistics can sometimes cause as big a performance problem as not having those statistics at all, depending of course on how relatively out of date statistics are. If statistics are out of date or not present then dynamic sampling may be used. Dynamic sampling reads a small number of blocks in a table to make a best guess at statistics.

Tip

The configuration parameter controlling dynamic sampling is OPTIMIZER_DYNAMIC_SAMPLING and is set on by default (1).
Note

Oracle Database 10 Grid OPTIMIZER_DYNAMIC_SAMPLING is defaulted to 2.

One of the most significant issues with respect to statistics is matching consistency of statistics between production, testing, and development databases. Most SQL code tuning is usually performed in development databases, but preferably in a test or dedicated tuning database. At least it should be! Test databases are more easily copied from production databases or parts of production databases. Test databases are often copied from production databases to get a realistic snapshot of a live production database. Development databases are often very messy and convoluted. Tuning on a production database, particularly tuning involving SQL code changes, is extremely risky for the stability and current level of performance for a 24 × 7 × 365 OLTP Oracle installation.

The OPTIMIZER_DYNAMIC_SAMPLING parameter sets dynamic sampling of data, substituting for cost-based statistics, from between a setting of 0 and 10. Set to 0 dynamic sampling is disabled and set to 10 the entire table is read. Settings in between simply change the number of blocks read for the sample.

Generating Statistics

Statistics can be generated using the ANALYZE command or the DBMS_STATS package. The ANALYZE command is potentially subject to deprecation in a future version of Oracle Database. In fact Oracle Corporation discourages use of the ANALYZE command for statistics generation.

What to Generate Statistics For

Statistics can be generated for tables and indexes plus general system statistics. System-level statistics will be covered in Part III. Schema-level statistics are generated for tables, indexes, and clusters for each schema. Schema statistics are accessible through the views USER_TABLES, USER_INDEXES, and USER_TAB_COLUMNS.

Tables

  • NUM_ROWS. Rows in table.

  • BLOCKS. Blocks used by a table.

  • EMPTY_BLOCKS. Blocks not used between highest appended block and high water mark. Reclaim these blocks by resizing the datafile.

  • AVG_SPACE. Average empty space for all blocks in the table. Depends on PCT_FREE updates reserved space and PCT_USED after deletions.

  • CHAIN_CNT. Rows spread across multiple blocks (row chaining) or having their data moved (migrated) retaining the ROWID pointer in the original block as well.

  • AVG_ROW_LEN. Average byte length of a row.

  • SAMPLE_SIZE. Sample size of most recent statistics generation.

  • LAST_ANALYZED. Date of last statistics generation.

Indexes

  • NUM_ROWS. Rows in index.

  • LEAF_BLOCKS. Total leaf blocks containing actual ROWID and indexed column values.

  • BLEVEL. Depth of a BTree index from root to leaf blocks, usually 0 for small static tables and ranging from 1 to 2 (3 levels) for very large tables.

  • DISTINCT_KEYS. Unique values for a key.

  • AVG_LEAF_BLOCKS_PER_KEY. Ratio of DISTINCT_KEYS to leaf blocks. How many rows are pointed to by each key value.

  • AVG_DATA_BLOCKS_PER_KEY. Ratio of DISTINCT_KEYS to table blocks. How many rows in a table are pointed to by each unique index value.

  • CLUSTERING_FACTOR. Randomness between index and table blocks. A reverse key index will be more random.

  • SAMPLE_SIZE. Sample size of most recent statistics generation.

  • LAST_ANALYZED. Date of last statistics generation.

Columns

  • NUM_DISTINCT. Unique column values.

  • LOW_VALUE. Lowest column value.

  • HIGH_VALUE. Highest column value.

  • DENSITY. Column density.

  • NUM_NULLS. Null values in column.

  • NUM_BUCKETS. Histogram buckets.

  • SAMPLE_SIZE. Sample size of most recent statistics generation.

  • LAST_ANALYZED. Date of last statistics generation.

The ANALYZE Command

Let's look at syntax first. The highlighted parts in the following syntax diagram are the interesting parts because they deal with Optimizer statistics.

ANALYZE { TABLE | INDEX | CLUSTER }
            [PARTITION ( partition ) | SUBPARTITION
               ( subpartition )]
      COMPUTE [ SYSTEM ] STATISTICS [ FOR ¼ object
         specifics ¼ ]
      ESTIMATE [ SYSTEM ] STATISTICS [ FOR ¼ object
         specifics ¼ ]
            [ SAMPLE n { ROWS | PERCENT } ]
      [ ¼ referential integrity | structural ¼ validation ]
      LIST CHAINED ROWS INTO table
DELETE [ SYSTEM ] STATISTICS;

  • COMPUTE. Calculates statistics for all rows and columns in an object (table, index, or cluster). Precise results are produced.

  • ESTIMATE. Calculates statistics on a sample number or percentage of rows, defaulted to 1,064 rows.

  • DELETE. Clears statistics.

  • SYSTEM. Collects only system statistics, not user schema statistics.

COMPUTE can be time consuming and must be executed in times of low activity. Executing full statistics computations during highly active concurrent activity cycles could cause performance issues, perhaps even unpopular response from end users. ESTIMATE is far less intrusive but less accurate and is also preferably executed at times of low activity.

A statistics generating script is included in Appendix B.

The DBMS_STATS Package

The DBMS_STATS package is more versatile and potentially faster than using the ANALYZE command to generate statistics. It is a little more complex to use. There are some non-Optimizer statistics that can only be gathered with the ANALYZE command. However, DBMS_STATS has parallel execution and Oracle Partitioning benefits plus performance tuning capabilities that the ANALYZE command does not have.

The DBMS_STATS package can even be used to copy statistics between databases thus allowing accurate simulated tuning between testing and production databases.

Generating Statistics Automatically

Some Optimizer statistics generation can be performed automatically using the MONITORING clause of the CREATE TABLE or ALTER TABLE commands and otherwise using the DBMS_STATS package.

The MONITORING clause may be a performance risk for very busy databases. Automated statistics gathering monitors table DML activity and automatically invokes statistics generation when a small percentage of table rows have been changed. Automatic statistics generation can cause performance problems for highly active concurrent databases. Automating generation of statistics is thus not recommended. Generation of statistics should be scheduled using scheduling software or the DBMS_JOBS package at times of known low activity.

Timed Statistics

Timed statistics gathering is controlled by the TIMED_STATISTICS and TIMED_OS_STATISTICS configuration parameters. Setting this parameter has negligible effect on performance and will be covered in Part III.

Histograms

A histogram is a mechanism used to provide an accurate mathematical picture to the Optimizer of the distribution of values across a range of values. A traditional mathematical statistics histogram looks something like that shown in Figure 8.1.

Click To expand
Figure 8.1: An Example Histogram

Oracle Database uses histograms to allow faster access to skewed distributions. The histogram in Figure 8.1 is a skewed or unbalanced distribution because most of the values are to the right of the diagram. A balanced distribution would have all the columns being of exactly the same height. In that case we would not need a histogram. It follows that histograms are completely pointless on unique indexes, since all the columns would be the same height and there would be a single value in each column. Additionally use of bind variables in SQL statements will ignore histograms. Histograms are sometimes used where SQL code cannot or should not be altered to lower shared pool SQL code reparsing.

The Accounts schema GeneralLedger table has a non-unique foreign key index on the COA# column. Looking at Figure 7.3 in the previous chapter we can see that the GeneralLedger.COA# foreign key index is potentially skewed, or at least heavier in certain parts. Figure 7.3 from the previous chapter is included in this chapter for convenience as Figure 8.2.

Click To expand
Figure 8.2: A Skewed BTree Index

Following is the count of the rows in the GeneralLedger table. Notice how COA# = '30001' has almost 400,000 rows and COA# = '60001' has only 75,000 rows. As a result of these varying row counts it is likely that there are a lot more leaf index blocks referenced by the branch containing '30001' than that of '60001'.

This is a skewed or unbalanced distribution, leading to a potentially skewed BTree index.

Tip

These counts are somewhat different to those in the previous chapter since the database is highly active and has grown.

SELECT coa#, COUNT(coa#) FROM generalledger GROUP BY coa#;

COA#     COUNT(COA#)
------   -----------
30001         383357
40003         151107
41000         252552
50001         206357
50028              1
60001          75555

Let's compare the costs of three queries reading rows of three different COA# column values. Before doing this I will refresh statistics for the GeneralLedger table, making sure we have no histograms to begin with.

ANALYZE TABLE generalledger DELETE STATISTICS;
ANALYZE INDEX xfk_coa# DELETE STATISTICS;
ANALYZE INDEX xpk_generalledger DELETE STATISTICS;
ANALYZE TABLE generalledger COMPUTE STATISTICS;
ANALYZE INDEX xfk_coa# COMPUTE STATISTICS;
ANALYZE INDEX xpk_generalledger COMPUTE STATISTICS;

Now let's compare the costs. Note that all cost, row, and byte values are identical.

EXPLAIN PLAN SET statement_id='TEST' FOR
      SELECT coa# FROM generalledger WHERE coa# = '30001';

Query                                 Cost     Rows     Bytes
-----------------------------------   ----   ------   -------
1. SELECT STATEMENT on                 425   178155    890775
2. INDEX RANGE SCAN on XFK_COA#        425   178155    890775

EXPLAIN PLAN SET statement_id='TEST' FOR
      SELECT coa# FROM generalledger WHERE coa# = '50028';

Query                                 Cost     Rows     Bytes
-----------------------------------   ----   ------   -------
1. SELECT STATEMENT on                 425   178155    890775
2. INDEX RANGE SCAN on XFK_COA#        425   178155    890775

EXPLAIN PLAN SET statement_id='TEST' FOR
      SELECT coa# FROM generalledger WHERE coa# = '60001';

Query                                 Cost     Rows     Bytes
-----------------------------------   ----   ------   -------
1. SELECT STATEMENT on                 425   178155    890775
2. INDEX RANGE SCAN on XFK_COA#        425   178155    890775

Let's create a histogram on the GeneralLedger.COA# column. Histograms can be generated using the ANALYZE command or the DBMS_STATS package. I will stick with the ANALYZE command for now.

ANALYZE TABLE generalledger COMPUTE STATISTICS FOR COLUMNS
coa#;
Tip

Histograms require regeneration just like statistics do and can become stale and useless.

Created histograms can be examined using the USER_HISTOGRAMS view. Let's look at the query plans for each of the statements separately. Here we can see a dramatic difference in cost, rows, and bytes values as a result of creating a histogram. These query plan figures are now consistent, relative to each other, with the ratios of the numbers of rows for each COA# column value.

EXPLAIN PLAN SET statement_id='TEST' FOR
      SELECT coa# FROM generalledger WHERE coa# = '30001';

Query                                 Cost     Rows     Bytes
-----------------------------------   ----   ------   -------
1. SELECT STATEMENT on                 609   383357   1916785
2. INDEX FAST FULL SCAN on XFK_COA#    609   383357   1916785

EXPLAIN PLAN SET statement_id='TEST' FOR
      SELECT coa# FROM generalledger WHERE coa# = '50028';

Query                                 Cost     Rows     Bytes
-----------------------------------   ----   ------   -------
1. SELECT STATEMENT on                   3        1         5
2. INDEX RANGE SCAN on XFK_COA#          3        1         5

EXPLAIN PLAN SET statement_id='TEST' FOR
      SELECT coa# FROM generalledger WHERE coa# = '60001';

Query                                 Cost     Rows     Bytes
-----------------------------------   ----   ------   -------
1. SELECT STATEMENT on                 182    75555    377775
2. INDEX RANGE SCAN on XFK_COA#        182    75555    377775

Histograms can be very useful.


8.5 Data Access Methods

This section on data access methods will cover a lot of ground. General areas covered are as follows:

  • Tables, Indexes, and Clusters. There are numerous ways to access data objects.

  • Joins. The Optimizer has specific methods of joining tables, largely determined by table sizes.

  • Sorting. The Optmizer sorts SQL statement output in various ways.

  • Special Cases. Concatenation, IN list operators, and UNION operators warrant special mention.

  • Hints. Hints allow overriding of a path of execution selected by the Optimizer for an SQL statement.

8.5.1 Accessing Tables and Indexes

In this section we examine the different ways in which the Optimizer accesses tables and indexes. Tables can be full scanned, sampled, or scanned by a ROWID passed from an index. There are a multitude of ways in which indexes can be accessed and many ways in which index scanning can be affected. There are some important things to note about indexes:

  • An index will occupy a small percentage of the physical space used by a table. Thus searching through an index either precisely or fully is usually much faster than scanning a table.

  • Reading an index to find a ROWID pointer to a table can speed up access to table rows phenomenally.

  • Sometimes tables do not have to be read at all where an index contains all the retrieved column data.

Full Table Scans

Reading Many Blocks at Once

A full table scan can be speeded up by reading multiple blocks for each read. This can be done using the DB_FILE_MULTIBLOCK _READ_COUNT parameter. This parameter is usually set between 4 and 16, typically 4, 8, or 16. The default value is 8. You might want to consider setting it to 4 or less for OLTP systems and 16 or higher for data warehouses.

Tip

I set the value of DB_FILE_MULTIBLOCK_READ_COUNT on my Win2K server to various values between 1 and 128 and had no problems. This parameter is not restricted to specific values.

A full table scan may result if the number of blocks occupied by a table is less than DB_FILE_MULTIBLOCK_READ_COUNT. Please note the emphasis on the term "may result"; it is not an absolute. The Optimizer is very sophisticated in the current version of Oracle Database and is unlikely to conclude that a full table scan is faster than an index plus ROWID pointer table read, especially where a unique index hit is involved on a large table.

Small Static Tables

For a small, static table a full table scan can often be the most efficient access method. A static table is obviously not changed and thus there are never deletions. In many cases an index unique hit will be preferred by the Optimizer since the cost will be lower, particularly in mutable joins. Let's examine small static tables. Let's start by showing the row and block counts from the table statistics in the Accounts schema.

SELECT table_name, num_rows, blocks FROM user_tables ORDER
BY 1;
TABLE_NAME         NUM_ROWS   BLOCKS
----------------   --------   ------
CASHBOOK             188185     1517
CASHBOOKLINE         188185      616
CATEGORY                 13        1
COA                      55        1
CUSTOMER               2694       60
GENERALLEDGER        752741     3244
ORDERS               172304      734
ORDERSLINE           540827     1713
PERIOD                   72        1
PERIODSUM               221        1
POSTING                   8        1
STOCK                   118        4
STOCKMOVEMENT        570175     2333
STOCKSOURCE           12083       30
SUBTYPE                   4        1
SUPPLIER               3874       85
TRANSACTIONS         188185     1231
TRANSACTIONSLINE     570175     1811
TYPE                      6        1

Looking at the block space occupied by the Accounts schema tables we can see that the COA and Type tables occupy a single block. First of all let's show the relative costs between a unique index hit and full table scan on the Type table. The Type table is very small and occupies considerably less space than the single block it uses.

A full table scan on the Type table has a cost of 2.

EXPLAIN PLAN SET statement_id='TEST' FOR SELECT * FROM
type;

Query                                 Cost     Rows     Bytes
-----------------------------------   ----   ------   -------
SELECT STATEMENT on                      2        6        48
 TABLE ACCESS FULL on TYPE               2        6        48

The following query will access both the index and the table. Accessing a single row reads 1/6th of the bytes and the cost is halved.

EXPLAIN PLAN SET statement_id='TEST' FOR
      SELECT * FROM type WHERE type = 'A';

Query                                 Cost     Rows     Bytes
-----------------------------------   ----   ------   -------
SELECT STATEMENT on                      1        1         8
 TABLE ACCESS BY INDEX ROWID on TYPE     1        1         8
  INDEX UNIQUE SCAN on XPK_TYPE                   6

Even a full scan on just the index still has half the cost. The full scan of the index also reads fewer bytes than the unique index scan.

EXPLAIN PLAN SET statement_id='TEST' FOR SELECT type FROM
type;

Query                                 Cost     Rows     Bytes
-----------------------------------   ----   ------   -------
SELECT STATEMENT on                      1        6         6
 INDEX FULL SCAN on XPK_TYPE             1        6         6

Now let's go completely nuts! Let's read the same table a multitude of times within the same query. We can see from the query plan following that even though the table is read many times the Optimizer will still assess the cost as 1. There are a multitude of unique index hits, even though the COA table is read in its entirety. We could run timing tests. Results do not imply that the Optimizer is giving us the fastest option.

EXPLAIN PLAN SET statement_id='TEST' FOR
 SELECT * FROM orders WHERE type IN
  (SELECT type FROM type WHERE type IN
   (SELECT type FROM type WHERE type IN
    (SELECT type FROM type WHERE type IN
     (SELECT type FROM type WHERE type IN
      (SELECT type FROM type WHERE type IN
       (SELECT type FROM type WHERE type IN
        (SELECT type FROM type WHERE type IN
         (SELECT type FROM type WHERE type IN
          (SELECT type FROM type WHERE type IN
           (SELECT type FROM type))))))))));

Query                                 Cost     Rows     Bytes
-----------------------------------   ----   ------   -------
SELECT STATEMENT on                      2       55      1870
 NESTED LOOPS on                         2       55      1870
  NESTED LOOPS on                        2       55      1815
   NESTED LOOPS on                       2       55      1760
    NESTED LOOPS on                      2       55      1705
     NESTED LOOPS on                     2       55      1650
      NESTED LOOPS on                    2       55      1595
       NESTED LOOPS on                   2       55      1540
        NESTED LOOPS on                  2       55      1485
         NESTED LOOPS on                 2       55      1430
          NESTED LOOPS on                2       55      1375
          TABLE ACCESS FULL on COA       2       55      1320
         INDEX UNIQUE SCAN on XPK_TYPE            1         1
        INDEX UNIQUE SCAN on XPK_TYPE             1         1
       INDEX UNIQUE SCAN on XPK_TYPE              1         1
      INDEX UNIQUE SCAN on XPK_TYPE               1         1
     INDEX UNIQUE SCAN on XPK_TYPE                1         1
    INDEX UNIQUE SCAN on XPK_TYPE                 1         1
   INDEX UNIQUE SCAN on XPK_TYPE                  1         1
  INDEX UNIQUE SCAN on XPK_TYPE                   1         1
 INDEX UNIQUE SCAN on XPK_TYPE                    1         1
INDEX UNIQUE SCAN on XPK_TYPE                     1         1

Now we will attempt something a little more sensible by running the EXPLAIN PLAN command for a join accessing the COA table four times and then time testing that query. There are four unique scans on the COA table.

EXPLAIN PLAN SET statement_id='TEST' FOR
      SELECT t.*, cb.*
      FROM transactions t JOIN cashbook cb
            ON(t.transaction_id = cb.transaction_id)
      WHERE t.drcoa# IN (SELECT coa# FROM coa)
      AND t.crcoa# IN (SELECT coa# FROM coa)
      AND cb.drcoa# IN (SELECT coa# FROM coa)
      AND cb.crcoa# IN (SELECT coa# FROM coa);

Query                                 Cost     Rows     Bytes
-----------------------------------   ----   ------   -------
1. SELECT STATEMENT on                7036   188185  22394015
2.  NESTED LOOPS on                   7036   188185  22394015
3.   NESTED LOOPS on                  7036   188185  21264905
4.    NESTED LOOPS on                 7036   188185  20135795
5.     NESTED LOOPS on                7036   188185  19006685
6.      MERGE JOIN on                 7036   188185  17877575
7.       TABLE ACCESS BY INDEX
           ROWID on CASH               826   188185   9973805
8.        INDEX FULL SCAN on
            XFK_CB_TRANS                26   188185
7.       SORT JOIN on                 6210   188185   7903770
8.      TABLE ACCESS FULL on
          TRANSACTIONS                 297   188185   7903770
6.     INDEX UNIQUE SCAN on
         XPK_COA                                  1         6
5.    INDEX UNIQUE SCAN on
        XPK_COA                                   1         6
4.   INDEX UNIQUE SCAN on XPK_COA                 1         6
3.  INDEX UNIQUE SCAN on XPK_COA                  1         6

In the next query plan I have suggested a full table scan to the Optimizer on the COA table with an overriding hint. The cost has a 20% increase.

EXPLAIN PLAN SET statement_id='TEST' FOR
      SELECT t.*, cb.*
      FROM transactions t JOIN cashbook cb
            ON(t.transaction_id = cb.transaction_id)
      WHERE t.drcoa# IN (SELECT /*+ FULL(coa) */ coa#
         FROM coa)
      AND t.crcoa# IN (SELECT /*+ FULL(coa) */ coa#
         FROM coa)
      AND cb.drcoa# IN (SELECT /*+ FULL(coa) */ coa#
         FROM coa)
      AND cb.crcoa# IN (SELECT /*+ FULL(coa) */ coa#
         FROM coa);

Query                                 Cost     Rows     Bytes
-----------------------------------   ----   ------   -------
1. SELECT STATEMENT on                8240   188185  22394015
2.  HASH JOIN on                      8240   188185  22394015
3.   TABLE ACCESS FULL on COA            2       55       330
3.   HASH JOIN on                     7916   188185  21264905
4.    TABLE ACCESS FULL on COA           2       55       330
4.    HASH JOIN on                    7607   188185  20135795
5.     TABLE ACCESS FULL on COA          2       55       330
5.     HASH JOIN on                   7314   188185  19006685
6.      TABLE ACCESS FULL on COA         2       55       330
6.      MERGE JOIN on                 7036   188185  17877575
7.       TABLE ACCESS BY INDEX
           ROWID on CASH               826   188185   9973805
8.        INDEX FULL SCAN on
            XFK_CB_TRANS                26   188185
7.       SORT JOIN on                 6210   188185   7903770
8.        TABLE ACCESS FULL
            on TRANSACTIONS            297   188185   7903770

Now let's do some timing tests. Even though query plan costs are higher for full table scans the timing tests following show that the overriding hint full table scans are actually almost twice as fast. Perhaps the Optimizer should have been intelligent enough in the case of this query to force all the COA tables to be read as full table scans.

SQL> SELECT COUNT(*) FROM(SELECT t.*, cb.*
  2 FROM transactions t JOIN cashbook cb
  3 ON(t.transaction_id = cb.transaction_id)
  4 WHERE t.drcoa# IN (SELECT coa# FROM coa)
  5 AND t.crcoa# IN (SELECT coa# FROM coa)
  6 AND cb.drcoa# IN (SELECT coa# FROM coa)
  7 AND cb.crcoa# IN (SELECT coa# FROM coa));

   COUNT(*)
-----------
     188185

Elapsed: 00:00:11.08

SQL> SELECT COUNT(*) FROM(SELECT t.*, cb.*
  2 FROM transactions t JOIN cashbook cb
  3 ON(t.transaction_id = cb.transaction_id)
  4 WHERE t.drcoa# IN (SELECT /*+ FULL(coa) */ coa#
      FROM coa)
  5 AND t.crcoa# IN (SELECT /*+ FULL(coa) */ coa#
      FROM coa)
  6 AND cb.drcoa# IN (SELECT /*+ FULL(coa) */ coa#
      FROM coa)
  7 AND cb.crcoa# IN (SELECT /*+ FULL(coa) */ coa#
      FROM coa));

   COUNT(*)
-----------
      188185

Elapsed: 00:00:07.07
Reading Most of the Rows

Reading a large percentage of the rows in a table can cause the Optimizer to perform a full table scan. This percentage is often stated as being anywhere between 5% and 25%. The following example against the Stock table switches from an index range scan to a full table scan at 30 rows: 30 out of 118 rows which is about 25%. Note how the cost of the two queries is the same.

EXPLAIN PLAN SET statement_id='TEST' FOR
      SELECT * FROM stock WHERE stock_id <= 29;

Query                                 Cost     Rows     Bytes
-----------------------------------   ----   ------   -------
SELECT STATEMENT on                      2       29      6757
 TABLE ACCESS BY INDEX ROWID
   on STOCK                              2       29      6757
  INDEX RANGE SCAN on XPK_STOCK          1       29

EXPLAIN PLAN SET statement_id='TEST' FOR
      SELECT * FROM stock WHERE stock_id <= 30;

Query                                 Cost     Rows     Bytes
-----------------------------------   ----   ------   -------
SELECT STATEMENT on                      2       30      6990
 TABLE ACCESS FULL on STOCK              2       30      6990

Reading Deleted Rows

A full table scan simply reads the entire table. The biggest potential problem with a full table scan is that all blocks ever used by the table will be read, not just all the rows in the table. What does this mean? The high water mark bears mentioning. What is the high water mark? The high water mark for a table is the largest number of blocks ever allocated to a table. As rows are added to a table the high water mark (number of blocks used) increases. When data is deleted from that table using the DELETE command the high water mark does not decrease. Only the TRUNCATE command or recreating the table with the CREATE TABLE command will decrease the high water mark.

If a table that had many rows added is then deleted from completely and full table scanned, what will be the result? All the blocks deleted from will be read even though the table has no rows at all. The following example proves this. Full table scans on very large tables subject to large amounts of deletion can cause serious performance problems. Let's look at deleting from tables. The first query plan reads all data from the OrdersLine table. Note the cost, rows, and bytes read.

EXPLAIN PLAN SET statement_id='TEST' FOR SELECT * FROM
      ordersline;

Query                                 Cost     Rows     Bytes
-----------------------------------   ----   ------   -------
SELECT STATEMENT on                    765    997903 15966448
 TABLE ACCESS FULL on ORDERSLINE       765    997903 15966448

Now I delete all the rows from the table and run the EXPLAIN PLAN command again. The rows and bytes values have decreased dramatically but the cost is identical regardless of all the rows being deleted. Additionally I have updated statistics. I am copying my OrdersLine table to a temporary table so that I can retrieve the OrdersLine table rows.

DROP TABLE tmp;
CREATE TABLE tmp AS SELECT * FROM ordersline;
DELETE FROM ordersline;
COMMIT;
ANALYZE TABLE ordersline COMPUTE STATISTICS;
EXPLAIN PLAN SET statement_id='TEST' FOR SELECT * FROM
ordersline;

Query                                 Cost     Rows     Bytes
-----------------------------------   ----   ------   -------
SELECT STATEMENT on                    765        1        52
 TABLE ACCESS FULL on ORDERSLINE       765        1        52

Now I will TRUNCATE the table and run my EXPLAIN PLAN command again. I must not forget to compute the statistics again otherwise the Optimizer will use previously calculated statistics, namely statistics showing almost a million rows in the OrdersLine table, when it is now empty. The query plan is drastically different now with the cost much reduced.

TRUNCATE TABLE ordersline;
ANALYZE TABLE ordersline COMPUTE STATISTICS;
EXPLAIN PLAN SET statement_id='TEST' FOR SELECT *
   FROM ordersline;

Query                                 Cost     Rows     Bytes
-----------------------------------   ----   ------   -------
SELECT STATEMENT on                      2        1        52
 TABLE ACCESS FULL on ORDERSLINE         2        1        52

A serious potential problem with full table scans is scanning up to the high water mark to include blocks which once contained rows that have now been deleted.

Now let's retrieve the OrdersLine table.

INSERT INTO ordersline SELECT * FROM tmp;
COMMIT;

So what can we conclude about full table scans. In short they are usually best avoided. In rare circumstances they can be faster than index scans, particularly for small static tables. A full table scan will result when no index is available or when an index is missed accidentally by, for instance, including a function on a column in a WHERE clause where no matching function-based index exists.

Parallel Full Table Scans

Let's try scanning a large table in parallel. Parallel scans can be induced on full table scans by using the PARALLEL hint or by setting the PARALLEL attribute when creating or altering a table. Let's look at changing a table. The GeneralLedger table is the largest table in the Accounts schema and is thus the most appropriate for parallel processing. Also I have a dual CPU machine. I do not however have Oracle Partitioning at this point. Oracle Partitioning is covered in Part III.

ALTER TABLE generalledger PARALLEL 2;

The degree of parallelism can be verified by using the following query.

SELECT table_name, degree FROM USER_TABLES:

Now let's look at a query plan. Note I am using an adjusted PLAN_TABLE query including the OTHER_TAG column (see Appendix B).

EXPLAIN PLAN SET statement_id='TEST' FOR SELECT *
   FROM generalledger;

Query                                       Cost     Rows
-----------------------------------------   ----   ------
1. SELECT STATEMENT on                       195   752740
2.  TABLE ACCESS FULL on GENERALLEDGER
      PARALLEL_TO_SERIAL                     195   752740

Now let's remove the parallelism from the GeneralLedger table again.

ALTER TABLE generalledger NOPARALLEL;

Now let's check the query plan again. There is no difference in cost.

EXPLAIN PLAN SET statement_id='TEST' FOR SELECT *
   FROM generalledger;

Query                                 Cost     Rows     Bytes
-----------------------------------   ----   ------   -------
1. SELECT STATEMENT on                 195   752740  19571240
2.  TABLE ACCESS FULL on
      GENERALLEDGER                    195   752740  19571240

Let's do some timing tests. Once again we set the degree of parallelism on the table to 2.

ALTER TABLE generalledger PARALLEL 2;

SQL> SELECT COUNT(*) FROM(SELECT /*+ FULL(gl) */ * FROM
generalledger gl);

   COUNT(*)
-----------
     752741

Elapsed: 00:00:05.06

Now increase the degree of parallelism on the table.

ALTER TABLE generalledger PARALLEL 4;

The time taken is slightly less with the degree of parallelism increased.

SQL> SELECT COUNT(*) FROM(SELECT /*+ FULL(gl) */ *
   FROM generalledger gl);

   COUNT(*)
-----------
     752741

Elapsed: 00:00:05.02

Lastly, remove parallelism from the table altogether. The elapsed time is much faster without parallelism as shown in the query plan following. Parallel queries should only be used on very large row sets. OLTP databases should avoid large row sets and therefore parallelism is most appropriate to large data warehouses and not OLTP databases. Parallelism in general applies to full table scans, fast full index scans, and on disk sorting, in other words physical I/O activity, and preferably using Oracle Partitioning.

ALTER TABLE generalledger NOPARALLEL;

SQL> SELECT COUNT(*) FROM(SELECT /*+ FULL(gl) */ *
   FROM generalledger gl);

   COUNT(*)
-----------
     752741

Elapsed: 00:00:01.01

Sample Table Scans

A sample scan can be used to sample a percentage of a table either by rows or blocks, useful if you want to examine a small part of a table. The row count is much lower in the second 0.001% sampled query following.

EXPLAIN PLAN SET statement_id='TEST' FOR SELECT *
   FROM generalledger;

Query                                 Cost     Rows     Bytes
-----------------------------------   ----   ------   -------
SELECT STATEMENT on                    715  1068929  24585367
 TABLE ACCESS FULL on GENERALLEDGER    715  1068929  24585367

EXPLAIN PLAN SET statement_id='TEST' FOR
      SELECT * FROM generalledger SAMPLE(0.001);

Query                                 Cost     Rows     Bytes
-----------------------------------   ----   ------   -------
SELECT STATEMENT on                    715       11       253
 TABLE ACCESS SAMPLE on GENERALLEDGER  715       11       253
Note

Oracle Database 10 Grid A seeding value option has been added to the sample clause to allow Oracle Database to attempt to return the same sample of rows or blocks across different executions as shown in the syntax following.

SELECT ¼ SAMPLE( % ) [ SEED n ];

ROWID Scans

A ROWID is an Oracle Database internal logical pointer to a row in a table. An index stores index column values plus a ROWID pointing to a row in a table. When an index is read the ROWID found for an index value is used to retrieve that row from the table directly. For a unique index hit at most three block reads are made in the index space and one block read in the table space. An index to table ROWID scan is shown in Figure 8.3.

Click To expand
Figure 8.3: Passing a ROWID Pointer from an Index to a Table
EXPLAIN PLAN SET statement_id= 'TEST' FOR
      SELECT * FROM coa WHERE coa# < '50000';

Index Scans

An index is a subset of a table containing a smaller number of columns. Each row in an index is linked to a table using a ROWID pointer. A ROWID pointer is an Oracle Database internal logical representation based on a datafile, a tablespace, a table, and block. Effectively a ROWID is an address pointer and is the fastest method of access for rows in the Oracle database.

There are numerous different types of index scans:

  • Index Unique Scan. Uses a unique key to return a single ROWID, which in turn finds a single row in a table. A unique scan is an exact row hit on a table and the fastest access for both index and table.

  • Index Range Scan. Scans a range of index values to find a range of ROWID pointers.

    • Reverse Order Index Range Scan. This is an index range scan in descending (reverse) order.

  • Index Skip Scan. Allows skipping of the prefix of a composite index and searches the suffix index column values.

  • Index Full Scan. Reads the entire index in the key order of the index.

    • Fast Full Index Scan. Reads an entire index like an index full scan but reads the index in physical block rather than key order. Thus when reading a large percentage of a table without a requirement for sorting a fast full index scan can be much faster than an index full scan.

      Tip

      A fast full index scan is executed in the same manner as that of a full table scan in that it reads data directly from disk and bypasses the buffer cache. As a result a fast full index scan will absolutely include I/O activity and load into the buffer cache but not read the buffer cache.

    • Index Join. This is a join performed by selecting from multiple BTree indexes.

    • Bitmap Join. Joins using bitmap key values, all read from indexes.

Let's demonstrate the use of each type of index scan.

Index Unique Scan

A unique scan or exact index hit reads a unique index and finds a single row. Unique scans usually occur with equality comparisons and IN or EXISTS subquery clauses.

EXPLAIN PLAN SET statement_id= 'TEST' FOR
      SELECT * FROM coa WHERE coa# = '60001';

Query                                 Cost     Rows     Bytes
-----------------------------------   ----   ------   -------
SELECT STATEMENT on                      1        1        24
 TABLE ACCESS BY INDEX ROWID on COA      1        1        24
  INDEX UNIQUE SCAN on XPK_COA 55

EXPLAIN PLAN SET statement_id= 'TEST' FOR
      SELECT * FROM coa WHERE coa# = '60001'
      AND type IN (SELECT type FROM type);

Query                                 Cost     Rows     Bytes
-----------------------------------   ----   ------   -------
SELECT STATEMENT on                      1        1        25
 NESTED LOOPS on 1 1 25
  TABLE ACCESS BY INDEX ROWID on COA     1        1        24
   INDEX UNIQUE SCAN on XPK_COA 55
  INDEX UNIQUE SCAN on XPK_TYPE 1 1

EXPLAIN PLAN SET statement_id= 'TEST' FOR
      SELECT * FROM coa WHERE coa# = '60001'
      AND EXISTS (SELECT type FROM type
         WHERE type = coa.type);

Query                                 Cost     Rows     Bytes
-----------------------------------   ----   ------   -------
SELECT STATEMENT on                      1        1        25
 NESTED LOOPS SEMI on                    1        1        25
  TABLE ACCESS BY INDEX ROWID
    on COA                               1        1        24
   INDEX UNIQUE SCAN on XPK_COA                  55
  INDEX UNIQUE SCAN on XPK_TYPE                   6         6

Index Range Scan

Range scans typically occur with range comparisons <, >, <=, >=, and BETWEEN. A range scan is less efficient than a unique scan because an index is traversed and read many times for a group of values rather than up to the three reads required for a unique scan.

EXPLAIN PLAN SET statement_id= 'TEST' FOR
      SELECT * FROM generalledger
         WHERE generalledger_id < 10;

Query                                 Cost     Rows     Bytes
-----------------------------------   ----   ------   -------
SELECT STATEMENT on                      4        1        23
 TABLE ACCESS BY INDEX ROWID on
   GENERALLEDGER                         4        1        23
  INDEX RANGE SCAN on
    XPK_GENERALLEDGER                    3        1

The next query reads the GeneralLedger table as opposed to the COA table and reads a large percentage of a very large table. The Optimizer switches from an index range scan to a full table scan since most of the table is read.

EXPLAIN PLAN SET statement_id= 'TEST' FOR
      SELECT * FROM generalledger
         WHERE generalledger_id > 10;

Query                                 Cost     Rows     Bytes
-----------------------------------   ----   ------   -------
SELECT STATEMENT on                    715  1068929  24585367
 TABLE ACCESS FULL on GENERALLEDGER    715  1068929  24585367

Forcing index use also avoids a range scan but the cost is very much higher than scanning the whole table because an ordered full index scan is used. Suggesting a fast full index scan using the INDEX_FFS hint would probably be faster than a full table scan.

EXPLAIN PLAN SET statement_id='TEST' FOR
      SELECT /*+ INDEX(generalledger,
         XPK_GENERALLEDGER) */ *
      FROM generalledger WHERE coa# < '60001';

Query                                 Cost     Rows     Bytes
-----------------------------------   ----   ------   -------
SELECT STATEMENT on                   7052   890774  20487802
 TABLE ACCESS BY INDEX ROWID on
   GENERALLEDGER                      7052   890774  20487802
  INDEX FULL SCAN on
    XPK_GENERALLEDGER                 2309  1068929
Reverse Order Index Range Scan

A reverse order index range scan is used when rows are sorted in descending order.

EXPLAIN PLAN SET statement_id='TEST' FOR
      SELECT * FROM generalledger WHERE
         generalledger_id < 10
      ORDER BY 1 DESC;

Query                                 Cost     Rows     Bytes
-----------------------------------   ----   ------   -------
SELECT STATEMENT on                      4        1        23
 TABLE ACCESS BY INDEX ROWID on
   GENERALLEDGER                         4        1        23
  INDEX RANGE SCAN DESCENDING on
    XPK_GENERALLEDGER                    3        1

Index Skip Scan

An index skip scan will allow skipping of a prefix column in a composite index and still utilize the index. This assumes that reading that index is faster than scanning the whole table. The Accounts schema does not have an appropriate index to demonstrate an index skip scan so we can create one. The Orders table has a type column set to "S" for "Sales" or "P" for "Purchase". An index skip scan is used when the prefix of a composite index has low cardinality (few values) and the suffix has high cardinality (many different values).

CREATE INDEX xak_o_type_customer ON
   orders(type, customer_id) COMPUTE STATISTICS;

Now let's find some values.

SELECT * FROM(
SELECT type, customer_id, COUNT(customer_id) FROM orders
WHERE customer_id IS NOT NULL GROUP BY type, customer_id)
WHERE ROWNUM <= 10;

T       CUSTOMER_ID       COUNT(CUSTOMER_ID)
-       -----------       ------------------
S                 1                       22
S                 2                        8
S                 3                       47
S                 4                        8
S                 5                       41
S                 6                       15
S                 7                       15
S                 8                       55
S                 9                       17
S                 10                      10

Firstly, use the index as a composite.

EXPLAIN PLAN SET statement_id='TEST' FOR
      SELECT type, customer_id FROM orders
      WHERE type = 'S' AND customer_id = 8;

Query                                 Cost     Rows     Bytes
-----------------------------------   ----   ------   -------
SELECT STATEMENT on                      1       12        36
 INDEX RANGE SCAN on
   XAK_O_TYPE_CUSTOMER                   1       12        36

Now remove the type column index prefix from the WHERE clause. The same index is used with an index skip scan, skipping the prefix.

EXPLAIN PLAN SET statement_id='TEST' FOR
      SELECT type, customer_id FROM orders
         WHERE customer_id = 8;

Query                                 Cost     Rows     Bytes
-----------------------------------   ----   ------   -------
SELECT STATEMENT on                      3       24        72
 INDEX SKIP SCAN on
   XAK_O_TYPE_CUSTOMER                   3       24        72

Selecting all columns uses a range scan on a different index because the CUSTOMER_ID column has an index. There is a foreign key index on the CUSTOMER_ID column.

EXPLAIN PLAN SET statement_id='TEST' FOR
      SELECT * FROM orders WHERE customer_id = 8;

Query                                 Cost     Rows     Bytes
-----------------------------------   ----   ------   -------
SELECT STATEMENT on                      7       24       552
 TABLE ACCESS BY INDEX ROWID
   on ORDERS                             7       24       552
  INDEX RANGE SCAN on
    XFK_O_CUSTOMER                       1       24

Index Full Scan

A full index scan reads the entire index in key order and not the physical block order of the index. Retrieved columns must be in the index with one non-NULL column, as can be seen in the following two examples. The first example shows a single-column primary key index and the second a multiple-column composite primary key index.

EXPLAIN PLAN SET statement_id='TEST' FOR SELECT
   stock_id FROM stock;

Query                                 Cost     Rows     Bytes
-----------------------------------   ----   ------   -------
SELECT STATEMENT on                      1      118       354
 INDEX FULL SCAN on XPK_STOCK            1      118       354

EXPLAIN PLAN SET statement_id='TEST' FOR
      SELECT year, period, coa# FROM periodsum;

Query                                 Cost     Rows     Bytes
-----------------------------------   ----   ------   -------
SELECT STATEMENT on                      1       45       450
 INDEX FULL SCAN on XPK_PERIODSUM        1       45       450

The benefit of a full index scan is a sorted result since the index is scanned in sorted key order. Sometimes a full index scan is used by the Optimizer to remove the need for sorting when an SQL statement includes an ORDER BY or GROUP BY clause. Let's look into use of the ORDER BY clause first. This query, as shown in the query plan, executes a full index scan on the Stock table primary key index. Sorting for the ORDER BY clause is not shown in the query plan because the rows are read from the index using the index full scan in the required STOCK_ID column sorted order.

EXPLAIN PLAN SET statement_id='TEST' FOR
      SELECT stock_id FROM stock ORDER BY stock_id;

Query                                 Cost     Rows     Bytes
-----------------------------------   ----   ------   -------
SELECT STATEMENT on                      1      118       354
 INDEX FULL SCAN on XPK_STOCK            1      118       354

The inherent sorting of the full index scan can be proved again by the following example. The INDEX_FFS (fast full index scan) hint has been used to remove the ORDER BY clause index sorted order. We will cover the fast full index scan shortly. A fast full index scan reads the index in physical block order and not sorted key order. Thus the query below being read in physical order requires STOCK_ID column sorting to satisfy the ORDER BY clause. Notice the increase in cost from 2 to 4 to execute the SORT ORDER BY step in the query plan.

EXPLAIN PLAN SET statement_id='TEST' FOR
      SELECT /*+ INDEX_FFS(stock, xpk_stock) */
         stock_id FROM stock
      ORDER BY stock_id;

Query                                 Cost     Rows     Bytes
-----------------------------------   ----   ------   -------
SELECT STATEMENT on                      4      118       354
 SORT ORDER BY on                        4      118       354
  INDEX FAST FULL SCAN on XPK_STOCK      2      118       354

Now let's take a look at GROUP BY clause sorting catered for using a full index scan. The following two examples show no increase in cost for sorting and distinctly show that NOSORT is required. There is no extra sorting.

EXPLAIN PLAN SET statement_id='TEST' FOR
      SELECT order_id, COUNT(order_id) FROM ordersline
      GROUP BY order_id;

Query                                 Cost     Rows     Bytes
-----------------------------------   ----   ------   -------
SELECT STATEMENT on                   2784   226703    906812
 SORT GROUP BY NOSORT on              2784   226703    906812
  INDEX FULL SCAN on XFK_OL_ORDERS    2784   997903   3991612

EXPLAIN PLAN SET statement_id='TEST' FOR
      SELECT order_id, seq#, COUNT(order_id) FROM ordersline
      GROUP BY seq#, order_id;

Query                                 Cost     Rows     Bytes
-----------------------------------   ----   ------   -------
SELECT STATEMENT on                   3134   997903   5987418
 SORT GROUP BY NOSORT on              3134   997903   5987418
  INDEX FULL SCAN on
    XPK_ORDERSLINE                    3134   997903   5987418

Sometimes a full index scan will be used, passing ROWID pointers to a table to retrieve rows in that table as opposed to only reading the index. Again there is no sorting required.

EXPLAIN PLAN SET statement_id='TEST' FOR
      SELECT * FROM stock ORDER BY stock_id;

Query                                 Cost     Rows     Bytes
-----------------------------------   ----   ------   -------
SELECT STATEMENT on                      5      118     27612
 TABLE ACCESS BY INDEX ROWID
   on STOCK                              5      118     27612
  INDEX FULL SCAN on XPK_STOCK           1      118

The following example is interesting because the key ordered index full scan is used to find both minimum and maximum values at either ends of the sorted key value range.

EXPLAIN PLAN SET statement_id='TEST' FOR
      SELECT MIN(stock_id), MAX(stock_id) FROM stock;

Query                                 Cost     Rows     Bytes
-----------------------------------   ----   ------   -------
SELECT STATEMENT on                      1        1         3
 SORT AGGREGATE on                       1        3
  INDEX FULL SCAN on XPK_STOCK           1      118       354
Fast Full Index Scan

A fast full index scan is a variation on and in some respects an improvement to a full index scan. Fast full index scans are only available for cost-based optimization. A full index scan reads an index in the sorted key index order. A fast full index scan is faster because it reads the index in physical order directly from the disk rather than in key order. A full index scan reading in key order will likely be a random physical search. When reading the entire index the physical index may as well be read in physical sequential order unless the index order is required. The following example shows use of a fast full index scan where no ordering is specified.

Tip

An additional benefit of a fast full index scan over a full index scan is that the DB_FILE_MULTIBLOCK_READ_COUNT can be utilized since the index is read in physical order.

EXPLAIN PLAN SET statement_id='TEST' FOR
      SELECT stock_id, supplier_id FROM stocksource;

Note that if the sequence of the columns were reversed the fast full index scan would still be used.

Query                                 Cost     Rows     Bytes
-----------------------------------   ----   ------   -------
SELECT STATEMENT on                      5    12083     72498
 INDEX FAST FULL SCAN on
   XPK_STOCKSOURCE                       5    12083     72498

This second example is a modification of the first where an ORDER BY clause is added for the same columns retrieved, in the same sequence. The Optimizer reverts back to a full index scan since the ORDER BY clause specifies the index be read in sorted key order rather than unsorted physical order as the fast full index scan does.

EXPLAIN PLAN SET statement_id='TEST' FOR
      SELECT stock_id, supplier_id FROM stocksource
      ORDER BY stock_id, supplier_id;

Query                                 Cost     Rows     Bytes
-----------------------------------   ----   ------   -------
SELECT STATEMENT on                     25    12083     72498
 INDEX FULL SCAN on XPK_STOCKSOURCE     25    12083     72498

Now let's force use of the unordered fast full index scan for the query shown previously with the ORDER BY clause added to show the difference in cost. Note the use of temporary sort space on disk to execute the sort plus the increase in cost for sorting denoted by SORT ORDER BY in the query plan.

EXPLAIN PLAN SET statement_id='TEST' FOR
      SELECT /*+ INDEX_FFS(stocksource, xpk_stocksource) */
      stock_id, supplier_id FROM stocksource
      ORDER BY stock_id, supplier_id;

Query                            Cost     Rows     Bytes      Sort
------------------------------   ----   ------   -------   -------
SELECT STATEMENT on                53    12083    72498
 SORT ORDER BY on                  53    12083    72498     402000
  INDEX FAST FULL SCAN on
    XPK_STOCKSOURCE                 5    12083    72498

Therefore, since a fast full index scan does not read an index in key order overhead for ORDER BY or GROUP BY clause sorting would apply as shown in the previous example.

Now let's take a look at some interesting effects of query variations on full and fast full index scans.

  • The DISTINCT clause.

  • The COUNT function.

  • Retrieving from an index with NOT NULL.

  • Parallel execution with multiple CPUs.

The DISTINCT Clause

DISTINCT always executes an inherent sort.

EXPLAIN PLAN SET statement_id='TEST' FOR
      SELECT DISTINCT(stock_id) FROM stockmovement;

Query                                 Cost     Rows     Bytes
-----------------------------------   ----   ------   -------
SELECT STATEMENT on                   1506      118       354
 SORT UNIQUE on                       1506      118       354
  INDEX FAST FULL SCAN on
    XFK_SM_STOCK                       312  1043460   3130380

Even when using the presorted full index scan as in the next example the DISTINCT clause still forces a sort. Thus, the previous query using the fast full index scan has a lower cost, even though the query plan specifies SORT UNIQUE NOSORT as opposed to SORT UNIQUE in the previous example.

EXPLAIN PLAN SET statement_id='TEST' FOR
      SELECT /*+ ORDERED INDEX(stockmovement, xfk_sm_stock) */
         DISTINCT(stock_id) FROM stockmovement ORDER BY 1;

Query                                 Cost     Rows     Bytes
-----------------------------------   ----   ------   -------
SELECT STATEMENT on                   3243      118       354
 SORT UNIQUE NOSORT on                3243      118       354
  INDEX FULL SCAN on XFK_SM_STOCK     2049  1043460   3130380
The COUNT Function

The COUNT function executes directly from the fast full index scan step without sorting requirements. The step called SORT AGGREGATE is not a sort operation since there is no cost increase associated with it.

EXPLAIN PLAN SET statement_id='TEST' FOR
      SELECT COUNT(*) FROM generalledger;

Query                                 Cost     Rows     Bytes
-----------------------------------   ----   ------   -------
SELECT STATEMENT on                    352        1
 SORT AGGREGATE on                                1
  INDEX FAST FULL SCAN on
    XPK_GENERALLEDGER                  352  1068929
Retrieving with NOT NULL

The IS NOT NULL comparison allows use of the fast full index scan option reading just non-null value rows from the index. If the query was sorted with the ORDER BY clause the fast full index scan would revert to a full index scan to take advantage of the full index scan sorted key traversal.

EXPLAIN PLAN SET statement_id='TEST' FOR
      SELECT customer_id FROM transactions
      WHERE customer_id IS NOT NULL;

Query                                 Cost     Rows     Bytes
-----------------------------------   ----   ------   -------
SELECT STATEMENT on                     29    64599    129198
 INDEX FAST FULL SCAN on
   XFK_T_CUSTOMER                       29    64599    129198
Parallel Index Scan

Parallel scans can be executed on fast full index scans (block order index scans) and not on full index scans (key order) or index range scans. An index can have a degree of parallelism set for it using the CREATE INDEX or ALTER INDEX commands. Hints can also be used. Once again queries in this section are executed on a dual CPU machine without Oracle Partitioning. Oracle Partitioning is covered in Part III. Let's do the same thing we did with parallel full table scans.

ALTER INDEX XFK_GL_COA# PARALLEL 2;

EXPLAIN PLAN SET statement_id='TEST' FOR
      SELECT coa# FROM generalledger;

Query                                           Cost     Rows
---------------------------------------------   ----   ------
1. SELECT STATEMENT on                             4   752740
2.  INDEX FAST FULL SCAN on XFK_GL_COA#
      PARALLEL_TO_SERIAL                           4   752740

Remove parallelism from the index.

ALTER INDEX XFK_GL_COA# NOPARALLEL;

Now let's check the query plan again. We have an increase in cost without parallelism.

EXPLAIN PLAN SET statement_id='TEST' FOR
      SELECT coa# FROM generalledger;

Query                                 Cost     Rows     Bytes
-----------------------------------   ----   ------   -------
1. SELECT STATEMENT on                   8   752740   4516440
2.  INDEX FAST FULL SCAN on
      XFK_GL_COA#                        8   752740   4516440

Since we have an increase in cost let's increase the degree of parallelism on the index. It seems that unlike full table scans, fast full index scans are complemented by use of parallelism even in a small database with no Oracle Partitioning.

ALTER INDEX XFK_GL_COA# PARALLEL 4;

EXPLAIN PLAN SET statement_id='TEST' FOR
      SELECT coa# FROM generalledger;

The higher we set the degree of parallelism for the index the lower the cost for the index fast full scan.

Query                                           Cost     Rows
---------------------------------------------   ----   ------
1. SELECT STATEMENT on                             2   752740
2.  INDEX FAST FULL SCAN on XFK_GL_COA#
      PARALLEL_TO_SERIAL                           2   752740

Let's do some timing tests. The degree of parallelism is set first to NOPARALLEL, then to 2 and finally to 4.

ALTER INDEX xfk_gl_coa# NOPARALLEL;

SQL> SELECT COUNT(*) FROM(SELECT coa# FROM generalledger);

   COUNT(*)
-----------
     752741

Elapsed: 00:00:00.05

ALTER INDEX xfk_gl_coa# PARALLEL 2;

SQL> SELECT COUNT(*) FROM(SELECT coa# FROM generalledger);

   COUNT(*)
-----------
     752741

Elapsed: 00:00:03.01

ALTER INDEX xfk_gl_coa# PARALLEL 4;

SQL> SELECT COUNT(*) FROM(SELECT coa# FROM generalledger);

   COUNT(*)
-----------
     752741

Elapsed: 00:00:03.01

The timing tests show that using no parallelism on the fast full index scan is faster than degrees of parallelism of both 2 and 4.

Therefore, in contradiction to the query plans, like fast full table scans, fast full index scan parallelism is not helpful for performance in any database type other than probably a data warehouse or unless Oracle Partitioning is used.

Index Join

Index joins use a hashing function to join scans of multiple indexes together if all columns can be retrieved from the indexes, no matter what type of index scans they are. Index joins are only available with cost-based optimization and statistics.

EXPLAIN PLAN SET statement_id='TEST' FOR
      SELECT
      /*+ INDEX_JOIN(generalledger xpk_generalledger
         xfx_gl_coa#) */
      generalledger_id, coa# FROM generalledger;

Query                                 Cost     Rows     Bytes
-----------------------------------   ----   ------   -------
SELECT STATEMENT on                 474818  1068929  10689290
 VIEW on index$_join$_001           474818  1068929  10689290
  HASH JOIN on                              1068929  10689290
   INDEX FAST FULL SCAN on
     XFK_COA#                         2886  1068929  10689290
    INDEX FAST FULL SCAN on
      XPK_GENERALLEDGER               2886  1068929  10689290

Following is the same query with a filtering restriction on the COA# column, changing one of the fast full index scans to an index range scan.

EXPLAIN PLAN SET statement_id='TEST' FOR
      SELECT
      /*+ INDEX_JOIN(generalledger xpk_generalledger
         xfx_gl_coa#) */
      generalledger_id, coa# FROM generalledger
      WHERE coa# = '60001';

Query                                 Cost     Rows     Bytes
-----------------------------------   ----   ------   -------
SELECT STATEMENT on                  83648   178155   1781550
 VIEW on index$_join$_001            83648   178155   1781550
  HASH JOIN on 178155 1781550
   INDEX RANGE SCAN on XFK_COA#        531   178155   1781550
    INDEX FAST FULL SCAN on
      XPK_GENERALLEDGER                531   178155   1781550

Bitmap Join

The Accounts schema has no bitmap indexes so I will create some temporarily on a copy of an existing table. Bitmap joins are only available with cost-based optimization and statistics.

CREATE TABLE tmp AS SELECT * FROM periodsum;
ANALYZE TABLE tmp COMPUTE STATISTICS;
CREATE BITMAP INDEX xfk_tmp_year ON tmp(year)
   COMPUTE STATISTICS;
CREATE BITMAP INDEX xfk_tmp_period ON tmp(period)
   COMPUTE STATISTICS;
CREATE BITMAP INDEX xfk_tmp_coa# ON tmp(coa#)
   COMPUTE STATISTICS;

EXPLAIN PLAN SET statement_id='TEST' FOR
      SELECT /*+ INDEX_JOIN(tmp) */ year, period,
         coa# FROM tmp;

Query                                 Cost     Rows     Bytes
-----------------------------------   ----   ------   -------
SELECT STATEMENT on                      6       45       405
 VIEW on index$_join$_001                6       45       405
  HASH JOIN on                                   45       405
   HASH JOIN on                                  45       405
    BITMAP CONVERSION TO ROWIDS on
     BITMAP INDEX FULL SCAN on
       XFK_TMP_COA#
    BITMAP CONVERSION TO ROWIDS on
     BITMAP INDEX FULL SCAN on
       XFK_TMP_PERIOD
   BITMAP CONVERSION TO ROWIDS on
    BITMAP INDEX FULL SCAN on
      XFK_TMP_YEAR

Let's clean up and drop the temporary table.

DROP TABLE tmp;

Cluster and Hash Scans

Cluster and hash scans access clusters and hash clusters, respectively. Both work in a similar fashion to that of accessing a ROWID from one object and retrieving from another object using the ROWID pointer. Clusters are generally read-only types of objects and since read-only objects are part of data warehousing they will not be covered in this book in detail. Let's take a very brief look at a cluster.

Let's create a cluster. Since a cluster is a read-only object we will take advantage of the physical storage capabilities of Oracle Database and squash as much into each block as possible. Additionally before creating the cluster I have to log in as SYSTEM and explicitly grant the CREATE TABLE and CREATE CLUSTER privileges to my schema.

GRANT CREATE TABLE TO accounts;
GRANT CREATE CLUSTER TO accounts;

Now back to my Accounts schema.

CREATE CLUSTER clu_tb (coa# CHAR(5))
      PCTFREE 0 PCTUSED 99 SIZE 512 TABLESPACE DATA
      STORAGE (INITIAL 1M NEXT 1M);
CREATE INDEX xclu_tb ON CLUSTER clu_tb TABLESPACE INDX
      COMPUTE STATISTICS;

Now let's add a table to the cluster.

CREATE TABLE trialbalance CLUSTER clu_tb(coa#) AS SELECT *
   FROM coa;
ANALYZE TABLE trialbalance COMPUTE STATISTICS;

EXPLAIN PLAN SET statement_id='TEST' FOR
      SELECT /*+ CLUSTER(trialbalance) */ *
         FROM trialbalance
      WHERE coa# = '50028';

Query                                 Cost     Rows     Bytes
-----------------------------------   ----   ------   -------
1. SELECT STATEMENT on                   1        1        24
2.  TABLE ACCESS CLUSTER on
      TRIALBALANCE                       1        1        24
3.   INDEX UNIQUE SCAN on
       XCLU_TB                                   55

Let's clean up.

DROP TABLE trialbalance;
DROP CLUSTER clu_tb;

8.5.2 Joining Tables

Oracle Database takes specific steps internally when joining tables together in SQL statements. The Optimizer decides on different types of join methods in cost-based optimization using statistics on tables and indexes. When joining tables, Oracle Database calls a table or part of a table a row source. A row source is a chunk of memory, much like a cursor in PL/SQL, containing a pointer to a current position in a set of rows. When multiple row sources are joined the Optimizer makes an assessment based on statistics and other factors to decide how two tables should be joined. These different join methods are listed and described in this section and have various different aspects and applications.

Tip

What Oracle Corporation calls a row source I like to call a row set or set of rows, being a subset of a table.

Join Order Execution

In joins with more than two tables, one table is joined to the second, which is in turn joined to the third table, and so on. The Optimizer will approach a join in this manner, adapting use of join methods accordingly. If the first join results in a small row set and the second join a large row set, the result of the first join effectively joins to a large set of rows. Therefore, it is extremely important how tables are placed into the FROM clause both in the Oracle SQL proprietary and ANSI standard join format syntax for SQL join statements. A join statement, much like any SQL statement, is parsed from left to right and from top to bottom. This is not a hard and fast rule, as applied to joins and any SQL statement in general.

Types of Joins

  • Nested Loop Join. A nested loop is usually used between a sorted and an unsorted set of rows. Nested loops generally involve the joining of a small row set to a large row set but are also the most efficient join for small row sets on both sides of the join.

  • Hash Join. Hash joins are utilized between two large sets of rows where a smaller row set has a temporary hash table generated for it. Generation of the hash table creates extra overhead. A hash join is only faster than a nested loop when there are a large number of rows in the smaller row set. The Optimizer will use a hash join rather than a nested loop join when the smaller row set is too large to warrant using a nested loop. When joining, the hash table is read and each entry prompts a search into the larger table.

  • Sort Merge Join. Sort merge joins are used between two large row sets where one or both row sets are unsorted. A sort merge join is chosen when row sets are too large for both nested loop and hash joins to be efficient. In a sort merge join row sets are sorted first, separately, and then merged together.

  • Other Join Types. Joins classified as "other" are generally joins affecting or affected by the use of nested loop, hash, and sort merge joins. These types include semi-joins, bitmaps, star queries (data warehouse), multiple table (mutable) nested joins, and some types of joins best avoided.

A hash join is the next worst option compared to that of a nested loop join. A sort merge join is the least efficient option of all three. A nested loop join is what should be aimed at preferably because it operates on small row sets. Small row sets lead to fast performance of joins. Any type of SQL code performs better when retrieving small row sets.

A nested loop join is typically used between an index and a full scan of some type, associating a value from the index to the full scan, or vice versa. A hash join is the same but often replaces a nested loop when two full scans are involved or when large row sets force the Optimizer to decide that a hash join is faster than a nested loop. A hash join is the second-best option because a hash join creates a hash table. The process of creating a hash table is an extra processing step, as a replacement for the index used in a nested loop. A sort merge join is used in favor of a nested loop or a hash join when two large row sets are involved or sorting is required.

The choice the Optimizer makes between the three join types can depend on relative configuration parameter values. Configuration parameters affecting join type selection are as follows:

  • SORT_AREA_SIZE. An increase favors join types requiring sorting.

  • HASH_AREA_SIZE. Increases the likelihood of hash joins.

    Tip

    Set the HASH_JOIN_ENABLED configuration parameter to TRUE to enable the use of hash joins by the Optimizer.

  • DB_FILE_MULTIBLOCK_READ_COUNT. Increases the likelihood of join types operating on large amounts of data such as a sort merge join.

Now let's examine and demonstrate each join method.

Nested Loop Join

A nested loop join is used between a sorted table and an unsorted table. Thus the sorted table is scanned with a sorted index scan where the unsorted table is accessed by a ROWID pointer from the sorted table. A nested loop is literally a loop within a loop, thus the term "nested". The cursor for one SQL statement is executed within a driving SQL statement. Therefore, a nested loop contains an outer and an inner loop where the inner loop is driven by each row found by the outer loop. Obviously the fewer rows in either loop or the smaller the loops, the more efficient a nested loop is.

Examine the following two queries and their query plans. Note two things. Firstly, both query plans are identical even though the second query has the sequence of the tables reversed in the FROM clause. The Optimizer is intelligent enough to figure this out. Secondly, some changes have been made to the output of the query plan, in addition to query plans shown previously in this book. The LEVEL pseudocolumn is shown on the left. The LEVEL pseudocolumn shows the branch layer in a hierarchical tree structured query result. In this case LEVEL shows the layer in which the Optimizer executes each function. Another addition to the query plans shown is the position column (Pos). The position column represents a sequence of execution in reverse. Thus in the first query plan shown following, the unique index scan is executed first on the COA table, for each row found by the fast full index scan in the GeneralLedger table. The nested loops join is driving the application of row selection from the GeneralLedger table accessing each match in the COA table. Thus, the nested loop and index fast full scan are executed at the same time. The SELECT statement is obviously executed on the result of the nested loop join.

Tip

Remember that a fast full index scan is a physically sequential complete read of an index data space in block order. It is not sorted in index order.

EXPLAIN PLAN SET statement_id='TEST' FOR
      SELECT coa.coa#, gl.coa#
      FROM coa JOIN generalledger gl
         ON(coa.coa# = gl.coa#);

Query                              Cost     Rows     Bytes   Pos
--------------------------------   ----   ------   -------   ---
1. SELECT STATEMENT on              609  1068929  10689290     0
2.  NESTED LOOPS on                 609  1068929  10689290     1
3.   INDEX FAST FULL
       SCAN on XFK_COA#             609  1068929   5344645     1
3.    INDEX UNIQUE SCAN
        on XPK_COA                             1         5     2

EXPLAIN PLAN SET statement_id='TEST' FOR
      SELECT coa.coa#, gl.coa#
      FROM generalledger gl JOIN coa
         ON(gl.coa# = coa.coa#);

Query                              Cost     Rows     Bytes   Pos
--------------------------------   ----   ------   -------   ---
1. SELECT STATEMENT on              609  1068929  10689290     0
2.  NESTED LOOPS on                 609  1068929  10689290     1
3.   INDEX FAST FULL SCAN
       on XFK_COA#                  609  1068929   5344645     1
3.    INDEX UNIQUE SCAN
        on XPK_COA                             1         5     2

All rows in the inner loop row set will be accessed by each row in the outer loop row set. Notice I am not using the term table but the term row set. Once again what is a row set? When filtering is included in a query then the row set is a subset of the rows in a table or from a query result. Following is a modification to the previous example using filtering and retrieving non-indexed rows from the GeneralLedger table. This should provoke a full table scan on the GeneralLedger table. Note that the filtering is applied to the largest table, the GeneralLedger table.

EXPLAIN PLAN SET statement_id='TEST' FOR
      SELECT coa.coa#, gl.dte, gl.dr, gl.dr FROM coa
            JOIN generalledger gl ON(coa.coa# = gl.coa#)
      WHERE gl.coa# = '60001';

This query plan is ugly! On the basis of an outer table driving an inner table, where all rows in the inner table are accessed by each row in the outer table, this join should be very inefficient. The unique scan on the COA table is accessing every row in the GeneralLedger table. It might make sense if the tables in the nested loop were reversed. The reason for the full table scan is because GL.COA# = '60001' accesses 7% of the GeneralLedger table.

Query                              Cost     Rows     Bytes   Pos
--------------------------------   ----   ------   -------   ---
1. SELECT STATEMENT on             1129    29692    593840     0
2.  NESTED LOOPS on                1129    29692    593840     1
3.   INDEX UNIQUE SCAN on
       XPK_COA                                 1         5     1
3.    TABLE ACCESS FULL
        on GENERALLEDGER           1128    29693    445395     2

In the next example, the filter is placed on the smaller table pushing the full table access into the outer loop. This is a very slightly better performing join. The cost is the same and timing tests following are the same as well. The row and byte values are much higher which is significant. More data read is higher I/O but that is physical tuning and will be covered in Part III.

EXPLAIN PLAN SET statement_id='TEST' FOR
      SELECT coa.coa#, gl.dte, gl.dr, gl.dr
      FROM coa JOIN generalledger gl ON(gl.coa# = coa.coa#)
      WHERE coa.coa# = '60001';

Query                                 Cost     Rows     Bytes   Pos
-----------------------------------   ----   ------   -------   ---
1. SELECT STATEMENT on                1128  1068929  21378580     0
2.  NESTED LOOPS on                   1128  1068929  21378580     1
3.   TABLE ACCESS FULL
       on GENERALLEDGER               1128  1068929  16033935     1
3.    INDEX UNIQUE SCAN
        on XPK_COA                                1         5     2

SQL> SELECT COUNT(*) FROM(SELECT coa.coa#, gl.dte, gl.dr,
   gl.dr FROM coa

2 JOIN generalledger gl ON(coa.coa# = gl.coa#)
3 WHERE gl.coa# = '60001');

   COUNT(*)
-----------
      75555

Elapsed: 00:00:00.01

SQL> SELECT COUNT(*) FROM(SELECT coa.coa#, gl.dte,
   gl.dr, gl.dr
2 FROM coa JOIN generalledger gl ON(gl.coa# =
      coa.coa#)
3 WHERE coa.coa# = '60001');

   COUNT(*)
-----------
      75555

Elapsed: 00:00:00.01

It should now be obvious that a nested loop join is used by the Optimizer between a small, sorted row set and a larger, unsorted row set. Let's abuse the Optimizer a little further and partially contradict this. The following example finds a single row in the GeneralLedger table and thus the COA table as well. Thus we are finding two small row sets. A nested loop in this situation is the most efficient option. A sort merge would ignore the indexes and a hash join would create a hash key table as an extra superfluous step.

EXPLAIN PLAN SET statement_id='TEST' FOR
      SELECT coa.*, gl.*
      FROM generalledger gl JOIN coa ON(gl.coa# = coa.coa#)
      WHERE generalledger_id = 10;

Query                            Cost     Rows   Bytes   Pos
------------------------------   ----   ------   -----   ---
1. SELECT STATEMENT on              4        1      47     0
2.  NESTED LOOPS on                 4        1      47     1
3.   TABLE ACCESS BY
       INDEX ROWID on GE            3        1      23     1
4.    INDEX UNIQUE SCAN
        on XPK_GENERAL              2  1068929             1
3.   TABLE ACCESS BY
       INDEX ROWID on CO            1        1      24     2
4.    INDEX UNIQUE SCAN
        on XPK_COA                           1             1

A nested loop join is used where a small and large row set are joined but also where two very small row sets are joined. As you will see in the next section on hash joins, a hash join is often selected by the Optimizer to replace a nested loop join when there are two large row sets. The following query uses a nested loop rather than a hash join because it reads a unique index and physically covers too few blocks to make a hash join more efficient than a nested loop. A hash join will be selected as being faster than a nested loop join if the joined row sets get larger. This join is shown here because there is a related example in the next section on hash joins.

EXPLAIN PLAN SET statement_id='TEST' FOR
      SELECT s.stock_id, sm.stockmovement_id
      FROM stock s JOIN stockmovement sm
         ON(s.stock_id = sm.stock_id)
      WHERE s.stock_id < 10;

Query                            Cost     Rows   Bytes   Pos
------------------------------   ----   ------   -----   ---
1. SELECT STATEMENT on           1032     6174   67914     0
2.  NESTED LOOPS on              1032     6174   67914     1
3.   TABLE ACCESS FULL
       on STOCKMOVEMENT          1032    80266  642128     1
3.    INDEX UNIQUE SCAN
        on XPK_STOCK                         1       3     2

Hash Join

Hash joins join large row sets together. A temporary hash table for the outer or smaller table is used for matching rows in the inner table. The HASH_AREA_SIZE configuration parameter determines the size of the hash table to be placed into cache. When available cache space is exceeded hash tables are split apart and stored temporarily in temporary sort space. This is detrimental for performance.

A hash key is mathematically most efficient for doing exact searches. Consequently equi-joins apply to hash joins and range or anti-joins do not. This is not always strictly true.

The second-last join in the previous section on nested loops used a nested loop on a single row found in the larger table. Removing the filtering clause as in the following join makes the outer table too large for a nested loop and thus the Optimizer switches the nested loop to a hash join. In this case it is all about physical space covered. The Optimizer considers it faster to create and read a hash table rather than use a nested loop because a lot of rows are accessed.

EXPLAIN PLAN SET statement_id='TEST' FOR
      SELECT coa.*, gl.*
      FROM generalledger gl JOIN coa
         ON(gl.coa# = coa.coa#);

Query                            Cost     Rows     Bytes   Pos
------------------------------   ----   ------   -------   ---
1. SELECT STATEMENT on           1642  1068929  50239663     0
2.  HASH JOIN on                 1642  1068929  50239663     1
3.   TABLE ACCESS FULL
       on COA                       2       55      1320     1
3.    TABLE ACCESS FULL
        on GENERALLEDGE          1128  1068929  24585367     2

The last join in the previous section on nested loop joins selected indexed column values only. The next query plan changes from a nested loop to a hash join because non-indexed table columns are selected, reading too much physical space to make a nested loop efficient.

EXPLAIN PLAN SET statement_id='TEST' FOR
      SELECT s.text, sm.qty, sm.price
      FROM stock s JOIN stockmovement sm
      ON(s.stock_id = sm.stock_id)
      WHERE s.stock_id < 10;

Query                            Cost     Rows   Bytes   Pos
------------------------------   ----   ------   -----   ---
1. SELECT STATEMENT on           1060     6174  345744     0
2.  HASH JOIN on                 1060     6174  345744     1
3.   TABLE ACCESS BY
       INDEX ROWID on ST            2        9     405     1
4.    INDEX RANGE SCAN
        on XPK_STOCK                1        9             1
3.     TABLE ACCESS FULL
         on STOCKMOVEMEN         1032    80266  882926     2

Sort Merge Join

A sort merge join executes three separate steps. The first two steps sort both tables separately and the third step merges them together into a sorted result. Typically a sort merge join is used to join tables where no indexes are used on either table or when sorting is imposed using ORDER BY or GROUP BY clauses. The first query following shows both the sorting and I/O columns. Notice the large values in those columns.

EXPLAIN PLAN SET statement_id='TEST' FOR
      SELECT * FROM transactionsline tl NATURAL JOIN
         stockmovement sm;

Query                            Cost     Sort      IO
------------------------------   ----   ------   -----
1. SELECT STATEMENT on          25616            25616
2.  MERGE JOIN on               25616            25616
3.   TABLE ACCESS BY INDEX
      ROWID on ST                6619             6619
4.    INDEX FULL SCAN on
        XPK_STOCKMOVE            2189             2189
3.    SORT JOIN on              18997  57869000  18997
4.     TABLE ACCESS FULL on
         TRANSACTION              800              800

A sort merge join is likely to involve extensive I/O activity and use of temporary sort space on disk. Adjusting the SORT_AREA_SIZE configuration parameter can help but increasing it will have adverse effects in other respects, to be discussed in Part III.

The next join is an alteration to the second-last hash join query shown in the previous section on hash joins, with the addition of an ORDER BY clause. The application of the ORDER BY clause persuades the Optimizer to switch from a hash join to a sort merge join, thereby avoiding the sort required by the ORDER BY clause. The sort part of the sort merge join takes care of the ORDER BY sorting required.

EXPLAIN PLAN SET statement_id='TEST' FOR
      SELECT coa.*, gl.*
      FROM generalledger gl JOIN coa ON(gl.coa# = coa.coa#)
      ORDER BY coa.coa#;

Note the two separated (by the position column) steps for sorting and merging in the next query plan. The sort is executed on the COA table and the merge joins the sorted result to the index full scan of the GeneralLedger table. Also note that the concept of an inner and an outer table do not apply as in hash and nested loop joins. A sort merge join has two separate steps: sorting of two separate row sets, followed by merging them together.

Query                                 Cost     Rows     Bytes   Pos
-----------------------------------   ----   ------   -------   ---
1. SELECT STATEMENT on               20782  1068929  50239663     0
2.  MERGE JOIN on                    20782  1068929  50239663     1
3.   TABLE ACCESS BY
       INDEX ROWID on GE             20778  1068929  24585367     1
4.    INDEX FULL SCAN
        on XFK_COA#                   2536  1068929               1
3.   SORT JOIN on                        4       55      1320     2
4.     TABLE ACCESS FULL on COA          2       55      1320     1

A similar Optimizer decision would apply for a GROUP BY clause.

EXPLAIN PLAN SET statement_id='TEST' FOR
      SELECT o.order_id, SUM(ol.amount)
      FROM ordersline ol JOIN orders o
            ON(o.order_id = ol.order_id)
      GROUP BY o.order_id;

A sort is not required for the GROUP BY clause as indicated by the SORT GROUP BY NOSORT on in the query plan.

Query                            Cost     Rows     Bytes   Pos
------------------------------   ----   ------   -------   ---
1. SELECT STATEMENT on          12243   226703   2947139     0
2.  SORT GROUP BY
      NOSORT on                 12243   226703   2947139     1
3.   MERGE JOIN on              12243   997903  12972739     1
4.    INDEX FULL SCAN
        on XPK_ORDERS             479   226703    906812     1
4.     SORT JOIN on             11764   997903   8981127     2
5.      TABLE ACCESS FULL
          on ORDERSLINE           765   997903   8981127     1

Sorting is also not required when a full index scan is used. Full index scans are read in index order and are thus pre-sorted. Fast full index scans are read in physical order and are not sorted in index order. A fast full index scan would require sorting.

EXPLAIN PLAN SET statement_id='TEST' FOR
      SELECT tl.transaction_id, sm.stockmovement_id
      FROM transactionsline tl JOIN stockmovement sm
            ON(tl.stockmovement_id = sm.stockmovement_id);

In this query plan the sort part of the sort merge join is applied to the fast full index scan in order to sort it into the required order.

Query                            Cost     Rows     Bytes   Pos
------------------------------   ----   ------   -------   ---
1. SELECT STATEMENT on          10552  1027251  14381514     0
2.  MERGE JOIN on               10552  1027251  14381514     1
3.   TABLE ACCESS BY
       INDEX ROWID on TR          826  1027251  9245259     1
4.    INDEX FULL SCAN on
        XFK_TL_STOCKM              26  1027251              1
3.   SORT JOIN on                9726  1043460  5217300     2
4.    INDEX FAST FULL
        SCAN on XPK_STOC          526  1043460  5217300     1

Here is a sort merge join where a sort is executed on both row sets in the join. This join is completely ridiculous and overzealous but is demonstrative in getting the point across; both row sets are sorted first prior to being joined. The cost is noticeably monstrous!

EXPLAIN PLAN SET statement_id='TEST' FOR
      SELECT * FROM generalledger a, generalledger
         b WHERE a.dte=b.dte;

Query                            Cost     Rows      Bytes   Pos
------------------------------   ----   ------    -------   ---
1. SELECT STATEMENT on          50949  ########  #########    0
2.  MERGE JOIN on               50949  ########  #########    1
3.   SORT JOIN on               25475   1068929   24585367    1
4.    TABLE ACCESS FULL
        on GENERALLEDGER         1128   1068929   24585367    1
3.   SORT JOIN on               25475   1068929   24585367    2
4.    TABLE ACCESS FULL
        on GENERALLEDGER         1128   1068929   24585367    1

Sort merge joins can be used over nested loops or even hash joins for larger data sets or when range or anti-join comparisons and operators are involved. Anti-joins will be covered shortly.

Mutable Join Nesting

The three basic join types of nested loops, hash joins, and sort merge joins can be nested together and even mixed into mutable joins. Each join in a mutable join is processed sequentially where the result of one join is joined to the next. Thus each separate join will be chosen by the Optimizer based on the content, size, and sorted nature of the row sets both retrieved from tables and produced by nested multilayered joins.

Tip

A mutable join is a join of more than two tables. A complex mutable join is a mutable join with filtering.

Here is a very simple example joining three small tables.

EXPLAIN PLAN SET statement_id='TEST' FOR
      SELECT t.text, coa.text
      FROM type t JOIN coa USING(type)
            JOIN subtype st USING(subtype);

Full table access on the first two tables causes the hash join. The result is joined to the next table using a nested loop due to the use of the index.

Query                            Cost     Rows     Bytes   Pos
------------------------------   ----   ------   -------   ---
1. SELECT STATEMENT on              5       55      1375     0
2.  NESTED LOOPS on                 5       55      1375     1
3.   HASH JOIN on                   5       55      1320     1
4.    TABLE ACCESS FULL
        on TYPE                     2        6        48     1
4.   TABLE ACCESS FULL
       on COA                       2       55       880     2
3.    INDEX UNIQUE SCAN
        on XPK_SUBTYPE                       1         1     2

This next example now includes a very large table. The index fast full scan and the size of the row set produced by the first two joins persuades the use of a hash join rather than a nested loop for the final join.

EXPLAIN PLAN SET statement_id='TEST' FOR
      SELECT t.text, coa.text
      FROM type t JOIN coa USING(type)
            JOIN subtype st USING(subtype)
                  JOIN generalledger gl ON(gl.coa# = coa.coa#);
                     Query  Cost

Query                            Cost     Rows      Bytes   Pos
------------------------------   ----   ------    -------   ---
1. SELECT STATEMENT on            863  1068929   37412515     0
2.  HASH JOIN on                  863  1068929   37412515     1
3.   NESTED LOOPS on                5       55       1650     1
4.    HASH JOIN on                  5       55       1595     1
5.     TABLE ACCESS FULL
         on TYPE                    2        6         48     1
5.     TABLE ACCESS FULL
         on COA                     2       55       1155     2
4.    INDEX UNIQUE SCAN
        on XPK_SUBTYPE                       1          1     2
3.   INDEX FAST FULL SCAN
       on XFK_COA#                609  1068929    5344645     2

Here is a large join retrieving a small number of rows by using a WHERE clause filter. Nested loops are evident in abundance in the query plan due to the small row sets and all the indexes used.

EXPLAIN PLAN SET statement_id='TEST' FOR
      SELECT t.text, coa.text
      FROM type t JOIN coa USING(type)
            JOIN subtype st USING(subtype)
                  JOIN generalledger gl ON(gl.coa# =
                     coa.coa#)
      WHERE gl.coa# = '40003';

Query                            Cost     Rows      Bytes   Pos
------------------------------   ----   ------    -------   ---
1. SELECT STATEMENT on            425    29692    1039220     0
2.  NESTED LOOPS on               425    29692    1039220     1
3.   NESTED LOOPS on                2        1         30     1
4.    NESTED LOOPS on               2        1         29     1
5.     TABLE ACCESS BY
         INDEX ROWID on CO          1        1         21     1
6.      INDEX UNIQUE
          SCAN on XPK_COA                   55                1
5.     TABLE ACCESS BY
         INDEX ROWID on TY          1        1          8     2
6.    INDEX UNIQUE SCAN
        on XPK_TYPE                          1                1
4.   INDEX UNIQUE SCAN
       on XPK_SUBTYPE                        1          1     2
3.  INDEX RANGE SCAN
      on XFK_COA#                 423    29693     148465     2

And here is a really huge and nasty join. This is a poorly tuned query. The first filter is against the TransactionsLine table and that table should be selected from first. However, this query serves a purpose of showing all the three join types in a single query plan.

EXPLAIN PLAN SET statement_id='TEST' FOR
      SELECT c.name, tl.amount FROM customer c
       JOIN orders o USING(customer_id)
        JOIN ordersline ol USING(order_id)
         JOIN transactions t USING(customer_id)
          JOIN transactionsline tl USING(transaction_id)
           JOIN stockmovement sm
            ON(tl.stockmovement_id = sm.stockmovement_id)
             JOIN stock s USING(stock_id)
      WHERE tl.amount > 3170
      AND c.balance > 0;

Query                            Cost     Rows      Bytes   Pos
------------------------------   ----   ------    -------   ---
1. SELECT STATEMENT on          16978   271539   17650035     0
2.  NESTED LOOPS on             16978   271539   17650035     1
3.   MERGE JOIN on              16978   271539   16835418     1
4.    SORT JOIN on               8062    61688    3577904     1
5.     HASH JOIN on              5917    61688    3577904     1
6.      TABLE ACCESS FULL
          on CUSTOMER              16     2693      64632     1
6.      HASH JOIN on             5427    61688    2097392     2
7.       HASH JOIN on            4976     2621      73388     1
8.        HASH JOIN on           4403     2621      57662     1
9.         TABLE ACCESS
             FULL on
               TRANSACT           800     2621      36694     1
9.         TABLE ACCESS
             FULL on
               STOCKMOV          1032  1043460    8347680     2
8.        TABLE ACCESS
            FULL on
            TRANSACTI             390    64599     387594     2
7.       TABLE ACCESS
           FULL on ORDERS         238    63392     380352     2
4.     SORT JOIN on              8917   997903    3991612     2
5.      INDEX FAST
          FULL SCAN on
          XFK_OL_OR               668   997903    3991612     1
3.    INDEX UNIQUE SCAN
        on XPK_STOCK                         1          3     2

Here is a somewhat tuned version of the previous query removing from the join tables not having any selected columns. This query can be tuned further.

EXPLAIN PLAN SET statement_id='TEST' FOR
SELECT c.name, tl.amount
FROM customer c
  JOIN transactions t ON(t.customer_id = c.customer_id)
    JOIN transactionsline tl ON (tl.transaction_id =
      t.transaction_id)
WHERE c.balance > 0
AND tl.amount > 3170
AND EXISTS(
      SELECT o.customer_id FROM orders o
      WHERE o.customer_id = c.customer_id
      AND EXISTS(
            SELECT order_id FROM ordersline
            WHERE order_id = o.order_id))
AND EXISTS(
      SELECT sm.stockmovement_id FROM stockmovement sm
      WHERE sm.stockmovement_id = tl.stockmovement_id
      AND EXISTS(
            SELECT stock_id FROM stock
            WHERE stock_id = sm.stockmovement_id));

Query                                     Cost   Rows   Bytes
---------------------------------------   ----   ----   -----
1. SELECT STATEMENT on                       8      1      61
2.  FILTER on
3.   TABLE ACCESS BY INDEX ROWID on
       TRANSACTIONSL                         2      1      16
4.    NESTED LOOPS on                       19      1      61
5.     NESTED LOOPS on                      17      1      45
6.      TABLE ACCESS FULL on CUSTOMER       15      1      38
6.      TABLE ACCESS BY INDEX ROWID on
          TRANSACTION                        2     12      84
7.       INDEX RANGE SCAN on
           XFK_T_CUSTOMER                    1     12
5.     INDEX RANGE SCAN on
         XFK_TL_TRANSACTION                  1      3
3.    NESTED LOOPS on                       66    201    2412
4.     TABLE ACCESS BY INDEX ROWID on
         ORDERS                              2     64     448
5.      INDEX RANGE SCAN on XFK_O_CUSTOMER   1     12
4.     INDEX RANGE SCAN on XFK_OL_ORDERS     1      3      15
3.    NESTED LOOPS on                        1      1       9
4.     INDEX UNIQUE SCAN on
         XPK_STOCKMOVEMENT                   1      1       5
4.     INDEX UNIQUE SCAN on XPK_STOCK               1       4

Semi-Join

In the previous chapter we used what Oracle Corporation calls semi-joins to tune complex mutable joins. Semi-joins use the EXISTS and IN operators to join tables. It is called a semi-join because it is not really a join: columns cannot be returned from the embedded subquery.

EXPLAIN PLAN SET statement_id='TEST' FOR
      SELECT * FROM coa WHERE EXISTS
            (SELECT type FROM type WHERE type = coa.type);

Query                            Cost     Rows      Bytes   Pos
------------------------------   ----   ------    -------   ---
1. SELECT STATEMENT on              2       55       1375     0
2.  NESTED LOOPS SEMI on            2       55       1375     1
3.    TABLE ACCESS FULL on COA      2       55       1320     1
3.    INDEX UNIQUE SCAN on XPK_TYPE          6          6     2

EXPLAIN PLAN SET statement_id='TEST' FOR
      SELECT * FROM coa WHERE coa# IN
      (SELECT coa# FROM generalledger);

Query                            Cost     Rows      Bytes   Pos
------------------------------   ----   ------    -------   ---
1. SELECT STATEMENT on            860        6        174     0
2. HASH JOIN SEMI on              860        6        174     1
3. TABLE ACCESS FULL
     on COA                         2       55       1320     1
3. INDEX FAST FULL SCAN on
     XFK_COA#                     609  1068929    5344645     2

Joins to Avoid

Some joins are best avoided because they can perform very poorly. We have covered all the aspects of tuning SQL code in previous chapters. There are still some things we need to watch out for which can be detected at this level.

Cartesian Join

A Cartesian join is a Cartesian product. A Cartesian product joins two tables without a match condition. Thus every row in one table is matched with every row in the other table. The resulting row set is the multiplication of the rows in both tables, thus Cartesian product. In mathematics, a product of two numbers is a multiplication of those two numbers.

EXPLAIN PLAN SET statement_id='TEST' FOR
      SELECT * FROM coa, generalledger;

Query                            Cost     Rows      Bytes   Pos
------------------------------   ----   ------    -------   ---
1. SELECT STATEMENT on          62042  ########  #########    0
2. MERGE JOIN CARTESIAN on      62042  ########  #########    1
3. TABLE ACCESS FULL
     on COA                         2        55       1320    1
3. BUFFER SORT on               62040   1068929   24585367    2
4. TABLE ACCESS FULL on
     GENERALLEDG                 1128   1068929   24585367    1

Outer Join

Perhaps this is somewhat of a sweeping and opinionated statement but I have always found outer joins to be a problem, and they have been discussed as such in previous chapters. The crux of the problem with outer join use is that they retrieve data from two tables where some of the data is not matched. Outer join functionality is often applicable to reporting and data warehouses and can sometimes be an indication of poor data model structure or orphaned rows. These types of problems can necessitate the use of outer joins. Performance can be seriously affected. Yes, outer joins can perform well. The issue is should they exist in your application SQL code? High occurrence of SQL code outer joins is sometimes indicative of data model or data problems, especially in an OLTP database.

Tip

Nested loop and hash joins are completely contradictory to the objective of outer joins. Why? Nested loop and hash joins require an inner loop to be dependent on an outer loop. An outer join is completely contradictory to this because there is not a match between the inner and outer join tables for rows outside of the intersection. Additionally an outer join generally retrieves large row sets to be useful because it usually has to retrieve a larger row set in order to cater for missing rows. Full table scans, full index reads, and range scans are more likely using outer joins in SQL code.

EXPLAIN PLAN SET statement_id='TEST' FOR
      SELECT c.name, COUNT(o.customer_id)
      FROM customer c LEFT OUTER JOIN orders o
            ON(c.customer_id = o.customer_id)
      WHERE c.lastdate = '24-MAY-02'
      GROUP BY c.name;

Query                            Cost     Rows      Bytes   Pos
------------------------------   ----   ------    -------   ---
1. SELECT STATEMENT on             25        7        217     0
2.  SORT GROUP BY on               25        7        217     1
3.   NESTED LOOPS OUTER on         23      156       4836     1
4.    TABLE ACCESS FULL on
        CUSTOMER                   16        7        203     1
4.    INDEX RANGE SCAN on
        XFK_O_CUSTOM                1       24         48     2

EXPLAIN PLAN SET statement_id='TEST' FOR
      SELECT s.text, sm.qty, sm.price
      FROM stock s LEFT OUTER JOIN stockmovement sm
            ON(s.stock_id = sm.stock_id)
      WHERE s.stock_id < 10;

Query                            Cost     Rows      Bytes   Pos
------------------------------   ----   ------    -------   ---
1. SELECT STATEMENT on           1060     6174     345744     0
2.  HASH JOIN OUTER on           1060     6174     345744     1
3.   TABLE ACCESS BY INDEX
       ROWID on ST                  2        9        405     1
4.    INDEX RANGE SCAN on
        XPK_STOCK                   1        9                1
3.   TABLE ACCESS FULL on
       STOCKMOVEMEN              1032    80266     882926     2

EXPLAIN PLAN SET statement_id='TEST' FOR
      SELECT tl.transaction_id, sm.stockmovement_id
      FROM transactionsline tl RIGHT OUTER JOIN
         stockmovement sm
               ON(tl.stockmovement_id = sm.stockmovement_id);

Query                            Cost     Rows      Bytes   Pos
------------------------------   ----   ------    -------   ---
1. SELECT STATEMENT on          10552  1027251   14381514     0
2.  MERGE JOIN
      OUTER on                  10552  1027251   14381514     1
3.   TABLE ACCESS BY INDEX
       ROWID on TR               826   1027251    9245259     1
4.     INDEX FULL SCAN on
         XFK_TL_STOCKM            26   1027251                1
3.    SORT JOIN on              9726   1043460    5217300     2
4.     INDEX FAST FULL SCAN
         on XPK_STOC             526   1043460    5217300     1

The worst type of outer join is a full outer join. A full outer join is not a Cartesian product. It is actually a join but retrieves the intersection and all rows not in either table.

EXPLAIN PLAN SET statement_id='TEST' FOR
      SELECT tl.transaction_id, sm.stockmovement_id
      FROM transactionsline tl FULL OUTER JOIN
         stockmovement sm
            ON(tl.stockmovement_id = sm.stockmovement_id);

As can be seen from the query plan a full outer join creates a complete cost-excessive nightmarish mess! Avoid full outer joins altogether if possible.

Query                            Cost     Rows      Bytes   Pos
------------------------------   ----   ------    -------   ---
1. SELECT STATEMENT on          21808  1043460   27129960     0
2.  VIEW on                     21808  1043460   27129960     1
3.   UNION-ALL on                                             1
4.    MERGE JOIN OUTER on       10552  1027251   14381514     1
5.     TABLE ACCESS BY
         INDEX ROWID on           826  1027251    9245259     1
6.      INDEX FULL SCAN on
          XFK_TL_STOC              26  1027251                1
5.     SORT JOIN on              9726  1043460    5217300     2
6.      INDEX FAST FULL
          SCAN on XPK_ST          526  1043460    5217300     1
4.    MERGE JOIN ANTI on        11256    16209     162090     2
5.     INDEX FULL SCAN on
         XPK_STOCKMOV            2189  1043460    5217300     1
5.     SORT UNIQUE on            9067  1027251    5136255     2
6.      INDEX FAST FULL
          SCAN on XFK_TL            8  1027251    5136255     1
Grouped Outer Join

Oracle Database 10 Grid A grouped outer join allows joining of group aggregations of rows as outer joins, where a group can exist in one row set of a join but not another.

You might be asking at this stage why the different join types were not explained in previous chapters. This is because this depth level was not really required to explain how to tune SQL code from the point of view of writing properly tuned SQL code from a programming perspective. Most SQL code tuning is common sense and does not really require the level of analytical complexity presented in this chapter. Query plans can be used for verification and perhaps even to make things like spotting of missing indexes more likely. The point to make at this stage is that previous chapters did not require an understanding of the different ways in which the Optimizer joined tables in order to completely understand all the best ways to tune SQL code from a purely SQL coding perspective.


8.6 Sorting

There are different types of sorting operations used by the Optimizer. Numerous different types of operations trigger sorts, both implicit and explicitly specified.

  • SORT UNIQUE. Caused by the use of DISTINCT or a list of unique values.

  • SORT ORDER BY. The ORDER BY clause forces a sort operation when indexes or other aspects in the query plan do not sort the rows.

  • SORT GROUP BY. The GROUP BY clause groups or aggregates rows into a summarized set of rows. Sorting is required to aggregate duplicate column values into their distinct groups.

  • SORT JOIN. A sort on a column to be joined in a sort merge join.

  • SORT AGGREGATE. Aggregation across many rows.

8.6.1 Unique Sort

Here is a unique sort.

EXPLAIN PLAN SET statement_id='TEST' FOR
      SELECT DISTINCT(order_id) FROM ordersline;

Query                            Cost     Rows      Bytes   Pos
------------------------------   ----   ------    -------   ---
1. SELECT STATEMENT on           8209   226703     906812     0
2.  SORT UNIQUE on               8209   226703     906812     1
3.   INDEX FAST FULL SCAN
       on XFK_OL_ORDER            668   997903    3991612     1

The query below requires that QTY values in the subquery must be unique prior to being validated against QTY values in the calling query. There is no sense in checking the same value twice.

EXPLAIN PLAN SET statement_id='TEST' FOR
      SELECT * FROM stockmovement sm WHERE qty IN
            (SELECT qty FROM stockmovement);

Query                            Cost     Rows     Bytes       sort
------------------------------   ----   ------   --------   --------
SELECT STATEMENT on             16353   570175   15964900
 MERGE JOIN SEMI on             16353   570175   15964900
  SORT JOIN on                  11979   570175   13684200   45802000
  TABLE ACCESS FULL on
     STOCKMOVEMENT                355   570175   13684200
    SORT UNIQUE on               4374   570175    2280700   13755000
      TABLE ACCESS FULL on
         STOCKMOVEMENT            355   570175    2280700

8.6.2 ORDER BY Sort

In this first example the entire table is read and the ORDER BY clause forces a sort on one of the columns.

EXPLAIN PLAN SET statement_id='TEST' FOR
      SELECT * FROM customer ORDER BY name;

Query                       Cost   Rows    Bytes   Pos     Sort
-------------------------   ----   ----   ------   ---   ------
1. SELECT STATEMENT on       149   2694   360996     0
2.  SORT ORDER BY on         149   2694   360996     1   877000
3.   TABLE ACCESS FULL
       on CUSTOMER            16   2694   360996    1

This second example is different as the sort specified in the ORDER BY clause is catered for by the index full scan, a key value ordered index scan. Thus there is no SORT ORDER BY step in the query plan as the index is already sorted.

EXPLAIN PLAN SET statement_id='TEST' FOR
      SELECT customer_id FROM customer ORDER BY
            customer_id;

Query                       Cost   Rows    Bytes   Pos     Sort
-------------------------   ----   ----   ------   ---   ------
1. SELECT STATEMENT on         7   2694     8082     0
2.  INDEX FULL SCAN on
      XPK_CUSTOMER             7   2694     8082     1

Here is another example of the same thing. The GROUP BY clause causes a sort. The ORDER BY clause sorts the same thing and is thus superfluous.

EXPLAIN PLAN SET statement_id='TEST' FOR
      SELECT amount, COUNT(amount) FROM ordersline
      GROUP BY amount ORDER BY amount;

Query                     Cost   Rows     Bytes   Pos       Sort
-----------------------   ----   ----    ------   ---   --------
1. SELECT STATEMENT on    8682   97713   488565     0
2.  SORT GROUP BY on      8682   97713   488565     1   13214000
3.   TABLE ACCESS FULL
       on ORDERSLINE       765  997903  4989515     1

8.6.3 GROUP BY Sort

In this case the GROUP BY clause is required to sort by the aggregated column.

EXPLAIN PLAN SET statement_id='TEST' FOR
      SELECT amount, COUNT(amount) FROM ordersline
         GROUP BY amount;

Query                     Cost    Rows     Bytes   Pos       Sort
-----------------------   ----    ----    ------   ---   --------
1. SELECT STATEMENT on    8682   97713    488565     0
2.  SORT GROUP BY on      8682   97713    488565     1   13214000
3.   TABLE ACCESS FULL
       on ORDERSLINE       765  997903   4989515     1

Here is another example of a sort not required and indicated so by the Optimizer.

EXPLAIN PLAN SET statement_id='TEST' FOR
      SELECT customer_id, COUNT(customer_id) FROM customer
      GROUP BY customer_id;

Query                     Cost   Rows     Bytes   Pos       Sort
-----------------------   ----   ----    ------   ---   --------
1. SELECT STATEMENT on       7   2694      8082     0
2.  SORT GROUP BY NOSORT on  7   2694      8082     1
3.   INDEX FULL SCAN on
       XPK_CUSTOMER          7   2694      8082     1

8.6.4 Sort Merge Join Sort

In this example the ORDER BY clause is executing the sort on the COA table as indicated by the SORT JOIN in the query plan, allowing the MERGE JOIN to occur on the COA# column between the two row sets.

EXPLAIN PLAN SET statement_id='TEST' FOR
      SELECT coa.*, gl.*
      FROM generalledger gl JOIN coa ON(gl.coa# = coa.coa#)
      ORDER BY coa.coa#;

Query                     Cost     Rows       Bytes   Pos       Sort
-----------------------   ----   ------    --------   ---   --------
1. SELECT STATEMENT on   20782   1068929   50239663     0
2.  MERGE JOIN on        20782   1068929   50239663     1
3.   TABLE ACCESS BY INDEX
       ROWID on GE       20778   1068929   24585367     1
4.    INDEX FULL SCAN on
        XFK_COA#          2536   1068929                1
3.   SORT JOIN on            4        55       1320     2
4.    TABLE ACCESS FULL
        on COA               2        55       1320     1

8.6.5 Aggregate Sort

An aggregate sort is shown in the query plan for the following query.

EXPLAIN PLAN SET statement_id='TEST' FOR
      SELECT AVG(balance) FROM customer;

Query                     Cost   Rows     Bytes   Pos
-----------------------   ----   ----    ------   ---
1. SELECT STATEMENT on      16      1         2     0
2.  SORT AGGREGATE on               1         2     1
3.   TABLE ACCESS FULL
       on CUSTOMER          16   2694      5388     1


8.7 Special Cases

There are some special cases. The ones of interest for discussion involve concatenation, IN list operators, and the UNION set operator.

8.7.1 Concatenation

Concatenation is used when OR separated conditions do not occur on the same column. The following query uses a hint to force concatenation, making the cost of the query slightly higher. The cost is higher because the PeriodSum table is small and a full table scan is quicker as can be seen in the second example.

EXPLAIN PLAN SET statement_id='TEST' FOR
      SELECT /*+ USE_CONCAT */ * FROM periodsum
      WHERE year = 2001 OR period = 10 OR
         coa# = '60001';

Query                        Cost   Rows     Bytes   Pos
--------------------------   ----   ----    ------   ---
1. SELECT STATEMENT on          6     11       165     0
2.  CONCATENATION on                                   1
3.   TABLE ACCESS BY INDEX
       ROWID on PE              2      1        15     1
4.    INDEX RANGE SCAN on
        XFK_PS_COA              1      1               1
3.   TABLE ACCESS FULL on
       PERIODSUM                2      1        15     2
3.   TABLE ACCESS BY INDEX
       ROWID on PE              2      1        15     3
4.    INDEX RANGE SCAN on
        XPK_PERIODSU            1      1               1

EXPLAIN PLAN SET statement_id='TEST' FOR
      SELECT * FROM periodsum
      WHERE year = 2001 OR period = 10 OR coa# = '60001';

Query                        Cost   Rows     Bytes   Pos
--------------------------   ----   ----    ------   ---
1. SELECT STATEMENT on          2     31       465     0
2.  TABLE ACCESS FULL on
      PERIODSUM                 2     31       465     1

Concatenation only becomes a performance advantage when row sets are extremely large. The next two examples use a very large query where the cost is decreased substantially by application of the USE_CONCAT hint. In this case the Optimizer fails to find the lowest cost option. However, timing tests executed on these two queries make the query with the hint slower so the Optimizer does not fail. Timing is shown after the queries. On the contrary, the query plan cost estimate fails because the cost values are spurious.

EXPLAIN PLAN SET statement_id='TEST' FOR
      SELECT * FROM transactions t
      JOIN transactionsline tl USING(transaction_id)
            JOIN stockmovement sm USING(stockmovement_id)
                  JOIN stock USING(stock_id)
      WHERE t.type = 'S' OR tl.amount < 100 OR sm.qty = 0;

Query                        Cost   Rows     Bytes   Pos
--------------------------   ----   ----    ------   ---
1. SELECT STATEMENT on 71985 1020650 ######### 0
2.  HASH JOIN on 71985 1020650 ######### 1
3.   TABLE ACCESS FULL
       on STOCK  2   118   27612 1
3.   MERGE JOIN on 67130 1020650 76548750 2
4.    SORT JOIN on 60057 1027251 40062789 1
5.     MERGE JOIN on 25616 1027251 40062789 1
6.      SORT JOIN on 6619 1043460 23999580 1
7.     TABLE ACCESS FULL
         on STOCKMOV 6619 1043460 23999580 1
6.    SORT JOIN on 18997 1027251 16436016 2
7.     TABLE ACCESS FULL
         on TRANSACT 800 1027251 16436016 1
4.  FILTER on 2
5.   SORT JOIN on 1
6.    TABLE ACCESS FULL
        on TRANSACTI 390 242584 8733024 1

EXPLAIN PLAN SET statement_id='TEST' FOR
      SELECT /*+ USE_CONCAT */ * FROM transactions t
      JOIN transactionsline tl USING(transaction_id)
            JOIN stockmovement sm USING(stockmovement_id)
                  JOIN stock USING(stock_id)
      WHERE t.type = 'S' OR tl.amount < 100 OR sm.qty = 0;

Query                        Cost   Rows     Bytes   Pos
--------------------------   ----   ----    ------   ---
1. SELECT STATEMENT on      20861  104406  32261454    0
2.  CONCATENATION on                                   1
3.   NESTED LOOPS on         6885   51363   3852225    1
4.    HASH JOIN on           7134   51363  15871167    1
5.     TABLE ACCESS FULL
         on STOCK               2     118     27612    1
5.     MERGE JOIN on         2587   51363   2003157    2
6.      TABLE ACCESS BY
           INDEX ROWID on     826   51363    821808    1
7.       INDEX FULL SCAN
           on XFK_TL_STO       26  1027251             1
6.      SORT JOIN on         1761    52173  1199979    2
7.      TABLE ACCESS FULL
          on STOCKMOV        1032    52173  1199979    1
4.    TABLE ACCESS BY INDEX
        ROWID on T            390    21292  4366512    2
5.     INDEX UNIQUE SCAN
         on XPK_TRANSA                   2             1
3.   HASH JOIN on            7134    51363  15871167   2
4.    TABLE ACCESS FULL
        on STOCK                2      118     27612   1
4.    MERGE JOIN on          6885    51363   3852225   2
5.     SORT JOIN on          3646    51363   2003157   1
6.      MERGE JOIN on        2587    51363   2003157   1
7.       SORT JOIN on         826    51363    821808   1
8.      TABLE ACCESS FULL
           on TRANSAC         826    51363    821808   1
7.     SORT JOIN on          1761    52173   1199979   2
8.      TABLE ACCESS FULL
          on STOCKMO         1032    52173   1199979   1
5.    SORT JOIN on           3239   121292   4366512   2
6.     TABLE ACCESS FULL
         on TRANSACTI         390   121292   4366512   1
3.  HASH JOIN on             7134    51363  15871167   3
4.   TABLE ACCESS FULL
       on STOCK                 2      118     27612   1
4.   MERGE JOIN on           6885    51363   3852225   2
5.    SORT JOIN on           3646    51363   2003157   1
6.    MERGE JOIN on          2587    51363   2003157   1
7.     SORT JOIN on           826    51363    821808   1
8.      TABLE ACCESS FULL
          on TRANSAC          826    51363    821808   1
7.    SORT JOIN on           1761    52173   1199979   2
8.     TABLE ACCESS FULL
         on STOCKMO          1032    52173   1199979   1
5.  SORT JOIN on             3239   121292   4366512   2
6.   TABLE ACCESS FULL
       on TRANSACTI           390   121292   4366512   1

These are the timing tests.

SQL> SELECT COUNT(*) FROM(
  2 SELECT * FROM transactions t
  3 JOIN transactionsline tl USING(transaction_id)
  4  JOIN stockmovement sm USING(stockmovement_id)
  5   JOIN stock USING(stock_id)
  6 WHERE t.type = 'S' OR tl.amount < 100 OR sm.qty = 0);

   COUNT(*)
-----------
    4989625

Elapsed: 00:22:08.08

SQL> SELECT COUNT(*) FROM(
  2 SELECT /*+ USE_CONCAT */ * FROM transactions t
  3 JOIN transactionsline tl USING(transaction_id)
  4 JOIN stockmovement sm USING(stockmovement_id)
  5 JOIN stock USING(stock_id)
  6 WHERE t.type = 'S' OR tl.amount < 100 OR sm.qty = 0);

   COUNT(*)
-----------
    4989625

Elapsed: 00:28:23.09

8.7.2 The IN List Operator

Concatenation is used to merge OR conditions on different columns. The IN List operator is used to merge conditions on the same column as shown in the following query.

EXPLAIN PLAN SET statement_id='TEST' FOR
      SELECT * FROM coa NATURAL JOIN generalledger
      WHERE coa.type = 'A' OR coa.type = 'L'
      OR coa.type = 'E' OR coa.type = 'I';

Query                        Cost   Rows     Bytes   Pos
--------------------------   ----   ------  -------  ---
1. SELECT STATEMENT on       1642  1068929  50239663   0
2.  HASH JOIN on             1642  1068929  50239663   1
3.   INLIST ITERATOR on                                1
4.    TABLE ACCESS BY INDEX
        ROWID on C              2       55      1320   1
5.     INDEX RANGE SCAN on
         XFK_COA_TYP            1       55             1
3.   TABLE ACCESS FULL on
       GENERALLEDGE          1128  1068929  24585367   2

The following query is a variation on the previous query, forcing concatenation by using a hint. It is much higher in cost.

EXPLAIN PLAN SET statement_id='TEST' FOR
      SELECT /*+ USE_CONCAT */ * FROM coa NATURAL JOIN
         generalledger
      WHERE coa.type = 'A' OR coa.type = 'L'
      OR coa.type = 'E' OR coa.type = 'I';

Each of four joins for COA.TYPE column values are concatenated together.

Query                        Cost   Rows     Bytes      Pos
--------------------------   ----   ------  ---------   ---
1. SELECT STATEMENT on       6568  4199364  #########     0
2.  CONCATENATION on                                      1
3.   HASH JOIN on            1642  1049841   49342527     1
4.    TABLE ACCESS BY INDEX
        ROWID on C              2       14        336     1
5.     INDEX RANGE SCAN on
         XFK_COA_TYP            1       14                1
4.   TABLE ACCESS FULL on
        GENERALLEDG          1128  1068929   24585367     2
3.    HASH JOIN on           1642  1049841   49342527     2
4.     TABLE ACCESS BY INDEX
         ROWID on C             2       14        336     1
5.      INDEX RANGE SCAN on
          XFK_COA_TYP           1       14                1
4.    TABLE ACCESS FULL on
        GENERALLEDG          1128  1068929   24585367     2
3.   HASH JOIN on            1642  1049841   49342527     3
4.    TABLE ACCESS BY INDEX
         ROWID on C             2       14        336     1
5.      INDEX RANGE SCAN on
          XFK_COA_TYP           1       14                1
4.     TABLE ACCESS FULL on
          GENERALLEDG        1128  1068929   24585367     2
3.    HASH JOIN on           1642  1049841   49342527     4
4.     TABLE ACCESS BY INDEX
          ROWID on C            2       14        336     1
5.       INDEX RANGE SCAN on
           XFK_COA_TYP          1       14                1
4.     TABLE ACCESS FULL on
         GENERALLEDG         1128  1068929   24585367     2

The previous query plan shows a concatenation of four separate joins. This leads us to the UNION set operator.

8.7.3 UNION, MINUS, and INTERSECT

The following UNION set operator concatenated query is the same query as the previous query, which used the USE_CONCAT hint. Concatenation is faster than using UNION.

EXPLAIN PLAN SET statement_id='TEST' FOR

   SELECT * FROM coa NATURAL JOIN generalledger WHERE
      coa.type = 'A'
   UNION
   SELECT * FROM coa NATURAL JOIN generalledger WHERE
      coa.type = 'L'
   UNION
   SELECT * FROM coa NATURAL JOIN generalledger WHERE
      coa.type = 'E'
   UNION
   SELECT * FROM coa NATURAL JOIN generalledger WHERE
      coa.type = 'I';

Note the enormously higher cost as a result of using UNION rather than concatenation.

Query                          Cost      Rows       Bytes   Pos
--------------------------   ------   -------   ---------   ---
1. SELECT STATEMENT on       171272   4199364   #########     0
2.  SORT UNIQUE on           171272   4199364   #########     1
3.   UNION-ALL on 1
4.    HASH JOIN on             1642   1049841    49342527     1
5.     TABLE ACCESS BY
         INDEX ROWID on           2        14         336     1
6.      INDEX RANGE SCAN
          on XFK_COA_TY           1        14                 1
5.     TABLE ACCESS FULL
         on GENERALLED         1128   1068929    24585367     2
4.   HASH JOIN on              1642   1049841    49342527     2
5.    TABLE ACCESS BY INDEX
         ROWID on                 2        14         336     1
6.     INDEX RANGE SCAN on
         XFK_COA_TY               1        14                 1
5.    TABLE ACCESS FULL on
        GENERALLED             1128   1068929    24585367     2
4.  HASH JOIN on               1642   1049841    49342527     3
5.   TABLE ACCESS BY INDEX
       ROWID on                   2        14         336     1
6.    INDEX RANGE SCAN on
        XFK_COA_TY                1        14                 1
5.   TABLE ACCESS FULL on
        GENERALLED             1128   1068929    24585367     2
4.  HASH JOIN on               1642   1049841    49342527     4
5.   TABLE ACCESS BY INDEX
       ROWID on                   2        14         336     1
6.    INDEX RANGE SCAN on
        XFK_COA_TY                1        14                 1
5.  TABLE ACCESS FULL on
      GENERALLED               1128   1068929    24585367     2

UNION is useful in a situation such as the following query where ranges of values are split and displayed, each with a different concatenated string. Using UNION generally creates very poorly performing SQL code.

SELECT 'OUT OF STOCK: '||text FROM stock WHERE qtyonhand <=0
UNION
SELECT 'UNDER STOCKED: ' ||text FROM stock WHERE qtyonhand
BETWEEN 1 AND min-1
UNION
SELECT 'STOCKED: ' ||text FROM stock WHERE qtyonhand
BETWEEN min AND max
UNION
SELECT 'OVER STOCKED: ' ||text FROM stock WHERE qtyonhand > max;
'OUTOFSTOCK:'||TEXT
-------------------
OUT OF STOCK: iRock MP3 Digital Audio Player by First
   International
OVER STOCKED: Compaq Presario 6016US Minitower
OVER STOCKED: Epson Perfection 2450 Photo Color Scanner
OVER STOCKED: Internet Design Shop XL by Boomerang Software
OVER STOCKED: Memorex 24X CD-R Media,700MB/80Min, Spindle,
   30 Pack
OVER STOCKED: Palm m505 Handheld
OVER STOCKED: Sonnet Technologies HARMONi G3 Processor
Upgrade/FireWire
STOCKED: Corex CardScan Executive (600c/V6) Business Card
   Scanner
STOCKED: Epson Perfection 1250 Color Flatbed Scanner
STOCKED: Epson Perfection 1650 Color Flatbed Scanner
STOCKED: FrontPage 2002 by Microsoft
STOCKED: Hewlett-Packard Photo Scanner 1000 Color Scanner
STOCKED: Imation 32x Neon CD-R 700 MB/80 Min 50-pk Spindle
STOCKED: Logitech Cordless Freedom Optical Keyboard/Mouse
STOCKED: Logitech Internet Navigator Keyboard
STOCKED: Memorex 4.7GB DVD+R Media, 3-pack
STOCKED: TDK DVD-R Media, 4.7GB, 10 Pack
UNDER STOCKED: Apple Studio 17-inch TFT LCD Monitor
UNDER STOCKED: Envision Peripherals EN-5100 15-inch LCD
   Monitor
UNDER STOCKED: Hewlett-Packard ScanJet 7400c Flatbed Scanner

The MINUS and INTERSECT operators behave similarly to that of UNION and also have very high cost. Avoid using UNION, UNION ALL, MINUS, and INTERSECT if possible. First let's show the MINUS operator.

EXPLAIN PLAN SET statement_id='TEST' FOR
      SELECT * FROM coa NATURAL JOIN generalledger
      MINUS
      SELECT * FROM coa NATURAL JOIN generalledger
      WHERE coa.type IN('E','I');

Query                            Cost      Rows       Bytes
----------------------------   ------   -------   ---------
1. SELECT STATEMENT on         143333   1577577   #########
2.  MINUS on
3.   SORT UNIQUE on             72290   1577577    74146119
4.    HASH JOIN on               2434   1577577    74146119
5.     TABLE ACCESS FULL
         on COA                     2        55        1320
5.      TABLE ACCESS FULL on
          GENERALLEDGER          1677   1577577    36284271
3.   SORT UNIQUE on             71043   1549406    72822082
4.    HASH JOIN on               2434   1549406    72822082
5.     INLIST ITERATOR on
6.      TABLE ACCESS BY INDEX
          ROWID on COA              2        28         672
7.       INDEX RANGE SCAN on
           XFK_COA_TYPE             1        28
5.     TABLE ACCESS FULL on
         GENERALLEDGER           1677   1577577    36284271

Now the INTERSECT operator.

EXPLAIN PLAN SET statement_id='TEST' FOR
      SELECT * FROM coa NATURAL JOIN generalledger WHERE
         coa.type = 'A'
      INTERSECT
      SELECT * FROM coa NATURAL JOIN generalledger WHERE
         coa.type = 'L';

Query                            Cost      Rows       Bytes
----------------------------   ------   -------   ---------
1. SELECT STATEMENT on         142086   1549406   #########
2.  INTERSECTION on
3.   SORT UNIQUE on             71043   1549406    72822082
4.    HASH JOIN on               2434   1549406    72822082
5.     TABLE ACCESS BY INDEX
         ROWID on COA               2        14         336
6.      INDEX RANGE SCAN on
          XFK_COA_TYPE              1        14
5.     TABLE ACCESS FULL on
         GENERALLEDGER           1677   1577577    36284271
3.   SORT UNIQUE on             71043   1549406    72822082
4.    HASH JOIN on               2434   1549406    72822082
5.     TABLE ACCESS BY INDEX
         ROWID on COA               2        14         336
6.      INDEX RANGE SCAN on
          XFK_COA_TYPE              1        14
5.     TABLE ACCESS FULL on
         GENERALLEDGER           1677   1577577    36284271


8.8 Overriding the Optimizer using Hints

As we have seen in numerous places in this book so far, a hint can be used to suggest an alteration to the way that the Optimizer creates a query plan for a query. OLTP databases rarely require use of hints if statistics can be regularly maintained. Hints are more commonly used in large data warehouse or reporting databases, rule-based databases or those lacking current or any statistics. Generating statistics can be time consuming. Using the DBMS_STATS package instead of the ANALYZE command may help to alleviate some potential problems. Hints can be used to control the Optimizer and to a certain extent freeze execution plans much like outlines were used in older versions of Oracle Database. Not all available Optimizer hints will be covered in this book, only those hints which are most useful.

The syntax of a hint is such that it is placed after a DML command between comments as shown, including the plus (+) sign.

SELECT /*+ RULE */ * FROM customer;
INSERT /*+ RULE */ INTO customer(customer_id, name, ticker)
      VALUES(1000000, 'New Customer', 'TICK');
UPDATE /*+ RULE */ customer SET name = 'A New Customer'
      WHERE customer_id = 1000000;
DELETE /*+ RULE */ FROM customer WHERE customer_id = 1000000;
Note

Oracle Database 10 Grid The RULE hint is desupported and scheduled for deprecation in a future release of Oracle Database.

If an alias is used in an SQL statement then any hints must refer to the alias and not the table. The first example following does not use any indexes on the customer table due to the NO_INDEX hint. The second does use an index and the hint suggested it not do as such. The hint in the second query is incorrectly specified since it uses the table name and not the alias for the Customer table, "c".

EXPLAIN PLAN SET statement_id='TEST' FOR
      SELECT /*+ NO_INDEX(c) */ customer_id FROM customer c
            NATURAL JOIN transactions;

Query                               Cost      Rows       Bytes
-------------------------------   ------   -------   ---------
1. SELECT STATEMENT on               126     64599      322995
2.  HASH JOIN on                     126     64599      322995
3.   TABLE ACCESS FULL on CUSTOMER    16      2694        8082
3.   INDEX FAST FULL SCAN on
       XFK_T_CUSTOM 44 64599 129198

EXPLAIN PLAN SET statement_id='TEST' FOR
      SELECT /*+ NO_INDEX(customer) */ customer_id FROM
         customer c
            NATURAL JOIN transactions;

Query                            Cost      Rows       Bytes
----------------------------   ------   -------   ---------
1. SELECT STATEMENT on             44     64599      322995
2.  NESTED LOOPS on                44     64599      322995
3.   INDEX FAST FULL SCAN on
       XFK_T_CUSTOM                44     64599      129198
3.   INDEX UNIQUE SCAN on
       XPK_CUSTOMER                           1           3

It is always a good idea to check query plans for SQL statements if only to insure that hints are coded in a syntactically correct manner. Incorrectly coded hints do not produce SQL code parser errors and may never be detected. This SQL statement will execute.

SELECT /*+ STUPIDHINT */ * FROM customer;

Here is a query plan using an index fast full scan.

EXPLAIN PLAN SET statement_id='TEST' FOR
      SELECT customer_id FROM customer;

Query                            Cost      Rows       Bytes
----------------------------   ------   -------   ---------
1. SELECT STATEMENT on              3      2694        8082
2.  INDEX FAST FULL SCAN on
      XPK_CUSTOMER                  3      2694        8082

Applying the RULE hint suggests that the Optimizer ignore statistics and use rule-based optimization.

EXPLAIN PLAN SET statement_id='TEST' FOR
      SELECT /*+ RULE */ customer_id FROM customer;
Note

Oracle Database 10 Grid The RULE hint is desupported and scheduled for deprecation in a future release of Oracle Database.

Note that cost, row, and byte figures are not shown using rule- based optimization because there are no statistics to work with. A full table scan is assumed to be the fastest option.

Query                                  Cost      Rows       Bytes
----------------------------------   ------   -------   ---------
1. SELECT STATEMENT on
2.  TABLE ACCESS FULL on CUSTOMER

We can influence the rule-based Optimizer to use any index on the Customer table.

EXPLAIN PLAN SET statement_id='TEST' FOR
      SELECT /*+ RULE INDEX(customer) */ customer_id FROM
         customer;

Once again we get the index fast full scan except that the cost is a little higher to override the rule-based Optimizer.

Query                                  Cost      Rows       Bytes
----------------------------------   ------   -------   ---------
1. SELECT STATEMENT on                    7      2694        8082
2.  INDEX FULL SCAN on XPK_CUSTOMER       7      2694        8082

8.8.1 Classifying Hints

There are all sorts of hints for persuading the Optimizer to do things differently. I like to categorize hints as follows.

  • Influence the Optimizer.

  • Change table scans.

  • Change index scans.

  • Change joins.

  • Parallel SQL.

  • Change queries and subqueries.

  • Oracle Database 10 Grid Other hints.

This book will not cover all of the available hints in Oracle Database, only the interesting ones. Most importantly we will examine hints that are potentially useful within the scope of the subject matter presented in this book. Some hints have already been covered in this book, some even in prior chapters. Repetition is avoided where possible.

8.8.2 Influence the Optimizer

Available hints:

  • ALL_ROWS. Suppresses indexes and favors full table scans to find all rows.

  • FIRST_ROWS(n). More likely to use indexing since favoring a few rows.

  • CHOOSE. Choose cost-based optimization when statistics present; otherwise use rule-based optimization.

  • RULE. Rule-based optimization.

  • CURSOR_SHARING_EXACT. Overrides behavior of configuration parameter CURSOR_SHARING=FORCE which changes literal values to bind variables.

  • DYNAMIC_SAMPLING. "On the fly" sampling if statistics not present or out of date.

Setting OPTIMIZER_MODE=CHOOSE is the most common setting for optimization in an OLTP database. The CHOOSE option will favor cost-based optimization and use of statistics. However, CHOOSE will tend to favor full scans rather than more precise index scans.

Tip

The OPTIMIZER_MODE parameter can be altered at the session level using a command such as ALTER SESSION SET OPTIMIZER_MODE=RULE;
Note

Oracle Database 10 Grid The RULE and CHOOSE hints are desupported and scheduled for deprecation in a future release of Oracle Database. The default for the OPTIMIZER_MODE parameter is now ALL_ROWS.

Let's experiment a little. My database is set to CHOOSE. Note the difference in cost in the two following queries. The chances are the second query will be slower since it is reading all the rows from the table and the hint is stating otherwise. These two examples demonstrate the difference made by the hint.

EXPLAIN PLAN SET statement_id='TEST' FOR SELECT * FROM
   generalledger;

Query                            Cost      Rows       Bytes
----------------------------   ------   -------   ---------
1. SELECT STATEMENT on           1128   1068929    24585367
2.  TABLE ACCESS FULL on
      GENERALLEDGER              1128   1068929    24585367

EXPLAIN PLAN SET statement_id='TEST' FOR
      SELECT /*+ FIRST_ROWS(1) */ * FROM generalledger;

Using the hint the cost is misleading since the query will still return all rows in the GeneralLedger table, which is very large.

Query                            Cost      Rows       Bytes
----------------------------   ------   -------   ---------
1. SELECT STATEMENT on              2         1           2
2.  TABLE ACCESS FULL on
      GENERALLEDGER                 2         1           2

8.8.3 Change Table Scans

Available hints:

  • FULL. Suggest a full table scan. Typically the FULL hint could be used on small static tables since it might sometimes be faster to read the entire table rather than read both the index and the table.

Let's play. The first query uses an index range scan. The second query suggests a full table scan, has fewer steps and lowers the number of bytes read.

EXPLAIN PLAN SET statement_id='TEST' FOR
      SELECT * FROM category WHERE category_id < 50;

Query                            Cost      Rows       Bytes
----------------------------   ------   -------   ---------
1. SELECT STATEMENT on              2        13         143
2.  TABLE ACCESS BY INDEX ROWID
      on CATEGO                     2        13         143
3.   INDEX RANGE SCAN on
       XPK_CATEGORY 1 13

EXPLAIN PLAN SET statement_id='TEST' FOR
      SELECT /*+ FULL(category) */ category_id FROM
         category
      WHERE category_id < 50;

Query                            Cost      Rows       Bytes
----------------------------   ------   -------   ---------
1. SELECT STATEMENT on              2        13          26
2.  TABLE ACCESS FULL on CATEGORY   2        13          26

8.8.4 Change Index Scans

Available hints:

  • INDEX[_ASC | _DESC]. Forces use of an index where if multiple indexes are specified then the lowest-cost index is selected. The selected index is scanned as it is sorted, or as specified in ascending or descending order.

  • INDEX_FFS. Forces a fast full index scan, reading the index in physical block order not key order.

  • NO_INDEX. Ignores the named indexes.

  • INDEX_COMBINE. Typically used to combine use of single-column bitmap indexes.

  • INDEX_JOIN. Suggests a join between indexes alone.

  • AND_EQUAL. Join multiple index single-column scans.

    Note

    Oracle Database 10 Grid The AND_EQUAL hint is deprecated.

  • Oracle Database 10 Grid INDEX_SS[_ASC | DESC]. Suggests an index skip scan.

  • Oracle Database 10 Grid NO_INDEX_FFS and NO_INDEX_SS. These hints suggest the Optimizer does not use index fast full scans or index skip scans respectively.

  • Oracle Database 10 Grid USE_NL_WITH_INDEX. Suggests a query uses a specific table as the inner loop of a nested loops join, with the option of using a particular index.

These are the indexes on the PeriodSum table.

PERIODSUM    BTree    XFK_PS_COA          COA#       1
PERIODSUM    BTree    XFK_PS_YEARPERIOD   YEAR       1
PERIODSUM    BTree    XFK_PS_YEARPERIOD   PERIOD     2
PERIODSUM    BTree    XPK_PERIODSUM       YEAR       1
PERIODSUM    BTree    XPK_PERIODSUM       PERIOD     2
PERIODSUM    BTree    XPK_PERIODSUM       COA#       3

The query plan for this SQL statement uses a full table scan because all the columns in all the rows are being read.

EXPLAIN PLAN SET statement_id= 'TEST' FOR SELECT * FROM
      periodsum;

Query                            Cost      Rows       Bytes
----------------------------   ------   -------   ---------
1. SELECT STATEMENT on              2        45         675
2.  TABLE ACCESS FULL on PERIODSUM  2        45         675

The next query suggests use of the most efficient index available. The primary key index is unique and thus the most efficient. There is no change in cost. The Optimizer was correct to choose the full table scan by default since reading index and table is probably slightly slower than reading just the table.

EXPLAIN PLAN SET statement_id= 'TEST' FOR
      SELECT /*+ INDEX(periodsum) */ * FROM periodsum;

Query                            Cost      Rows       Bytes
----------------------------   ------   -------   ---------
1. SELECT STATEMENT on              2        45         675
2.  TABLE ACCESS BY INDEX ROWID
      on PERIOD                     2        45         675
3.   INDEX FULL SCAN on
       XPK_PERIODSUM                1        45

This query removes the primary key index from the equation by suggesting that the Optimizer chooses between the more efficient of two others. The cost is slightly higher.

EXPLAIN PLAN SET statement_id= 'TEST' FOR
      SELECT /*+ INDEX(periodsum, xfk_ps_coa
         xfk_ps_yearperiod) */ *
      FROM periodsum;

Query                               Cost      Rows       Bytes
-------------------------------   ------   -------   ---------
1. SELECT STATEMENT on                 4        45         675
2.  TABLE ACCESS BY INDEX ROWID
      on PERIOD 4 45 675
3.   INDEX FULL SCAN on XFK_PS_COA     3        45

Now we suggest the use of the foreign key to the PeriodSum table and show it is higher in cost than using the foreign key to the COA table. Thus the Optimizer selected the best key as being the foreign key to the COA table in the previous query.

EXPLAIN PLAN SET statement_id= 'TEST' FOR
      SELECT /*+ INDEX(periodsum, xfk_ps_yearperiod) */
            * FROM periodsum;

Query                            Cost      Rows       Bytes
----------------------------   ------   -------   ---------
1. SELECT STATEMENT on              5        45         675
2.  TABLE ACCESS BY INDEX ROWID
      on PERIOD                     5        45         675
3.   INDEX FULL SCAN on
       XFK_PS_YEARPERIOD            4        45

We could also use the INDEX_DESC hint to scan an index in reverse order.

EXPLAIN PLAN SET statement_id= 'TEST' FOR
      SELECT /*+ INDEX_DESC(customer) */ customer_id FROM
         customer;

Query                            Cost      Rows       Bytes
----------------------------   ------   -------   ---------
1. SELECT STATEMENT on              7      2694        8082
2.  INDEX FULL SCAN DESCENDING on
      XPK_CUS                       7      2694        8082

The AND_EQUAL hint has not been covered as of yet. This first example uses the foreign key index on the ORDER_ID column.

Note

Oracle Database 10 Grid The AND_EQUAL hint is deprecated.

EXPLAIN PLAN SET statement_id= 'TEST' FOR
      SELECT * FROM transactions WHERE type = 'S' AND
         order_id = 10;

Query                               Cost      Rows       Bytes
-------------------------------   ------   -------   ---------
1. SELECT STATEMENT on                 2         1          36
2.  TABLE ACCESS BY INDEX ROWID
      on TRANSA                        2         1          36
3.   INDEX RANGE SCAN on XFK_T_ORDERS            1           1

The AND_EQUAL hint causes an alteration to the previous example by suggesting use of both the TYPE and ORDER_ID foreign key indexes. It does not help the cost but this is what the hint does.

Tip

The cost-based Optimizer is very intelligent and sophisticated in Oracle9i Database. Do not ever assume that by using hints your SQL code will be faster. ALWAYS check the query plan with the EXPLAIN PLAN command when using hints.

EXPLAIN PLAN SET statement_id='TEST' FOR
      SELECT /*+ AND_EQUAL(transactions xfk_t_type
         xfk_t_orders) */ *
      FROM transactions WHERE type = 'S' AND order_id = 10;

Query                                 Cost      Rows       Bytes
---------------------------------   ------   -------   ---------
1. SELECT STATEMENT on                 258         1          36
2.  TABLE ACCESS BY INDEX ROWID
      on TRANSA                        258         1          36
3.   AND-EQUAL on
4.    INDEX RANGE SCAN on
        XFK_T_ORDERS
4.    INDEX RANGE SCAN on XFK_T_TYPE   256    121292
Note

Oracle Database 10 Grid Index hints include index names or table names with column lists. Table.COLUMN name settings can also be used even with columns in separate tables for join indexes.

8.8.5 Change Joins

Available hints:

  • ORDERED. Makes the Optimizer join tables in the sequence that tables appear in the FROM clause of an SQL statement, amongst other uses (refer to the section on DISTINCT on p. 276).

  • LEADING. Makes the Optimizer use a named table as the first table in the join, regardless of which table occurs first in the FROM clause.

  • USE_NL, USE_HASH and USE_MERGE. Suggest nested loops, hash join, or sort merge join, respectively.

  • Oracle Database 10 Grid NO_USE_NL, NO_USE_HASH, and NO_USE_MERGE. These hints suggest that the Optimizer does not use nested loops, hash joins, or sort merge joins respectively in a particular query.

  • NL_AJ, HASH_AJ, MERGE_AJ, NL_SJ, HASH_SJ, MERGE_SJ. All these hints change the join type for semi-and anti-join subqueries.

Oracle Database 10 Grid All of the nested loop, hash, and sort merge join anti-and semi-join hints are deprecated.

We have already seen the ORDERED hint in this chapter but I will reiterate with a different example because it is important. This first example does not use the ORDERED hint and accesses tables with the smaller of the COA and GeneralLedger tables in the outer loop.

EXPLAIN PLAN SET statement_id='TEST' FOR
      SELECT * FROM generalledger gl, coa WHERE coa.coa# =
         gl.coa#;

Query                            Cost      Rows       Bytes
----------------------------   ------   -------   ---------
1. SELECT STATEMENT on           1642   1068929    50239663
2.  HASH JOIN on                 1642   1068929    50239663
3.   TABLE ACCESS FULL on COA       2        55        1320
3.   TABLE ACCESS FULL on
       GENERALLEDGER             1128   1068929    24585367

This second example applies the ORDERED hint and changes the order in which the Optimizer accesses tables based on the sequence of tables in the FROM clause.

EXPLAIN PLAN SET statement_id='TEST' FOR
      SELECT /*+ ORDERED */ * FROM generalledger gl, coa
      WHERE coa.coa# = gl.coa#;

Query                            Cost      Rows       Bytes
----------------------------   ------   -------   ---------
1. SELECT STATEMENT on           4026   1068929    50239663
2.  HASH JOIN on                 4026   1068929    50239663
3.   TABLE ACCESS FULL on
       GENERALLEDGER             1128   1068929    24585367
3.   TABLE ACCESS FULL on COA       2        55        1320

In the next example not all tables are switched according to the FROM clause. The ORDERED hint manages to switch the Transactions and TransactionsLine tables. It does not change the sequence for the Customer table.

EXPLAIN PLAN SET statement_id='TEST' FOR
      SELECT /*+ ORDERED */ *
      FROM transactionsline tl, transactions t, customer c
      WHERE t.transaction_id = tl.transaction_id
      AND c.customer_id = t.customer_id;

Query                            Cost      Rows       Bytes
----------------------------   ------   -------   ---------
1. SELECT STATEMENT on          27327    273552    50880672
2.  HASH JOIN on                27327    273552    50880672
3.   TABLE ACCESS FULL on
       CUSTOMER                    16      2694      360996
3.   MERGE JOIN on              20905    273552    14224704
4.    SORT JOIN on              18997   1027251    16436016
5.     TABLE ACCESS FULL on
         TRANSACTIONSLINE         800   1027251    16436016
4.    SORT JOIN on               1908     64599     2325564
5.     TABLE ACCESS FULL on
         TRANSACTIONS             390     64599     2325564

The LEADING hint uses a named table as the first table in the join.

EXPLAIN PLAN SET statement_id='TEST' FOR
      SELECT /*+ LEADING(tl) */ *
      FROM transactionsline tl, transactions t, customer c
      WHERE t.transaction_id = tl.transaction_id
      AND c.customer_id = t.customer_id;

Query                            Cost      Rows       Bytes
----------------------------   ------   -------   ---------
1. SELECT STATEMENT on          26143    273552    50880672
2.  HASH JOIN on                26143    273552    50880672
3.   MERGE JOIN on              20905    273552    14224704
4.    SORT JOIN on              18997   1027251    16436016
5.     TABLE ACCESS FULL on
         TRANSACTIONSLINE         800   1027251    16436016
4.    SORT JOIN on               1908     64599     2325564
5.     TABLE ACCESS FULL on
         TRANSACTIONS             390     64599     2325564
3.  TABLE ACCESS FULL on
      CUSTOMER                     16      2694      360996

8.8.6 Parallel SQL

Available hints:

  • [NO]PARALLEL. Parallel execution on multiple CPUs or servers.

  • PQ_DISTRIBUTE. Improves a parallel join.

  • [NO]PARALLEL_INDEX. Process index scans for partitions in parallel.

    Note

    Oracle Database 10 Grid The NOPARALLEL and NOPARALLEL_INDEX hints are renamed to NO_PARALLEL and NO_PARALLEL_INDEX, respectively.

Previously in this chapter we examined performance using parallel settings on both full table scans and fast full index scans. Parallelism was detrimental to performance. Parallel SQL statements are only beneficial in very large data warehouse databases. Using parallelism with Oracle Partitioning and multiple disks will be examined in Part III.

8.8.7 Changing Queries and Subqueries

Available hints:

  • [NO]CACHE. CACHE can be used to suggest that data be forced into a Most Recently Used list (MRU). NOCACHE pushes data to a Least Recently Used list (LRU).

  • ORDERED_PREDICATES. Preserves SQL statement precedence of evaluation such as the sequence of comparisons in a WHERE clause.

  • [NO_]UNNEST. Undoes subquery layers by allowing the Optimizer to attempt to merge subqueries into a calling query. This is probably contrary to performance tuning of multi-layered SQL statements.

  • [NO_]PUSH_SUBQ. Resolves subqueries first or last (NO_).

8.8.8 Other Hints

Oracle Database 10 Grid The CPU_COSTING hint fills a column in the PLAN_TABLE called CPU_COST based on CPU cycles and I/O operations.

That is enough about hints. Remember one thing. A hint suggests rather than instructs the Optimizer. Therefore, the Optimizer may not necessarily take notice of the hint.

Tip

A hint is only a suggestion to the Optimizer, not an instruction.

This concludes this chapter of the Oracle Database internal specifics of constructing efficient SQL code. The next chapter will look at how to detect problems with SQL code in the database.


Chapter 9: How to Find Problem Queries

Overview

Problem queries are queries taking too long to run or hoarding too many resources, or both. Problem queries are common in ad hoc user SQL environments and sometimes a result of SQL coding by inexperienced applications developers. Many years ago databases contained small amounts of information and thus ad hoc environments were common. MS-Access is a clear example of an ad hoc single-user environment. In general, a multi-user database environment is not as susceptible to ad hoc SQL user queries since larger databases tend to be more tightly controlled and managed. It is common for inexperienced applications developers to submit inappropriately structured queries to the database. Database administrators have a responsibility to ensure that these queries are located and tuned and that the application developers concerned are advised as to how to build efficient queries. This chapter will discuss some of the various tools that can be used to find problem queries.

Let's look at available tools.


9.1 Tools to Detect Problems

This book is intended as a step-by-step learning guide teaching tuning of Oracle installations. You should now have an understanding of SQL code tuning from the most important perspective. What is that perspective? You should have an understanding of how to tune SQL code without having to use all sorts of fancy and complicated SQL code tuning tools. Why is this approach better? This approach is better because it should always be viewed as a last resort to fall back on detailed analysis with specific tools. It is better to know how to build properly tuned SQL code in the first place without having to take drastic measures, such as use of the EXPLAIN PLAN command and more significantly tracing. Tracing is highly complex, very time consuming, produces a significant performance drain on a database and generally provides far more information than is ever necessary!

There are various Oracle Corporation and non-Oracle Corporation tools which can be used to find and tune problematic and poorly performing SQL code. What are these tools?

  • EXPLAIN PLAN.

  • SQL Trace with TKPROF.

  • Oracle Database 10 Grid TRCSESS.

  • Autotracing in SQL*Plus.

  • Oracle Trace.

    Note

    Oracle Database 10 Grid Oracle Trace has been deprecated including all ORACLE_ TRACE_% configuration parameters.

  • Oracle Performance views.

  • Oracle Enterprise Manager and other non-Oracle tuning tools. These will be covered in later chapters depending on the type of tuning.

Now let's examine the use of some Oracle Database tuning tools in detail. Let's start with the EXPLAIN PLAN command.


9.2 EXPLAIN PLAN

We have seen extensive use of the EXPLAIN PLAN command in this book so far. Thus far it has not really been essential to describe the nitty-gritty of what precisely the EXPLAIN PLAN command does. From my perspective this type of information falls under the guise of reference material. I find that initially bombarding the reader with too much reference material can cause a lot of confusion and an even larger amount of boredom.

Once again so far in this book you have learned about data model tuning and SQL code tuning. SQL code tuning has been covered from the perspective of writing usable SQL code rather than an in-depth analysis of the tools used to find problems. Therefore, writing SQL code properly in the first place might help to avoid having to use in-depth and overbearing SQL code tuning tools. It is better by far to build properly tuned SQL code without having to immediately resort to masses of interpretive output such as that produced by trace files or STATSPACK.

9.2.1 What Does EXPLAIN PLAN Produce?

The Optimizer creates a query plan for every SQL statement, be it a DML command or a SELECT statement. DML commands are INSERT, UPDATE, DELETE, and MERGE commands. A query plan is selected by the Optimizer as a method of best performance for the execution of an SQL statement. The EXPLAIN PLAN command simply creates readable descriptions of the steps in the query plan and inserts those steps as entries into a table called the PLAN_TABLE.

An SQL statement query plan contains a number of general items. These items describe how an SQL statement should be executed.

  • The sequence in which tables are accessed.

  • Probably most importantly an estimated cost of the query plan and a cost for each step.

  • Less importantly details such as I/O estimates, temporary sort space usage, rows accessed and bytes read, amongst other details.

  • A method of access for each table.

  • Join methods for any joins.

  • Filtering and sorting operations.

  • Other specialized factors such as parallel processing or concatenation.

Note that these operations are not necessarily executed on a table itself but rather what is effectively a row set. A row set is either a filtered or an indexed subset of a table or a resulting set of rows produced by a join. A join is a join between two tables, a table and a row set or even two row sets.

Three things are important to note. Firstly, the Optimizer is not always right. Secondly, the query plan can be changed or at least influenced using hints. Thirdly, the EXPLAIN PLAN command makes a best guess of how the Optimizer should execute a query. This best guess is not the actual query execution plan. Actual query execution plans can be found in the V$SQL_PLAN performance view.

9.2.2 What to Look for in Query Plans

What should one be searching for in query plans with respect to creating high-performance SQL code statements? Bad things! Well what are "bad things"? This is colloquial but it is descriptive. We need to search for anything potentially problematic, slowing down the execution of an SQL statement. We have already seen a lot of potential problems in previous chapters. What are some common "red flags" to watch out for?

  • Full table scans and poor use of indexing, or lack of use of indexing.

    • Inappropriate use of indexing where full table scans can sometimes be faster.

    • Over-indexing on individual tables or too many indexes in a database in general.

  • Large selections from large tables sometimes caused by filtering too late.

  • Unnecessary sorting.

  • Joins.

    • Sort merge joins where nested loops or hash joins can be used.

    • Cartesian joins, outer joins, and anti-joins.

    • Highly complex mutable (multiple table) joins.

    • Noncorrelated subquery semi-joins.

  • Using views, especially multiple layers of views calling subviews.

  • Inherently poorly performing steps like UNION and concatenation operations.

9.2.3 Problems Producing Query Plans

Cost-based optimization is far more efficient than rule-based optimization. Statistics must exist for cost-based optimization to be used. If statistics are out of date then statistics may be inconsistent with data in the database. This can lead to incorrect assessment of the best-performing query plan for SQL code by the Optimizer.

Note

Oracle Database 10 Grid Rule-based optimization is desupported.

Statistics can be copied between different databases using the export (exp) and import (imp) utilities or even using the DBMS_STATS package. There are a number of things of paramount importance to tuning of SQL code with respect to statistics.

  • The existence of statistics in all tables and indexes.

  • Realistic statistics perhaps matching production database environments. Development, testing, tuning, and production databases can often be vastly different physically. Therefore, statistics may vary wildly.

  • Different databases can be placed on different hardware platforms where those different hardware platforms have differing configurations. Those differing configurations can affect how the Oracle Optimizer creates query plans. Different configurations can be differences in numbers of CPUs, storage media such as RAID arrays or not, and perhaps most importantly different Oracle installation configuration parameters. For instance, differences between the SORT_AREA_SIZE and HASH_AREA_SIZE parameters on different database servers can affect Optimizer choices between nested loops and hash joins.

9.2.4 EXPLAIN PLAN Command Syntax

EXPLAIN PLAN
      [ SET STATEMENT_ID = 'string' ]
      [ INTO [schema.]table[@dblink] ]
      FOR sql_statement;

This simplest version of the EXPLAIN PLAN command generates entries directly into the PLAN_TABLE.

EXPLAIN PLAN FOR SELECT * FROM generalledger;

Using this query we can examine the PLAN_TABLE entries.

COL Cost FORMAT 9990;
COL Rows FORMAT 999990;
COL Bytes FORMAT 99999990;
COL Query FORMAT a40;
SELECT operation||' '||options||' on '||object_name "Query"
      ,cost "Cost"
      ,cardinality "Rows"
      ,bytes "Bytes"
FROM plan_table ORDER BY id;

This is the result of the query.

Query                                Cost      Rows       Bytes
--------------------------------   ------   -------   ---------
SELECT STATEMENT on                   778    752741    17313043
TABLE ACCESS FULL on GENERALLEDGER    778    752741    17313043

The EXPLAIN PLAN command does not delete any previous entries from the PLAN_TABLE. Thus execution of the previous EXPLAIN PLAN command will add the same two rows again. Thus deletion of the previously added rows is essential to clarity. In this case, we can simply issue the DELETE or TRUNCATE commands to clear the PLAN_TABLE completely. In a multi-user development database this would be a problem. Multiple coders could be generating PLAN_TABLE entries. It would probably irritate other developers to continuously be removing all entries from the PLAN_TABLE. We can use specific identifying values to remove only the PLAN_TABLE rows generated. The SET_STATEMENT_ID = 'string' option can be used to generate session-specific PLAN_TABLE entries and isolate specific output, assuming no other coders are using the same string value.

EXPLAIN PLAN SET STATEMENT_ID = 'TEST'
      FOR SELECT * FROM generalledger;

Change the PLAN_TABLE query accordingly.

COL Cost FORMAT 9990;
COL Rows FORMAT 999990;
COL Bytes FORMAT 99999990;
COL Query FORMAT a40;
SELECT operation||' '||options||' on '||object_name "Query"
      ,cost "Cost"
      ,cardinality "Rows"
      ,bytes "Bytes"
FROM plan_table WHERE statement_id = 'TEST' ORDER BY id;

The optional INTO table clause can be used to place PLAN_TABLE output into a table other than the PLAN_TABLE.

The FOR clause part of the EXPLAIN PLAN command is passed an SQL statement. That SQL statement can be any SQL statement type for which the Optimizer generates a query plan.

  • SELECT.

  • INSERT, UPDATE, DELETE, MERGE.

  • CREATE TABLE AS SELECT ¼

  • CREATE INDEX.

  • ALTER INDEX ¼ REBUILD ¼

Following are some examples. At this stage we will use a more sophisticated version of the PLAN_TABLE query already used in previous chapters (see Appendix B).

EXPLAIN PLAN SET STATEMENT_ID = 'TEST' FOR
      INSERT INTO coa(coa#, type, subtype, text) VALUES(
         '60007', 'I', 'S', 'Consulting Fees');

Query                            Cost      Rows       Bytes
----------------------------   ------   -------   ---------
1. INSERT STATEMENT on              2        55        1320

EXPLAIN PLAN SET STATEMENT_ID = 'TEST' FOR
      UPDATE coa SET text = 'Consulting Fees' WHERE coa# =
         '60007';

Query                            Cost      Rows       Bytes
----------------------------   ------   -------   ---------
1. UPDATE STATEMENT on              1         1          19
2.  UPDATE on COA
3.   INDEX UNIQUE SCAN on XPK_COA             1          19

EXPLAIN PLAN SET STATEMENT_ID = 'TEST' FOR
      DELETE FROM coa WHERE coa# = '60007';

Query                               Cost      Rows       Bytes
-------------------------------   ------   -------   ---------
1. DELETE STATEMENT on                 1         1           7
2.  DELETE on COA
3.   TABLE ACCESS BY INDEX ROWID
       on COA                          1         1           7
4.     INDEX UNIQUE SCAN on XPK_COA             55

CREATE TABLE tmp AS
      SELECT coa#, type, subtype, text FROM coa WHERE
         ROWNUM < 1;
INSERT INTO tmp VALUES( '60008', 'I', 'S', 'Other Income');
INSERT INTO tmp VALUES( '60001', 'I', 'S', 'Primary
   Sales');
EXPLAIN PLAN SET STATEMENT_ID = 'TEST' FOR
      MERGE INTO coa USING tmp ON(coa.coa# = tmp.coa#)
      WHEN MATCHED THEN UPDATE SET coa.text = tmp.text
      WHEN NOT MATCHED THEN
            INSERT VALUES(tmp.coa#, tmp.type, tmp.subtype,
               tmp.text,
                     NULL, NULL, NULL);

Query                            Cost      Rows       Bytes
----------------------------   ------   -------   ---------
1. MERGE STATEMENT on            9189    369820    26996860
2.  MERGE on COA
3.   VIEW on
4.    HASH JOIN OUTER on            5        82        4920
5.     TABLE ACCESS FULL on TMP     2        82        2542
5.     TABLE ACCESS FULL on COA     2        55        1595

EXPLAIN PLAN SET STATEMENT_ID = 'TEST' FOR
      CREATE TABLE tmp AS SELECT * FROM generalledger;

Query                            Cost      Rows       Bytes
----------------------------   ------   -------   ---------
1. CREATE TABLE STATEMENT on      778    752741    17313043
2.  LOAD AS SELECT on
3.   TABLE ACCESS FULL on         778    752741    17313043
       GENERALLEDGER

EXPLAIN PLAN SET STATEMENT_ID = 'TEST' FOR
      CREATE INDEX xak_gl_coa#_dte ON generalledger
         (coa#, dte);

Query                            Cost      Rows       Bytes
----------------------------   ------   -------   ---------
1. CREATE INDEX STATEMENT on      778    752741     9032892
2.  INDEX BUILD NON UNIQUE on
      XAK_GL_COA#_DT
3.   SORT CREATE INDEX on                752741     9032892
4.    TABLE ACCESS FULL on
        GENERALLEDGER             778    752741     9032892

EXPLAIN PLAN SET STATEMENT_ID = 'TEST' FOR
      ALTER INDEX xak_gl_coa#_dte REBUILD ONLINE;

Query                            Cost      Rows       Bytes
----------------------------   ------   -------   ---------
1. ALTER INDEX STATEMENT on       101    752741     9032892
2.  INDEX BUILD NON UNIQUE on
      XAK_GL_COA#_DT
3.   SORT CREATE INDEX on                752741     9032892
4.    TABLE ACCESS FULL on
        GENERALLEDGER             101    752741     9032892

9.2.5 How to Create the PLAN_TABLE

Use the script called UTLXPLAN.SQL to create the PLAN_TABLE. This script is located on the database server in the $ORACLE_HOME/rdbms/admin directory on a Unix box and in the ORACLE_HOME\rdbms\admin directory on a Windows machine. The UTLXPLAN.SQL script contains a simple table creation command.

Also in that directory are two other scripts called UTLXPLS.SQL and UTLXPLP.SQL. The latter version involves a reference to a column called OTHER_TAG containing details of parallel execution. Thus use UTLXPLS.SQL for serial execution and UTLXPLP.SQL for parallel execution. These two scripts can be used to display PLAN_TABLE query plans using the DBMS_XPLAN package. I prefer to use a hierarchical query as seen in Appendix B.

Note

Oracle Database 10 Grid Two new columns in the PLAN_TABLE are ACCESS_ PREDICATES and FILTER_PREDICATES.

9.2.6 What is Not Provided in Query Plans?

A query plan will not necessarily tell you everything about an SQL code statement. There are things not included in query plans potentially drastically affecting performance. It is always best to attempt to tune SQL code from the perspective of the SQL code itself and use query plans for verification and fine-tuning. In the extreme, tracing can be used. Timing tests can be used to assess the speed of execution of SQL statements; however, be careful of executing large SQL code statements and killing database performance, particularly on a production database. Small portions of tables can be used for testing and counts using the COUNT function as a wrapper function to minimize performance impact.

SQL> SELECT COUNT(*) FROM(SELECT * FROM generalledger);

   COUNT(*)
-----------
     752741
Elapsed: 00:00:00.03

Using the COUNT function as a wrapper is a simple and effective timing mechanism. It avoids having to include the display of data into a tool such as SQL*Plus. Displaying thousands or even millions of rows in SQL*Plus will take forever to display and could slow down your database and flood your network. Note that every SQL statement tested in this manner should probably be executed at least twice. This allows loading into buffer caches perhaps accounting for differences in retrieval from buffers and disk. Note that a method of timing such as this may be ineffective running against a highly active concurrent database, especially if concurrent activity fluctuates within the time taken to execute such timing SQL code statements.

So query plans do not tell you everything. Tracing and TKPROF is the next step from using EXPLAIN PLAN. SQL Trace and TKPROF can also be used to determine SQL statement performance on a deeper level by allowing analysis of resource consumption.


9.3 SQL Trace and TKPROF

SQL Trace causes trace files to be produced by Oracle Database. Various items of interest are generated into trace files. This information comprises details of CPU and I/O needs plus parsing and execution information, processed rows, commits, and rollbacks. The output in trace files is more or less impossible to interpret by hand. TKPROF is a formatting utility which is used to format trace file output into easily readable output. TKPROF can also be used to produce EXPLAIN PLAN query plans and store past output from statistics collections in trace files, into a file called TKPROF_TABLE.

9.3.1 Setting up SQL Trace

SQL Trace can be configured for instance-level use or for a session, for a session either within a current open session or from one session to another. Setting instance-wide tracing can cause serious performance problems. It is better for performance to enable tracing only from a current session or for a specific session. Not using instance-wide tracing can miss a lot of useful information. I am using instance-level tracing since I am not working on a production server database. Start by setting the TIMED_STATISTICS, TIMED_OS_STATISTICS, and MAX_DUMP_FILE_SIZE parameters.

TIMED_STATISTICS = TRUE
TIMED_OS_STATISTICS = 5
MAX_DUMP_FILE_SIZE = 1M

The effect on performance of setting the TIMED_STATISTICS parameter is negligible. The parameters previously shown enable collection of statistics including CPU and I/O statistics. TIMED_STATISTICS writes information to both trace files plus the V$SYSSTATS and V$SESSSTATS performance views. TIMED_OS_STATISTICS is a timing mechanism for statistics collection, triggering system statistics collection periodically.

The MAX_DUMP_FILE_SIZE parameter will truncate each trace file in the USER_DUMP_DEST directory, the default target directory for trace files, usually the $ORACLE_BASE/admin/$ORACLE_SID/ udump directory.

Next set the SQL_TRACE parameter to switch on SQL Trace. The TRACE_ENABLED parameter could also be set to TRUE but we will avoid that. Setting TRACE_ENABLED tends to produce copious amounts of detail, more relevant to Oracle support personnel than anything else.

SQL_TRACE = TRUE

Trace files contain generated statistics. In past versions of Oracle trace levels could be set. Oracle9i Database has a new parameter called STATISTICS_LEVEL which can be set to BASIC, TYPICAL, or ALL. The default is TYPICAL. Setting ALL would incur significant overhead and a noticeable effect on performance. In my database I will set the STATISTICS_LEVEL parameter to ALL to gather as much information as possible. Note that setting this parameter to ALL automatically sets the TIMED_OS_STATISTICS parameter to 5.

STATISTICS_LEVEL = ALL

Session-Level Tracing

Specific sessions can be traced. Tracing at the session level can help to remove tracing overhead from the database in general and focus on a specific session. Personally I have never seen the value in using the session level of tracing. There is far too much information produced. There are easier ways of tracking down problems than using session-level tracing. This command traces the current session. Tracing a specific session may be most applicable to tracking down some really nasty SQL code such as in a pre-release application, or one which should not have been released, or perhaps in an ad hoc SQL environment.

ALTER SESSION SET SQL_TRACE = TRUE;

This command will trace a session other than the current session, someone else's session.

DBMS_SYSTEM_SET_SQL_TRACE_IN SESSION(SID, SERIAL#, TRUE);

Finding Trace Files

One of the biggest problems with trace files is finding them. Many files can be generated. You could find the files applicable to specific processes by including tags in your SQL code such as SELECT 'my trace file' FROM DUAL; or by checking datestamps in the operating system. Typically a trace file is named in the following format:

$ORACLE_SID_ora_<pid>.trc

"pid" represents the process ID of the process which generated the trace file. The process ID can be found with the following query, which finds the process identifier using the process and session addresses and the session auditing identifier as the session identifier.

SELECT spid FROM v$process
WHERE addr = (SELECT paddr FROM v$session
      WHERE audsid = USERENV('sessionid'));

9.3.2 Using SQL Trace

So the next logical step is to make use of SQL Trace. I have now set up my Oracle Database configuration parameter file (init.ora) with various parameter changes.

TIMED_STATISTICS = TRUE
TIMED_OS_STATISTICS=5
MAX_DUMP_FILE_SIZE = 1M
SQL_TRACE = TRUE
STATISTICS_LEVEL = ALL

Now I bounce my database to instantiate these configuration parameter changes.

Tip

Some of these parameters can be updated online using the ALTER SYSTEM command.

The next step is starting up my highly active concurrent database scripting (see Appendix B). These scripts use the DBMS_JOBS package to repetitively execute a slew of database changes to my Accounts schema, from multiple concurrently executed database jobs using the DBMS_JOBS package. Since I am running sessions using the DBMS_JOBS package I have to be able to find trace files for each session. So let's change the process identifier query to access multiple sessions, and find a trace file that way. So how do I find the appropriate sessions? Log into a database administrator user such as SYSTEM and execute a query something like this.

SELECT audsid, username, saddr, paddr, sid, serial#
FROM v$session WHERE username = 'ACCOUNTS' AND audsid != 0;

Then I change my process identifier finding query and find a filename.

SELECT 'c:\oracle\admin\test\udump\test_ora_'||spid||'.trc'
      "Trace File"
FROM v$process
WHERE addr = (
      SELECT paddr FROM V$SESSION
      WHERE username = 'ACCOUNTS' AND audsid != 0);
Trace File
---------------------------------------------------
c:\oracle\admin\test\udump\test_ora_972.trc

Trace files can look like the gibberish shown below. Not really very easy to interpret is it? Yes that is sarcastic; certainly the very lowest form of wit but apt in this case. This is where TKPROF comes into play.

======================
PARSING IN CURSOR #3 len=56 dep=1 uid=0 oct=3 lid=0
   tim=13822711723 hv=4049165760 ad='67f67898'
select order#,columns,types from access$ where d_obj#=:1
END OF STMT
PARSE #3:c=0,e=145,p=0,cr=0,cu=0,mis=0,r=0,dep=1,og=4,
   tim=13822711703
EXEC #3:c=0,e=136,p=0,cr=0,cu=0,mis=0,r=0,dep=1,og=4,
   tim=13822714183
FETCH #3:c=0,e=114,p=0,cr=3,cu=0,mis=0,r=1,dep=1,og=4,
   tim=13822714826
FETCH #3:c=0,e=78,p=0,cr=2,cu=0,mis=0,r=1,dep=1,og=4,
   tim=13822715459
FETCH #3:c=0,e=67,p=0,cr=2,cu=0,mis=0,r=1,dep=1,og=4,
   tim=13822716065
FETCH #3:c=0,e=64,p=0,cr=2,cu=0,mis=0,r=1,dep=1,og=4,
   tim=13822716662
FETCH #3:c=0,e=65,p=0,cr=2,cu=0,mis=0,r=1,dep=1,og=4,
   tim=13822717260
FETCH #3:c=0,e=42,p=0,cr=1,cu=0,mis=0,r=0,dep=1,og=4,
   tim=13822717850
STAT #3 id=1 cnt=5 pid=0 pos=1 obj=97 op='TABLE ACCESS BY
   INDEX ROWID ACCESS$ '
STAT #3 id=2 cnt=5 pid=1 pos=1 obj=129 op='INDEX RANGE
   SCAN I_ACCESS1 '

9.3.3 TKPROF

TKPROF is essentially a text-formatting or text-parsing utility used to massage trace files into a readable format.

Syntax of TKPROF

TKPROF is available on the database server in the $ORACLE_HOME\rdbms\admin directory. To examine TKPROF utility syntax type TKPROF or TKPROF HELP=Y at the command line and hit the return key, as with many other Oracle Database utilities. TKPROF accepts an input trace file and outputs a formatted file.

Execution plans can also be generated using TKPROF from trace files for all SQL code in the trace input file. Additionally TKPROF can generate a historical record of both all recursive and nonrecursive SQL executed by a session. This is the TKPROF command syntax.

TKPROF tracefile formatted
      [ SORT = { option | ( option list ) } ]
      [ PRINT = n ]
      [ AGGREGATE = { YES | NO } ]
      [ INSERT = scriptfile ]
      [ SYS = { YES | NO } ]
      [ [ TABLE = [schema.]table ] EXPLAIN = username/
         password ]
      [ RECORD = scriptfile ]

  • SORT. Sorts by specified options, there are a multitude of them.

  • PRINT. Sorts the first n SQL statements in the trace file.

  • AGGREGATE. Groups multiple users using the same SQL text.

  • INSERT. Writes out an SQL script to generate statistics into the database.

  • TABLE. The name of the table to which query plans are written.

  • EXPLAIN. Creates SQL statement query plans.

  • RECORD. Writes out an SQL script usable to replay all nonre-cursive SQL events.

  • SYS. SYS user SQL statements and recursive SQL. A recursive SQL statement occurs when, for instance, an SQL statement requires a new extent and an extent must be created prior to completion of the SQL statement.

Using TKPROF

Now I am simply executing TKPROF on the trace file I found earlier using all useful options. On Windows a script like this could go into a batch called RTK.BAT for instance.

TKPROF c:\oracle\admin\test\udump\test_ora_%1.trc
  c:\tmp\%1.out
 INSERT = c:\tmp\%1.ins
 TABLE = PLAN_TABLE EXPLAIN = accounts/accounts
 RECORD = c:\tmp\%1.rec
 SYS = YES

The batch file would be executed like this.

RTK 972

This produces files called 972.OUT, 972.INS, and 972.REC. The .OUT file contains the formatted output, the .INS file contains a statistics generation script and the .REC file contains a record of nonrecursive SQL commands executed.

Interpretation of TKPROF Output

Following are sections of sample TKPROF output. Some comment is required. The first section of the TKPROF output shown is an SQL code SELECT statement contained in the trace file.

************************************************************

SELECT a.supplier_id, a.price from (
select supplier_id, min(price) as price
from stocksource
where stock_id = :b1
group by supplier_id
order by price
) a where rownum = 1

The next section from the TKPROF output shows statistics generated by the execution of this SELECT statement expressed in three separate steps.

call     count    cpu  elapsed  disk  query  current  rows
-------  -----  -----  -------  ----  -----  -------  ----
Parse        1   0.07     0.08     0     56        0     0
Execute     10   0.02     0.01     0      0        0     0
Fetch       10   0.10     0.17    36    256        0    10
-------  -----  -----  -------  ----  -----  -------  ----
total       21   0.19     0.27    36    312        0    10

Let's examine the three separate steps.

  • Parse. Checks validity of syntax, objects and security, and generation of the query plan by the Optimizer.

  • Execute. Step for execution of SQL code. DML changes rows and SELECT builds appropriate cursors in memory.

  • Fetch. A cursor created by a SELECT statement is used to fetch the rows.

Repeating the second part of the TKPROF output for clarity let's follow it by examining the separate columns.

call     count    cpu  elapsed  disk  query  current  rows
-------  -----  -----  -------  ----  -----  -------  ----
Parse        1   0.07     0.08     0     56        0     0
Execute     10   0.02     0.01     0      0        0     0
Fetch       10   0.10     0.17    36    256        0    10
-------  -----  -----  -------  ----  -----  -------  ----
total       21   0.19     0.27    36    312        0    10

  • Count. Parse, execute, and fetch calls or executions of each type.

  • CPU. Processor time.

  • Elapsed. Total time elapsed.

  • Disk. Physical disk reads.

  • Query. Read-only query block retrievals.

  • Current. Read-write block retrievals or changes.

  • Rows. Rows processed.

The importance of the preceding TKPROF output is the ratios between the columns. What are these ratios and why are they significant?

  • Physical Disk Reads vs. Logical Reads. This is disk/(query+ current). The lower this ratio is the better, but not always. An extremely low ratio could indicate poorly tuned SQL code, an extremely high ratio could be normal depending on the application type. There is current thought in the Oracle community that logical reads in the extreme are not as efficient as once thought. Use of huge amounts of RAM and loading entire databases into the database buffer cache is often nowhere near as efficient as we are led to believe. On the contrary, LRU and MRU lists are significant because, for instance, reading a large table can push everything useful out of memory. More in Part III.

  • Block Reads vs. Rows. Block reads are often stated as being logical reads only: (query + current). I like to lump together the logical and the physical block reads: (query + current + disk) because they are all reads. The more blocks required to be read for each row the worse performance could possibly be.

  • Parse vs. Execute. We have a ratio of 1/10, which is good. If there were more parses than executions it would imply that SQL code is being re-parsed unnecessarily and the shared pool is not being shared enough, for a multitude of reasons but most likely due to poorly tuned SQL code.

The rest of the output shows a general analysis plus a query plan.

Misses in library cache during parse: 1
Optimizer goal: CHOOSE
Parsing user id: 88 (ACCOUNTS) (recursive depth: 1)

Rows  Execution Plan
----  ----------------------------------------------------
   0  SELECT STATEMENT GOAL: CHOOSE
   0   COUNT (STOPKEY)
   0    VIEW
   0     SORT (ORDER BY STOPKEY)
   0      SORT (GROUP BY)
   0       TABLE ACCESS GOAL: ANALYZED (BY INDEX ROWID) OF
               'STOCKSOURCE'
   0        INDEX (RANGE SCAN) OF 'XFK_SS_STOCK'
               (NON-UNIQUE)

Personally I look at using tracing of any form and TKPROF as a very last resort. The amount of extra tuning information gained from TKPROF output is minimal if SQL code is written properly in the first place. Using TKPROF can be useful in situations where you cannot get at the SQL code, for instance in situations where all SQL code is embedded in applications code or stored procedures where an Oracle database administrator is brought in on a temporary basis, or simply because the database administrator cannot get access to SQL code or there is lack of communication with application developers. Tuning SQL code without any knowledge of the application can be very difficult. I have used TKPROF in the past to examine all trace files available on database servers and produce TKPROF EXPLAIN option generated query plans for all SQL code. I then passed recommendations onto developers for potential embedded SQL code changes. Before resorting to TKPROF I found and tuned the worst-performing SQL code statements using other tools and Oracle performance views. Example scripting for performing TKPROF EXPLAIN option query plan analysis for all SQL code executed and recorded in trace files can be found in Appendix B.

Let's go through an example showing tuning using SQL Trace and TKPROF. This is a very nasty join query from a previous chapter.

EXPLAIN PLAN SET statement_id='TEST' FOR
      SELECT c.name
      FROM customer c JOIN orders o USING(customer_id)
            JOIN ordersline ol USING(order_id)
            JOIN transactions t USING(customer_id)
                  JOIN transactionsline tl
                     USING(transaction_id)
      WHERE c.balance > 0;

This is its query plan. Note the high cost.

Query                             Cost       Rows       Bytes
----------------------------   -------   --------   ---------
1. SELECT STATEMENT on         1345027   ########   #########
2.  HASH JOIN on               1345027   ########   #########
3.   TABLE ACCESS FULL on           16       1795       44875
       CUSTOMER
3.   MERGE JOIN on               37726   ########   #########
4.    SORT JOIN on               18298   ########   #########
5.     MERGE JOIN on             18298   ########   #########
6.      SORT JOIN on             16864     934659     9346590
7.       MERGE JOIN on            6044     934659     9346590
8.        SORT JOIN on            4388    1916444     7665776
9.         INDEX FAST FULL        4388    1916444     7665776
             SCAN on XFK_TL_
8.         SORT JOIN on           1656     166323      997938
9.          TABLE ACCESS FULL      556     166323      997938
              on TRANSACTIO
6.       SORT JOIN on             1434     165116      990696
7.        TABLE ACCESS FULL        344     165116      990696
            on ORDERS
4.  SORT JOIN on                 19428    1887096     7548384
5.   TABLE ACCESS FULL            1518    1887096     7548384
       on ORDERSLINE

Here is a tuned version of the same query.

EXPLAIN PLAN SET statement_id='TEST' FOR
      SELECT c.name FROM customer c
      WHERE c.balance > 0
      AND EXISTS(
            SELECT o.order_id FROM orders o
            WHERE o.customer_id = c.customer_id
            AND EXISTS(
                  SELECT order_id FROM ordersline
                  WHERE order_id = o.order_id
            )
)
      AND EXISTS(
            SELECT t.transaction_id FROM transactions t
            WHERE t.customer_id = c.customer_id
            AND EXISTS(
                  SELECT transaction_id FROM
                     transactionsline
                  WHERE transaction_id = t.transaction_id
            )
);

Here is the query plan for the tuned version of the query. This tuned version is more than 500 times lower in cost. That is quite a difference.

Query                                      Cost    Rows    Bytes
-------------------------------------   -------   -----   ------
1. SELECT STATEMENT on                     2228       4      100
2.  FILTER on
3.   TABLE ACCESS FULL on CUSTOMER           16       4      100
3.   NESTED LOOPS on 271 700 7000
4.    TABLE ACCESS BY INDEX ROWID on         29     121      726
        ORDER
5.     INDEX RANGE SCAN on XFK_O_CUSTOMER     1      61
4.    INDEX RANGE SCAN on XFK_OL_ORDERS       2       6       24
3.   NESTED LOOPS on                        282     711     7110
4.    TABLE ACCESS BY INDEX ROWID
        on TRANS 28 127 762
5.     INDEX RANGE SCAN on XFK_T_CUSTOMER     1      62
4.    INDEX RANGE SCAN on XFK_TL_TRANSACTI    2       6       24

Now let's use SQL Trace and TKPROF to show what changed when tuning this query. Here is the query not tuned.

EXPLAIN PLAN SET statement_id='TEST' FOR
SELECT c.name
FROM customer c JOIN orders o USING(customer_id)
JOIN ordersline ol USING(order_id)
JOIN transactions t USING(customer_id)
JOIN transactionsline tl USING(transaction_id)
WHERE c.balance > 0

call       count    cpu  elapsed  disk  query  current  rows
---------  -----  -----  -------  ----  -----  -------  ----
Parse          1   0.52     1.20     0    818        0     0
Execute        1   0.09     0.09     0     24        0     0
Fetch          0   0.00     0.00     0      0        0     0
---------  -----  -----  -------  ----  -----  -------  ----
total          2   0.61     1.30     0    842        0     0

Misses in library cache during parse: 1
Optimizer goal: CHOOSE
Parsing user id: 84 (ACCOUNTS)

Rows  Row Source Operation
----  -------------------------------------------
   0 HASH JOIN
   0  TABLE ACCESS FULL CUSTOMER
   0  MERGE JOIN
   0   SORT JOIN
   0    MERGE JOIN
   0     SORT JOIN
   0      MERGE JOIN
   0       SORT JOIN
   0        INDEX FAST FULL SCAN XFK_TL_TRANSACTION
              (object id 34989)
   0       SORT JOIN
   0        TABLE ACCESS FULL TRANSACTIONS
   0     SORT JOIN
   0      TABLE ACCESS FULL ORDERS
   0   SORT JOIN
   0    TABLE ACCESS FULL ORDERSLINE

Here is the tuned version of the query.

EXPLAIN PLAN SET statement_id='TEST' FOR
SELECT c.name FROM customer c
WHERE c.balance > 0
AND EXISTS(
   SELECT o.order_id FROM orders o
   WHERE o.customer_id = c.customer_id
   AND EXISTS(
      SELECT order_id FROM ordersline
      WHERE order_id = o.order_id
   )
)
AND EXISTS(
SELECT t.transaction_id FROM transactions t
WHERE t.customer_id = c.customer_id
AND EXISTS(
      SELECT transaction_id FROM transactionsline
WHERE transaction_id = t.transaction_id
)
)

call       count    cpu  elapsed  disk  query  current  rows
---------  -----  -----  -------  ----  -----  -------  ----
Parse          1   0.02     0.01     0      0        0     0
Execute        1   0.03     0.02     0      0        0     0
Fetch          0   0.00     0.00     0      0        0     0
---------  -----  -----  -------  ----  -----  -------  ----
total          2   0.05     0.04     0      0        0     0

Misses in library cache during parse: 1
Optimizer goal: CHOOSE
Parsing user id: 84 (ACCOUNTS)

Rows  Row Source Operation
----  --------------------------------------------------
0     FILTER
0      TABLE ACCESS FULL CUSTOMER
0      NESTED LOOPS
0       TABLE ACCESS BY INDEX ROWID ORDERS
0        INDEX RANGE SCAN XFK_O_CUSTOMER (object id 32584)
0       INDEX RANGE SCAN XFK_OL_ORDERS (object id 32588)
0      NESTED LOOPS
0       TABLE ACCESS BY INDEX ROWID TRANSACTIONS
0        INDEX RANGE SCAN XFK_T_CUSTOMER (object id 32633)
0       INDEX RANGE SCAN XFK_TL_TRANSACTION (object
          id 34989)

All that is a lot of information. What I want to do is focus on the statistics section produced by SQL Trace and TKPROF. This is the statistics section for the query not tuned.

call       count    cpu  elapsed  disk  query  current  rows
---------  -----  -----  -------  ----  -----  -------  ----
Parse          1   0.52     1.20     0    818        0     0
Execute        1   0.09     0.09     0     24        0     0
Fetch          0   0.00     0.00     0      0        0     0
---------  -----  -----  -------  ----  -----  -------  ----
total          2   0.61     1.30     0    842        0     0

Next is the statistics section for the query tuned. Obviously we can see that both CPU usage and elapsed times have decreased substantially. Parse time is substantially decreased and execute time is decreased to a lesser degree. The different ratios are less distinct than the cost differences for the query plans. This probably implies that the query plan is not accurate, perhaps because statistics are out of date. Parse time is reduced because fewer rows are accessed and probably less SQL code library cache is being used. The execution time is decreased because fewer rows are retrieved and thus the query is reading less physical space. In short, the tuned query is a lot faster.

call       count    cpu  elapsed  disk  query  current  rows
---------  -----  -----  -------  ----  -----  -------  ----
Parse          1   0.02     0.01     0      0        0     0
Execute        1   0.03     0.02     0      0        0     0
Fetch          0   0.00     0.00     0      0        0     0
---------  -----  -----  -------  ----  -----  -------  ----
total          2   0.05     0.04     0      0        0     0

The only point to note in the case of tuning this query with SQL Trace and TKPROF is that the query was tuned in a previous chapter just by recoding the SQL statement in a more efficient manner. The extreme of utilizing SQL Trace and TKPROF was not required to tune this SQL code statement. Additionally examining this TKPROF output there is much detailed information provided but there is no information as to how to tune the SQL code.

Leaving SQL Trace on for extended periods of time can allow gathering of large amounts of analyzable trace files. However, tracing will affect performance quite drastically, so it is better switched off when not being used. Trace files can also use a lot of disk space over a period of time.


9.4 TRCSESS

Oracle Database 10 Grid The TRCSESS utility is new to Oracle Database 10g and can be used to amalgamate information from multiple trace files into a single output file, based on grouping criteria such as a session or a database, amongst other things. This utility is useful in a number of ways such as in a shared server environment, merging information about one specific session from multiple trace files.


9.5 Autotrace

The AUTOTRACE command is a SET command option in SQL*Plus. Every SQL statement issued in SQL*Plus can get some of the information that can be found using SQL Trace, TKPROF, and the EXPLAIN PLAN command. The syntax for AUTOTRACE is as follows:

SET AUTOTRACE { OFF | ON EXPLAIN | ON STATISTICS | ON |
   TRACEONLY }

I find that if I am using AUTOTRACE I should probably be using the TRACEONLY option, which suppresses display of returned rows. Also AUTOTRACE will not function without the PLAN_TABLE in place. Here is an example of what AUTOTRACE produces in SQL*Plus.

SET AUTOTRACE TRACEONLY;
SELECT coa.coa#, coa.text, SUM(gl.dr + gl.cr)
FROM coa JOIN generalledger gl ON(coa.coa# = gl.coa#)
WHERE coa.type = 'A'
GROUP BY coa.coa#, coa.text;

Execution   Plan
------------------------------------------------
   0      SELECT STATEMENT Optimizer=CHOOSE (Cost=420
            Card=139 Bytes=7
   1    0   SORT (GROUP BY) (Cost=420 Card=139 Bytes=7784)
   2    1     TABLE ACCESS (BY INDEX ROWID) OF
                'GENERALLEDGER'(Cost=2
   3    2       NESTED LOOPS (Cost=30 Card=66243
                  Bytes=3709608)
   4    3         TABLE ACCESS (FULL) OF 'COA' (Cost=2
                    Card=14 Bytes=3
   5    3         INDEX (RANGE SCAN) OF 'XFK_COA#'
                    (NON-UNIQUE)(Cost=

Statistics
--------------------------------------------------------
      0   recursive calls
      0   db block gets
   4876   consistent gets
      0   physical reads
      0   redo size
    565   bytes sent via SQL*Net to client
    499   bytes received via SQL*Net from client
      2   SQL*Net roundtrips to/from client
      1   sorts (memory)
      0   sorts (disk)
      2   rows processed

Autotrace can be useful.


9.6 Oracle Database Performance Views for Tuning SQL

Most performance views are applicable to physical and configuration tuning and will be covered in Part III. Additionally the Oracle Database Wait Event Interface will be covered in Part III, which utilizes the SQL code views described here.

There are four views we will examine in this section: V$SQLAREA, V$SQL, V$SQLTEXT, and V$SQL_PLAN. These performance views can be used to find poorly performing queries. Once again most of the problems that can be highlighted with these performance views should not exist if SQL code is properly built in the first place, as described in previous chapters and reiterated in this one. The first three of these performance views access SQL statements parsed and resident in the SGA library cache. V$SQL_PLAN contains currently cached query execution plans or actual execution plans.

9.6.1 Finding Cached SQL Code

  • V$SQLAREA. Contains part of the text of each SQL query up to 1,000 characters plus a lot of statistics. Interesting columns in this performance view are:

    • SQL_TEXT

    • SORTS

    • FETCHES

    • EXECUTIONS

    • LOADS

    • PARSE_CALLS

    • DISK_READS

    • BUFFER_GETS

    • ROWS_PROCESSED

  • V$SQL. This view is similar to V$SQLAREA except with more statistics and without aggregation on rows as in V$SQLAREA. Interesting columns in this performance view in addition to those in V$SQLAREA are:

    • OPTIMIZER_COST

    • CPU_TIME

    • ELAPSED_TIME

  • V$SQLTEXT. Contains the entire SQL query text divided up into separate rows of 64 characters each.

    Tip

    Much of the statistical information in these previous three performance views is part of physical and configuration tuning and will be covered in Part III if appropriate. Additionally a lot of information that can be gathered from these performance views can be retrieved using SQL Trace and TKPROF, amongst various other tools.

  • V$SQL_PLAN. Cached execution plans of recently executed SQL statements can be viewed using the V$SQL_PLAN performance view. The EXPLAIN PLAN command generates potential query plans into the PLAN_TABLE. On the contrary, the V$SQL_PLAN view contains actual query execution plans for recently executed SQL statements. Three other related performance views are V$SQL_PLAN_STATISTICS, V$SQL_PLAN_ STATISTICS_ALL, and the V$SQL_WORKAREA queries.

Examining SQL Code

Let's start with V$SQLAREA. I could hook from V$SQLAREA into currently active Accounts schema sessions but I am not doing that because the queries simply get horrendously complicated. This information is reserved for an examination of the Oracle Database Wait Event Interface in Part III. At this stage we will focus on SQL code issues exclusively. Find SQL text for a particular user as follows:

SELECT * FROM v$sqlarea WHERE parsing_user_id =
      (SELECT user_id FROM dba_users WHERE username =
         'ACCOUNTS');

Now we could retrieve specific columns and sort in descending order to get the hardest-hitting SQL code statements listed first. Quite often the number of executions is the most significant factor.

SELECT executions, disk_reads, buffer_gets, rows_processed,
   sorts, sql_text
FROM v$sqlarea WHERE parsing_user_id =
         (SELECT user_id FROM dba_users WHERE username =
            'ACCOUNTS')
ORDER BY executions DESC;

The V$SQL performance view gives us some slightly different information. The following query is sorted in decreasing Optimizer cost order.

SELECT sa.rows_processed, s.optimizer_cost, s.cpu_time,
   s.elapsed_time, sa.sql_text
FROM v$sqlarea sa, v$sql s
WHERE sa.parsing_user_id =
      (SELECT user_id FROM dba_users WHERE username =
         'ACCOUNTS')

AND s.sql_text = sa.sql_text
ORDER BY s.optimizer_cost DESC;

Now let's join the V$SQLAREA and V$SQLTEXT views to retrieve the entire SQL code text string for each SQL code statement in V$SQLAREA. We will also include some data: the first five rows.

SELECT * FROM(
SELECT st.sql_text
FROM v$sqlarea sa, v$sqltext st
WHERE sa.hash_value = st.hash_value AND sa.address =
   st.address
AND sa.parsing_user_id =
      (SELECT user_id FROM dba_users WHERE username =
         'ACCOUNTS')
ORDER BY st.hash_value, st.address, st.piece)
WHERE ROWNUM <= 5;

SQL_TEXT
-----------------------------------------------------------
INSERT into stockmovement values(stockmovement_seq.nextval,
   :b4,(:b3*-1),:b2,:b1) returning stockmovement_id INTO :b0
INSERT into generalledger values (generalledger_seq.nextval,
   :b1, ABS(:b2 -:b3 ),0,:b4 )
SELECT transaction_id from transactions where
   order_id =:b1

I can also use the V$SQLTEXT_WITH_NEWLINES view to make the output a little more readable. Again the first five rows have been included.

SELECT * FROM(
SELECT st.sql_text
FROM v$sqlarea sa, v$sqltext_with_newlines st
WHERE sa.hash_value = st.hash_value AND sa.address =
   st.address
AND sa.parsing_user_id =
      (SELECT user_id FROM dba_users WHERE username =
         'ACCOUNTS')
ORDER BY st.hash_value, st.address, st.piece)
WHERE ROWNUM <= 5;

SQL_TEXT
-----------------------------------------------------------
INSERT into stockmovement values(stockmovement_seq.nextval,
   :b4,(:b3*-1),:b2,:b1)
SELECT * FROM(
ith_newlines st
ss = st.address

Hard-Hitting SQL Code

Significant factors for finding poorly performing SQL code measurements are rows processed, disk and buffer reads per row, sorts per row, parses per execution, amongst others. So let's find some simple ratios using the various performance views to find the worst-performing SQL code. Let's start with V$SQLAREA.

Using V$SQLAREA

V$SQLAREA gives us a window into parsing and execution statistics.

Executions

The following query finds the 10 queries executed the most often.

SELECT * FROM (
SELECT executions "Execs", rows_processed "Rows", sql_text
FROM v$sqlarea WHERE parsing_user_id =
      (SELECT user_id FROM dba_users WHERE username =
         'ACCOUNTS')
ORDER BY executions DESC
) WHERE ROWNUM <= 10;

 Execs    Rows  SQL_TEXT
------  ------  -----------------------------------------------
107785  107785  INSERT into cashbookline values(:b3,:b2,:b1)
107785  107785  INSERT into ordersline values(:b4,:b3,
                  :b2,:b1)
107785  107782  INSERT into transactionsline values(:b4,
                  :b3,:b2,:b
107785  107785  INSERT into stockmovement values
                  (stockmovement_seq
 81823   81823  SELECT to_number(to_char
                  (SYSTIMESTAMP,'FF3'))+1 fr
 53909   53909  SELECT systimestamp FROM sys.dual
 34208       0  COMMIT
 32391   32391  INSERT into generalledger values
                  (generalledger_se
 32391   32391  INSERT into generalledger values
                  (generalledger_se
 22438   22438  INSERT into cashbookline values(:b3,:b2,
                  (:b1*-1))
Disk + Buffer Reads per Row

This query looks at highest data access queries.

SELECT * FROM(
SELECT ROUND((disk_reads + buffer_gets)/rows_processed)
   "Reads/Row"
      ,disk_reads + buffer_gets "Reads"
      ,rows_processed "Rows", sql_text
FROM v$sqlarea WHERE parsing_user_id =
      (SELECT user_id FROM dba_users WHERE username =
         'ACCOUNTS')
AND rows_processed > 0 AND (disk_reads + buffer_gets) > 0
ORDER BY ROUND((disk_reads + buffer_gets)/rows_processed)
   DESC
) WHERE ROWNUM <= 10;

Reads/Row     Reads   Rows  SQL_TEXT
---------  --------   ----  ----------------------------------
     9971     29912      3  select count(*) from generalledger
     5562      5562      1  select min(id),min(parent_id)
                              from v$sq
     5265      5265      1  select count(*) from v$sql_plan
     1330      3991      3  SELECT TRIM(LEVEL)||'. '||LPAD
                              (' ', LE
     1325      3974      3  SELECT TRIM(LEVEL)||'. '||LPAD
                              (' ', LE
     1310      3929      3  select p.operation, p.options,
                              p.objec
     1308      3925      3  SELECT TRIM(LEVEL)||'. '||LPAD
                              (' ', LE
     1303      3909      3  select s.sql_text, p.operation,
                              p.opti
      897   3719117   4148  SELECT min(order_id),max(order_id)
                              from
      841  14626647  17386  DECLARE job BINARY_INTEGER:= :job;
                              nex
Rows per Sort

Fewer rows per sort may indicate a lack of indexing, lack of use of indexes or superfluously ordered SQL code statements. Too much sorting!

SELECT * FROM(
SELECT ROUND(rows_processed/sorts) "Rows/Sort"
      ,rows_processed "Rows"
      ,Sorts "Sorts", sql_text
FROM v$sqlarea WHERE parsing_user_id =
      (SELECT user_id FROM dba_users WHERE username =
         'ACCOUNTS')
AND rows_processed > 0 AND sorts > 0
ORDER BY ROUND(rows_processed/sorts) ASC
) WHERE ROWNUM <= 10;

Reads/Row     Reads   Rows  SQL_TEXT
---------  --------   ----  ----------------------------------
        1     15616  31232  SELECT a.supplier_id, a.price from (
        1         3      3  SELECT (disk_reads + buffer_gets)/
                              row
        2         3      2  select p.operation, p.options,
                              p.objec
        2         3      2  select s.sql_text, p.operation,
                              p.opti
        2        27     12  SELECT * FROM ( SELECT executions,
                              rows
        3        10      4  SELECT * FROM( SELECT ROUND((sorts)/r
        3        10      4  SELECT * FROM ( SELECT ROUND
                              (parse_cal
        3        10      4  SELECT * FROM ( SELECT ROUND
                              (fetches/r
        3        20      8  SELECT * FROM ( SELECT ROUND
                              (buffer_ge
        3        10      4  SELECT * FROM ( SELECT ROUND
                              ((buffer_g
Rows per Fetch

The fewer rows fetched for each fetch, the worse the performance. We are looking for problems not successes.

SELECT * FROM (
SELECT ROUND(rows_processed/fetches) "Rows/Fetch"
      ,rows_processed "Rows", fetches "Fetches",
         sql_text
FROM v$sqlarea WHERE parsing_user_id =
      (SELECT user_id FROM dba_users WHERE username =
         'ACCOUNTS')
AND rows_processed > 0 AND fetches > 0
ORDER BY ROUND(rows_processed/fetches) ASC
) WHERE ROWNUM <= 10;

Rows/Fetch   Rows  Fetches  SQL_TEXT
----------  -----  -------  ------------------------------
         1      2        2  select user from dual
         1      4        4  SELECT USER FROM DUAL
         1  14532    14532  SELECT max(stock_id) from stock
         1      1        1  select count(*) from v$sql_plan
         1     10       16  SELECT job from user_jobs
         1  19060    19060  select SYSDATE+1/86400 from dual
         1      4        4  SELECT DECODE('A','A','1','2')
                              FROM DUAL
         1 14532 14532 SELECT max(customer_id) from
                              customer
         1 3 3 select count(*) from
                              generalledger
         1 59213 59213 SELECT systimestamp FROM sys.dual
Parses per Execution

More parsing means that previously parsed SQL code in the shared pool is not being reused efficiently. This can indicate lack of SQL code bind variables. The CURSOR_SHARING parameter SIMILAR or FORCE settings can help.

SELECT * FROM (
SELECT ROUND(parse_calls/executions) "Parses/Exec"
      ,parse_calls "Parses", executions "Execs", sql_text
FROM v$sqlarea WHERE parsing_user_id =
      (SELECT user_id FROM dba_users WHERE username =
         'ACCOUNTS')
AND parse_calls > 0 AND executions > 0
ORDER BY ROUND(parse_calls/executions) ASC
) WHERE ROWNUM <= 10;

Parses/Exec  Parses   Execs  SQL_TEXT
-----------  ------  ------  ---------------------------------
          0   17490   54306  SELECT systimestamp FROM sys.dual
          0   13315  108638  INSERT into cashbookline
                               values(:b3,
          0   13315  108638  INSERT into ordersline
                               values(:b4,:b
          0   13315  108638  INSERT into transactionsline
                               values(
          0    2364   22646  INSERT into ordersline
                               values(:b4,:b
          0    2364   22646  INSERT into cashbookline
                               values(:b3,
          0    2364   22646  INSERT into stockmovement
                               values(sto
          0     685    3757  INSERT into stockmovement values
          0   17490   82425  SELECT to_number(to_char
                               (SYSTIMESTAM
          0    2364   22646  INSERT into transactionsline
                               values(
Disk versus Logical Reads

Traditionally disk I/O is slower than reading data from memory. However, logical memory reads involve CPU and potential cache latch contention overhead to maintain consistency between multiple sessions accessing the same buffer blocks. Logical versus physical disk reads is known as the Buffer Cache Hit Ratio. Changing this ratio is quite often not a performance solution in itself but more likely a symptom or indication of other potential problems. Try not to take this ratio as seriously as most Oracle Database and Oracle-oriented documentation has described in the past. In the extreme a 95%+ Buffer Cache Hit Ratio is an unreliable measure of database performance health. There will be more on this in Part III. In general use any type of ratio as a symptom or an indicator of a possible problem. Do not attempt to tune the ratio, rather find the problem.

Tip

Current research indicates that logical reads are perhaps as little as only 30 times faster than physical reads, not thousands or hundreds of times faster as previously thought.

COLUMN bchr HEADING "Buffer Cache Hit Ratio" FORMAT a24;
SELECT * FROM (
SELECT ROUND((buffer_gets/(buffer_gets+disk_reads))*100)||'%'
   AS bchr
      ,sql_text
FROM v$sqlarea WHERE parsing_user_id =
      (SELECT user_id FROM dba_users WHERE username =
         'ACCOUNTS')
AND buffer_gets > 0 AND disk_reads > 0
ORDER BY ROUND((buffer_gets / (buffer_gets +
   disk_reads))*100) ASC
) WHERE ROWNUM <= 10;

Buffer Cache Hit Ratio  SQL_TEXT
----------------------  -------------------------------------
29%                     SELECT * FROM ( SELECT ROUND
                          (buffer_gets / (
30%                     SELECT * FROM ( SELECT ROUND
                          ((buffer_gets /
31%                     SELECT * FROM ( SELECT ROUND
                          (fetches/rows_pr
31%                     SELECT * FROM ( SELECT ROUND
                          (parse_calls/exe
32%                     SELECT sa.rows_processed,
                          s.optimizer_cost, s
32%                     SELECT * FROM ( SELECT executions
                          "Execs", ro
33%                     SELECT * FROM ( SELECT ROUND
                          ((buffer_gets /
33%                     SELECT * FROM ( SELECT executions,
                          rows_proce
33%                     SELECT * FROM ( SELECT executions,
                          rows_proce
33%                     SELECT * FROM ( SELECT ROUND
                          ((disk_reads + b

Using V$SQL

Now let's use V$SQL to look at Optimizer cost, CPU, and elapsed time.

Optimizer Cost
SELECT * FROM(
SELECT s.optimizer_cost "Cost", sa.rows_processed "Rows",
   sa.sql_text
FROM v$sqlarea sa, v$sql s
WHERE sa.parsing_user_id =
      (SELECT user_id FROM dba_users WHERE username =
         'ACCOUNTS')
AND s.sql_text = sa.sql_text
ORDER BY s.optimizer_cost DESC
) WHERE ROWNUM <= 10;

Cost    Rows  SQL_TEXT
----  ------  -----------------------------------------------
2019    4160  INSERT into stockmovement values (:b5,:b4,:b3,:
2019  122208  INSERT into stockmovement values
                (stockmovement_seq.
2019   25474  INSERT into stockmovement values
                (stockmovement_seq.
1677   36755  INSERT into generalledger values
                (generalledger_seq
1677    7394  INSERT into generalledger values
                (generalledger_seq
1677   36755  INSERT into generalledger values
                (generalledger_seq
1677    7394  INSERT into generalledger values
                (generalledger_seq
1540   25474  INSERT into transactionsline values(:b4,:b3,
                (:b2*-1
1540  122205  INSERT into transactionsline values(:b4,:b3,
                :b2,:b1
1518   25474  INSERT into ordersline values(:b4,:b3,
                (:b2*-1),:b1)
CPU Time
SELECT * FROM(
SELECT s.cpu_time "CPU", sa.rows_processed "Rows",
   sa.sql_text
FROM v$sqlarea sa, v$sql s
WHERE sa.parsing_user_id =
      (SELECT user_id FROM dba_users WHERE username =
         'ACCOUNTS')
AND s.sql_text = sa.sql_text
ORDER BY s.cpu_time DESC
) WHERE ROWNUM <= 10;

CPU           Rows  SQL_TEXT
----------  ------  -----------------------------------------
5216000189   19747  DECLARE job BINARY_INTEGER := :job;
                      next_date DATE
2578878295    4689  SELECT min(order_id),max(order_id) from
                      orders
310826983   122566  INSERT into transactionsline values(:b4,
                      :b3,:b2,:b
302695184   122573  INSERT into ordersline values(:b4,:b3,
                      :b2,:b1)
297647959    17735  SELECT a.supplier_id, a.price from
                      ( select supp
260374370   122588  INSERT into stockmovement values
                      (stockmovement_seq
244912201   122569  INSERT into cashbookline values(:b3,
                      :b2,:b1)
155974301    15062  INSERT into transactions values
                      (transactions_seq.n
147301740    15062  INSERT into cashbook values
                      (cheque_seq.nextval,:b3
100884998    36889  INSERT into generalledger values
                      (generalledger_se
Elapsed Time
SELECT * FROM(
SELECT s.elapsed_time "Time", sa.rows_processed "Rows",
   sa.sql_text
FROM v$sqlarea sa, v$sql s
WHERE sa.parsing_user_id =
      (SELECT user_id FROM dba_users WHERE username =
         'ACCOUNTS')
AND s.sql_text = sa.sql_text
ORDER BY s.elapsed_time DESC
) WHERE ROWNUM <= 10;

Time          Rows  SQL_TEXT
----------  ------  ---------------------------------------------
4.4771E+10   19792  DECLARE job BINARY_INTEGER := :job;
                      next_date DATE
1.7155E+10    4707  SELECT min(order_id),max(order_id) from
                      orders
2809286653   15091  INSERT into transactions values
                      (transactions_seq.n
2739120692  122853  INSERT into transactionsline values(:b4,
                      :b3,:b2,:b
2649237617  122857  INSERT into ordersline values(:b4,:b3,
                      :b2,:b1)
2061587320  122854  INSERT into cashbookline values(:b3,
                      :b2,:b1)
1974169102   15091  INSERT into orders values
                      (orders_seq.nextval,:b3,n
1885794542   17769  SELECT a.supplier_id, a.price from
                      ( select supp
1731725989   36965  INSERT into generalledger values
                      (generalledger_se
1601419096  122865  INSERT into stockmovement values
                      (stockmovement_seq

So what was the point in showing all these different sorted versions of the same query? If you look closely you might see some patterns. A number of the sections have a lot of INSERT statements. My high concurrency DBMS_JOBS executed OLTP activity does a lot of INSERT plus a fair amount UPDATE and DELETE activity. Also notice some versions of this query have a lot of SELECT activity. DML and particularly INSERT DML activity is obvious in queries dealing with a high number of executions and CPU time. Most of all the other measurements involve amounts of data found and thus include SELECT statements.

The high INSERT statement activity begs the following question. Why is INSERT activity using up so many resources? Probably because I am inserting multiple rows into the same blocks in the buffer cache and the database files, probably involving both index and data spaces. I am probably getting locking problems with contention for buffer and disk blocks; hot blocks. I could potentially resolve these issues by perhaps changing some primary key integer identifier indexes to reverse key indexes. However, this would probably result in my database growing so fast that I eventually run out of disk space. So for the purposes of maintaining my simulation I will leave my current configuration the way it is.

Examining Cached Query Plans

Now let's examine cached query execution plans in V$SQL_PLAN. This performance view is best queried by matching SQL statements from V$SQLAREA or V$SQL views using the common ADDRESS and HASH_VALUE columns. Using V$SQL_PLAN can be similar to this:

SELECT TRIM(depth)||'. '||LPAD (' ', depth - 1)||operation
   ||' '||options||' on '||object_name "Query"
      ,cost "Cost"
      ,cardinality "Rows"
      ,bytes "Bytes"
FROM v$sql_plan
ORDER BY address, hash_value, parent_id, id;

Query
----------------------------------------
0. SELECT STATEMENT on
1.  COUNT STOPKEY on
2.   VIEW on
3.    SORT ORDER BY STOPKEY on
4.     NESTED LOOPS OUTER on
5.      NESTED LOOPS OUTER on
5.      TABLE ACCESS CLUSTER on USER$
6.       FIXED TABLE FULL on X$KQLFXPL
6.       TABLE ACCESS BY INDEX ROWID on OBJ$
7.        INDEX UNIQUE SCAN on I_OBJ1
6.       INDEX UNIQUE SCAN on I_USER#
0. SELECT STATEMENT on
1.  NESTED LOOPS OUTER on
2.   NESTED LOOPS OUTER on
2.   TABLE ACCESS CLUSTER on USER$
3.    FIXED TABLE FULL on X$KQLFXPL
3.    TABLE ACCESS BY INDEX ROWID on OBJ$
4.     INDEX UNIQUE SCAN on I_OBJ1
3.    INDEX UNIQUE SCAN on I_USER#

That concludes this discussion of how to find problem queries and the tools used to accomplish this. In the next chapter we will look at how to tune SQL code in Oracle Database using Oracle Enterprise Manager tuning tools.


Chapter 10: Tuning SQL with Oracle Enterprise Manager

Oracle Enterprise Manager is a large suite of software tools used to manage and tune Oracle databases across an enterprise. This chapter will briefly cover parts of Oracle Enterprise Manager useful to tuning of SQL code. More will be covered in Part III when discussing the Oracle Database Wait Event Interface.

10.1 The Tuning Pack

The Tuning Pack contains several applications for both preemptive and reactive tuning of an Oracle installation. With respect to SQL code tuning the parts we are interested in at present are as follows:

  • Index Tuning Wizard. Searches for inefficient indexes and makes suggestions for new indexes, removal of superfluous indexing and changes to existing indexes.

  • SQL Analyze. Allows SQL code analysis, potential optimization, SQL statement editing, and even some capability for automated tuning.

  • Oracle Expert. Allows recommendations and implementation of possible tuning approaches.

  • Outlines. Outline performance assessment and management is also included in the Tuning Pack. Outlines will eventually be deprecated from a future version of Oracle Database and are thus not covered in this book.

Let's start by taking a look at the Index Tuning Wizard.

10.1.1 Index Tuning Wizard

The Index Tuning Wizard attempts to find inefficient indexes from a cost-based optimization perspective. Inefficient indexing was covered in Chapter 7 on Common Sense Indexing. The Index Tuning Wizard makes recommendations. These recommendations include recommendations to add new indexes, rebuild current indexes, and index type changes such as changing from a Bitmap to a BTree.

Any changes with the database can warrant running the Index Tuning Wizard. If your database consistently changes over time this tool could be executed periodically.

On a more practical note, using a tool such as this to automatically regenerate indexes can be detrimental to performance in itself. Interrogation of database statistics should suffice to interrogate indexing capabilities. Also when I have used this tool in the past recommendations to create new and rebuild existing indexes have been somewhat "wild" to say the least. I would propose extreme caution when taking heed of recommendations reported by this tool. In general, OLTP systems are highly active and any index rebuilds should be executed with the ONLINE option to avoid application errors. The mere act of rebuilding an index can cause a severe performance hit. For an OLTP database any recommendation to change an index type from a BTree to any other index type should probably be completely ignored. Bitmap indexes are not really useful for anything but read-only environments. I have even heard complaints about bitmap index use in data warehouses. Additionally BTree indexing only becomes inefficient when skewed. Skewing is generally as a result of data content and not much can be done about that unless the data model can be altered. The only case where BTree indexes should occasionally be rebuilt is when a table undergoes a lot of deletion. Deleted leaf blocks are not reclaimed in BTree indexes.

Note

Oracle Database 10 Grid General recommendation and acceptance of use of automated segment management may make occasional rebuilding of BTree indexing less necessary.

I executed the Index Tuning Wizard on my highly active database. It appears from the following example that the Index Tuning Wizard searches through past SQL code statements attempting to match WHERE, ORDER BY, and GROUP BY clauses against indexes. If an index does not exist it suggests creating one. The composite index suggested for creation in the following example is not strictly necessary. Let's examine this example. Figure 10.1 shows the initial screen for the Index Tuning Wizard.

Click To expand
Figure 10.1: The Index Tuning Wizard

The next screen shown in Figure 10.2 allows an option of selecting different database types. Index requirements between the two extremes of OLTP and data warehousing are totally different. OLTP databases require exact hits on single rows and data warehouses require retrieval of large numbers of rows at once.

Click To expand
Figure 10.2: Select the Application Type

Then either a single schema or multiple schemas can be selected for index analysis, as shown in Figure 10.3.

Click To expand
Figure 10.3: Select Schemas to Analyze

Figure 10.4 shows a recommendation to modify an index on the Transactions table. This index is suggested as being altered from a single-column to a multiple-column index. The suggestion is sensible because as shown in Index Tuning Wizard output, included later in this chapter, this index is accessed by an SQL statement using the two columns suggested for the composite index. However, this index is a foreign key index on the TransactionsLine table pointing to the primary key on the Transactions table. This index can absolutely not be altered in my Accounts schema. Note that Oracle Database does not create indexes on foreign key constraints automatically, thus Oracle Database is not aware that this index is actually a single-column only foreign key index. Or, more succinctly, the Index Tuning Wizard ignores foreign keys. Additionally this recommendation is poor for two further reasons. Firstly, the number of values in the suffix (second column) of the composite index is minimal. As a result, indexing the composite would have little effect on any SQL accessing both columns at once. The single-column index would suffice for good performance. Secondly, my application code simply does not require a composite index in this case; a composite index is totally inappropriate.

Click To expand
Figure 10.4: The Index Tuning Wizard Makes Suggestions

The following script shows the Index Tuning Wizard output suggesting why the index should be changed. Note the composite column access in the WHERE clause.

A change to an existing B*-tree index is recommended for
the table ACCSTEST.TRANSACTIONSLINE.

Index before recommendations
----------------------------

TRANSACTION_ID

Recommended columns
-------------------

TRANSACTION_ID
      There was at least one reference by an equality
         operator.

STOCKMOVEMENT_ID
      There was at least one reference by an equality
         operator.
      There was at least one reference by SELECT, ORDER-BY
         or GROUP-BY clauses.
      Number of rows in the table: 1551030
      Distinct values in the index: 1551030

      ************** SQL Statement Information **************

      The recommendation above is based on the following SQL:

      SQL: SELECT stockmovement_seq.NEXTVAL, sm.stock_id,
            sm.qty * -1 qty, sm.price, :b1
         FROM stockmovement sm
         WHERE EXISTS (SELECT stockmovement_id
                  FROM transactionsline
                  WHERE stockmovement_id =
                     sm.stockmovement_id
                  AND transaction_id = :b2)

The Index Tuning Wizard also produces the code necessary to regenerate the index. Note one thing about this code. Performing these DDL statements in a highly available OLTP database is a very bad idea because the ONLINE option was excluded. If this table is large, which it is, and there is access to the table between dropping and completion of the CREATE INDEX command, and SQL code accesses the index, any rows not yet built in the index will simply not be found by the SQL statements.

--CREATE Index Recommendations
DROP INDEX ACCSTEST.XFK_TL_TRANSACTION;
CREATE INDEX ACCSTEST.XFK_TL_TRANSACTION
      ON ACCSTEST.TRANSACTIONSLINE
         (TRANSACTION_ID, STOCKMOVEMENT_ID)
      LOGGING
      INITRANS 2
      MAXTRANS 255
      TABLESPACE INDX
      PCTFREE 10
      STORAGE (
         INITIAL 1024K
         NEXT 1024K
         PCTINCREASE 0
         MINEXTENTS 1
         MAXEXTENTS 2147483645) ONLINE;

I have grave doubts about the usefulness of the Index Tuning Wizard in any highly active OLTP database. Most OLTP databases are required to be available permanently. Perhaps this tool would be more useful for data warehouses? For the CREATE INDEX command shown previously at least the ONLINE option should be used. To reiterate, the ONLINE option was not generated for the commands created by the Index Tuning Wizard; I added it. Additionally better availability would be gained by using the ALTER INDEX command with the REBUILD and ONLINE options. ALTER INDEX REBUILD ONLINE would never create a situation of the index not existing. In the preceding script the index does not exist between the DROP and CREATE commands.

Note

Oracle Database 10 Grid Over the years the parts and pieces in Oracle Enterprise Manager have become much better as tuning tools. The problem with making index recommendations in this context is that indexing is abstracted from applications and data model to a database engine. The database engine is not an expert on what applications are attempting to achieve.

10.1.2 SQL Analyze

Tuning SQL code is one of the most effective methods of tuning an Oracle installation and its attached applications. However, tuning of SQL code can sometimes be difficult and very time consuming. SQL code embedded in applications can involve application code changes, which may not be too difficult but may involve resistance from developers, and quite understandably so. Developers are of course correct to resist any code changes. Code changes can cause a stable application to rapidly become very unstable. In a production environment the cliché "If it works don't fix it" holds true in many cases. Additionally tuning of SQL code can require that the personnel tuning that code have extensive knowledge of all aspects of the database and any applications. This is often a difficult if not impossible scenario.

SQL Analyze is a tool which can be used to assist in tuning SQL code by allowing SQL code analysis, potential optimization, SQL statement editing, and even some capability for automated tuning. Note that allowing SQL Analyze to automatically tune SQL code could be risky. In the past when using automated tuning features in Oracle Enterprise Manager I have found the more automated aspects to be somewhat dubious at best. Making automated changes in production environments is contrary to the approach of most database administrators. The unpredictability of automated changes is relatively unknown. Database administrators are generally exceedingly cautious and for a good reason. Production databases are often required to be running constantly and any potential changes could cause loss of service. For most OLTP database installations downtime is absolutely not acceptable. Customers can get very upset. When customers are upset they go to the competition.

SQL Analyze can be used to do a large number of things. SQL Analyze allows access to initialization parameters, current cache "TopSQL" (hardest-hitting SQL statements) plus stored historical SQL statements. In general, SQL Analyze can be divided into two main sections. The first as shown in Figure 10.5 shows the analysis stage.

Click To expand
Figure 10.5: SQL Analyze and the Analysis Stage

As can be seen in Figure 10.6 the analysis stage allows retrieval of the worst-performing SQL statements. Note all the different sort orders. The sort orders are consistent with SQL Trace and TKPROF output sort orders described in the previous chapter. Disk reads per execution is the default because it shows disk access for the execution of an SQL statement. In terms of data retrieval I prefer to simply add both disk and buffer gets and compare the sum to rows processed, giving an assessment of database access required for each row. Very often the actual executions of an SQL statement is the most significant factor. Sometimes a few SQL statements can be executed much more frequently than any other SQL code. Obviously the most frequently executed SQL code could potentially have the most profound effect on performance and should be the most highly tuned SQL statements in the database. Focus on the biggest problems!

Click To expand
Figure 10.6: SQL Analyze Single SQL Statement Assessment and Testing

Figure 10.6 shows a selection of TopSQL selected SQL statements in the top right box, sorted in order of disk reads per execution. Note access to configuration parameters in the bottom right box.

The second main section of SQL Analyze has rich capabilities and is very useful indeed, as seen in Figure 10.7 through to Figure 10.12. SQL Analyze allows for numerous actions and assessments on individual SQL code statements, providing a graphical user interface into both query plans and execution statistics.

Click To expand
Figure 10.7: SQL Analyze SQL Text Display

Figure 10.7 shows the SQL code text in the bottom right box for a highlighted SQL statement.

Figure 10.8 shows the EXPLAIN PLAN command query plan for an SQL statement.

Click To expand
Figure 10.8: SQL Analyze Query Plan

Figure 10.9 shows the ability in SQL Analyze for index recommendations.

Click To expand
Figure 10.9: SQL Analyze Index Recommendations

Figure 10.10 shows that a query plan can be assessed for all Optimizer modes: RULE, FIRST_ROWS, ALL_ROWS, and CHOOSE. This allows testing of different modes of optimization.

Click To expand
Figure 10.10: SQL Analyze Query Plan Assessment

Figure 10.11 shows menu access routes to the Virtual Index Wizard, the Hint Wizard, and the SQL Tuning Wizard. These tools are extremely useful for doing SQL code tuning, interactively and with ease of use.

Click To expand
Figure 10.11: SQL Analyze Virtual Indexing, Hints, and Tuning

Having used SQL Analyze quite extensively I am rather enthused with its very rich capabilities. I am still noticing a few bugs, even in Oracle9i Database, but nothing debilitating. SQL Analyze can be used to compare different versions of the same SQL statement as shown in Figure 10.12. This multiple SQL statement functionality is completely interactive and changes as the SQL programmer changes things.

Click To expand
Figure 10.12: Comparing Different Versions of an SQL Statement

Figures 10.13 and 10.14 show two query plans for a rather "busy" SQL join statement. Note the difference in cost between the ALL_ROWS and FIRST_ROWS Optimizer mode versions, respectively. The ALL_ROWS version is much lower in cost because the mutable join is retrieving all the rows in all the tables. This is more apparent when examining the query plans, showing that the FIRST_ROWS option in Figure 10.14 uses all nested loop joins with no hash joins. Hash joins are efficient for joining large row sets and nested loop joins are better for joining a very small row set with another small or a large row set. Forcing FIRST_ROWS optimization as in Figure 10.14 makes the Optimizer think it is not accessing many rows and thus it defaults to nested loop joins.

SQL Analyze is very rich in functionality. Spotlight and TOAD are non-Oracle tools of the same caliber with similar capabilities but SQL Analyze is a richer environment with respect to tuning SQL code. The only problem using any tool within Oracle Enterprise Manager is that you need to be a rocket scientist to set it up properly.

Keep trying. Practice makes perfect! Parts of Oracle Enterprise Manager are extremely useful and informative.

Tip

Drilling down into the Oracle Database Wait Event Interface is covered extremely well by Oracle Enterprise Manager. The Oracle Database Wait Event Interface will be covered in Part III.

In the past Oracle Enterprise Manager using the Management Server has had serious security issues. In the past Oracle Enterprise Manager has been bug-ridden and known to induce a certain amount of frustration, sometimes most immense frustration. The word is that these problems have apparently largely been resolved.

What can SQL Analyze be used for in general?

  • Search for and change poorly performing SQL code.

  • Assess existing indexing.

  • TopSQL can be used to show high-impact SQL statements from both current cache and historical SQL code. SQL code statements can be sorted on a large number of criteria much like SQL Trace files can be sorted with TKPROF.

  • A high-impact SQL statement can be selected. That selected statement can be analyzed in its entirety including query plans. SQL statements can even be imported or typed in from scratch and tested.

  • Query plans and execution statistics can be generated for all potential Optimizer modes.

  • A Hint Wizard allows suggested and tested alterations using hints.

  • Virtual indexing and index recommended alterations and additions can be utilized to produce varying query plans.

  • There is some automated change capability in SQL Analyze. My experience with Oracle Enterprise Manager in general makes it necessary to recommend not executing any automation-type features which change anything in the database without a database administrator being made aware of potential changes before changes are made.

  • Allow comparisons between different query plans for the same SQL statement both in textual and graphical form. This is a really "cute" feature but not necessarily all that useful.

  • Allow a step-by-step walk-through of query plans giving a detailed, easy-to-understand explanation of each step.

10.1.3 Oracle Expert

Oracle Expert is a tool allowing recommendations and implementation of possible tuning approaches. This tool applies to SQL code tuning plus physical and configuration tuning. In this chapter we will concentrate on the SQL code tuning aspects of Oracle Expert. As far as SQL code tuning is concerned Oracle Expert can produce what is called an Optimal Index Analysis. An excerpt of one of these analyses executed on the Accounts schema is shown on the next page.

Click To expand
Figure 10.13: An ALL_ROWS Optimizer Query Plan
----------------------------------------------
-- DROP Index Recommendations
----------------------------------------------
DROP INDEX ACCSTEST.XFK_CBL_CHEQUE;
DROP INDEX ACCSTEST.XFK_O_TYPE;
DROP INDEX ACCSTEST.XAK_O_TYPE_CUSTOMER;
DROP INDEX ACCSTEST.XFK_OL_ORDERS;
DROP INDEX ACCSTEST.XFK_TL_TRANSACTION;

Note that all the Oracle Expert has advised that I do is to drop foreign key indexes from the Accounts schema. Those foreign key indexes are not accessed by any SQL code. Shriek! That is definitely not an intelligent idea. Why? Firstly, foreign key indexes will not necessarily be used by SQL code. Secondly, removing indexes from foreign keys will create a lot of locking problems and probably result in some severe performance issues. Foreign key indexes are used by Oracle Database internal Referential Integrity checks. Those Referential Integrity checks are very likely ignored by Oracle Expert since no explicit SQL code is involved. This is not good. Be very careful using Oracle Expert in this particular respect.

Click To expand
Figure 10.14: A FIRST_ROWS Optimizer Query Plan

The Oracle Expert on-screen visual of the above script is shown in Figure 10.15.

Click To expand
Figure 10.15: Oracle Expert Recommendations for the Accounts Schema

10.1.4 SQL Access Advisor

Oracle Database 10 Grid SQL Access Advisor is an Oracle Enterprise Manager tool allowing identification and resolution of problematic SQL code statements.

That is about all we need to cover on the subject of tuning SQL code using Oracle Enterprise Manager. Further discussion of Oracle Database physical and configuration tuning using Oracle Enterprise Manager will be covered in Part III. Oracle Enterprise Manager is effectively an excellent teaching tool and one can learn a lot about the Oracle Database and tuning using Oracle Enterprise Manager. There is much in Oracle Enterprise Manager which is very useful.

This concludes Part II of this book and the discussion of SQL code tuning. The next chapter is the first chapter in Part III, beginning the subject of physical and configuration tuning in Oracle Database. We will start Part III with a chapter covering Oracle installation and database creation, allowing for the best database performance from first acquisition and installation of Oracle software. Subsequent chapters in Part III will proceed into physical and configuration tuning in much greater depth.


Part III: Physical and Configuration Tuning

Chapter List

Chapter 11: Installing Oracle and Creating a Database
Chapter 12: Tuning Oracle Database File Structures
Chapter 13: Object Tuning
Chapter 14: Low Level Physical Tuning
Chapter 15: Hardware Resource Usage Tuning
Chapter 16: Tuning Network Usage
Chapter 17: Oracle Partitioning and Parallelism
Chapter 18: Ratios: Possible Symptoms of Problems
Chapter 19: Wait Events
Chapter 20: Latches
Chapter 21: Tools and Utilities
Chapter 22: Tuning with the Wait Event Interface and STATSPACK


Chapter 11: Installing Oracle and Creating a Database

So far in this book we have covered both data model and SQL code tuning. Part III deals exclusively with physical and configuration tuning. Probably the most essential parts of this type of tuning Oracle Database could be the first two steps you ever take with Oracle Database. These two steps are installing Oracle software and creating a database; in this chapter we will focus on just that. The configuration parameter file is an integral part of that process.

11.1 Installing Oracle Database

Oracle9i Database installs a lot of extra options if you let it. One way to install Oracle software without the extra options is to do a custom installation and select only the required options. Avoiding installing unwanted options will not necessarily help performance but it can make administration and ultimately tuning a lot easier in the long run. However, this can cause problems because any missing components could have to be added at a later stage.

Note

Oracle Database 10 Grid The general trend is towards more automation, both with installation, configuration and the underlying physical architecture of an Oracle database in general. On the whole, Oracle Database 10g is very much simplified from Oracle9i Database.

11.1.1 Different Editions of Oracle Database

The three different editions are Personal Oracle, Standard Edition, and Enterprise Edition. Personal Oracle is intended for use on a single machine and Standard Edition is a trimmed down, cheaper version of Enterprise Edition. There are also additional optional components available with Enterprise Edition, such as Oracle Partitioning, which require additional licensing fees. Enterprise Edition is the best and the easiest to manage and use, and is well-worth its slightly higher price tag.

There are a multitude of installable options which can be added to an Oracle installation at the time of installation or at a later stage. Simply run the Oracle installation software again to add further Oracle Database binary options. Some options added into a database itself can be added at a later stage using scripts contained in the $ORACLE_HOME/rdbms/admin directory.

11.1.2 Oracle Enterprise Manager

Oracle Enterprise Manager is an excellent Oracle database management and tuning tool with a very user-friendly interface. Oracle Enterprise Manager allows for easy administration and some tuning. Additionally search the Internet for a tool called Spotlight. In my opinion Spotlight is better as a real-time monitoring tool for busy production databases. Oracle Enterprise Manager is perhaps a little easier to use and more versatile in finding problems with the Oracle Database Wait Event Interface. Oracle Enterprise Manager consists of a number of parts:

  • Enterprise Manager Web Site. This option allows Oracle Enterprise Manager functionality to be used over the web.

  • Enterprise Manager Console and Client. The console provides centralized access and control to all the parts of Oracle Enterprise Manager.

  • Oracle Management Server and Agent Processes. Some parts of Oracle Enterprise Manager require two extra processes: the Management Server and the Agent.

  • Add-on Packages. Extra packages involve change management, diagnostics, and tuning capabilities. These add-on packages are extremely useful. The only point to note is that perhaps any automated functionality should be tested first before using it in production.


11.2 Basic Configuration

Let's take a brief look at Oracle networking software configuration files and the Oracle instance configuration parameter file. Any details left out at this stage will be explained later in this book. Oracle instance parameters are covered throughout this book but are described in some detail here.

Let's start with the networking files since they are fairly simple.

11.2.1 Basic Network Files

The listener runs on the database server. It "listens" for requests from network processes, namely Oracle client processes. The database can be connected to a session using a tool such as SQL*Plus through a network protocol such as TCP/IP. A connection can be made on the server itself using something called the bequeath protocol which is a direct connection to the database. The ORACLE_SID variable must be set to use the bequeath protocol. Using the bequeath protocol will probably be a little faster than passing the connection through TNS.

Configuring the Listener

The file below is an example Listener configuration file. This file is installed in the $ORACLE_HOME/network/admin directory. Note that using the IP-Address instead of the hostname for the database server machine may be noticeably faster. However, subsequent changes to IP-Addresses on your network will require changing the Listener configuration file and bouncing the Listener. This is a problem if your database is not allowed any downtime.

LISTENER =
   (DESCRIPTION_LIST = (DESCRIPTION =
      (ADDRESS_LIST =
         (ADDRESS = (PROTOCOL = TCP)(HOST = <hostname>)
            (PORT = 1521))
      )
   )
)
SID_LIST_LISTENER =
   (SID_LIST =
      (SID_DESC =
         (GLOBAL_DBNAME = <SID>)
         (ORACLE_HOME = /<path>/oracle/ora92)
         (SID_NAME = <SID>)
      )
   )

Start the Listener by using the lsnrctl utility. Enter Listener control commands within the utility or as a parameter to the utility. The commands lsnrctl start starts the Listener and lsnrctl stop stops the Listener. The command lsnrctl status will show status of database services served by the Listener. Typing help within the Listener control utility will display a summary of all available Listener control commands.

Configuring the Client

<TNS Name> =
   (DESCRIPTION =
      (ADDRESS_LIST =
         (ADDRESS = (PROTOCOL = TCP)(HOST = <hostname>)
            (PORT = 1521))
      )
      (CONNECT_DATA = (SERVICE_NAME = <SID>)
      )
   )

Shown above is a simple example tnsnames.ora file placed on a client machine. To test the connection with the database server execute the command tnsping <TNS Name> from a shell on the client machine.

There is little "tuned" about these two network configuration files here. Much of tuning the performance of Oracle networking software involves tweaking of these two and other configuration files, depending on requirements. Make sure all the parentheses match and are in the correct places otherwise you will not be able to connect to the database. There are other options such as Oracle Names and Connection Manager allowing for scalability but not necessarily better performance.

11.2.2 The Parameter File

This section includes a simple example Oracle instance configuration parameter file with numerous comments. I use a file such as this when creating a database manually. The parameter file is usually placed in the $ORACLE_HOME/database directory or otherwise in the $ORACLE_BASE/admin/<SID>/pfile directory with a pointer file in the $ORACLE_HOME/database directory. This example is a text format version of the parameter file, which in Oracle9i Database and after can also be stored as a binary file called the SPFILE file. The SPFILE binary file was introduced in Oracle9i Database and allows changing of many configuration parameters with the database online.

Tip

The parameter file contains Oracle Database parameter values overriding default values. The parameter file is parsed and loaded when the Oracle instance is started up. If the parameter file contains errors the database will not start!
Note

Oracle Database 10 Grid Configuration parameters have been simplified in Oracle Database 10g by categorizing them. For the simplest configurations only the basic parameters need to be changed. More advanced parameters can be altered for more complex and demanding environments. Many parameters have been greatly simplified, particularly the shared server parameters.

The following description of the parameter file contains descriptions of different variations for different types of databases plus notes describing various settings. Comment out or delete the notes if you want to use this parameter file. There are many more configuration parameters in Oracle Database. The following query finds all the parameters and their settings.

SELECT name, value FROM v$parameter ORDER BY name;

Let's look at some of the configuration parameters.

Database Identification

db_name = "<SID >"
db_domain = ""
instance_name = <SID >
service_names = <SID >

Control Files

Place multiple control files onto different mount points to ensure that all control files are not lost. This is called multiplexing of control files and has negligible effect on performance.

control_files =
(
      "/<mp1>/oracle/oradata/<SID>/control01a.ctl"
      ,"/<mp1>/oracle/oradata/<SID>/control02a.ctl"
      ,"/<mp2>/oracle/oradata/<SID>/control01b.ctl"
      ,"/<mp2>/oracle/oradata/<SID>/control02b.ctl"
)

Block Size

OLTP databases are typically set to a block size of 8K. Increase the block size to 16K or even 32K for data warehouses and read-only type databases. Block sizes of 2K or 4K are not common even for OLTP databases except in older versions of Oracle Database.

db_block_size = 8192
Tip

Available block sizes may vary for different operating systems.

Block checking can cause a performance overhead. It can help prevent data block level corruption. Data corruption is more common when you have not spent enough money on disk storage media. Use of RAID arrays makes use of parameters such as this unnecessary.

db_block_checking = TRUE

Memory Buffers and I/O Behavior

The Database Buffer Cache

Oracle9i Database replaces the DB_BLOCK_BUFFERS parameter with the DB_CACHE_SIZE parameter. The database buffer cache should not be sized to a ridiculously high amount. A very high database buffer cache hit ratio is not an indicator of good performance. Very large database buffer cache sizes can actually hurt performance and sometimes be an indicator of poor performance.

db_cache_size = 80M

Separate database buffer caches must also be set for table-spaces of non-standard database block size. Thus with a block size of 8K, DB_2K_CACHE_SIZE, DB_4K_CACHE_SIZE, DB_16K_CACHE_SIZE, and DB_32K_CACHE_SIZE parameters with their related tablespaces can be set.

db_2K_cache_size = 2M
db_4K_cache_size = 4M
db_16K_cache_size = 16M
db_32K_cache_size = 32M
Tip

Available block sizes and thus DB_nK_CACHE_SIZE parameters may vary for different operating systems. A 32K block size is not available for Win2K SP3 running Oracle9i Database Release 2 (9.2).

Multiple blocks can be read at once. Typically OLTP databases are set to 8K or lower and data warehouses are set to 16K or higher. Setting the DB_FILE_MULTIBLOCK_READ_COUNT parameter to 16 would read 16 blocks at once for every read of database buffer cache or disk. This parameter must remain small for OLTP databases since it can make full table scans more palatable to the Optimizer. Multiple physically contiguous blocks (next to each other) are read at once. Data warehouses can benefit substantially from reading many blocks at once since they are more likely to have data physically sorted depending on how tables are created. Obviously reading large numbers of blocks at once in a highly active OLTP database could have ill effects; it is unlikely even when large chunks of data are required that everything will be physically contiguous.

db_file_multiblock_read_count = 8

The DB_KEEP_CACHE_SIZE and DB_RECYCLE_CACHE_SIZE parameters are parameters allowing forcing of MRU (most recently used) data to be retained and LRU (least recently used) data to be discarded with less interference from other data in the main database buffer cache. Depending on the size and function of the database, these parameters could be set at 5% and 25% of the DB_CACHE_SIZE parameter, respectively. These parameters must be a multiple of the standard block size as defined by the DB_BLOCK_SIZE parameter.

db_keep_cache_size = 5M
db_recycle_cache_size 20M

The FAST_START_MTTR_TARGET parameter is a crash recovery parameter.

fast_start_mttr_target = 300

The Shared Pool

The shared pool contains the library cache, the dictionary cache, language character set, and shared server queues. Shared server queues can contain some session connection memory. The library cache consists of parsed SQL code, optimization plans, and latches. The dictionary cache contains database metadata. Metadata is the data about the data. Like the database buffer cache the shared pool is often sized much too large, often because of SQL code lacking bind variables. SQL code without use of bind variables forces constant re-parsing, making reuse of SQL code non-existent and increasing use of CPU time substantially.

shared_pool_size = 80M

A small portion of the shared pool can be reserved for large operations creating a contiguous section of the buffer. This helps to avoid defragmentation in the shared pool by large operations, minimizing latch problems and a lot of bouncing all over a large, possibly highly fragmented buffer. Typically the SHARED_POOL_RESERVED_SIZE parameter is set within 5% and 10% of the size of the shared pool, but can be sized larger if circumstances and applications warrant it.

shared_pool_reserved_size = 8M

The processes parameter limits database connections, stored in the shared pool. Oracle8i Database required a minimum of 55 processes with 200 processes required when using Oracle Enterprise Manager software. Oracle9i Database and Oracle Database 10g values should be set higher as concurrent connections grow. The maximum processes for a dedicated server environment recommended by Oracle Corporation was set around 500. On the contrary, dedicated server environments can exceed 1,000 concurrent connections and still function efficiently. To get an idea of process memory usage there is 1 process per connection in a dedicated server environment. For 100 concurrent connections at 300K per connection, 30M of memory would be required. For 500 concurrent connections at 300K per connection 150M of memory would be required. A shared server configuration may decrease the amount of connection memory required because there is sharing of resources with sharing of server processes.

processes = 500

Connection Memory Requirements

For dedicated servers set the WORKAREA_SIZE_POLICY parameter to AUTO and set the PGA_AGGREGATE_TARGET parameter for automated session connection memory requirements. In the case of a shared server the WORKAREA_SIZE_POLICY parameter must be set to MANUAL.

Setting the SORT_AREA_SIZE parameter to 1M with 1,000 connections could claim 1 Gb of memory if every connection was sorting at once. To reiterate, 1 Gb of memory would only be claimed for sorting if every connection was performing an in-memory sort at the same time. This is unlikely but it is possible depending on what applications do. Each user connection is allocated up to a SORT_AREA_SIZE parameter specified chunk of memory whenever a sort is performed. Do not make the SORT_AREA_SIZE parameter too large because it will use a lot of memory if there are many concurrent connections performing sorting. Reporting, lots of joins, bad SQL code, and data warehousing could very well cause lots of sorting.

sort_area_size = 256K

The SORT_AREA_RETAINED_SIZE parameter is recommended to be set to 10% of SORT_AREA_SIZE when most transactions are small and set equal to SORT_AREA_SIZE when many large transactions are the norm. A small portion of memory sort space is retained by the connection for queries containing multiple sorts. This parameter can potentially speed up some queries, probably complex, poorly tuned or very large queries.

sort_area_retained_size = 6K

Other parameters in this group are as follows. These parameters are best not altered without proper planning and thought. Making the HASH_AREA_SIZE parameter larger tends to be useful when trying to persuade the Optimizer to favor hash joins rather than nested loop joins. I do not see the point because I never assume that I am more intelligent than the Optimizer unless statistics are completely useless, especially in Oracle9i Database and beyond.

  • HASH_AREA_SIZE. Used for SQL code Optimizer hash joins on larger row sets.

  • BITMAP_MERGE_AREA_SIZE. Bitmap merges.

  • CREATE_BITMAP_AREA_SIZE. Creating bitmaps.

The WORKAREA_SIZE_POLICY = AUTO and PGA_ AGGREGATE_TARGET parameters are intended as replacements for the previous five parameters controlling session shared pool memory usage. Oracle Corporation recommends use of these parameters in Oracle9i Database to automate PGA memory management.

Tip

WORKAREA_SIZE_POLICY = AUTO will only function in a dedicated-server not a shared-server environment.

The OPEN_CURSORS parameter sets a limit on the maximum number of open cursors allowed per session. Setting this parameter is application dependent and being a limitation does not reserve space for each session.

open_cursors = 100

The Large Pool

The large pool is required for use of tools such as RMAN and parallel queries, and could be set to much less than 200M, often much less.

Shared server configuration pushes some connection session memory requirements from the shared pool into the large pool. Increase large pool size to accommodate use of shared servers. The shared pool can be decreased accordingly.

large_pool_size = 200M

The Java Pool

The Java pool is used for the Oracle JVM if you choose to write stored procedures in Java. This parameter must be set if the Oracle JVM is installed into a database. Otherwise this parameter can be set to 0 and is a complete waste of resources unless Java is being used to write stored procedures.

java_pool_size = 0
Note

Oracle Database 10 Grid JAVA_POOL_SIZE can be altered using the ALTER SYSTEM command and a binary parameter file, SPFILE.

The Redo Log Buffer Cache

Oracle9i Database defaults the log buffer cache to 512K. This could be too high for low-activity OLTP databases but it is not necessary to change the default. Perhaps set the LOG_BUFFER parameter higher for the biggest, highly concurrent databases or large transactional databases with a lot of throughput and throughput changing rather than just reading data.

log_buffer = 524288

Small increases in the size of the log buffer can occasionally produce big increases in performance. However, I have seen databases with 500 Mb log files and log buffer caches of 100 Mb. Suffice it to say that these databases had serious performance problems, not only with little use of the log buffer cache but a lot of time spent copying redo log files to archive logs. The log buffer cache does not need to be enormous; it is recycled rapidly and written to disk.

Duplexing of redo logs is efficient, especially on servers with more than one CPU, and will generally not affect performance. It is always prudent to create multiple copies of redo logs no matter what your disk storage structure is. Redo logs files cannot be included in backups. Duplexing the redo logs provides for a second copy.

SQL Code Optimization

There are various parameters classifiable as influencing the Optimizer.

  • CURSOR_SHARING. This parameter is a temporary solution helping to alleviate performance problems when not using bind variables in SQL code. The CURSOR_SHARING parameter can be set to SIMILAR or FORCE, partially deteriorating the accuracy of statistics usage. The FORCE option will parse only the first SQL statement submitted. Since all literals are replaced with bind variables, subsequent submissions of similar SQL code may not match the current execution plan very well at all, depending on the structure of data. The SIMILAR option is the same as FORCE except histograms are used to help tune the execution plan better than when using the FORCE option. Use CURSOR_SHARING settings of FORCE or SIMILAR in OLTP systems but not in data warehouses. A data warehouse database will perform better using exact matches on statistics with literal values in SQL code since data volumes are much greater. In a data warehouse environment, perhaps set the CURSOR_SHARING parameter to EXACT.

  • HASH_JOIN_ENABLED. This parameter must be set to TRUE to allow use of hash joins in SQL code join optimization. It is extremely unlikely that any OLTP database would ever need to only use sort merge joins unless your OLTP database has very poorly tuned SQL join code or there is extensive reporting. If this parameter is not set to TRUE inefficient sort merge joins will always be used in favor of hash joins. Sort merge joins are very bad for performance. Hash joins are potentially much faster than sort merge joins.

  • QUERY_REWRITE_ENABLED. Set to TRUE to enable function-based indexing and query rewrites when using materialized views. Materialized views are generally only used in data warehouses.

  • QUERY_REWRITE_INTEGRITY. Set to TRUSTED to enable function-based indexing.

  • STAR_TRANSFORMATION_ENABLED. Set to TRUE in data warehouses to enable star query transformations.

  • PARALLEL_AUTOMATIC_TUNING. Set to TRUE if you have more than one CPU. If there is only one CPU this parameter can still help to automate semi-parallel processed Optimizer tuning. Rely on the Optimizer; setting this parameter to TRUE helps the Optimizer.

    Note

    Oracle Database 10 Grid The PARALLEL_AUTOMATIC_TUNING parameter is deprecated.

  • COMPATIBLE. Set to the version of Oracle Database installed to take advantage of newly introduced optimization features.

    compatible = 9.2.0.1.0
    Tip

    Some of the connection memory parameters, named as %_AREA_SIZE could also be included here since they can affect Optimizer behavior.

Auditing and Tracing

Auditing and tracing allows for tracking, auditing, and tuning of database use.

Auditing

Database auditing can be performed at the operating system, database or even DML statement history levels. Switch auditing off if it is no longer required. It is a performance hog!

audit_trail = DB

Tracing

Tracing will affect performance drastically. I like to categorize levels of tracing into six groups:

  1. No tracing.

    timed_statistics = TRUE
timed_os_statistics = 0
sql_trace = FALSE
    Tip

    The overhead for setting the TIMED_STATISTICS = TRUE parameter is negligible.

  2. Some tracing.

    timed_statistics = TRUE
timed_os_statistics = 5
sql_trace = FALSE
max_dump_file_size = 1M
statistics_level = BASIC or TYPICAL

  3. SQL tracing.

    timed_statistics = TRUE
timed_os_statistics = 5
sql_trace = TRUE
max_dump_file_size = 1M
statistics_level = TYPICAL

  4. Too much tracing.

    timed_statistics = TRUE
timed_os_statistics = 5
sql_trace = TRUE
max_dump_file_size = 1M
statistics_level = ALL

  5. Oracle support tracing. This level of tracing you should never need. It produces a large amount of superfluous information and is utilized by setting the TRACE_ENABLED parameter to TRUE. This option is intended for the use of Oracle Corporation in analyzing potential bugs in Oracle software.

    trace_enabled = FALSE
    Note

    Oracle Database 10 Grid Oracle Trace and all ORACLE_TRACE_% parameters have been deprecated.

  6. Session-level tracing. Specific sessions can be traced. Tracing at the session level can help to remove tracing overhead from the database in general and focus on a specific session.

    However, the value of session-level tracing is relative to how easy it is to track when poorly performing SQL code occurs. It is true that instance-level tracing is instantiated for the entire database and can hurt performance severely. Session-level tracing is easier to use in large transactional environments as opposed to sharing of sessions between shared servers in an OLTP environment. Using the Oracle Database Wait Event Interface may provide better possibilities for finding problems at this level of detail.

Archiving, Check Points, the Alert Log, and Trace Files

Archive Logs

log_archive_start = true
log_archive_dest_1 = 'LOCATION =<mp1>/oracle/oradata/<SID>/
   archive'
log_archive_format = Arc_%s.arc
log_archive_start = false

Check Pointing

A checkpoint forces writing of all dirty buffer cache blocks to data files. Setting the LOG_CHECKPOINT_INTERVAL parameter to 0 will ignore checkpoints altogether.

log_checkpoint_interval = 0

Checkpoints are executed every 5 min with this setting.

log_checkpoint_timeout = 300
The Alert Log and Trace Files

The BACKGROUND_DUMP_DEST parameter determines the location of the alert log file and process trace files. User trace files are placed in the directory specified in the USER_DUMP_DEST parameter.

background_dump_dest = /<mp1>/oracle/admin/<SID>/bdump
user_dump_dest = /<mp1>/oracle/admin/<SID>/udump

Rollback and Undo

Oracle9i Database and beyond allows for automated undo in addition to manually configured rollback segments. My test database with my Accounts schema uses both manual and automated undo at different points in time. This book contains tuning information on both since many existing pre-Oracle Database 10g installations use manual rollback. Growth appears to be properly controlled using automated undo segments in highly active concurrent OLTP databases. Automated undo should be used if possible in Oracle9i Database.

Note

Oracle Database 10 Grid Manual rollback is deprecated.

Manual Rollback Segments

OLTP databases require many small rollback segments. Many small rollback transactions for an OLTP database will increase database performance drastically for an OLTP database because transactions will spread evenly across all online rollback segments. It may even be intelligent to make many rollback segments as small as 16K initial and next. Set optimal size depending on transaction size.

Tip

Separate large rollback segments placed offline by default can be placed online and utilized for large transactions, and then placed offline on transaction completion.

OLTP databases can perform well with a large number of small rollback segments. The ROLLBACK_SEGMENTS parameter forces rollback segments online at database startup and is advisable even for public rollback segments. Otherwise unused rollback segments may never be placed online and used. The number of rollback segments placed online at startup is dependent on potential transactions, not the sizes of rollback segments. When using large numbers of rollback segments make sure that next extent values are very small to avoid rollback segments sharing too many transactions at once. The idea is to separate small transactions into separate rollback segments to reduce contention between different transactions using the same rollback segment.

max_rollback_segments 50
rollback_segments =
(
       RB00,RB01,RB02,RB03,RB04,RB05,RB06,RB07,RB08,RB09
      ,RB10,RB11,RB12,RB13,RB14,RB15,RB16,RB17,RB18,RB19
      ,RB20,RB21,RB22,RB23,RB24,RB25,RB26,RB27,RB28,RB29
      ,RB30,RB31,RB32,RB33,RB34,RB35,RB36,RB37,RB38,RB39
      ,RB40,RB41,RB42,RB43,RB44,RB45,RB46,RB47,RB48,RB49
)
Note

Oracle Database 10 Grid Manual rollback is deprecated.

Automated Undo Segments

Databases with small transactions can have lower values for the UNDO_RETENTION parameter.

undo_management = AUTO
undo_retention = 900
undo_tablespace = UNDOTBS
Note

Oracle Database 10 Grid The UNDO_SUPPRESS_ERRORS parameter is deprecated.

Resource Management

Resource Management allows the distribution of resources such as CPU time between different groups of sessions. The RESOURCE_MANAGER_PLAN parameter sets the default resource management plan or group. Resource plans can be altered depending on processing requirements during the course of a day, allowing for variable performance requirements. For instance, during high activity times an OLTP grouping plan can be used allowing for fast reaction times and during low activity times a BACKUP or batch-type grouping could be used allowing for faster throughput.

resource_manager_plan = 'OLTP'

Job Scheduling

This parameter has an effect on how many jobs can be scheduled and executed at once. I have some highly active code executed on my database using the DBMS_JOBS package to produce a tunable database for writing this book. This code kicks off between 10 and 100 jobs per second in order to produce highly concurrent activity. The intention was to create a tunable database. The default value for the JOB_QUEUE_PROCESSES parameter is 10.

job_queue_processes = 10

Networking

Networking parameters only affect Oracle shared servers (formerly MTS or Multi-Threaded Server). Shared servers allows for connection pooling, load balancing, and sharing of database connections. Shared servers perform the same function that an application server or web server would. A word of advice: If you can afford a connection pooling application server or web server avoid shared servers; they have not necessarily always been problematic but perhaps misused and misunderstood.

Tip

Shared servers were called MTS or Multi-Threaded Server prior to Oracle9i Database.

dispatchers = "(PROTOCOL = TCP) (DISPATCHERS = 1) (POOL = ON)
   (PORT = 1521)"
shared_servers = 2
circuits = 10
shared_server_sessions = 10
max_dispatchers = 2
max_shared_servers = 5
Note

Oracle Database 10 Grid Shared server parameter configuration is simplified.

A simple explanation of shared servers is as shown in Figure 11.1. In a dedicated server environment, client sessions connect directly to the database via a dedicated server process. In a shared servers configuration, client processes can share connections to shared server processes, having messages passed among themselves and those shared server processes using dispatcher processes. Shared servers can therefore service a lot more connections since idle sessions are not always connected to the database, particularly when a client connection is idle. Idle connections through dedicated server processes waste resources better used otherwise. A shared server is more scalable than a dedicated server environment in terms of the number of concurrent connections to the database.

Click To expand
Figure 11.1: Dedicated and Shared Servers Architecture


11.3 Creating a Database

Like initial Oracle binary software installation, well-structured and thought out initial database creation is essential to a properly physically tuned Oracle database. There are two different methods of creating a database. The first is using the Database Configuration Assistant and the second manually using command line scripts. The Database Configuration Assistant is easy to use and can be utilized to create a set of adjustable scripts. Let's look at the Database Configuration Assistant first.

11.3.1 The Database Configuration Assistant

I generally use the Database Configuration Assistant in two ways: firstly, when creating a database initially using a new version of Oracle Database and secondly, as a database creation script generator.

Tip

Selecting the New Database option as shown in Figure 11.2 will allow selection of explicit options for database creation, one of the most important being the removal of all the sample tablespaces as shown in Figure 11.3. Extra options shown in Figure 11.4 such as Oracle JVM and Intermedia can also be removed and added later using $ORACLE_HOME/rdbms/admin scripts if required.
Click To expand
Figure 11.2: Choosing the Appropriate Database Template
Click To expand
Figure 11.3: Options and Examples when Creating a Database
Click To expand
Figure 11.4: More Optional Database Features

Many of the steps involved in database creation are database administration and not tuning functions. However, in the interest of creating a properly physically and configured database architecture it is prudent to go through some of the steps of the Database Configuration Assistant.

Make sure you select the correct database creation template. An OLTP database is equivalent to a Transaction Processing database as shown in Figure 11.2. However, in order to create a database and remove the example schemas it is best to select the New Database option.

Figure 11.3 shows various options which can be removed from a database creation. All options can be retained. Bear in mind that general use of added options could incur higher Oracle software licensing fees.

Figure 11.4 allows selection of standard included optional features. None of these options absolutely has to be retained; their selection depends on application requirements.

Even for highly active, concurrent, very large OLTP databases dedicated servers are still commonly used, sometimes with as many as 1,000 concurrent connections. A shared server configuration will ultimately be more scalable. Once again if your database concurrency is high it may be more cost effective in the long term to invest in a connection pooling application or web server machine, allowing more performance capacity for your database itself. The choices of dedicated and shared server modes are shown in Figure 11.5.

Click To expand
Figure 11.5: Choosing Dedicated or Shared Servers

Oracle Installer tends to maximize the use of memory resources on a database server, to the detriment of everything else on that server. Figures 11.6 and 11.7 show that I have altered buffer values to very low values. I am running Oracle Database on a low-powered, single CPU, Pentium II, Intel Windows 2K box. Additionally I have multiple databases running on that machine.

Click To expand
Figure 11.6: Oracle Database Memory Buffer Parameters
Click To expand
Figure 11.7: The Sort Buffer

Note in Figure 11.6 that PGA is automatically entered. If this value is set to 0 the tool will prompt to set the PGA value to an amount greater than 0 when clicking the next tab. Note that in Figure 11.5 that Shared Server mode is selected. Shared servers do not allow use of automatically managed PGA cache in Oracle9i Database.

I have two gripes with using this tool to create databases.

My first gripe is as shown in Figure 11.8 showing the database creation tool creating a large number of tablespaces containing numerous different examples and Oracle options. These take up space and too many tablespaces is simply confusing. If you were to count all the users after creating a database with this tool you should find that Oracle9i Database Release 2 (9.2) creates almost 30 users, using all the extra tablespace datafiles shown in Figure 11.8. The most essential usernames in a newly created Oracle database are SYS, SYSTEM, probably OUTLN, and perhaps DBSNMP and maybe RMAN.

Click To expand
Figure 11.8: Creation of Tablespaces and Datafiles
Note

Oracle Database 10 Grid My first gripe has been resolved by the addition of a system-level auxiliary tablespace. This is very pleasing indeed. The SYSAUX tablespace will be created automatically along with the SYSTEM tablespace. The SYSAUX tablespace contains options and tools such as RMAN tools, Auditing, an Oracle Enterprise Manager repository (like the RMAN repository, best not included with the production database), plus various add-on Oracle options such as OLAP, Data Mining, XML, and many others.

My second gripe with the Database Configuration Assistant is as shown in Figure 11.9, where only three large redo log files are created. Rarely can an OLTP database survive with less than at least five and sometimes even as many as 10 redo log files. Also the redo logs are created at 100 Mb each. 100 Mb is a very big redo log file and can take time to archive, especially on a Windows box.

Click To expand
Figure 11.9: Creation of Redo Logs

A very active DML OLTP database with only three very large redo log files can result in the database periodically halting service whilst all redo log files are archived. Additionally redo log files are not duplexed and they probably should be for the sake of recoverability.

Current Trends

One particularly useful aspect of the Database Configuration Assistant is that it can be used to show current trends for Oracle database creation. New things recently introduced definitely work properly when used for the SYSTEM tablespace. Some of these physical aspects affect much of physical architecture even to the level of logical objects such as tables and indexes. Oracle8i Database defaulted to creation of all tablespaces as dictionary-managed except for the temporary sorting tablespace. Oracle9i Release 1 (9.1) made all tablespaces locally managed except for the SYSTEM tablespace. As can be seen in Figure 11.10 the recommended physical structure in Oracle9i Release 2 (9.2) is for all tablespaces to be locally managed with automatic allocation and automatic segment space management.

Click To expand
Figure 11.10: Physical Datafile Default Structure

Now let's take a look at manual database creation using command line instructions and scripting.

11.3.2 Manual Database Creation

The best way to show how to create a database manually is to present a set of scripts and describe the pieces. These scripts are by no means executable as they are, as with any of the scripts in this book. However, these scripts should give an overall picture as to how to create a physically neat and properly structured database which is easy to tune.

Tip

Scripts presented here were originally built for a Solaris box Oracle8i Release 2 (8.1.7) installation and upgraded for Oracle9i Release 2 (9.2). The Database Configuration Assistant can be used to generate scripts like these, which can then be altered to suit requirements. They are presented merely to show low-level physical structural details and are not necessarily applicable to the latest version of Oracle Database.

Create the Directory Structure

This script creates the directory structure for an Oracle database. This script is generally in compliance with the Oracle Optimal Flexible Architecture (OFA). A few other little bits and pieces are included such as extra variables, SQL*Net configuration files, most importantly a pre-prepared configuration parameter file, and the Oracle password file.

Tip

Once again scripts contained in this book are "as is". Please do not simply type them in without testing first, especially any Unix or Perl scripting.

#!/bin/ksh
ORACLE_BASE=/<mp1>/oracle
ORACLE_HOME=$ORACLE_BASE/ora92
TNS_ADMIN=/<mp1>/oracle/ora92/network/admin
ORACLE_SID=<SID>
ORACLE_DBF1=/<mp1>/oracle/oradata/$ORACLE_SID
ORACLE_DBF2=/<mp2>/oracle/oradata/$ORACLE_SID
ORACLE_BACKUPS=/<mp2>/backups
ORACLE_SBIN=$ORACLE_HOME/sbin
ORACLE_ALERT=$ORACLE_BASE/admin/$ORACLE_SID/bdump
PATH=/bin:/usr/bin:/etc:/usr/ccs/bin:/usr/openwin/bin:/usr
/ucb:/usr/local/bin:/usr/sbin:/usr/X/bin
PATH=${PATH}:$ORACLE_HOME/bin:$ORACLE_HOME/sbin:$JAVA_HOME
USAGE="$0: Incorrect arguments, Usage: $0 <password>"
if [ -z "$1" ]; then
      echo "$USAGE"
      exit 1
fi
if [ '/usr/ucb/whoami' != "oracle" ]; then
      echo "Must be oracle"
      exit 1
fi
orapwd file=$ORACLE_HOME/dbs/orapw$ORACLE_SID password=$1
entries=2
ln -s $ORACLE_BASE/admin/$ORACLE_SID/pfile/init$ORACLE_SID.ora
$ORACLE_HOME/dbs/init$ORACLE_SID.ora
mkdir $ORACLE_BASE/admin
mkdir $ORACLE_BASE/admin/$ORACLE_SID
mkdir $ORACLE_BASE/admin/$ORACLE_SID/adhoc
mkdir $ORACLE_BASE/admin/$ORACLE_SID/arch
mkdir $ORACLE_BASE/admin/$ORACLE_SID/bdump
mkdir $ORACLE_BASE/admin/$ORACLE_SID/cdump
mkdir $ORACLE_BASE/admin/$ORACLE_SID/create
mkdir $ORACLE_BASE/admin/$ORACLE_SID/exp
mkdir $ORACLE_BASE/admin/$ORACLE_SID/pfile
mkdir $ORACLE_BASE/admin/$ORACLE_SID/udump

mkdir $ORACLE_BASE/oradata
mkdir $ORACLE_BASE/oradata/$ORACLE_SID
mkdir $ORACLE_BASE/oradata/$ORACLE_SID/archive
mkdir /<mp2>/oracle/oradata
mkdir /<mp2>/oracle/oradata/$ORACLE_SID
mkdir $ORACLE_BACKUPS/$ORACLE_SID

cp ./listener.ora $TNS_ADMIN
cp ./tnsnames.ora $TNS_ADMIN
cp ./init$ORACLE_SID.ora
$ORACLE_BASE/admin/$ORACLE_SID/pfile
$ORACLE_HOME/bin/lsnrctl start

Create the Database

This script creates a SYSTEM tablespace and a single set of five duplexed redo log files of 10 Mb each. These redo log files are enough for a small-load OLTP database, perhaps even a little on the large size. A highly active large OLTP or data warehouse database would have many more, much larger redo logs.

Additionally I am not using a binary parameter file (SPFILE) in this case. Using a binary parameter file is very useful for maintaining high availability as it allows for changing of parameters online using the ALTER SYSTEM command. A deeper discussion on the subject of the SPFILE is database administration, not tuning, and therefore inapplicable to this book. The only problem I have found using a binary parameter file is if the file gets lost. Some clients tend to forget to back up binary files or make textual parameter file copies.

When something goes wrong with parameter settings they sometimes forget what they have changed online using the ALTER SYSTEM command.

Lastly, I create a temporary rollback segment. I am not using automated undo segments on my test database at this point. My test database is a very old server and has a serious lack of disk space; manual rollback segments are easier to control in this respect. Both automated undo and manual rollback are being covered in this book due to the existence of many pre-Oracle Database 10g installations.

spool ../logs/system.log;
connect sys/<password> as sysdba
startup nomount pfile =
   "/<mp1>/oracle/admin/<SID>/pfile/init<SID>.ora"
CREATE DATABASE "<SID>"
        maxdatafiles 400
        maxinstances 4
        maxlogfiles 16
        maxlogmembers 4
        maxloghistory 200
        character set US7ASCII
        national character set US7ASCII
        archivelog
DATAFILE '/<mp1>/oracle/oradata/<SID>/system01.dbf' SIZE 450M
      AUTOEXTEND ON NEXT 1M MAXSIZE UNLIMITED
      EXTENT MANAGEMENT LOCAL UNIFORM SIZE 1M
DEFAULT TEMPORARY TABLESPACE TEMP TEMPFILE
      '/<mp1>/oracle/oradata/<SID>/temp01.dbf' SIZE 250M
      AUTOEXTEND ON NEXT 1M MAXSIZE UNLIMITED
      LOGFILE GROUP 1 ('/<mp1>/oracle/oradata/<SID>/redo01a.log',
      '/<mp1>/oracle/oradata/<SID>/redo01b.log') SIZE 10M,
      GROUP 2 ('/<mp1>/oracle/oradata/<SID>/redo02a.log',
      '/<mp1>/oracle/oradata/<SID>/redo02b.log') SIZE 10M,
      GROUP 3 ('/<mp1>/oracle/oradata/<SID>/redo03a.log',
      '/<mp1>/oracle/oradata/<SID>/redo03b.log') SIZE 10M,
      GROUP 4 ('/<mp1>/oracle/oradata/<SID>/redo04a.log',
      '/<mp1>/oracle/oradata/<SID>/redo04b.log') SIZE 10M;
      GROUP 5 ('/<mp1>/oracle/oradata/<SID>/redo05a.log',
      '/<mp1>/oracle/oradata/<SID>/redo05b.log') SIZE 10M;

create rollback segment rbtemp tablespace system
storage (initial 64k next 64k minextents 2 maxextents 300);
alter rollback segment rbtemp online;

disconnect;
spool off;
Note

Oracle Database 10 Grid Manual rollback is deprecated.

If using automated undo include the following command in the previous script immediately following the command creating the temporary tablespace.

UNDO TABLESPACE UNDOTBS1 DATAFILE
      '/<mp1>/oracle/oradata/<SID>/undotbs01.dbf' SIZE 250M
      AUTOEXTEND ON NEXT 1M MAXSIZE UNLIMITED

Tablespace and Rollback Creation

The script in this section creates a lot of tablespaces plus manual rollback segments. In general, I have created a tablespace structure based on an expected rapid database growth rate rather than potential application functionality. This script is out of date intentionally to show the basics and is commented inline for clarity.

spool ../logs/tbs.log;
connect sys/<password> as sysdba

In the latest version of Oracle9i Database all tablespaces are recommended as being locally managed. The command following assumes the SYSTEM tablespace is dictionary managed.

Tip

Creation of a logical standby database requires use of a backup control file, which is problematic in Oracle9i Database with a default temporary tablespace created for a database with a locally managed SYSTEM tablespace.

Dictionary managed tablespaces will be phased out in a future version of Oracle Database but if not then they still have potential uses for very fine physical tuning. For both tablespace types, but more so for dictionary managed tablespaces, consistent sizes for all datafiles tends to minimize space wastage by allowing reuse of freed extents. New larger extents will not fit into smaller, older extents. This can result in a lot of wasted space through fragmentation not even recoverable by coalescence. Even automatic extent allocations for locally managed tablespaces have default uniform sizes of 1 M. I have not used automatic allocation of extents in these scripts.

Most OLTP databases grow at rapid rates. Small, variable extent sizes can lead to huge numbers of extents in time. Smaller tables can be placed into smaller extent sized tablespaces if so desired.

alter tablespace system default storage
(initial 1M next 1M minextents 1 maxextents unlimited
  pctincrease 0);
alter tablespace system minimum extent 1M;

Temporary sort space is allocated to the database using the default temporary sort space. This is an Oracle9i Database change and a very good one at that. In the past many databases used the SYSTEM tablespace for sorting. Many database administrators and developers were unaware of having to assign a user a temporary table-space for on disk buffer overflow sorting purposes when creating a user.

-Temporary sort
create temporary tablespace temp
      tempfile '/<mp1>/oracle/oradata/<SID>/temp01.dbf'
      size 300M autoextend on next 1M maxsize unlimited
      extent management local uniform size 1M;
ALTER DATABASE DEFAULT TEMPORARY TABLESPACE TEMP;

Oracle Enterprise Manager requires its own tablespace. It is best to install the Oracle Enterprise Manager repository in a separate database on a separate machine but licensing and security issues could make this a problem. Perhaps a development or a test database is a good place for an Oracle Enterprise Manager repository or even in the same database as an RMAN repository. An RMAN repository should be in a database separate to that of a production database; an RMAN database must be backed up separately from production databases. RMAN requires a backup of itself when not using a control file for the RMAN repository.

Note

Oracle Database 10 Grid The Oracle Enterprise Manager repository is included in the SYSAUX tablespace.

-Oracle Enterprise Manager
create tablespace oem datafile
      '/<mp1>/oracle/oradata/<SID>/oem01.dbf'
      size 500M
      AUTOEXTEND ON NEXT 1M MAXSIZE UNLIMITED
      EXTENT MANAGEMENT LOCAL UNIFORM SIZE 1M;

-Main rollbacks
create tablespace rbs1 datafile
      '/<mp1>/oracle/oradata/<SID>/rbs01.dbf'
      size 300M
      AUTOEXTEND ON NEXT 1M MAXSIZE UNLIMITED
      EXTENT MANAGEMENT LOCAL UNIFORM SIZE 1M;
Note

Oracle Database 10 Grid Manual rollback is deprecated.

Note the different block size specifications of the next two large rollback and LOB tablespaces. A large block size would be relevant to the content of these tablespaces. The DB_32K_CACHE_SIZE parameter would have to be specified in the configuration parameter file, requiring bouncing of the database prior to creation of these tablespaces.

Tip

A 32K block size is not available for Win2K SP3 running Oracle9i Database Release 2 (9.2).

-Batch rollbacks
create tablespace rbs2 datafile
      '/<mp1>/oracle/oradata/<SID>/rbs02.dbf'
      size 320M BLOCKSIZE 32K
      AUTOEXTEND ON NEXT 1M MAXSIZE UNLIMITED
      EXTENT MANAGEMENT LOCAL UNIFORM SIZE 8M;
Note

Oracle Database 10 Grid Manual rollback is deprecated.

-LOBs
create tablespace objects datafile
      '/<mp1>/oracle/oradata/<SID>/objects01.dbf'
      size 96M BLOCKSIZE 32K
      AUTOEXTEND ON NEXT 1M MAXSIZE UNLIMITED
      EXTENT MANAGEMENT LOCAL UNIFORM SIZE 4M;

RMAN functionality is often placed into a tablespace called Tools by the Oracle Installer. As already stated RMAN is always best installed into a separate database on a separate server. Putting the RMAN repository on a production server in a production database can risk losing both production database and backup recoverability. In other words, everything!

-Stored procedures
create tablespace tools datafile
      '/<mp1>/oracle/oradata/<SID>/tools01.dbf'
      size 100M
      AUTOEXTEND ON NEXT 1M MAXSIZE UNLIMITED
      EXTENT MANAGEMENT LOCAL UNIFORM SIZE 1M;

Small static data tables go into this tablespace. Note the nonstandard block size of 2K. If the DB_2K_CACHE_SIZE parameter was not set, the database may have to be bounced prior to the creation of this tablespace. The tablespace cannot be created without the existence of a value in the correct cache parameter.

-Static datafiles
create tablespace datastat datafile
      '/<mp1>/oracle/oradata/<SID>/datastat01.dbf'
      size 1M BLOCKSIZE 2K
      AUTOEXTEND ON NEXT 32K MAXSIZE UNLIMITED
      EXTENT MANAGEMENT LOCAL UNIFORM SIZE 32K;

Let's say that transaction datafiles are expected to grow rapidly and will need to reuse space. There are multiple transaction table-spaces shown below. The separation of these tablespaces is based on growth rates and not application functionality. Many Internet startup companies using OLTP databases never got past the starting post. The startup companies that did succeed often had physical storage structure problems within a short period of time, especially when their databases grew rapidly. Some companies had rapid growth from a few megabytes up to a few hundred gigabytes and even more. There is one often forgotten fact. In the past migration between versions of Oracle Database required exporting from the old version and into the new version. The Oracle export and import utilities allow restructuring into a single extent for each datafile. It all depends on how fast a database grows. Is a database growing uncontrollably between new Oracle Database version releases?

Tip

Oracle9i Database and Oracle Database 10g have excellent interversion upgrade facilities.

-Transaction datafiles are separated based on expansion
  potential create tablespace datatrn1 datafile
      '/<mp1>/oracle/oradata/<SID>/datatrn101.dbf'
      size 50M
      AUTOEXTEND ON NEXT 1M MAXSIZE UNLIMITED
      EXTENT MANAGEMENT LOCAL UNIFORM SIZE 1M;
create tablespace datatrn2 datafile
      '/<mp1>/oracle/oradata/<SID>/datatrn102.dbf'
      size 100M
      AUTOEXTEND ON NEXT 1M MAXSIZE UNLIMITED
      EXTENT MANAGEMENT LOCAL UNIFORM SIZE 2M;
create tablespace datatrn4 datafile
      '/<mp1>/oracle/oradata/<SID>/datatrn104.dbf'
      size 200M
      AUTOEXTEND ON NEXT 1M MAXSIZE UNLIMITED
      EXTENT MANAGEMENT LOCAL UNIFORM SIZE 4M;

A tablespace such as this logging tablespace has rapid growth and generally involves only DML insertion and deletion activity plus of course selection of data. Thus its extent sizes are even larger than the transactional tablespaces.

create tablespace logs datafile
      '/<mp1>/oracle/oradata/<SID>/logtrn01.dbf'
      size 1000M BLOCKSIZE 32K
      AUTOEXTEND ON NEXT 1M MAXSIZE UNLIMITED
      EXTENT MANAGEMENT LOCAL UNIFORM SIZE 8M;

Tables and indexes should usually be separated into different tablespaces. Even when RAID arrays are used, index datafiles should be accessed sequentially and table datafiles should be accessed mostly in a random fashion from the index datafile. Some applications and database types require full scans of indexes. Potentially full scanned indexes could be placed into randomly accessed storage media structures as long as you are sure that those indexes are always accessed with physical fast full index scans.

The ratio between sizes of table and index tablespaces depends on how you index your data model with respect to Referential Integrity, use of sequence identifiers and how much alternate indexing there is. A properly designed OLTP data model will have a minimum of indexing and will likely use much less space for indexing than for tables. A data warehouse, more traditional or perhaps poorly designed OLTP database could need a lot more index space than table space. Some data models have many indexes for each table and most of those indexes are often composite-column indexes. These types of data models will usually have much bigger index tablespaces than table tablespaces. This structure tends to partially negate the performance benefits of indexing. As a side issue physical space problems are often common in OLTP databases using bitmap and not BTree indexes.

Once again the non-standard block size of 2K is used. The DB_2K_CACHE_SIZE parameter is required.

-Static indexes
create tablespace indxstat datafile
      '/<mp1>/oracle/oradata/<SID>/indxstat01.dbf'
      size 1M BLOCKSIZE 2K
      AUTOEXTEND ON NEXT 32K MAXSIZE UNLIMITED
      EXTENT MANAGEMENT LOCAL UNIFORM SIZE 32K;
-Transactional indexes
create tablespace indxtrn1 datafile
      '/<mp1>/oracle/oradata/<SID>/indxtrn101.dbf'
      size 50M
      AUTOEXTEND ON NEXT 1M MAXSIZE UNLIMITED
      EXTENT MANAGEMENT LOCAL UNIFORM SIZE 1M;
create tablespace indxtrn2 datafile
      '/<mp1>/oracle/oradata/<SID>/indxtrn201.dbf'
      size 100M
      AUTOEXTEND ON NEXT 1M MAXSIZE UNLIMITED
      EXTENT MANAGEMENT LOCAL UNIFORM SIZE 2M;
create tablespace indxtrn4 datafile
      '/<mp1>/oracle/oradata/<SID>/indxtrn401.dbf'
      size 200M
      AUTOEXTEND ON NEXT 1M MAXSIZE UNLIMITED
      EXTENT MANAGEMENT LOCAL UNIFORM SIZE 4M;

It is likely that a logging index tablespace will be smaller than a logging table tablespace. Thus this tablespace has a smaller initial size and extent incremental size than the logging table tablespace.

create tablespace logsidx datafile
      '/<mp1>/oracle/oradata/<SID>/logtidx01.dbf'
      size 200M
      AUTOEXTEND ON NEXT 1M MAXSIZE UNLIMITED
      EXTENT MANAGEMENT LOCAL UNIFORM SIZE 4M;
alter user sys temporary tablespace temp;
alter user system temporary tablespace temp;

In general, OLTP databases require many small rollback segments. Data warehouses and batch processing environments require a smaller number of the much larger rollback segments. The larger the transactions are, the larger rollback segments should be. Note that the MAXEXTENTS parameter is set for all rollback segments to prevent runaway transactions. If your customers are executing ad hoc queries or development is being done on a production database this is prudent.

create public rollback segment rb00 tablespace rbs1 storage
      (initial 256K next 256K optimal 512K minextents 2
        maxextents 64);
create public rollback segment rb01 tablespace rbs1 storage
      (initial 256K next 256K optimal 512K minextents 2
        maxextents 64);
create public rollback segment rb02 tablespace rbs1 storage
      (initial 256K next 256K optimal 512K minextents 2
        maxextents 64);
.
.
.
create public rollback segment rb49 tablespace rbs1 storage
      (initial 256K next 256K optimal 512K minextents 2
        maxextents 64);

Explicitly force all rollback segments online to make sure that Oracle Database is more likely to spread transaction load across all of those rollback segments.

alter rollback segment rb00 online;
alter rollback segment rb01 online;
alter rollback segment rb02 online;
.
.
.
alter rollback segment rb49 online;

Creating large rollback segments can help separate small OLTP functionality from batch processing.

create public rollback segment large00 tablespace rbs2
  storage (initial 8M next 8M optimal 16M minextents 2
  maxextents 32);
create public rollback segment large01 tablespace rbs2
storage
      (initial 8M next 8M optimal 16M minextents 2
        maxextents 32);
alter rollback segment large00 offline;
alter rollback segment large01 offline;

alter rollback segment rbtemp offline;
drop rollback segment rbtemp;

disconnect;
spool off;
Note

Oracle Database 10 Grid Note Manual rollback is deprecated.

The Catalog and the SPFILE

Creating the database catalog and the binary SPFILE parameter file is not directly relevant to tuning. A binary parameter file makes administration easier. The only danger is that it makes random "guesstimate" changes to configuration parameters easier to do. This is always dangerous. Research any changes first.

spool ../logs/catalog.log;
connect sys/<password> as sysdba;
-database creation of data dictionary views amongst many
  other scripts
@/<mp1>/oracle/ora92/rdbms/admin/catalog.sql;
-oracle lock views
@/<mp1>/oracle/ora92/rdbms/admin/catblock.sql;
-database creation of PL/SQL
@/<mp1>/oracle/ora92/rdbms/admin/catproc.sql;
-export utility ovm_sys user
@/<mp1>/oracle/ora92/rdbms/admin/owminst.plb;
-pinning objects
@/<mp1>/oracle/ora92/rdbms/admin/dbmspool.sql;
-heterogeneous services
@/<mp1>/oracle/ora92/rdbms/admin/caths.sql;
-prepare database for direct path SQL*Loader
@/<mp1>/oracle/ora92/rdbms/admin/catldr.sql;
-audit trails views
@/<mp1>/oracle/ora92/rdbms/admin/cataudit.sql;
-parallel server views
@/<mp1>/oracle/ora92/rdbms/admin/catparr.sql;
-tkprof use
@/<mp1>/oracle/ora92/rdbms/admin/utltkprf.sql;
connect system/manager;
-sqlplus user help files
  @/<mp1>/oracle/ora92/sqlplus/admin/pupbld.sql;
connect sys/<password> as sysdba;
-pl/sql recompilation
@/<mp1>/oracle/ora92/sqlplus/admin/utlrp.sql;

shutdown immediate;

create spfile='/<mp1>/oracle/database/spfile<SID>.ora'
FROM
      pfile='/<mp1>/oracle/admin/pfile/init<SID>.ora';
startup;
disconnect;
spool off;

I have provided these sample installation scripts here, and particularly the variable physical sizes for the tablespaces, for one reason only. Many of the client sites I have worked at in the last 3 years had serious problems with uncontrollable physical data-base growth. OLTP databases and data warehouses are becoming larger and larger all the time. This is one possible approach. Your structure does not have to be the same. Additionally there are many new features and changes in the latest versions of Oracle Database that will help with respect to rapid growth. Oracle Corporation generally responds very vigorously to current industry commercial requirements of its software.

Tip

Once again scripts presented here were originally built for a Solaris box Oracle8i Release 2 (8.1.7) installation and upgraded for Oracle9i Release 2 (9.2). The Database Configuration Assistant can be used to generate scripts like these, which can then be altered to suit requirements. They are presented to merely show low-level physical structural details and are not necessarily applicable to the latest version of Oracle Database.

11.3.3 Automated Storage Management

Oracle Database 10 Grid What was Oracle Managed Files in Oracle9i Database is now Automated Storage Management in Oracle Database 10g. Various new features were introduced in Oracle9i Database with expansion in Oracle Database 10g. Automated Storage Management allows complete automated management of all physical Oracle Database files; this includes control files, datafiles, and all log files. When creating and dropping a tablespace, for instance, datafiles no longer have to be added or removed, respectively, physically in the operating system anymore. Creation of files will involve automatic file naming.

Only when using Automated Storage Management are Diskgroups introduced, allowing grouping of tablespaces and datafiles into groups such as allocation of multiple temporary tablespaces to a user from a Diskgroup.

Additionally Automated Storage Management can automatically balance and distribute files for optimal performance when configuration changes are made.

Tip

Automated Storage Management or Oracle Managed Files is reputed to have quite severe adverse effects on overall performance.

This brief introduction to the Oracle installation and database creation process I hope gives a quick glimpse into physical and configuration tuning. The next chapter will start to dig into physical tuning in more depth by examining datafiles, tablespaces, control files, redo and archive logs, manual rollback and automated undo plus temporary sorting on disk.


Chapter 12: Tuning Oracle Database File Structures

Overview

This chapter will examine the file structure layers of Oracle Database and the logical structures within the Oracle database controlling and utilizing the file structures. An Oracle database in its simplest form is made up of files stored in the file system of an operating system such as Unix, Linux, or Windows. The management, controlling and linking of all those files together is what is called the Oracle Instance. The Oracle Instance is effectively an instantiation on a server of the shared global area (the memory structures) and a number of processes.


12.1 Oracle Database Architecture and the Physical Layer

The architecture of an Oracle installation could be called the physical layer of Oracle Database. Architecturally Oracle Database is made up of an Oracle Instance and an Oracle database. The Instance contains the running Oracle binary software and the database contains the data for the Oracle Instance to access. As already stated the database itself is made of files resident in the file system of the operating system.

12.1.1 The Oracle Instance

The Oracle Instance is a part of Oracle Database sitting resident in memory when an Oracle database is active and available for use. That instance or instantiation of Oracle Database is made up of processes or programs and reserved memory buffer cache areas.

Figure 12.1: Oracle Instance Memory Cache Buffers

Buffers

Most of the aspects of tuning memory buffer cache areas are discussed in Chapter 15. The different buffers are shown in Figure 12.1.

Processes

There is nothing that can be done to speed up a process itself by changing anything in Oracle Database. However, Oracle processes can be examined at the operating system level to perhaps help determine the source of other problems. For instance, enormous amounts of I/O or CPU time could show various potential bottlenecks. In other words a process itself is not tunable, only factors the process is affecting or being affected by. Building on Figure 12.1 the diagram in Figure 12.2 shows some of the Oracle processes plus the previous picture of the Oracle Instance.

Click To expand
Figure 12.2: Oracle Instance Process and Memory Layers

There are a multitude of utilities in operating systems such as Unix and Linux which can be used to detect problems with processes. Some of those utilities available on a Solaris flavor Unix box are as follows:

  • top. This utility shows a real-time picture of processes, CPU, and memory usage.

    last pid: 15713; load averages: 0.01, 0.01, 0.01
58 processes: 57 sleeping, 1 on cpu
CPU states: 99.7% idle,0.0% user,0.3% kernel,0.0%
  iowait,0.0% swap
Memory: 1024M real, 34M free, 598M swap in use, 2263M swap
  free

PID    USERNAME  THR  PRI  NICE   SIZE    RES   STATE  TIME   CPU    COMMAND
15713    oracle    1   58     0  2136K  1664K    cpu0  0:00  0.08%       top
1732      build   26   58     0   195M    60M   sleep  5:45  0.05%      java
14478     user1   25   58     0   182M    40M   sleep  1:08  0.01%      java
15711      root    1   58     0  2152K  1728K   sleep  0:00  0.00%  sendmail
127        root    1   58     0  3208K  2672K   sleep  0:38  0.00%  in.named
235        root    1   58     0  1752K  1056K   sleep  0:20  0.00%     sshd1
9259     oracle    1   58     0  9144K  4544K   sleep  0:18  0.00%   tnslsnr
14829     user2   22   58     0    50M    25M   sleep  0:03  0.00%      java
14950     user2   21    0     0    46M    19M   sleep  0:01  0.00%      java
1609      build   20    0     0   167M    18M   sleep  0:01  0.00%      java
179        root    8   52     0  2376K  1944K   sleep  0:01  0.00%      nscd
143      daemon    4   44     0  2184K  1568K   sleep  0:00  0.00%     statd
138        root    1   48     0  2008K  1584K   sleep  0:00  0.00%     inetd
231        root    1   48     0  2096K  1552K   sleep  0:00  0.00%  sendmail

  • sar. The System Activity Reporter examines CPU, swapping, paging, memory, and buffer usage for specified time periods.

  • sadc. Server activity statistics collection examining CPU, buffers, I/O, switching, system calls, file access, queues, IPCs, amongst other things.

  • vmstat. Virtual memory statistics.

  • iostat. I/O statistics.

There are numerous other Unix and Linux utilities and tools for doing all sorts of things to inspect hardware resource usage at the operating system level. Most of these utilities can provide clues as to performance problems within both an Oracle Instance and an Oracle database. What can be done on a Windows server to examine processing in the way that Unix utilities can? Windows Performance Manager can be used.

Within the Oracle database itself operating system-level statistics can be found in the V$SYSSTAT and V$SESSSTAT views. The V$SYSTEM_EVENT and numerous other related views can be used to track bottlenecks through analysis of wait events: the Oracle Database Wait Event Interface. This subject is covered in detail in later chapters.

12.1.2 The Oracle Database or File System Layer

An Oracle database is effectively the file system layer in the operating system plus the Oracle Instance which comprises processes and memory buffers. An active and accessible Oracle database is an Oracle database using the Oracle Instance to access Oracle database files as shown in Figure 12.3.

Click To expand
Figure 12.3: The Oracle Database and the Oracle Instance

Figure 12.3 also shows different types of Oracle database files. These files are as follows:

  • Datafiles. Contain all physical data such as tables, indexes, database metadata, procedural code and anything stored in the database as accessible or usable by an Oracle database user.

  • Redo Logs. Transaction records of all database changes.

  • Archive Logs. Historical copies of cycled redo logs maintaining a complete history of all database change activity.

  • Control File. Contains pointers to all datafiles and log files used for synchronization between all of those files.

  • Parameter File. The configuration parameter file is discussed extensively in Chapter 11 and throughout this book.

How Oracle Database Files Fit Together

As already discussed there are a number of different types of files at the file system level that a running Oracle Instance uses to control and gain access to data in the database. Datafiles contain all table and index data. Redo log files contain a record of all transactions or metadata database changes to the database. Redo logs are cyclic such that they are reused. When a redo log file is full it can be copied to an archive log file. An archive log file is a historical redo log file. The copied redo log file is once again ready for reuse to contain new database activity records.

The most important file in the file system structure of Oracle database files is a file called the control file. The control file contains pointers to all datafiles, redo logs, and the archived redo logs. The control file stores the current time state of a particular datafile by matching up what are called system change numbers (SCN) between datafiles and log files. An SCN is a sequential number used to synchronize changes between datafiles and redo logs. If SCN values are higher in redo logs than datafiles then the control file knows that datafiles are older versions than they should be: either the datafiles are recovered backups or the redo log entries, always written before datafiles, are updated and the datafiles have not as yet been synchronized with the latest changes.

Special Types of Datafiles

Three special types of datafiles are the SYSTEM datafile, rollback datafiles, and temporary datafiles. The SYSTEM datafile stores all the metadata or the data about the data.

Note

Oracle Database 10 Grid The SYSAUX tablespace is created automatically along with the SYSTEM tablespace when a database is created. The SYSAUX tablespace contains options, Oracle tools, repositories, and even some types of metadata previously stored in the SYSTEM tablespace.

Rollback datafiles allow for undoing of certain types of previously processed but uncommitted transactions. Rollbacks also cater for the multi-user ability of sessions to "snapshot" data at a specific point in time regardless of changes made by other sessions.

Tip

Automatic undo, a more sophisticated type of rollback released in Oracle9i Database, allows retention of uncommitted as well as already committed data, permitting queries which can "flashback" providing a snapshot of data at a previous point in time.

Temporary datafiles allow for large sorts to be sorted using disk space when memory resident sort space is used up. Figure 12.4 shows a general map of Oracle database file system structure plus pointer links and flow of data between them.

Click To expand
Figure 12.4: Relationships Between Oracle Database Files

Tuning Datafiles

There is little which can be done to tune datafiles. Most Oracle Database physical tuning with datafiles is performed from the table-spaces overlaying those datafiles. A datafile can have its extent growth parameter changed using the ALTER DATABASE command and Oracle Partitioning can be implemented.

Tip

Any changes to datafiles such as an extent size change using the ALTER DATABASE command will only affect new extents not existing extent sizes. The same applies to any block changes. Block level tuning will be covered in Chapter 14.

Datafiles can be tuned much more effectively at the file system level in the operating system. Factors such as striping and RAID arrays can help to tune datafile I/O access rates enormously.

Oracle Partitioning allows the separation of large sets of data into separate datafiles, perhaps placing different partition datafiles on to different disks.

Manual striping is a primitive implementation of the functionality of RAID arrays. Striping includes various methods of splitting datafiles into parts and spreading them across separate storage areas, or even within the same storage area, for rapid random concurrent access.

Tip

Separation of table and index tablespaces onto separate disks or even the same disk is a form of striping since the two datafiles underlying the tablespaces are likely to be accessed concurrently due to the nature of their content.

Modern RAID arrays have all sorts of bells and whistles allowing striping, mirroring, Oracle Partitioning, and other aids to faster I/O such as dissimilation between sequential and random access. Indexes and log files should be accessed and written sequentially. Tables should be accessed and written randomly since index scans help to pinpoint table access. This is especially true of OLTP databases.

As already mentioned Oracle Partitioning allows the splitting of datafile content into separate datafiles. What this means is that a very large table and its associated indexes could be separated into separate datafiles. As regards reading data from the datafile on a heavily striped RAID array partitioning of datafiles might be irrelevant. Oracle Partitioning certainly is irrelevant as far as striping of datafiles onto an already striped RAID array is concerned. However, partitioning of datafiles still has two uses, which can have a distinct effect on I/O and processing of datafiles, even on a striped RAID array. Firstly, separate partitions using separate datafiles can be unplugged and plugged into a set of datafiles in a partition set with the speed of a rename command. What this means is that a datafile partition within a large table can be added or removed without accessing any other datafiles in the partition. This leads us to the second datafile I/O benefit of using Oracle Partitioning: SQL code can be executed within the range of a partition datafile with physical access to only the datafile required to satisfy the SQL code. So SQL code reading partitioned datafiles can read single partitions in a set of partitions, reducing I/O rates drastically.

Control Files

The only thing to mention with respect to tuning of control files is that creation of multiple control files is a very sensible practice. Multiple control files on separate disks assures that a copy of the control file can be recovered should one be lost or become corrupted. The performance effect of multiple control files is negligible. However, utilizing modern RAID arrays the chances of corruption are very remote.

Tuning Redo Logs and Archive Logs

Tuning of redo logs and archive logs is important to Oracle Database efficiency. The size of redo logs must be specified when created by a database administrator. Both the number and size of redo logs are important.

The more redo logs created for a database the less likely writing of the redo logs is going to catch up with the archiving process. Let's say for example your database has three small redo logs. Heavy transactional activity will rapidly switch the redo logs but the archive process which copies redo logs to the archived log files may not be fast enough to keep up. What happens then is that all database transactional change activity halts until a redo log can be archived and recycled. A redo log will not be recycled if it has not as yet been archived. A halt will occur and effectively stop all database activity until all necessary archiving has completed and a redo log is cleared for reuse.

The same problem can occur if redo logs are too small. With high DML activity small redo logs can result in excessive switching. A redo log file is switched from CURRENT to ACTIVE or INACTIVE when it has filled. If redo log files are too small and a lot of switching is occurring with high transactional activity, once again the archive process may not be able to keep up, potentially temporarily halting the database.

A similar problem can occur with redo log and archiving performance when redo log files are too large. The archive process copies redo log files to archived log files. Sometimes, especially on Windows systems, very large redo log files can take so long to copy that transactional activity will once again catch up with the archive process and once again produce a temporary availability halt. I have seen this large-file copying problem occurring on a large Unix-based system, but only once. The redo log files were sized at 500 Mb. Yes they were half a gigabyte. The system was highly active and concurrent. It had a mixture of OLTP and data warehouse activity (Yoiks!) and it had enormous amounts of CPU time and space wastage as a result of terrible SQL code and bitmap indexes on transactional tables. This particular system had all sorts of problems, which I managed to resolve partially before I decided that the company was about to run out of money. I gladly accepted a better offer elsewhere. This particular company also had a history of database administrators dropping entire production databases by mistake, amongst other scary things. There was not really much that could be done in this situation. The fact is this. Someone had resized the redo logs up to a ridiculous amount in a vague trial and error approach of attempting to solve other problems. The result of this kind of experimentation is often further problems. Be warned! Be careful and do not change things without understanding potential effects first. Additionally it is also wise to record the state of a database before making changes. Simply writing down changes and making frequent backups can often help one to recover from critical situations rapidly without too much downtime.

Let's get back to logging. So what is the best recipe for efficient log writing and archiving of redo logs to prevent a database from halting, or perhaps worse? There is no hard and fast solution. Every database is different. Additionally the size of the redo log buffer can have a profound effect on redo log file writes and archiving performance. There is one important thing to remember. The redo log buffer should not be of the same size as the redo log files and should generally be a very small percentage of redo log file size. Redo log buffer tuning is discussed in Chapters 11 and 15. A number of things are important to remember. The redo log buffer is flushed or written to the current redo log file whenever a COMMIT or ROLLBACK is issued; the redo log buffer is partially filled to one-third full, every 3 s or if the redo log reaches 1 Mb.

Tip

A checkpoint is a process of flushing dirty buffers from database buffer cache to disk storage. The redo log buffer is not flushed to disk when a checkpoint occurs. This is the consensus of opinion from a number of database administrators and contrary to many Oracle software texts.

So how does the size of log buffer affect the redo log files? The most common and most successful sizes for the redo log buffer range from 32K to 512K and occasionally as much as 1M. I have seen many databases with problems when the redo log buffer is larger than 512K. Multiple archive processes can help to speed up archiving and alleviate pressure from elsewhere on highly active databases.

So what do we do with redo log files? How large should they be and how many should be created? Should redo logs be duplexed? What is duplexing? Duplexing is creation of multiple redo logs, which are written from the log buffer to multiple redo log files at the same time in parallel. Duplexed redo logs are used for the sake of recoverability. Redo log duplexing is not really necessary with the more sophisticated types of RAID arrays but is extra insurance. Generally flushing of the redo log buffer to duplexed redo logs on servers with more than a single CPU has a negligible impact on performance.

How large should redo log files be and how many should be created? The more transactional activity, the more redo logs you should have. If redo logs are continually switching then make the redo log files larger. Do not make redo log files so large that you get a performance bottleneck when redo logs are switched and copied to archive logs. If redo logs are getting too large then add more redo logs and perhaps try to tune other factors such as the log buffer. In short, turning off logging is a juggling process between multiple factors.

Another potential problem with very large redo log files is when using standby databases and passing log files over SQL*Net on a slow connection between two machines. The answer in this case would be to do something about your network connection. Changing the SQL*Net packet size can help your standby database log file transfers but may slow network performance for small transactions. If you are operating a fast-response OLTP database through to your customers, changing packet size is not a solution.

This leads us to the networking layer.

12.1.3 The Networking Layer

Another part of the architecture of Oracle software is the networking layer. Oracle Database has a proprietary networking software layer. This layer of software has multiple names in different versions of Oracle Database. They all do and are the same thing. SQL*Net sits on top of a protocol such as TCP/IP and allows for easy formatting and access for connections between client machines and an Oracle database server.

Network tuning will be covered in Chapter 16. There is much more to Oracle networking software than just SQL*Net. There is also much that can be done to tune Oracle networking software.


12.2 Tuning and the Logical Layer

I like to call anything in an Oracle database above the datafiles the logical layer. This layer is logical in nature because it overlays the underlying physical datafile structure in the operating system file system. The mapping between the physical and logical layers is not necessarily direct or one-to-one. Figure 12.5 shows a quick picture of the overall structure of both physical and logical Oracle database architecture.

Click To expand
Figure 12.5: Oracle Database Physical and Logical Structure

Figure 12.5 shows the various structural layers in an Oracle database.

  • Physical Layers. Datafiles, extents, and blocks are all contained within a datafile.

    • Datafiles. Datafiles contain data, metadata, rollback, sorting space, clusters, amongst other things.

    • Extent. An extent is a new chunk added to a datafile when the datafile is increased in size to accommodate new data.

    • Block. A block is the smallest physical structure in an Oracle datafile.

  • Logical Layers. All logical layers such as tables and indexes are contained within tablespaces. Also all physical layers, the datafiles, are accessible through tablespaces. Thus datafiles, extents and blocks are all accessible and tunable through tablespaces and their contained schema objects.

    • Tablespace. A tablespace can have many datafiles.

    • Schema. A schema is the same thing as an owner or username.

    • Segment. A segment can be a part of a tablespace, a datafile, and a schema. Table, index, rollback, temporary, partitions, and clusters are all different types of segments.

Now let's examine how we can tune tablespaces.

12.2.1 Tablespaces

A tablespace has a referencing map similar to the map on the boot sector of a hard disk drive. When opening a file on a hard disk drive the map is accessed in order to find the address at which the file begins physically on the disk. An Oracle tablespace contains a map in the headers of datafiles describing all extents within underlying datafiles, allowing rapid access to specific addresses in those datafiles. This map of extents can either be stored in database meta-data or in a bitmap attached to the header of a datafile. There are two different general datafile structures which can be created from a tablespace.

  • Dictionary-Managed Tablespace. Extent and block locations are stored in database metadata. Accessing data from a dictionary-managed tablespace is less efficient because database metadata must be searched internally in the Oracle Database with SQL type functionality, in order to find addresses of data in datafiles.

    Note

    Oracle Database 10 Grid Dictionary-managed tablespaces will be deprecated in a future version of Oracle Database.

  • Locally Managed Tablespace. Extent and block locations are managed by and stored in a bitmap contained within header portions of datafiles, not elsewhere in SYSTEM tablespace metadata. Accessing the extent map of a locally managed tablespace using the file header contained bitmap is much faster than searching through database metadata to find physical addresses of data within a datafile.

    Note

    Oracle Database 10 Grid Locally managed tablespaces with automatically allocated extents are the default.

In the latest version of Oracle Database all tablespaces are created as locally managed tablespaces. Conversion to locally managed tablespaces has been a gradual process in recent versions of Oracle Database. It is now safe to assume that locally managed tablespaces have been industry standard tested and should always be used.

Tip

All physical and storage parameters are inherited from datafile to tablespace and ultimately to all objects created in a tablespace. Many of these structural parameters can be overridden. Thus a tablespace can override datafile parameters and database objects such as tables and indexes can override tablespace parameters.

Dictionary-Managed Tablespaces

PCTINCREASE should always be explicitly set to zero (0) for any type of tablespace. PCTINCREASE makes each new extent a percentage larger than the previous extent created. The problem with PCTINCREASE is that since each new extent is larger than the previous one any extent which has all of its contents completely deleted will probably never be reused. The result is a lot of empty space in a datafile, thus fragmentation. Fragmentation can be partially resolved by using the COALESCE option of the ALTER TABLESPACE command. Coalescence attempts to take smaller empty extents and merge them into a smaller number of larger extents. On a previous consulting job I coalesced a number of fragmented tablespaces in order to conserve and defragment space. The result was worse performance and larger datafiles. If you have to use dictionary-managed tablespaces never set PCTINCREASE to anything but zero.

The NEXT parameter is used to specify the size of each subsequently created extent. Setting small extent sizes can be used for tablespaces containing only very small tables but it might be better to override this parameter at the table and index object level. I usually set the size of NEXT in the order of megabytes such as 1M, depending on growth rate.

Do not use dictionary-managed tablespaces as they will eventually be deprecated from Oracle Database. Locally managed tablespaces have all the useful and well-performing attributes of dictionary managed tablespaces with parameters most likely to cause problems disallowed or having no effect. Both the PCTINCREASE and NEXT parameters are ignored for locally managed tablespaces.

Note

Oracle Database 10 Grid Dictionary-managed tablespaces will be deprecated in a future version of Oracle Database. Converting a dictionary tablespace to a locally managed tablespace requires use of the DBMS_SPACE_ADMIN package.

Locally Managed Tablespaces

What are the factors that help in tuning locally managed tablespaces? Let's start with a syntax diagram for the CREATE TABLESPACE command. Certain parts are highlighted.

CREATE [ UNDO ] TABLESPACE tablespace
      [ DATAFILE 'file' SIZE n[K|M] [ REUSE ]
            [ AUTOEXTEND { OFF
                  | ON [ NEXT n[K|M]
                        [ MAXSIZE { UNLIMITED | n[K|M] }
                           ]
                  ]
            } ]
      ]
      [ MINIMUM EXTENT n[K|M] ]
      [ BLOCKSIZE n[K] ]
      [ [NO]LOGGING ]
      [ EXTENT MANAGEMENT { DICTIONARY
            | LOCAL [ { AUTOALLOCATE | UNIFORM [ SIZE
               n[K|M] ] } ]
      } ]
      [ SEGMENT SPACE MANAGEMENT { AUTO | MANUAL } ]
      [
            DEFAULT [ [NO]COMPRESS ] STORAGE(
                  [ INITIAL n[K|M] ] [ NEXT n[K|M] ]
                     [ PCTINCREASE n ]
                  [ MINEXTENTS n ] [ MAXEXTENTS { n |
                     UNLIMITED } ])
      ];
Tip

If a database is created with a locally managed SYSTEM table-space then dictionary-managed tablespaces cannot be created in that database. Additionally the SYSTEM tablespace cannot be changed from local to dictionary-managed.

Now let's go through the highlighted parts of the CREATE TABLESPACE syntax.

Auto Extend

Tablespaces should always be automatically extended with a specified size for the NEXT parameter. Specifying the NEXT parameter assures that datafiles grow with consistent, reusable extent sizes. If extent sizes are too small then a large table could have so many extents to search through that performance will degrade seriously. The NEXT parameter is defaulted to the block size. Do not leave NEXT undeclared and defaulted. Defaulting to the block size is usually too small. Only small static tables and indexes could have sizes for NEXT of below 1M. For some tables well over 1M is prudent.

Setting MAXSIZE UNLIMITED is also sensible because if a maximum datafile size is specified the database will cease to function if datafiles ever reach a maximum size. It is better to extend Oracle tablespace datafiles automatically and monitor disk space usage with scripting in the operating system.

Minimum Extent Sizes

This option specifies that every extent in a datafile is at least the specified size, minimizing defragmentation.

Block Size

Permanent tablespaces can be created with block sizes different to the DB_BLOCK_SIZE database block size parameter. Appropriate DB_nK_CACHE_SIZE parameters must be created in the configuration parameter file to cater for such tablespaces. Obviously a tablespace with a smaller block size is useful to contain smaller tables. A tablespace with a larger block size is good for large tables when reading large amounts of data at once. I have personally never experimented with this aspect of Oracle9i Database in a highly active production environment but would certainly suggest using them for storage of large objects such as LOBs. LOBs contained within tables can be stored into a tablespace separate to the tablespace in which the table resides.

Logging

Switching off logging as the default for a tablespace will increase performance in your database. However, no logging will result in no redo log entries and an unrecoverable database. Do not switch off logging for the sake of performance. The only type of tablespace for which it is sensible to have no logging for is a read-only tablespace. Make a tablespace read-only using the ALTER TABLESPACE command. There is no logging for read-only tablespaces since read-only does not allow DML command changes. Thus there are no redo log entries. Do not use NOLOGGING on read-write table-spaces unless you are prepared to lose your entire database in the event of a disaster. Individual objects in tablespaces can be forced into logging mode when other tables are not if the tablespace is set to NOLOGGING. Once again any type of no logging setting is risky.

Extent Management

The EXTENT MANAGEMENT clause allows specification of a locally or dictionary-managed tablespace. Stick to locally managed tablespaces unless you have a very good reason.

Tip

When a database SYSTEM tablespace is created as locally managed, all tablespaces in that database must be locally managed.

Once again the most efficient datafile structure for growth is consistent extent sizes. Differing extent sizes in datafiles will result in fragmentation and resulting slow access times due to bouncing around a disk when searching for data. Coalescence is generally useless when trying to reclaim deleted space where extent sizes differ.

Note

Oracle Database 10 Grid In past versions of Oracle automatic extent allocation was not recommended as varying since documentation stated that very small variable extent sizes were created. In Oracle9i Release 2 (9.2) and beyond automatic extent allocation is recommended as its default extent sizes are uniformly set at 1M.

When using manual extent allocation always use the UNIFORM SIZE clause for extent management to ensure consistency of extent sizes across a datafile. The more growth you have the larger the extent size should be. I generally use 1M and larger values for very large databases or databases with high growth rates. Small objects can have smaller extent sizes as long as the database administrator is absolutely sure that large tables or high growth rate tables will never be placed in that tablespace. Database administrators are rarely involved in the development process and thus a database administrator is unlikely to have enough knowledge of applications to make these types of decisions.

Keep extent sizes consistent and never below 1M.

Segment Space Management

A segment space management specification is allowed for locally managed nontemporary tablespaces. Automatic segment space management as specified by the SEGMENT SPACE MANAGEMENT AUTO clause eliminates the need to declare PCTUSED, FREELIST, and FREELIST GROUPS values for objects created in a tablespace. These settings are now bitmap-managed automatically by Oracle Database.

Tip

Setting values for PCTUSED, FREELIST, and FREELIST_GROUPS in database objects such as tables will be ignored if the containing tablespace is set to automatic segment space management.

In past versions of Oracle Database manual free list management was usually only required for very highly concurrent active OLTP databases or Oracle RAC (Parallel Server) installations. Access to the PCTUSED parameter, on the other hand, helped immensely with tuning for performance in the following respect. The default value for PCTUSED is very low at 40%. Any database with heavy deletion activity could have a lot of wasted space with PCTUSED set to 40%. Deleted rows are still read by full scans and can ultimately result in serious performance problems. One of the biggest problems with manually setting the PCTUSED value for a table was that it was often set based on subjective guesswork due to a lack of understanding. Otherwise it was simply left at the default value. Automatic management of free space resolves these issues.

Tip

One of the reviewers for this book commented that automatic segment space management realizes up to 35% performance improvement, specifically in Oracle RAC environments.
Note

Oracle Database 10 Grid Locally managed tablespaces with automatic extent management and automatic segment management have been the default since Oracle9i Release 2 (9.2). This indicates that tablespaces should be created as such. In Oracle Database 10g the current accepted practice is to set the SYSTEM, UNDO, and TEMP tablespaces to manual segment space management and all other tablespaces to automated segment space management.

BIGFILE Tablespaces

Oracle Database 10 Grid Oracle Database 10g allows division of tablespace types for locally managed tablespaces into two categories: Smallfile tablespaces and Bigfile tablespaces. A Smallfile tablespace is the default.

Tip

Smallfile or Bigfile can be set as the default for a database as a whole using the CREATE DATABASE or ALTER DATABASE commands.

A Bigfile tablespace contains a single datafile, which can be up to 128 Tb for a block size of 32K. The general trend for database technology at the file structure level is fewer, larger files. Many years ago relational databases had a single datafile for every table or index; some still do. The most advanced database engines are object databases. The most recently developed object databases generally have a single large datafile. The performance benefit of maintaining a single large file is significant. I would suggest that it is likely that Oracle Database will continue on this path of development. In a future version of Oracle Database Bigfile tablespaces may be the default and perhaps eventually the only option, where all physical storage structure within that Bigfile is automatically managed, transparent and inaccessible to database administrators. Splitting files up physically using striping on a RAID array, for instance, is managed at the operating system level and is not a requirement from within the database. Automated Storage Management, introduced in Oracle Database 10g, can implement striping managed from within the Oracle database. This hints further at less emphasis on explicit management of physical storage structures.

Avoiding Datafile Header Contention

Datafile header contention can be avoided when using locally managed tablespaces in busy Oracle installations by creating a large number of small datafiles for each tablespace. Datafile header contention is avoided since extent bitmaps attached to each datafile are not overstressed in highly concurrent environments. This is a new-fangled way of doing things and is not to be confused with manual striping. It is not striping! Locally managed tablespaces have bitmaps in datafile headers in much the same way that dictionary-managed tablespaces use metadata storage to store extent maps. Since bitmaps compress far more than metadata storage, datafile header contention under highly concurrent conditions is more likely.

Now let's take a look at the different types of functions of table-spaces.

Temporary Sort Space

Temporary sort space is catered for in two ways: firstly, sort buffers declared by the SORT_AREA_SIZE parameter; secondly, a temporary sort space tablespace and datafile on disk. It might make sense to size temporary sort space to a multiple of the SORT_AREA_SIZE parameter.

A temporary tablespace is specially structured for sorting and is best implemented as a locally managed tablespace. This is the syntax for creating a temporary tablespace.

CREATE TEMPORARY TABLESPACE tablespace
      [ TEMPFILE 'file' [ SIZE n[K|M] ] ]
      [ EXTENT MANAGEMENT LOCAL ]
      [ UNIFORM [ SIZE n[K|M] ] ];

Below is an example temporary tablespace creation command. This is how I create temporary tablespaces with Oracle9i Database Release 2 (9.2), which may change with a future version of Oracle Database. This is the type of temporary tablespace structure that I trust completely for performance and stability in Oracle9i Database Release 2 (9.2).

CREATE TEMPORARY TABLESPACE TEMP TEMPFILE
      '$ORACLE_HOME/oradata/$ORACLE_SID/temp01.dbf' SIZE
         100M AUTOEXTEND ON NEXT 1M MAXSIZE UNLIMITED EXTENT
         MANAGEMENT LOCAL UNIFORM SIZE 1M;

It is important to allocate the use of a temporary tablespace automatically to a user either using the CREATE DATABASE command or when creating the user with the CREATE USER command. The ALTER USER command can be used later on.

A temporary tablespace is used by an Oracle database for various operations such as index creation and large sorts in SQL code. Temporary sort space is used to shift sorting operations from the sort buffer to disk when a sort operation does not fit into the maximum sort buffer size allocated for each session. Note that every session obtains a buffer area equal to the SORT_AREA_SIZE parameter for sorting purposes. Every session should have access to a temporary tablespace for sorting on disk when the sort buffer is too small for a particular sort operation. Sorting will be covered more later on.

One common mistake with respect to sorting can have severe implications on sorting. If a user does not have a temporary table-space allocated for on-disk sorting overflow from the sort buffer there will be a problem. Prior to Oracle9i Database the CREATE DATABASE command did not allow the setting of a default sorting temporary tablespace for automatic allocation to a newly created user. Quite often database administrators forgot about allocating a temporary tablespace to a user altogether. The result was that the SYSTEM tablespace contained overflow onto disk from the sort buffer.

The SYSTEM tablespace contains the database metadata and is probably the most critical tablespace in an Oracle database with respect to performance. Whenever a database administrator is confronted with a new database, perhaps in a new job, temporary sort space on disk is one of the first things to check.

The following queries check that users are using appropriate default and temporary tablespaces. The TEMP tablespace is my temporary sorting tablespace, the DATA tablespace contains application tables. The SYSTEM tablespace is used for Oracle Database metadata, Oracle applications, and add-on options such as DBSNMP (Oracle agent) and OUTLN (Outlines). The OEM user and OEM tablespace contain an Oracle Enterprise Manager repository (this is best stored in a separate database on a separate machine) and RMAN (Recovery Manager) is stored in the TOOLS tablespace. This structure is shown in the following script output.

Note

Oracle Database 10 Grid The SYSAUX tablespace is created automatically along with the SYSTEM tablespace when a database is created. The SYSAUX tablespace contains options, Oracle tools, repositories, and even some types of metadata previously stored in the SYSTEM tablespace.

COL username FORMAT a20;
COL default_tablespace FORMAT a20;
COL temporary_tablespace FORMAT a20;
SELECT username, default_tablespace, temporary_tablespace
      FROM dba_users ORDER BY 1;

USERNAME           DEFAULT_TABLESPACE  TEMPORARY_TABLESPACE
-----------------  ------------------  --------------------
ACCOUNTS           DATA                TEMP
DBSNMP             SYSTEM              TEMP
EMPLOYEES          DATA                TEMP
OEM                OEM                 TEMP
OUTLN              SYSTEM              TEMP
RMAN               TOOLS               TEMP
SYS                SYSTEM              TEMP
SYSTEM             SYSTEM              TEMP

One other thing that should be checked is that the temporary tablespace TEMP really is a temporary tablespace and not a permanent tablespace. This is a common error as well.

COL tablespace_name FORMAT a20;
SELECT tablespace_name, contents FROM dba_tablespaces ORDER
   BY 1;

TABLESPACE_NAME             CONTENTS
-------------------------  ---------
DATA                       PERMANENT
DRSYS                      PERMANENT
INDX                       PERMANENT
OBJECTS                    PERMANENT
OEM                        PERMANENT
RBS                        PERMANENT
SMALLDATA                  PERMANENT
SYSTEM                     PERMANENT
TEMP                       TEMPORARY
TOOLS                      PERMANENT
USERS                      PERMANENT

Tablespace Groups

Oracle Database 10 Grid On disk, temporary space can be allocated as tablespace groups where a tablespace group can contain multiple locally managed temporary tablespaces. A tablespace group will allow spreading of SQL execution sorting across multiple temporary tablespaces thereby potentially speeding up sorting operations. Processing is distributed. A user can be allocated a tablespace group as a temporary sort space as opposed to just a single temporary tablespace.

Manual Rollback and Automatic Undo

Manual rollback and automatic undo both do the same thing except that automated undo is the more recent and more sophisticated innovation.

Note

Oracle Database 10 Grid Manual rollback is deprecated and replaced by automatic undo (automatic rollback).

Rollback has two functions. The first is allowing the undoing of noncommitted transactions. The second is allowing for a consistent point-in-time snapshot of data. This snapshot capability allows different sessions to find data as it was at a specific time, regardless of any other uncommitted transactions which have already changed the database. Let's examine automatic undo first.

Automated Undo

The automated undo tablespace was introduced in Oracle9i Database. An undo tablespace automates rollback segment management so rollback segments no longer have to be created and tuned. Many database administrators had problems tuning rollback segments. There are a number of undo configuration parameters for controlling automated undo.

  • UNDO_MANAGEMENT. Switches automated undo on and off. Set to MANUAL to use manual rollback segments.

  • UNDO_TABLESPACE. The name of an undo tablespace must be specified.

  • UNDO_RETENTION. Places a time period for how much committed undo data is kept. When using the Database Configuration Assistant default parameters for this value appear to be set to over 10,000 for a data warehouse and under 1,000 for OLTP databases. Documentation on this parameter is vague and it is unclear whether if this parameter is set to too short a time period then uncommitted information is over written.

    Note

    Oracle Database 10 Grid An undo tablespace can be created with the tablespace retention clause. The tablespace retention clause can be set in two different ways: RETENTION_GUARANTEE or RETENTION_ NOGUARANTEE. The latter is the default where undo information not yet expired can be consumed by active transactions. Nonexpired undo data could be required for a rollback on a transaction. The RETENTION_GUARANTEE setting guarantees that this will never happen.

Let's look at rollback segments.

Manual Rollback Segments

In previous versions of Oracle Database rollback involved creation and tuning of rollback segments. Rollback segments are placed into a permanent tablespace. Rollback segments are separate physical areas having individual or multiple transactions allocated to them at the same time. In an OLTP system it is best to create a large number of small rollback segments, even up to the maximum of 50 rollback segments. This will help to divide up different transactions into different rollback segments and thus minimize contention between rollback segments. A database with large transactional throughput would be better served with fewer, much larger rollback segments. If a rollback segment expands up to a specified limited number of extents a transaction will fail and rollback. Sometimes in ad hoc SQL code and development environments it is best to limit rollback segment maximum extents in order to avoid runaway transactions. A runaway transaction can completely destroy database performance.

This command creates a rollback segment in a rollback tablespace called RBS.

create public rollback segment rb00 tablespace rbs storage
      (initial 16K next 16K optimal 64K minextents 2
         maxextents 64);

The OPTIMAL parameter retains space in a rollback segment after completion of a transaction. The setting of OPTIMAL is a debatable issue and depends on database activity and transaction size. When a transaction is committed and rollback is cleared the rollback segment will be cleared physically back to its optimal size; in the case of the rollback segment shown, OPTIMAL is set to 64K, where the initial size is 16K. This will assure that transactions larger than 16K will always have 64K of space available before having to extend and that rollback segments will be of consistent sizes. On the other hand, continual resizing of rollback segments can affect performance. Check for rollback resizing using this query.

SELECT n.name, s.shrinks, s.gets FROM v$rollname n,
   v$rollstat s
         WHERE n.usn = s.usn AND s.status = 'ONLINE';

So we set rollback sizing parameters according to the size of transactions in the database. For an OLTP database with very small transactions these values could be as small as the values shown of 16K extent size with an optimal value of 64K.

Do not forget to place rollback segments online. The same can be achieved by placing all rollback segments into the ROLLBACK_SEGMENTS configuration parameter, placing all listed rollback segments online at database startup.

alter rollback segment rb00 online;
Note

Oracle Database 10 Grid Manual rollback is deprecated.

The next chapter will delve a little deeper into the logical layers of the Oracle database and examine physical tuning of database objects such as tables and indexes.


Chapter 13: Object Tuning

Overview

There are two layers to object physical tuning. The first layer allows for special behavior applied to objects such as tables and indexes. These special behaviors will be covered in this chapter. The second layer is explicit block and extent level tuning within objects. Explicit block and extent level tuning is what I like to call low-level architecture and will be covered in Chapter 14.

This chapter looks at object-level physical tuning. What is object-level physical tuning? This subject encompasses changes to an object, such as a table, which involve changing the underlying physical structure without necessarily affecting block structure. This chapter is split into the different objects such as tables, indexes, and sequences. Much of the detail for many object types is covered in the first section on tables. Each subsequent section is expanded to cater for details specific to the different object types.


13.1 Tables

So what can be done with table objects?

13.1.1 Caching

This option is much like the keep and recycle buffer pools from a previous version of Oracle Database and applies to full table scans only. Caching will allow forcing of small static tables into MRU (most recently used) cache and large seldom used tables into LRU (least recently used) cache.

For my Accounts schema I could do something like that shown as follows. The GeneralLedger and StockMovement tables are large and I would like to force any full table scans on very large tables out of cache as soon as possible. Small static lookup tables in my Accounts schema are Type, Subtype, COA, Stock, and Category. I would want to force these lookup tables to remain in cache for as long as possible.

ALTER TABLE generalledger NOCACHE;
ALTER TABLE stockmovement NOCACHE;
ALTER TABLE type CACHE;
ALTER TABLE subtype CACHE;
ALTER TABLE coa CACHE;
ALTER TABLE stock CACHE;
ALTER TABLE category CACHE;

The Customer and Supplier tables are semi-static and are not cached because they are large and not generally full table scanned. Other static tables such as Period and PeriodSum are not cached because they are rarely used. Changes will be made to these pools with further discussion on caching tables and indexes in Chapter 15.

Tip

There are numerous methods of caching objects in Oracle Database. It is not necessarily particularly useful, perhaps causing more problems than it resolves.

13.1.2 Logging

Logging can be switched off where redo log entries are minimized. Switching off logging for an object such as a table will help performance but is not recommended because recoverability can be seriously affected. Logging applies to indexes as well as tables. Indexes are recoverable without logging because they can be rebuilt. Rebuilding indexes is not a problem as long as applications can account for slow access speeds whilst no indexes are available and indexes are rebuilt, as a result of recovery.

13.1.3 Table Parallelism

A table can be set as being accessible in parallel or not in parallel for DML and SELECT activity. Parallel execution applies to database servers with more than one CPU. I will experiment using a rather geriatric dual 300 MHz PII CPU server with minimal RAM (128 Mb).

Some points to note about parallelism:

  • Full Table Scan. A query must contain a full table scan to be executed in parallel.

  • Oracle Partitioning. It is most appropriate to use parallelism with Oracle Partitioning or a powerful machine with plenty of RAM and at least two CPUs.

  • CREATE TABLE AS SELECT. Often believed to be executable in parallel. It is, but only from a remote database.

  • Bitmaps. Using bitmap indexes in an OLTP database is unwise. Parallel operations on tables using bitmap indexes in data warehouses are commonly used.

So what does use parallelism? Let's use a large table in the Accounts schema. First of all make sure that the table has no parallelism.

ALTER TABLE generalledger NOPARALLEL;

Now create a temporary table and compute statistics.

CREATE TABLE tmp1 AS SELECT * FROM generalledger;
ANALYSE TABLE tmp1 COMPUTE STATISTICS;

Now count the rows with parallelism switched off. The time elapsed is less than 1 s.

SQL> SELECT COUNT(*) FROM (SELECT * FROM tmp1);

   COUNT(*)
-----------
     752741

Elapsed: 00:00:00.07

Here is the query plan.

Query                          Cost
-----------------------------  ----
1. SELECT STATEMENT on          495
2.  SORT AGGREGATE on
3.   TABLE ACCESS FULL on TMP1   495

Now set the temporary table to PARALLEL 2.

ALTER TABLE tmp1 PARALLEL 2;
Click To expand
Figure 13.1: Nonparallel Versus Parallel Processing

Count the rows again. Note how the time elapsed has more than quadrupled using parallelism. This same result was found in Chapter 8. Figure 13.1 shows a picture of CPU usage for both processors. Notice how the query executed in parallel is using both processors even though the elapsed time is slower. It is quite possible that my somewhat geriatric dual CPU machine is the problem but not necessarily.

SQL> SELECT COUNT(*) FROM (SELECT * FROM tmp1);

   COUNT(*)
-----------
     752741

Elapsed: 00:00:03.04

Here is the query plan. The cost for the parallel executed query is about half that of the serially executed query. My database server machine is probably just too old.

Query                                      Cost
-----------------------------------------  ----
1. SELECT STATEMENT on                      248
2.  SORT AGGREGATE on
3.   SORT AGGREGATE on PARALLEL_TO_SERIAL
4.    TABLE ACCESS FULL on TMP1
        PARALLEL_COMBINED_WITH_PARENT       248

It is sensible to conclude that parallel processing works best with Oracle Partitioning and very large tables. My Accounts schema tables, at two million rows, are simply not large enough to show any effect and my hardware platform is inadequate. Oracle Partitioning will be covered in detail in Chapter 17.

13.1.4 Storing LOBs Separately

LOB objects such as images are generally large. Storing a large chunk of space into a table row block is very likely to result in large numbers of rows being occupied by multiple blocks. Row chaining and row migration are inefficient. Oracle Database allows LOB objects to be stored in a tablespace physically separate to the tablespace in which the table row is stored.

I have two LOB objects in the Stock table in my Accounts schema. The first object is called Stock.DESCRIPTION and is a text CLOB column. The second is called Stock.IMAGE and is a BLOB column containing an image. How do I change my Stock table to store the contents of these columns into a specialized objects tablespace? One method of moving the image column to a specialized objects table-space called OBJECTS is as shown. Since I have not yet stored any images in the table there is no problem dropping and recreating the column.

ALTER TABLE stock DROP COLUMN image;
ALTER TABLE stock ADD (image BLOB) LOB (image)
      STORE AS image (TABLESPACE objects);

There are various special LOB parameters. These parameters can sometimes affect performance adversely.

  • STORAGE IN ROW. Set to DISABLE and stores only a pointer in the row. It is much more efficient to store only the pointer to a contiguous binary object in a table.

  • CHUNK. LOB manipulation disk space reserved chunk.

  • PCTVERSION. Multiple versions storage of the same LOB object.

  • RETENTION. How many old versions can be stored.

  • FREEPOOLS. LOB freelists for high concurrency access.

The most important aspect about LOB objects with respect to tuning is to store them out of line. Out of line implies separated from other data in the row. The most efficient method of storing multimedia type objects is to use a BFILE pointer. A BFILE pointer contains the full path and name of an object file. The object file is located somewhere on disk, outside of the database. BFILE pointers are the most efficient LOB-type object storage for both relational and object databases.

13.1.5 Dropping Columns

Dropping columns from very large tables can cause a problem. Simply dropping a column using the DROP COLUMN clause from a very large table will very likely cause an exclusive table lock. During the process of dropping a column both the column definition is removed from the metadata, and data values are physically removed from data blocks. In very high availability systems dropping a column from a table can cause serious performance problems.

There are two options for column removal from a table in addition to the DROP COLUMN option.

  • DROP COLUMN. Removes the column both physically and from metadata.

    ALTER TABLE table DROP { COLUMN column | ( column [,
   column ¼ ] ) };

  • SET UNUSED COLUMN. Removes access to the column to avoid interrupting service. Use this option for high usage, very large tables in highly available databases. Perhaps even small lookup tables could cause a problem when a column is dropped from them. It all depends on how much the table is accessed concurrently.

    ALTER TABLE table SET UNUSED
   { COLUMN column | ( column [, column ¼ ] ) };

  • DROP UNUSED COLUMNS. Destroys metadata and physical column values for any unused columns. Use this option in low activity times to remove the column entirely, after having used the SET UNUSED COLUMN option previously. Use the DROP UNUSED option to reclaim physical space.

    ALTER TABLE table DROP { UNUSED COLUMNS | COLUMNS
   CONTINUE };

13.1.6 Deallocating Unused Space

Unused space above the high water mark can be cleared for reuse. Deallocation can help to make a table smaller, particularly if extensive deletion has occurred.

ALTER TABLE table DEALLOCATE UNUSED [KEEP integer
   [ K | M ]];


13.2 Indexes

Many factors affecting indexes and types of indexes were discussed in detail in Chapter 7 including index compression. Even though there is some crossover with the previous section on tables there are a few other factors to consider.

13.2.1 Monitoring

The simplest use for monitoring is to tell if an index is used or not. When an index is monitored as being used the V$OBJECT_ USAGE.USED column will contain a YES value. An index not used after monitoring over a period of time could possibly be dropped; preferably do not drop foreign key indexes.

Tip

Monitoring applies to both tables and indexes. However, when testing on my database I got absolutely no sensible response to monitoring of tables for DML activity.

Let's experiment. This query creates a script allowing monitoring of indexes for all the tables in the Accounts schema.

DECLARE
      CURSOR cIndexes IS
            SELECT index_name FROM user_indexes
            WHERE index_type != 'LOB';
BEGIN
      FOR rIndex IN cIndexes LOOP
            EXECUTE IMMEDIATE 'ALTER INDEX
               '||rIndex.index_name ||' MONITORING
                     USAGE';
      END LOOP;
END;
/

The result will be something like this:

ALTER INDEX XFK_COA# MONITORING USAGE;
ALTER INDEX XFK_SM_STOCK MONITORING USAGE;
ALTER INDEX XPK_GENERALLEDGER MONITORING USAGE;
ALTER INDEX XPK_STOCKMOVEMENT MONITORING USAGE;
¼

Now I change all these tables with the previous script and then execute my DBMS_JOBS package scripting to start up my highly active concurrent OLTP database. After that I query the V$OBJECT_USAGE performance view and see if I can find any unused indexes.

SQL> SELECT COUNT(*) FROM v$object_usage WHERE used = 'NO';

   COUNT(*)
-----------
    42

Many of the indexes I have in my Accounts schema are not used.

Tip

Foreign key indexes will not be flagged as used when utilized for validation of Referential Integrity. Be careful not to remove foreign key indexes. You will get a lot of performance problems if you do. When foreign keys are not indexed, table exclusive locks are extremely likely. Most of the unused indexes in my Accounts schema are foreign key indexes.

I could use a query such as this to find exact details of monitoring.

SELECT start_monitoring "Start", end_monitoring "End",
   used
,index_name "Index" FROM v$object_usage WHERE used = 'NO';

It is important to tuning to drop unused indexes because a DML operation executed on a table results in a DML operation on the table plus all its indexes. The fewer the objects requiring changes the faster DML activity will be. Note that indexing usually speeds up data retrieval so it is sensible to weigh between DML, SELECT activity and the need for foreign key indexes when deciding on the necessity for an index.

13.2.2 Index Parallelism

Index creations and rebuilds can be executed in parallel. There are three indexes on the Accounts.GeneralLedger table. Let's rebuild one of them.

SQL> ALTER INDEX xfk_coa# REBUILD;

Index altered.

Elapsed: 00:00:30.00

Now let's rebuild in parallel.

SQL> ALTER INDEX xfk_coa# REBUILD PARALLEL 2;

Index altered.

Elapsed: 00:02:56.04

It appears that the parallel rebuild is quite a lot slower. Perhaps there is something very wrong with my machine, perhaps not!

Let's try creating the index from scratch, both serially and in parallel. First serially.

SQL> DROP INDEX xfk_coa#;

Index dropped.

Elapsed: 00:00:00.04

SQL> CREATE INDEX xfk_coa# ON generalledger(coa#);

Index created.

Elapsed: 00:05:24.01

Secondly, let's create the index in parallel.

SQL> DROP INDEX xfk_coa#;

Index dropped.

Elapsed: 00:00:00.01

SQL> CREATE INDEX xfk_coa# ON generalledger(coa#) PARALLEL 2;

Index created.

Elapsed: 00:06:02.06

Once again parallel processing on my two CPU machine is slower and once again my dual CPU box has limited RAM, limited disk capacity and I/O performance. It is possible that a combination of these factors causes parallel processing to be abnormally slow. Then again parallel processing requires extra processing and coordination between different processes. In my experience parallelism is often only effective in very large databases on Unix machines with at least four CPUs, or when Oracle Partitioning is utilized.

13.2.3 Fragmentation and Coalescing

Indexes can become defragmented, perhaps by large amounts of deletion. Space in index leaf blocks is not reclaimed for reuse. Coalescence of indexes can help to alleviate loss of space by attempting to merge neighboring blocks together where those blocks are empty.

ALTER INDEX index COALESCE;

Be aware that it is more effective to reclaim index space by rebuilding the index: slower but much more effective. Also indexes can be rebuilt with the ONLINE option. The ONLINE option does not disrupt DML activity using that index.

ALTER INDEX index REBUILD ONLINE;


13.3 Index-Organized Tables and Clusters

Some of what applies to tables and indexes applies to index-organized tables and clusters. Since these two types of objects have often been used in the past as read-only data warehouse objects they will not be examined in any detail.

In reality index-organized tables are sometimes used in OLTP-type databases. Many commercial OLTP databases are a combination of online small transactional DML activity with some DSS reporting or listing activity mixed in. Even though index-organized tables may be useful in OLTP databases, they are likely to become unbalanced as indexes faster than a simple index would.

Degradation in index-organized tables is possible over time. Any changes to data must slot into current index structure, rearrange the structure or overflow into nonoptimally physically located segments. Any of these three situations can degrade performance for any type of index. An index-organized table includes data within the structure of the index. An index-organized table is potentially much bigger than a normal BTree index and there is much less scope for UPDATE and DELETE DML activity not degrading that index structure substantially, fairly quickly. Rapidly degrading indexes have to be more frequently rebuilt to maintain their efficiency. Regularly rebuilding indexes can be executed with the index remaining available during the rebuild process. This is not the case with a table.

Index-organized tables are possibly most appropriate for OLTP databases servicing object-built applications, Java written perhaps, where sequence numbers are used to generate primary keys. Using reverse key indexes on the primary keys could help alleviate hot block issues if the index-organized table is subject to high concurrent insertion rates, especially in Oracle RAC environments.


13.4 Sequences

Sequences can be cached. Twenty sequences are cached by default. If a system failure occurs then all cached sequences are lost.

Tip

Only sequence numbers are lost if a database server crashes, not log file recoverable rows in tables, or any other operations using sequence values.

If required by local law that all sequence numbers added to something like invoices must exist then always create sequences with the NOCACHE option. Generally extensive sequence caching is useful for highly active concurrent OLTP or Oracle RAC installations. I have never seen extensive caching of sequences adversely affect performance except in circumstances where sequences were abused.


13.5 Synonyms and Views

Synonyms could potentially be used to help replace nested layers of views to enhance performance. Synonyms are also often used to allow creation of multiple users where access to data is pointed to a central schema. Profligate use of public synonyms can cause a security issue depending on what is granted to them; more important perhaps is potential contention in highly concurrent environments. On the other hand, there is little which does not result in contention in highly concurrent environments.

Using synonyms can have a similar effect to that of using views. Synonyms can help to organize but can ultimately cause metadata performance issues as a result of the extra layers of metadata. Generally, using views will always create performance problems, especially where there are multiple layers of views. Large amounts of metadata can cause problems in the shared pool and degrade the performance of the Optimizer with too much complexity, especially with multiple layers of views and synonyms.

Synonyms and views are often used for similar reasons. Views and synonyms are often used when a database is designed and built from the top down and the perspective of an application. A relational database has simplistic structure. Applications can handle complexity that relational database structure is not intended for. As a side issue an object database can handle complexity. Extensive use of views and synonyms is common for applications tending to attempt to overuse relational database structure, imposing complex structure onto that relational database. In other words, do not overuse views and synonyms to simplify application coding. Performance of your application will suffer immensely in the long run if too many layers of complexity are imposed on the database layer. A relational database is intended to organize, store, allow changes to and retrieve data. A relational database does not have the processing power of an application tool. Keep complex application functionality out of the database if you want good performance. A similar approach should apply to PL/SQL. Use PL/SQL to access data, not to perform nondata accessing calculations and processing. The Oracle Java JVM can be utilized for complex processing.

Views can be used to implement security. The biggest issue with views is that any filtering executed on a query of a view is applied to the result of the view. The query executed within a view will always be executed in its entirety, regardless of filtering on the query executed against the view.

Be careful using synonyms and views.


13.6 The Recycle Bin

Oracle Database 10 Grid Oracle Database 10g introduces a recycle bin, much like the recycle bin in the Windows operating system. This feature is certainly very useful from the perspective of database administration but could cause problems with space and perhaps performance if the recycle bin is not regularly monitored and cleared.

Check the contents of the recycle bin by selecting from the RECYCLEBIN or USER_RECYCLEBIN views.

This chapter covered some sometimes rather vague aspects of Oracle Database logical object tuning and had some necessary repetition from previous chapters. The next chapter will look into block-level tuning regardless of the object type.


Chapter 14: Low-Level Physical Tuning

Overview

This chapter will cover the second layer of object physical tuning, dealing with explicit block-and extent-level tuning within objects. I like to call block-and extent-level tuning low-level physical tuning.

There are a few things we need to reiterate and freshly discover before going too deeply into low-level Oracle Database tuning.

Note

Oracle Database 10 Grid The current trend for Oracle Database is towards automation with respect to physical storage structure. Some of the subject matter in this chapter is redundant in Oracle Database 10g and more may become so in future versions.


14.1 What is the High Water Mark?

Often a table, index, or database object is referred to as a segment. The high water mark of a segment is the maximum number of blocks ever used in that segment. Space can be removed from segments by deletions, fully or partially freeing up blocks. Therefore, the high water mark of a table is the highest block number ever used by that table for data, not allocated space. The high water mark is not reset when data is deleted. When a full table scan is performed each block is read up to the table's high water mark. Therefore, if a new table with 100 free blocks, has inserts filling 70 blocks, followed by deletes freeing 20 blocks, then that table's high water mark is 70 blocks and not 50 blocks (70 - 20). A full table scan on this table will physically read 70 blocks even though 20 of those 70 blocks have been deleted. This is why it is good for performance to avoid full table scans and use indexes, especially for large tables with extensive DML deletion activity.

It is true that full table scans will be faster than index plus table reads, particularly for index range scans, if enough rows are read. However, the point about the high water mark is that the more deletions occur, without reuse of fully or partially freed up blocks, the more likely a full table scan will read deleted blocks. Small transactional OLTP databases do not benefit generally from full scans of any kind unless tables are very small and static.


14.2 Space Used in a Database

There can be a lot of wasted space in datafiles because of varyingsized extents where new extents cannot fit into smaller, older empty extents. This type of behavior was common in older versions of Oracle Database where the default for PCTINCREASE for dictionary-managed tablespaces was set to 50%. Extents can be coalesced and possibly reused by joining adjacent extents together. In my experience coalescence and deallocation was not very effective.

Note

Oracle Database 10 Grid Dictionary-managed tablespaces will be deprecated in a future version of Oracle Database.

The query result shown notes the full database size of my testing database as 4,489 Mb, where about 75% of that space is used. This means that 25% of currently allocated extent space is wasted or not as yet used. This space could simply be allocated space but it could also be deleted space not reused as of yet.

*****          %Free  %Used  Mb Free  Mb Used   Size
-------------  -----  -----  -------  -------   ----
Database Size     22     78      998     3491   4489

Here is the same script expanded for all tablespaces in the database. All tablespaces are locally managed.

Tablespace  %Free  %Used  Mb Free  Mb Used
----------  -----  -----  -------  -------
SYSTEM          2     98        7      403
OEM             0    100        0      749
CWMLITE        53     47       11        9
DRSYS          52     48       10       10
RBS            95      5      690       33
USERS          48     52       30       32
ODM            53     47       11        9
TOOLS          39     61        4        6
DATA           17     83      165      812
XDB             0    100        0       38
INDX            5     95       66     1381
OBJECTS        31     69        4        9
SMALLDATA      75     25        0        0

The scripts to produce these queries, along with scripts displaying sizes for all table and index segments and extents, are included in Appendix B.


14.3 What are Row Chaining and Row Migration?

Row chaining and migration is overflow of blocks to other blocks because data cannot fit into a single block. Additionally chained blocks might be spread across disk storage haphazardly, somewhat like overflow to a separate part of a disk storage area. A lot of row chaining and row migration can make for very poor performance.

More specifically row chaining occurs when a row is too large for a single block and thus the row is "chained" or spread across multiple blocks. Row migration occurs when updating, increasing the row size, prohibiting the whole row from fitting into a single block. Thus the row is migrated or moved elsewhere leaving only a pointer in the original block. It is possible that a row migration can lead to a row chain as well if the row is large enough. Row chains can be removed by table reconstruction such as recreating the table or using export and re-import.

Row migration can be minimized by way of allocating more empty (free) space in each block when the block is first used for insertion. The downside to increasing free space on initial block insertion is that rows will be less compacted and thus there will be fewer rows per block on average. High row to block ratios or row density leads to faster access time because there are more rows read per block read.

Row chaining is obviously more likely with smaller block sizes, either at the database level or in the smaller of variable block sized tablespaces.


14.4 Different Types of Objects

The size and usage of tables affects the way that physical structure should be organized.

  • Transactional Data. Tables with high DML activity can have a profound effect on physical requirements. Updates on large datatypes making column values larger should have more free space available for future updates. High deletion activity dictates that blocks will become available for insertions again when a threshold is reached. Less space is wasted when blocks are more likely to be reused. Tables with high insertion activity and little update or deletion activity do not have to reserve free space.

  • Read-Only and Static Data. Read-only or static tables such as data warehouse or lookup tables do not require any reserved space for later changes. In these cases all block space can be used when initially filling the tables.

  • Large Tables. Regardless of volatility, large and small tables should be treated differently. Very large tables should use as much space as possible in each block in order to minimize disk space used. The more physical space that is read when data is retrieved the more I/O and CPU time used, and thus the worse the performance.

  • Small Tables. Very small tables with high concurrency rates can present problems all of their own. Small static tables, which are never changed, are generally not a problem as long as any foreign keys are indexed. Some small semi-static tables can present serious concurrency issues when they are subject to high amounts of DML activity. The Accounts schema has a number of tables of this nature, particularly the Customer, Supplier, and COA tables. Each of these tables has summarized values. Originally I used triggers in my DBMS_JOBS initiated highly active database code. I had serious problems with deadlocks updating the summary columns. I disabled the triggers and updated these columns in batch mode when my activity code was no longer running. High concurrency changes can be made to small tables at the block level to resolve this problem. Years ago when working in the Ingres relational database the only method of resolving this type of problem was by filling each row with an empty column, padding each row up to the block size. Thus each row occupied a single block. Of course in those days Ingres block-locked rather than row-locked; Oracle Database locks rows. This is an extreme solution but always a possibility, even with Oracle Database.


14.5 How Much Block and Extent Tuning?

The simple answer to this question is not too much. Look for extremes and adjust block-sizing parameters for specific tables and indexes. Also do not forget the Oracle9i Database's new feature allowing separation of problematic tables into separate block size spaces, both physically and in memory.


14.6 Choosing Database Block Size

In general larger block sizes are appropriate to read-only data warehouse-type databases. Smaller block sizes are appropriate to highly concurrent, highly active OLTP databases with small transactions. Block size can be 2K, 4K, 8K, 16K, and 32K. An OLTP database could have serious problems reading a few bytes of data from 32K blocks. A data warehouse would suffer immense performance problems reading huge volumes of data from block size of 2K because of the huge number of blocks it would have to read.

Tip

Available block sizes and DB_nK_CACHE_SIZE parameters may vary for different operating systems. A 32K block size is not available for Win2K SP3 running Oracle9i Release 2 (9.2).

A block size of 8K is the Oracle Database default. I have never seen an OLTP Oracle database with a block size of less than 8K other than older versions of Oracle Database. Definitely beware of an OLTP Oracle database with 16K or 32K block size.

One other important factor is the DB_FILE_MULTIBLOCK_ READ_COUNT parameter.

Tip

The SORT_MULTIBLOCK_READ_COUNT has been deprecated from Oracle9i Database.

The DB_FILE_MULTIBLOCK_READ_COUNT parameter allows the reading of multiple blocks at once. The default value for Oracle9i Database is 8 for OLTP databases. I generally set this value to 8 but it can be set to 4 or even 2 for OLTP databases depending on reporting and the nastiness of application SQL code. Any reporting or poorly written SQL code tends to warrant a higher setting for the DB_FILE_MULTIBLOCK_READ_COUNT parameter. Bad SQL code tends to execute more full scanning anyway. Setting the DB_FILE_MULTIBLOCK_READ_COUNT parameter higher will encourage more full physical scanning.

So beware of OLTP databases with block size larger than 8K and a high DB_FILE_MULTIBLOCK_READ_COUNT. Large blocks and large block read counts do not fit well with small transactions and high DML activity. The result could be wasted I/O and probably extensive locking and latch wait problems. Do not set high values for anything to do with block size for OLTP databases. Do the opposite for data warehouse databases. Let's look further at different types of databases briefly:

  • OLTP/DSS Database. A smaller block size of perhaps 8K is preferable. If some reporting is required then multiple block reads can be taken advantage of. Also poorly written SQL code is common, which may execute a lot of full scans. Locking and latch wait problems could result but it is best to keep to commonly used standards. A block size of 8K is a commonly used standard. A balance between low block contention and the ability to read large amounts of data is important. With larger block sizes lock levels can become incremental in that locks on rows can degenerate into block locks and sometimes even exclusive table locks.

  • High DML Concurrency OLTP Database with Small Transactions. Perhaps use a small block size, perhaps even 4K or 2K. These block sizes are generally not used but very good performance would be gained for small transactions. Additionally table row lengths must be short. Getting less block contention is important. A small block size is better for index searches. As a side issue very high concurrency databases can have concurrency parameters adjusted to help with multiple session access and updates. However, high concurrency settings reserve more block header space and reduce block space available for data.

  • Data Warehouse or Read-Only. Use the largest block size possible (32K) assuming there is low concurrency. Higher concurrency should probably have the block size changed to 16K. Reading large amounts of data at once is important with serial access. Larger block size is better for full table scans and tends to encourage them.

    Tip

    Oracle9i Database allows multiple block sizes so there is no reason why different tables in any type of database could not be placed into different block-sized tablespaces. Many issues can be resolved by using multiple block sizes effectively providing multiple, nonconflicting pools of data at both the physical I/O and memory buffer levels. Multiple block-sized tablespaces can replace the KEEP and RECYCLE database buffer pools of Oracle8i Database.


14.7 Physical Block Structure

I like to make a distinction between physical block attributes and extent attributes. A block is a subset of an extent. Blocks contain data values. Extents are added to datafiles when a database object such as a table requires more physical datafile space and none can be allocated from existing unused extents. Extents can be sized and blocks can have specific allocation behaviors.

In this section we will discuss what is known as either the segments clause or the physical attributes clause. We will cover the storage clause later in this chapter. Physical attributes affect the physical structure of blocks and apply to objects such as tables, indexes, or clusters. Some factors apply to all object types and some apply to specific object types

14.7.1 What is in a Block?

A data block is the smallest unit of storage in a database. The block format is the same regardless of the type of object the data block contains. In general, an Oracle database block is divided into specific parts. These parts could be described vaguely as follows:

  • Fixed Header. The fixed header space contains general block information such as block address and segment type. The fixed block header is as shown in Figure 14.1 and size may vary by segment type.

    Figure 14.1: Block Fixed Header

  • Table Directory. The table directory space stores information about the tables and rows stored in the block as shown in Figure 14.2. Only clusters allow two or more tables to share a block.

    Figure 14.2: Block Table Directory

  • Row Directory. The row directory area contains information about all rows in the block as shown in Figure 14.3. Each row requires a few bytes of overhead. The row directory grows as rows are added to the block.

    Figure 14.3: Block Row Directory

  • Free Space. The free space area is the amount of space available either for insertion of new rows into the block or updates to existing rows as shown in Figure 14.4.

    Figure 14.4: Block Free Space

  • Row Data. The row data space contains table or index data as shown in Figure 14.5.

    Figure 14.5: Block Row Data

14.7.2 Block Space Management

Two parameters or settings are used to control how physical space is used in a block. PCTUSED determines how much space is required before rows can be inserted into a block. PCTFREE determines how much space is left free in a block when that block is first added to.

Tip

Setting values for PCTUSED, FREELIST, and FREELIST_ GROUPS in database objects such as tables will be ignored if the containing tablespace is set to automatic segment space management.

Default values for PCTUSED and PCTFREE are 40% and 10%, respectively.

  • PCTUSED. When a block has rows deleted such that its capacity falls below 40%, it is placed onto a list of blocks available for insertion of new rows.

    Tip

    Indexes do not reclaim space at the leaf block level where the index column values are stored. Thus PCTUSED does not apply to indexes and index-organized tables. This makes it essential to occasionally rebuild BTree indexes if their tables are subject to heavy deletion activity.

  • PCTFREE. 10% of space is reserved in every block as empty. When a row is updated and a row's length is increased that free space is used by the extra bytes required.

By no means do the Oracle Database default values have to be used. In fact it is advisable to change these values for certain types of combinations of DML activities on table and indexes.

Let's go through the process of tuning the block structure in the Accounts schema. My Accounts schema is a heavily concurrent DML intensive database. I will also adjust the approach assuming that an application would be running against this schema, including some reporting based on content in tables. First of all take a look at Figure 14.6. Figure 14.6 shows all tables with rows and blocks occupied after statistics generation.

Figure 14.6: Accounts Schema Current Row and Block Numbers

The number of rows in a table and the number of blocks occupied are important with respect to both DML and SELECT activity. Searching fewer rows is faster. Searching less physical space is also better. On the contrary, a few rows being accessed by many sessions using DML statements, all at once, can cause performance problems. Accessing a lot of rows in a single block is good for data retrieval but not for concurrent changes.

Possible changes for PCTUSED and PCTFREE parameters are shown in Figure 14.7. Index block structures must be changed in the same way that their respective tables are changed. This is because indexes undergo the same DML and SELECT activity as their respective tables.

Click To expand
Figure 14.7: Changes to Block Structure Space Usage

What exactly have I done in Figure 14.7? That is the question that should be asked. I have done various things for various reasons. Let's work through a list to break it up into easily understandable pieces.

What are influential factors for altering PCTUSED and PCTFREE?

  • Type of Table. A static table either does not change, changes very little, and is usually very small but not necessarily too small. Static tables are usually lookup tables but they can be subject to small amounts of DML activity.

  • Number of Rows. Distinguish between extremely small, average, and very large tables. Extremely small tables generally fit into a single block. Average-size tables are often static but occupy less than 100 blocks. Extremely large tables can have millions of rows and potentially millions of blocks.

  • Insert Activity. Raise PCTUSED for high insert activity to allow for reuse of blocks more easily.

  • Update Activity. Raise PCTFREE for high update activity involving possible row length changes.

  • Delete Activity. Raise PCTUSED to allow for future reuse of blocks.

  • Select Activity. Lower PCTFREE and raise PCTUSED to squash more rows into each block and thus increase data retrieval performance.

  • Potential Concurrency. This is very important because the more rows in a single block having DML activity on them at once the more likely hot blocking and locking will occur. Locking can cause the biggest potential performance problems due to lock and latch waits, especially in highly active OLTP databases like my Accounts schema.

    Tip

    The biggest danger from setting of low PCTFREE values is chained and migrated rows. Check the CHAIN_CNT statistic column in the USER_TABLES view.

Now I will group the tables and examine why I changed values for each table grouping.

  • Static Lookup. Category, Period, Subtype, and Type. These tables have PCTFREE = 0 and PCTUSED = 99. There is no point reserving free space for tables not subject to any DML activity. Lookup tables are only ever read and may as well be squashed up as much as possible to ensure a single block read every time.

  • Static (Subject to DML). The Period and Posting tables are subject to very occasional DML activity thus settings other than 0 and 99 would probably be pointless. Additionally the Posting table is subject to intense SELECT activity, which is better served by ensuring fewer blocks are occupied. The PeriodSum, Stock, and COA tables have high PCTFREE and low PCTUSED values for two reasons. The first reason is because all three tables are subject to a lot of updates. The COA table is higher than the other two because those updates involve higher row length growth. PCTFREE and PCTUSED can only be applied to new blocks. Thus existing data cannot be affected. Therefore, it makes sense to change these values on empty tables and then fill the tables with rows. Thus columns will be null-valued to start with. Three columns on the COA table are updated constantly. The Stock table has more rows updated but not as often. The same occurs with the PeriodSum table since it is an aggregation of the COA table. The second reason for high PCTFREE and low PCTUSED is to attempt to alleviate potential DML activity concurrency issues. Oracle Database locks rows and unlike other relational databases does not escalate row locking to block locks, even if there is high concurrent DML activity. Concurrent capacity in a block can be increased using the INITRANS and MAXTRANS parameters. High concurrent block activity can however lead to hot blocks and contention. Therefore, it makes sense to attempt to distribute rows over more blocks. These tables are small and multiple block read counts will probably cater for a single read even though there are potentially more blocks. The chances are if all the block and row size values were calculated precisely PCTFREE would probably have to be a lot higher. It is best to utilize a combination of multiple parameters including PCTUSED, PCTFREE, and INITRANS.

    Note

    Oracle Database 10 Grid MAXTRANS is deprecated.

  • Semi-static (Large). The Customer and Supplier tables are updated frequently but are much larger than other static tables. The Customer and Supplier tables are subject to a lot of lookup activity thus the relatively high PCTUSED values. PCTFREE is low at 20% but allows for some updating row length growth. The StockSource table does not grow nearly as much as Customer and Supplier tables in terms of updates because only a single column is changed, rather than three columns. Also queries executing against the StockSource table tend to read large numbers of rows.

  • Transactional. All standard transaction tables involve three master detail relationships for orders, transactions, and cash-book entries. These tables are generally only subject to insertion activity due to the nature of their purpose. It is very unlikely that an accounting system would have frequent changes to invoices. Accounting practices are strictly controlled. Changes to transactions involve new transactions executing the opposite effect. In other words, a sale transaction is documented using an invoice. A return of sale does not change the invoice but creates a new transaction called a credit note. The invoice and credit note are separate transactions. Therefore, all these six tables have minimal PCTFREE and fairly high PCTUSED values. PCTUSED is not set to 99% because typically transactional files in accounting systems are purged or transferred to archives at the end of distinct periods, such as every quarter.

  • Transactional Large. The GeneralLedger and StockMovement tables are very similar to logs or audit trails. A log file is filled over a period of time and then purged. Purging processing may involve aggregation into other tables, archiving or complete destruction of data. These types of tables are usually subject to heavy insertion and full table scan data select activity. Therefore, PCTFREE is 0 and PCTUSED is high but not too high. The reason why PCTUSED is 80% and not 99% is because these tables are large and space must be conserved. On the contrary, INSERT commands are better executed into clean blocks if possible. Appending of data is faster than searching for usable blocks. When a table like this is purged it is probably best to TRUNCATE the entire table rather than delete, or perhaps even drop and recreate the table. If that is the case then PCTUSED can be set to 99%.

Assessing PCTFREE Settings

A vague attempt can be made at assessing PCTFREE settings using a query such as this.

COL Table FORMAT a16;
COL Rows FORMAT 9999999;
COL Blocks FORMAT 9999999;
COL RPB FORMAT 9999999 HEADING "Rows/Block";
COL SU FORMAT 9999999 HEADING "Space Free %";

SELECT table_name "Table", num_rows "Rows", blocks "Blocks"
      ,num_rows/blocks AS RPB, ROUND(avg_space/8192*100) AS SU
      ,chain_cnt "Chaining"
FROM user_tables ORDER BY table_name;

Here are the results. Note that no changes have been made to PCTUSED and PCTFREE values in the Accounts schema at the time of execution of this query. All tables over 1,000 rows have over 10% of free space. This is a particularly obvious problem for the GeneralLedger and StockMovement tables because they never have any update activity.

Table              Rows  Blocks  Rows/Block  Space Free %  Chaining
--------------  -------  ------  ----------  ------------  --------
CASHBOOK         450957    4067         111            13         0
CASHBOOKLINE    2529733    6557         386            11         0
CATEGORY             13       1          13            96         0
COA                  55       1          55            77         0
CUSTOMER           2694      60          45            17         0
GENERALLEDGER   2158809    9654         224            10         0
ORDERS           435076    1924         226            13         0
ORDERSLINE      2866927    9757         294            12         0
PERIOD               72       1          72            68         0
PERIODSUM           221       1         221            35         0
POSTING               8       1           8            94         0
STOCK               118       4          30            27         0
STOCK-          3032315   12937         234            10         0
   MOVEMENT
STOCKSOURCE       12083      30         403            15         0
SUBTYPE               4       1           4            98         0
SUPPLIER           3874      85          46            16         0
TRANSACTIONS     450957    3091         146            13         0
TRANSACTIONS-   2896275    9847         294            12         0
   LINE
TYPE                  6       1           6            98         0

14.7.3 Block Concurrency

Block-level concurrency determines the number of locks which can be placed on a block. Concurrency capacity is the ability for rows in the same block to be accessed by multiple sessions at the same time, for both DML and SELECT activity. At the block level the parameters INITRANS and MAXTRANS help to control block concurrency. The INITRANS parameter can be increased for higher levels of concurrency.

  • INITRANS. This parameter determines the initial concurrency allowance for a block. Increasing INITRANS increases block concurrency (active transactions per block) but decreases available row data block space; more space is reserved in each block for higher transactional concurrency rates.

    Tip

    INITRANS must always be at least double the value for an index as for its parent table. The default for a table is 1 and for an index is 2. Set INITRANS to more than double for indexes when indexes are nonunique. The more duplicated values in an index the higher INITRANS should be, because searches on those values will be more likely to be in the same block.

  • MAXTRANS. This parameter determines the maximum number of active transactions per block. The more active transactions the more space required in block header areas and the less space allowed for row data.

    Note

    Oracle Database 10 Grid MAXTRANS is deprecated.

Let's look once again at the Accounts schema to determine appropriate changes to INITRANS parameters. I will not alter MAXTRANS parameter values. Possible changes to INITRANS are shown in Figure 14.8.

Click To expand
Figure 14.8: Changes to Block Structure Concurrency

In Figure 14.8 INITRANS values have been adjusted to account for potential DML activity concurrency; multiple transactions are changing rows in the same block at once.

Most drastically changed are the COA and PeriodSum tables based on their high amount of DML activity. The Stock table INITRANS value is lower since it occupies more physical blocks than the other two tables (see Figure 14.6). The large static tables have INITRANS set to 2 because they are subject to concurrent UPDATE command activity. Setting INITRANS to 2 for the GeneralLedger and StockMovement tables is probably an unwise choice. These tables are subject to intense INSERT command activity. A reduction in concurrency problems could probably be resolved by changing their sequence number generated primary keys into reverse key indexes. Reverse key indexes would prevent concurrent sequential insertions of similar key values into the same block for the primary key index.

Tip

Blocks already created cannot have their PCTUSED, PCTFREE, INITRANS, and MAXTRANS parameters changed. Changes only apply to new blocks.


14.8 Extent-Level Storage Parameters

Extent-level storage parameters apply to groups of blocks created in an extent when a segment in a tablespace is increased in size. A segment is increased in size when more space is required by new data. There are a number of parameters using the following syntax. Note that some parameters apply to specific object types.

STORAGE({
      [ INITIAL n[K|M] ] [ NEXT n[K|M] ]
      [ MINEXTENTS n ] [ MAXEXTENTS { n | UNLIMITED } ]
      [ PCTINCREASE n ]
      [ FREELISTS n ] [ FREELIST GROUPS n ]
      [ OPTIMAL [ n[K|M] | NULL ] ]
      [ BUFFER_POOL { KEEP | RECYCLE | DEFAULT } ]});

Let's look at the details.

14.8.1 Setting Extent Sizes

Initial and next extent sizes can be set using the INITIAL and NEXT parameters.

Tip

Use of locally managed as opposed to dictionary-managed tablespaces avoids the overhead and complexity of extent management.

As far as physical tuning is concerned here are a number of pointers:

  • Use a minimal number of extent sizes between different table-spaces and database objects such as tables and indexes. For some databases all extent sizes can be consistent throughout.

  • If using different extent sizes standardize different objects according to a rule such as functionality or growth rates.

  • Do not use Oracle Database defaults. They are small and will ultimately give you a lot of problems if your database grows at a reasonable rate.

INITIAL sets the size of the first extent. NEXT sets the size for all but the first extent unless PCTINCREASE is used; PCTINCREASE is best avoided. Generally you can use kilobyte ranges for small tables but use nothing less than 1M for anything larger than that. If you do want to use sizes matching each table then do not use too many different sizes, especially in the same tablespace. Deleted extents are not reused unless new extent requirements can fit into the old extent. Defragmentation on large tables is a much more serious problem than fitting a 1 block static table into a 1M extent; space is wasted physically but access time is not increased. Oracle Database reads in blocks not extents, reading one block multiplied by the multi-block read count parameter value; 1M will not be read. If you must use multiple extent sizes perhaps separate different objects into different tablespaces based on differing extent sizes and perhaps even multiple block size tablespaces. These things will help to alleviate fragmentation in the future.

14.8.2 Minimum and Maximum Extents

The MINEXTENTS value specifies that any new extent added to a datafile must be created at a minimum value, perhaps helping to alleviate fragmentation.

MAXEXTENTS should be set to UNLIMITED for everything but objects you know are uncontrollable and you are not concerned about causing a problem when MAXEXTENTS is reached. Do not set MAXEXTENTS UNLIMITED for rollback segments if there is any development or ad hoc SQL activity. Runaway transactions do occur often and can cause rollback to resize a datafile upwards so far as to fill all disk space and potentially crash your database.

Note

Oracle Database 10 Grid Manual rollback is deprecated and replaced by automatic undo (automatic rollback).

14.8.3 Variable Extent Sizes

PCTINCREASE can be used to gradually increase the size of extents. Do not use it. In a past version of Oracle Database PCTINCREASE had to be set to at least 1 in order to help automate tablespace-level coalescence. The default for PCTINCREASE was 50% in past versions of Oracle Database. Manual coalescence never worked very well so I do not see why automated coalescence should either. Also the defragmentation problem will crop up once again.

PCTINCREASE increases each new extent as a percentage increase of the previous extent. Any deleted extents will likely never be used again. The PCTINCREASE parameter was the culprit of many Oracle databases in the past being much larger and therefore much slower than they should have been. Fortunately locally managed tablespaces ignore the PCTINCREASE parameter.

14.8.4 Managing Concurrency

The FREELIST GROUPS and FREELIST parameters are generally only meaningful in very highly concurrent environments such as Oracle RAC (Parallel Server) environments. These parameters are beyond the scope of this book.

14.8.5 Minimizing Rollback Resizing

The OPTIMAL parameter determines the number of extents to which a rollback segment will shrink after having extended and completed a transaction. If OPTIMAL is not set then no rollback segment shrinkage will occur. Shrinking rollback segments will result in those rollback segments potentially extending once again when reused. Continual extending and shrinking is a waste of time. Increase OPTIMAL until it matches the size of most transactions. This can be assessed by minimizing the SHRINKS column in the V$ROLLSTAT dynamic performance view.

Note

Oracle Database 10 Grid Manual rollback is deprecated and replaced by automatic undo (automatic rollback).

Setting this parameter is not required for automatic undo.

14.8.6 Different Cache Recently Used Lists

The BUFFER_POOL storage setting allows for objects such as tables to be placed into different parts of the database buffer cache or, more specifically, to have them placed into either the MRU or LRU lists. The most recently used (MRU) list allows faster access by retaining blocks in the database buffer cache. The least recently used (LRU) list forces blocks out of the database buffer cache faster.

Various other options exist in Oracle9i Database for the purposes of caching specific tables in the database buffer cache. One is the CACHE option in the CREATE TABLE and ALTER TABLE commands. Another is using multiple block sizes for tablespaces and their related multiple-sized database buffer caches. The parameters DB_KEEP_CACHE_SIZE and DB_RECYCLE_CACHE_SIZE are to be favored over the BUFFER_POOL storage setting.

This chapter covered the low-level aspects of physical tuning encompassing block and extent structures. The next chapter will cover hardware resource usage tuning in the areas of CPU, memory, and I/O usage.


Chapter 15: Hardware Resource Usage Tuning

This chapter will briefly introduce hardware resource usage tuning, specifically dealing with CPU usage, memory usage, and I/O usage. More of the details on how to tune these various areas of an Oracle installation will be covered in later chapters. Let's start with tuning CPU usage.

15.1 Tuning Oracle CPU Usage

CPU usage can be best assessed at peak times and thus peak CPU workload times. What are peak workloads? This depends entirely on your application. An OLTP application is generally active and available 24 h a day, 365 days a year. There may be peak times and quiet times for OLTP databases but any very large highly active OLTP database generally passes batch operation work onto a data warehouse database, wherever possible. DSS-type databases tend to be a mixture of OLTP, client-server, and data warehouse type functionality. Therefore, some databases may have peak workloads for different types of activity at different times of the day. It is quite possible that client-server or OLTP-type service is provided during the day and batch operations are processed at night. The point to be made is this. Do not waste your time looking for something wrong with CPU usage when the CPU is idle. Wait until the CPU is really busy and then try to figure out what is overloading it, if anything. The CPU could simply be busy because busy is what it is meant to be!

So what do we do about tuning CPU usage? Well we cannot actually tune CPU usage itself, only determine what is being processed at any particular time and try to figure out what is causing heavy load or spiking. Then we can perhaps tune what is being processed, and perhaps better distribute processing if there is conflict as a result of different types of processing executing concurrently. Obviously processors can be upgraded and multiple CPU hardware platforms can be purchased. The crux of tuning CPU usage is to find out what is causing the problem and then tune the cause rather than attempt to "do something about the CPU".

15.1.1 Busy I/O and Intense CPU Activity

Is I/O activity intense and are CPUs very busy? If this is the case then it means the I/O is the same speed as CPU processing time. I/O activity should be much slower than the CPU. The CPU is a much faster piece of hardware than disk storage, without question. Excessive CPU activity can be due to a number of causes. We will concentrate on things related to Oracle Database rather than digging into the operating system too much.

Swapping and Paging

Swapping and paging are not precisely the same thing. A swap or a page fault occurs when a chunk of data is required but that chunk is not in memory. The chunk of data is then retrieved from swap space on disk and exchanged with something else if there is not enough space in RAM. Thus the information is swapped between RAM and a swap file on disk.

Tip

A swap file in Windows is called a paging file. The Windows paging file is added to RAM and constitutes virtual memory. The Windows paging file can be found in the System icon on the Control Panel.

Swapping or paging between disk "memory" and RAM physical memory is not a problem unless it occurs frequently, generating large amounts of I/O. Sometimes a process can cause a lot of swapping over and over again.

Swapping and paging can cause unnecessary use of the CPU, with respect to Oracle Database, often as a result of something caused within an Oracle Database configuration or an application activity. The obvious solution to swapping and paging is purchasing more RAM, as an easy short-term solution. However, as a system scales, if other problems are continuously placated with hardware upgrades, those hardware upgrades may eventually become very expensive.

15.1.2 Possible Oracle Database Causes of High CPU Activity

Unnecessary CPU use can often be the result of Oracle Database activity and related applications behavior.

Poorly Tuned SQL Code

Poorly tuned SQL code can cause a lot of re-parsing, which can cause intense CPU activity. Following are parsing and execution statistics.

COL name FORMAT a20;
SELECT name, value FROM v$sysstat
WHERE name LIKE '%parse%' OR name LIKE '%execute count%';

NAME                           VALUE
--------------------------  --------
parse time cpu                  1618
parse time elapsed             15096
parse count (total)            82683
parse count (hard)               293
parse count (failure)              8
execute count                 184523

In the next query currently executing SQL code is interrogated. Note that the Product column is a product of both parses and executions. The frequency of parses is more important than just the parses or executions.

COL sql_text FORMAT A38;
SELECT * FROM(
SELECT parse_calls*executions "Product", parse_calls
"Parses"
   ,executions "Execs", sql_text FROM v$sqlarea ORDER BY 1 DESC)
WHERE ROWNUM <= 10;

Product  Parses  Execs  SQL_TEXT
--------  -----  -----  ------------------------
53740146  2454   21899  INSERT into cashbookline
                          values(:b3,:b
53740146  2454   21899  INSERT into ordersline
                          values(:b4,:b3,
53740146  2454   21899  INSERT into transactionsline
                          values(:b
53740146  2454   21899  INSERT into stockmovement
                          values(stock
49695750  3250   15291  SELECT to_number
                          (to_char(SYSTIMESTAMP,
40742689  6383   6383   COMMIT
37466641  6121   6121   INSERT into generalledger
                          values (gene
37466641  6121   6121   INSERT into generalledger
                          values (gene
32890000  3250   10120  SELECT systimestamp FROM sys.dual
10575504  3252   3252   select privilege#,level from sysauth$

Increasing the SESSION_CACHED_CURSORS parameter allows caching of more SQL statements per session, potentially helping to reduce parsing. However, use of bind variables in SQL code will reduce parsing much more significantly, especially for highly concurrent OLTP databases where SQL code is highly shareable and if SQL code is poorly tuned.

Poor Index Usage

Missing indexes or too many large composite indexes can cause a lot of full table scans, leading to high I/O, swapping and probably high CPU usage times. Lack of use of indexes is most likely indicated by too many full table scans, particularly those on large tables, as indicated in the following query by full table scans on long tables (table scans (long tables)).

COL name FORMAT a32;
SELECT name, value FROM v$sysstat
WHERE name IN ('table scans (short tables)'
  ,'table scans (long tables)','table fetch by rowid');

NAME                          VALUE
---------------------------  ------
table scans (short tables)    40420
table scans (long tables)       111
table fetch by rowed          85673

Rollback and Undo

Problems with overusage of rollback and undo can possibly be resolved by increasing COMMIT command frequency in SQL code and reducing the number of SQL statements in single transactions, reducing transaction size. However, higher commit rates will lead to higher CPU usage. The following query can be used to check rollback segment usage in relation to extension and shrinkage. Similar information can be obtained from the V$UNDOSTAT performance view for automated undo. My database is currently using manual rollback segments. Automated undo tends to be less controllable with disk space. I have very limited disk space. On the other hand, automated undo simply removes much of the tuning requirements for manual rollback. However, it does not hurt to understand how rollback works because rollback and automated undo are essentially performing the same function.

Note

Oracle Database 10 Grid Manual rollback is deprecated.

COL RBS FORMAT a4;
SELECT n.name "RBS", s.extends "Extends", s.shrinks
   "Shrinks"
  ,s.wraps "Wraps", s.aveshrink "AveShrink"
  ,s.aveactive "AveActive"
FROM v$rollname n JOIN v$rollstat s USING(usn)
WHERE n.name != 'SYSTEM';

This query shows wrapping on all manual rollback segments. Rollback segment wraps are not a serious problem. It is a commonly held misconception that wrapping is reuse of the first extent of the currently used rollback segment, causing a snapshot too old error. On the contrary, wrapping is expanding a transaction into a new extent inside the same rollback segment. Excessive wrapping could imply that storage parameters within a rollback segment might be adjusted upwards to prevent wrapping.

RBS    Extends  Shrinks  Wraps  AveShrink  AveActive
-----  -------  -------  -----  ---------  ---------
RB00         0        0     13          0     779254
RB01         0        0     12          0     749639
RB02         0        0     12          0     749639
RB03         0        0     12          0     749330
RB04         0        0     13          0     779254
RB05         0        0     12          0     749330
RB06         0        0     12          0     749330
RB07         0        0     12          0     749330
RB08         0        0     13          0     779254
RB09         0        0     11          0     716991
RB10         0        0     13          0     778713
RB11         0        0     13          0     779254
RB12         0        0     12          0     749330
RB13         0        0     12          0     749330
RB14         0        0     12          0     749639
RB15         0        0     12          0     749639

Now let's take a different perspective and examine the amount and spread of activity within rollback segments.

COL RBS FORMAT a4;
SELECT n.name "RBS", s.status, s.waits, s.gets, s.writes
      ,s.xacts "Active Trans"
FROM v$rollname n JOIN v$rollstat s USING(usn)
WHERE n.name != 'SYSTEM';

This query shows a few interesting things. Transactions are fairly well spread across all the rollback segments, which is good. However, it might be that there are too many transactions per rollback segment. The impact of transactions per rollback segment depends somewhat on transaction size, which is extremely small in my database. There appears to be a fair amount of waiting for space in rollback segments, which is not so good.

RBS    STATUS  WAITS   GETS    WRITES  Active Trans
-----  ------  -----  -----  --------  ------------
RB00   ONLINE      9   6452  10669902             5
RB01   ONLINE     15   6375  10485074             4
RB02   ONLINE     14   6614  10670958             5
RB03   ONLINE     14   6800  10670618             5
RB04   ONLINE     19   6872  10759932             4
RB05   ONLINE      7   6431  10527624             5
RB06   ONLINE     13   6478  10707842             4
RB07   ONLINE     13   6345  10324652             4
RB08   ONLINE     10   6662  11237024             4
RB09   ONLINE     16   6270   9979316             4
RB10   ONLINE     14   6668  11021116             5
RB11   ONLINE     11   6412  10775736             3
RB12   ONLINE     13   6762  10702146             4
RB13   ONLINE      6   6476  10279358             5
RB14   ONLINE     12   6497  10475656             5
RB15   ONLINE      9   6341  10398742             4

Since transaction waits on rollback segments appear to occur let's dig further in that direction. This query will give an impression of rollback segment contention or competition for use of rollback segments. At just over 3% it might be warranted to create more rollback segments. It certainly would not do any harm.

SELECT ROUND(SUM(waits/gets)*100,2)||'%' "Contention" FROM
  v$rollstat;

Contention
----------
3.34%

I originally had 15 consistently sized rollback segments, which has now been increased to 50 rollback segments. Let's do some further querying to see if there is any improvement.

COL RBS FORMAT a4;
SELECT n.name "RBS", s.status, s.waits, s.gets, s.writes
      ,s.xacts "Active Trans"
FROM v$rollname n JOIN v$rollstat s USING(usn)
WHERE n.name != 'SYSTEM' ORDER BY 1;

This query looks at the first 10 of 50 rollback segments only. Transactions are now better spread across rollback segments with fewer active transactions in each rollback segment. Additionally there are much fewer waits.

RBS    STATUS  WAITS   GETS    WRITES  Active Trans
-----  ------  -----  -----  --------  ------------
RB00   ONLINE      0    149    158860             1
RB01   ONLINE      0    134    167698             1
RB02   ONLINE      0    145    203588             1
RB03   ONLINE      0    155    186144             2
RB04   ONLINE      0    149    147222             1
RB05   ONLINE      2    276    166638             2
RB06   ONLINE      0    150    151052             1
RB07   ONLINE      0    116    171126             1
RB08   ONLINE      0    145    215020             1
RB09   ONLINE      3    133    202338             1

Looking at contention again, much to my surprise it has actually increased. This is a classic example of a ratio giving a false impression. I just restarted my database so all rollback statistics have been cleared. As time passes this ratio will decrease. When I first started up my database after adding new rollback segments the ratio was over 12%. It has now dropped to around 7%. Over time it will decrease further. Additionally I am currently running over 100 jobs whereas over the last few hours, prior to bouncing my database, I was only running 10 jobs per second. The contention percentage shown previously was misleading, as is the one below. The previous percentage for contention was based on a much smaller load on the database. This current contention value shown here is misleading because it requires a period of time to settle down and build up statistics.

COL contention FORMAT 9999999990;
SELECT       AVG(xacts) "Trans per RBS"
      ,ROUND(SUM(waits/gets)*100,2)||'%' "Contention"
FROM v$rollstat;

Trans per RBS   Contention
-------------   ----------
   1.35294118        6.92%

This query shows a picture of automated undo segment statistics when executing highly active concurrency processing in my Accounts schema.

COL undob FORMAT 99990;
COL trans FORMAT 99990;
COL snapshot2old FORMAT 9999999990;
SELECT undoblks "UndoB", txncount "Trans"
 ,maxquerylen "LongestQuery", maxconcurrency
  "MaxConcurrency"
 ,ssolderrcnt "Snapshot2Old", nospaceerrcnt "FreeSpaceWait"
FROM v$undostat;

UndoB   Trans   LongestQuery   MaxConcurrency
-----   -----   ------------   --------------
 186    24596             30               66
4152    24339             71               67
3703    18835             58               62
2991    13644            143               74
2734     9175             13                1
2696     5372             12                3
 975     1383             12                2

Serious problems with rollback segments can sometimes cause snapshot too old errors. Snapshot too old errors occur when a rollback segment runs out-of-space and uncommitted transactions are overwritten. Do not confuse snapshot too old errors with wrapping in manual rollback segments. Adjust the UNDO_RETENTION parameter to remove snapshot too old errors when using automated undo segments for rollback.

Temporary Sort Space

Temporary sort space can be interrogated using a query such as the following query. Increasing the SORT_AREA_SIZE parameter decreases requirements for temporary sort space on disk but potentially increases memory requirements for database connections.

COL username FORMAT a10;
COL segtype FORMAT a10;
SELECT username, segtype, extents "Extents Allocated"
 ,blocks "Blocks Allocated"
FROM v$tempseg_usage;

USERNAME   SEGTYPE   Extents Allocated   Blocks Allocated
--------   -------   -----------------   ----------------
ACCSTEST      SORT                  47               6016

Row Locks and Latch Waits

Table or row locking and latch waits can cause waiting, sleeping, and spinning of processes. Locks in the database are possibly indicative of poor data model design or perhaps use of trigger code for event detection. Latch waits can be caused by poor buffer configuration parameters and, once again, poor SQL coding. It is important to note that buffer cache areas can be inappropriately sized in one of four ways. They can be too small, they can be the wrong sizes in relation to other buffers, they can be sized incorrectly according to application requirements, and they can even be too big. Making buffers too small can cause latch contention issues. Very large buffers may require intense management by Oracle Database. There can come a point where the database buffer cache or the shared pool can become so large as to somewhat negate their usefulness. Additionally large buffers increase the temptation to create sloppy SQL code. Many database administrators simply increase buffer sizes when there is a problem, without ever solving the root cause of that problem. Quite often that problem is caused as a result of a low ratio such as the database buffer cache hit ratio. Ratios will be examined in Chapter 18. Ratios provide possible symptoms of problems but are not problems in themselves. Oracle Database tuning documentation often stresses the importance of ratios far too much. Ratios are symptomatic of problems and should not be used as a measurement of performance but rather an indicator of possible performance problems, which are not necessarily directly related to those ratios.

Various performance views can be used to track locks and latch wait events. Wait events, latches, and the Oracle Database Wait Event Interface will be covered in Chapters 19-22. The V$SYSTEM_EVENT performance view can be used to determine wait events for free latches.

COL event FORMAT a30;
COL waits FORMAT 9999990;
COL timeouts FORMAT 99999990;
COL average FORMAT 99999990;
SELECT event "Event", time_waited "Total Time", total_waits
   "Waits", average_wait "Average", total_timeouts
   "Timeouts"
FROM V$SYSTEM_EVENT
WHERE event = 'latch free'
ORDER BY EVENT;

Event        Total Time   Waits   Average   Timeouts
----------   ----------   -----   -------   --------
latch free        77421   23645         3       3304

There are many more events than this.

High Network Activity

High network activity with many small queries can cause problems with high CPU activity. Be aware that breaking SQL code into small constituent parts is more natural to object designs rather than relational design. On the contrary, reduction of complexity into smaller parts leads to simplicity in tuning, especially when it comes to SQL code. There is a fine balance between too many small transactions and large complex SQL code join statements. Do not be concerned about network activity with SQL*Net too much. If you are sure that your network is the bottleneck take a look at your network, never overcomplicate SQL code to compensate for other problems.

The next query shows idle SQL*Net events. These values can help to describe Oracle network usage activity. However, for an OLTP database server high network activity is normal.

SELECT name FROM v$EVENT_NAME
WHERE name LIKE '%SQL*Net%' OR name LIKE '%ispatcher%'
OR name LIKE '%virtual circuit%';

NAME
-------------------------------
alter system set dispatcher
dispatcher shutdown
virtual circuit status
dispatcher timer
dispatcher listen timer
SQL*Net message to client
SQL*Net message to dblink
SQL*Net more data to client
SQL*Net more data to dblink
SQL*Net message from client
SQL*Net more data from client
SQL*Net message from dblink
SQL*Net more data from dblink
SQL*Net break/reset to client
SQL*Net break/reset to dblink


15.2 How Oracle Database Uses Memory

Tuning the way that Oracle Database uses memory is a highly complex subject. This section will serve as an introduction to how Oracle Database uses memory with a brief description of tuning Oracle Database memory usage.

Oracle Database uses memory as a number of buffers to cache information, instructions, and data in. These buffers are placed into both RAM and virtual memory and generally exist to help speed up performance. Note that it is best to contain all buffers in RAM and not spread into virtual memory. Also do not neglect memory requirements for the operating system and anything else running on your database server. If something can be accessed from a memory buffer rather than from disk storage then I/O activity will be reduced since accessing memory is much faster than accessing from disk storage.

Tip

There is a limit to how large memory buffers can be. Managing the multi-user aspects of obtaining information from buffers is much more complex than managing locking of table data stored on disk. There is a point where buffers can become too big and become detrimental to performance.

The different buffers not only have to be tuned in relation to CPU usage and I/O activity but also in relation to each other. Some buffers depend on the proper sizes of other buffers to perform at optimal pace. Here we will briefly introduce tuning the buffer cache areas from a generalized perspective. What are the separate buffers areas?

  • The Database Buffer Cache. Contains database object data such as table and index rows for fast access in RAM, reducing I/O.

    Note

    Oracle Database 10 Grid The ALTER SYSTEM command allows flushing of the database buffer cache. This option should only be used for testing purposes.

  • The Shared Pool. Contains highly shareable information such as database metadata and parsed SQL code.

    Tip

    Flushing the shared pool is not an action which should be used on a production server. The need to flush the shared pool is probably due to oversizing of the shared pool, probably as a result of application issues. Flushing the shared pool will not resolve application issues and it absolutely will hurt performance because anything shareable must be reloaded.

  • Session Connection Buffers. Every database connection creates a session. Shared memory areas are allocated to each session both within the shared pool and the large pool depending on configuration. Some applications continually connect and disconnect from the database for every SQL code command executed. This type of activity is common in Java object applications. Objects tend to oversimplify complexity whereas a relational database can thrive on complexity.

  • The Large Pool. A cache area less structured than the shared pool with similar functions.

  • The Redo Log Buffer. Redo log entries record all database change activity.

  • The Java Pool. Used for the Oracle JVM.

All of the buffers together are known as the System Global Area and can be examined in general using the SHOW SGA command in SQL*Plus. The same information can be found by querying the V$SGA performance view. The V$SGASTAT performance view will provide memory allocations to all parts of the SGA.

SHOW SGA;

Total System Global Area   122755896   bytes
Fixed Size                    453432   bytes
Variable Size               88080384   bytes
Database Buffers            33554432   bytes
Redo Buffers                  667648   bytes

Here are some of the more interesting parts of the SGA using the V$SGASTAT performance view.

COL name FORMAT a32;
SELECT pool, name, bytes FROM v$sgastat
WHERE pool IS NULL OR pool = 'large pool'
OR (pool = 'shared pool'
      AND (name IN('dictionary cache','enqueue','library cache'
      ,'parameters','processes','sessions','free memory')))
ORDER BY pool DESC NULLS FIRST, name;

POOL           NAME                   BYTES
------------   ----------------   ---------
               buffer_cache        46137344
               fixed_sga             453492
               log_buffer            656384
shared pool    dictionary cache     1610880
shared pool    enqueue               223368
shared pool    free memory         58436864
shared pool    library cache        2497744
shared pool    parameters              6264
shared pool    processes             192000
shared pool    sessions              543600
large pool     PX msg pool           122880
large pool     free memory          4071424

It is best to make sure that the entire SGA fits into RAM and does not overflow into temporary disk space or virtual memory. As already stated, space should be reserved in RAM for other functions such as other processes, applications, and the operating system. The LOCK_SGA parameter forces the entire SGA to remain in RAM, excluding the use of virtual memory disk space. Only some platforms support this parameter.

Note

Oracle Database 10 Grid A new performance view called V$SGAINFO gives a better picture of the SGA.

Now let's look at the different pools in a little more detail.

15.2.1 The Database Buffer Cache

The database buffer cache is available in Oracle9i Database in two forms. The first form is the original form using the DB_BLOCK_BUFFERS parameter and various other parameters. The second form is using the DB_CACHE_SIZE parameter, along with its associated parameters. In general the same rules and methods govern the way in which the two database buffer caches are tuned. The DB_BLOCK_BUFFERS parameter will not be covered in this book.

Note

Oracle Database 10 Grid The DB_BLOCK_BUFFERS parameter was deprecated in Oracle9i Database but is still usable. This parameter is no longer available in Oracle Database 10g.

The database buffer cache can be bypassed. When it is bypassed then data is read directly from disk. Full table scans are read directly from disk amongst various other operations such as sorting and parallel processing.

Two further factors warrant discussion:

  • Multiple Block Sizes. Oracle9i Database allows for tablespaces with different block sizes. Large transactions can benefit from larger block size tablespaces and small transactions from smaller block sizes. The database buffer cache is thus split into separate caches involving all but the default, using parameters named DB_[ 2 | 4 | 8 | 16 | 32 ]K_CACHE_SIZE.

    Tip

    Available block sizes and thus DB_nK_CACHE_SIZE parameters may vary for different operating systems. A 32K block size is not available for Win2K SP3 running Oracle Database Release 2 (9.2).

  • Multiple Buffer Pools. Oracle9i Database handles the keep and recycle pools using the DB_KEEP_POOL_SIZE and DB_RECYCLE_POOL_SIZE parameters. The keep pool should be used to attempt to keep small static or frequently used tables and indexes in memory. The recycle pool should be used to remove large infrequently accessed row sets from memory as quickly as possible. Note that the keep and recycle pools can only contain blocks contained in tablespaces of block size defined by the DB_BLOCK_SIZE parameter; use the DB_nK_CACHE_SIZE buffers otherwise.

15.2.2 The Shared Pool

The shared pool allows for sharing and reuse of information and previously processed executions. The shared pool retains highly shareable data in memory used simultaneously by many database users. Things stored in the shared pool include items such as previously parsed and executed SQL code, latches protecting buffer areas and data dictionary metadata. All of these items are heavily used by all sessions connected to the database. The shared pool is set using the SHARED_POOL_SIZE parameter. Setting the SHARED_POOL_RESERVED_SIZE parameter to a small percentage of the shared pool, normally less than 10%, will reserve a small portion of the shared pool for contiguous large transactions. This can help to avoid large amounts of defragmentation in the shared pool when large transactions are executed. Defragmentation in the shared pool can cause serious wait event and latch contention problems.

The reserved section of the shared pool can be examined with the following query.

SELECT request_misses, request_failures, free_space
FROM v$shared_pool_reserved;

Even when running large queries on my database free space remains constant at 2 Mb, the setting for the SHARED_POOL_ RESERVED_SIZE parameter. My reserved shared pool is never used because the rest of the shared pool is large enough and not suffering from fragmentation as a result of poorly tuned SQL code. Request misses imply that flushing occurred because the shared pool reserved portion is not large enough for large transactions, or it is being used. Request failures are worse. If either of the misses or failures columns have a value greater than zero, perhaps increase the value of the parameter SHARED_POOL_RESERVED_ SIZE. I have seen some success setting this parameter to values sometimes 10% and over of the shared pool for databases executing large amounts of poorly tuned SQL code. The default setting is 5%. The better solution is to tune the SQL code but that is often not possible. If you are going to tell a customer to rewrite an application you might as well be telling them to reinvent the wheel. Using bind variables in SQL code or forcing their use with the CURSOR_SHARING parameter can alleviate these issues substantially but be cautious using cursor sharing in a data warehouse. As always there is no better solution than tuning SQL code. Using bind variables as a quick fix to avoid tuning poorly built SQL code is relative to how scalable the database needs to be in the long term. Additionally data warehouse databases perform best without bind variable use, due to matching of statistics over vast quantities of data and the use of histograms. Statistics are very precise, if they are kept up to date. Using bind variables in SQL code will offset the accuracy of statistics in large data warehouses.

REQUEST_MISSES   REQUEST_FAILURES   FREE_SPACE
--------------   ----------------   ----------
             0                  0      2014320

The shared pool is quite literally a shared area of memory containing various types of information in various caches used to help Oracle Database perform better. What are these different memory caches?

  • The Library Cache. Parsed SQL and PL/SQL code plus optimized query execution plans for SQL code. Also included is Java class code.

  • The Metadata Cache. Data about the data stored in memory for rapid access.

Tuning and sizing the shared pool correctly is critical to performance. Proper "sharing" of high-usage information between many users is more critical than proper tuning of other buffers, other than perhaps a completely inappropriately sized redo log buffer. The database buffer cache is not as important as the shared pool because data in the shared pool is more likely to be shared by more users.

The Library Cache

The library cache contains parsed SQL code and latches. It is important to note that efficiently tuned SQL code will not affect library cache performance directly. Properly tuned SQL code will help database performance in general. Bind variables used in SQL code will help to allow the library cache to share previously parsed SQL code. The problem with sharing using bind variables is that in very large or widely variant data sets bind variables can generalize statistics. This need to use statistics precisely is more important in data warehouses but exact value searches with statistics can potentially perform much faster.

SQL code tuning is covered in Part II of this book. Most of the aspects of SQL code tuning affecting performance of the library cache are based on the way in which SQL code is written and the ways in which applications manage connections to the database. Latches and locks will be covered later in this book. Note that management of latches can become difficult if shared pool space allocated to latch management is too low if the shared pool is either too small or filled with too many different things.

Examine library cache statistics using the V$LIBRARYCACHE performance view. The GETS column applies to locking requests, the PINS column to pins in the shared pool, RELOADS indicates I/O activity, and INVALIDATIONS denotes SQL code re-parsing.

SELECT namespace, gets, pins, reloads, invalidations
      FROM v$librarycache;

NAMESPACE          GETS      PINS   RELOADS   INVALIDATIONS
---------------   -----   -------   -------   -------------
SQL AREA          97385   1577407        16               0
TABLE/PROCEDURE   50741    494446         0               0
BODY              24624     24623         0               0
TRIGGER           52336     52336         0               0
INDEX                66        35         0               0
CLUSTER             167       221         0               0
OBJECT                0         0         0               0
PIPE                  0         0         0               0
JAVA SOURCE           0         0         0               0
JAVA RESOURCE         0         0         0               0
JAVA DATA             0         0         0               0

Here are a few pointers:

  • Maintain Connections. Do not continuously connect and disconnect from the database for every SQL code operation. Establishing and breaking a database connection involves starting up and stopping of processes and assignment and releasing of chunks of memory. It is better to share connections, especially with large numbers of users, preferably using a middle-tier server utilizing some type of connection pooling. Oracle shared servers is not as effective as a middle-tier since connection pool management takes processing power away from the database server.

  • Generated SQL. Generated SQL is SQL generated "on the fly" from within an application. This type of application-based SQL code is always unpredictable and is largely dependent on the coding skills of multiple programmers. Additionally programmers always have their own distinctive styles and it is very difficult to enforce standards.

  • Copying Schemas to Different Users. There is some weight to be granted for creating separate schemas for each user. However, this type of application-based database design tends to be top-down and not built from the perspective of database efficiency. See the next section on the metadata (dictionary) cache.

  • PL/SQL and DDL. PL/SQL can help since packages can be pinned into contiguous areas of memory at database startup. However, many developers will balk at the idea of placing too much business logic and application-type processing into the database. Executing DDL commands on highly active objects is detrimental to performance because many DDL commands will invoke table locks amongst other unpleasant things.

  • SESSION_CACHED_CURSORS. The library cache effectively caches cursors at the database level. The SESSION_CACHED_ CURSORS parameter can be used to cache cursors for each session. A cached cursor is a reusable cursor. Reusing a cursor removes the need for re-processing.

    SELECT name, value FROM v$sysstat WHERE name
      IN('session cursor cache count','session cursor
         cache hits');

NAME                          VALUE
---------------------------   -----
session cursor cache hits         0
session cursor cache count        0

    Search for unnecessary parse calls using the V$SQLAREA view.

    COL execs FORMAT 9999990;
COL parses FORMAT 99990;
COL fetches FORMAT 999990;
COL loads FORMAT 9990;
COL invalids FORMAT 9999990;
COL i/o FORMAT 99990;
COL hits FORMAT 9999990;
COL rows FORMAT 999990;
COL sorts FORMAT 9990;
COL sql FORMAT a32;

SELECT * FROM(
SELECT executions "Execs"
       --Executions
      ,parse_calls "Parses"         --Parses
      ,fetches "Fetches"            --Fetches
      ,loads     "Loads"            --Loads and reloads
      ,invalidations "Invalids"     --Invalidations
      ,disk_reads "I/O"             --I/O
      ,buffer_gets "Hits"           --Buffer hits
      ,rows_processed "Rows"        --Rows
      ,sorts       "Sorts"          --Sorts
      ,sql_text "SQL"
FROM v$sqlarea
ORDER BY executions DESC
) WHERE ROWNUM <= 10;

Execs    Parses   Fetches   Loads     I/O     Rows   SQL
------   ------   -------   -----   -----   ------   ---------
210140    24788         0     1      1477   210139   INSERT in
210140    24788         0     1      2205   210140   INSERT in
210140    24788         0     1      2081   210131   INSERT in
210137    24789         0     1      3022   210142   INSERT in
154342    32775    154342     1       164   154342   SELECT to
101918    32773    101918     1       180   101918   SELECT sy
64544     64544         0     1         0        0   COMMIT
61054     61056         0     1     19950    61054   INSERT in
61050     61050         0     1     14588    61050   INSERT in
45193      4603         0     1       330    45193   INSERT in

  • CURSOR_SPACE_FOR_TIME. This parameter can be set to TRUE to only remove cursors from the shared pool when they are closed rather than aging them out according to MRU and LRU lists. A parameter like this can cause problems if for instance cursors are left open by applications or ad hoc SQL code power users. What happens if someone goes to lunch?

The Metadata or Dictionary Cache

The metadata cache contains all of the database definitional data such as table and index structural definitions in memory in the shared pool. Metadata is generally accessed much more frequently than anything else in the database and is absolutely critical for efficiency.

Too much metadata in the database can cause serious performance problems. Let's once again examine a top-down database design built from the perspective of an application. Sometimes these types of database designs are characterized by profligate use of schema copies, views, and synonyms, increasing memory requirements for metadata even further. It is always best to access objects directly. Be aware of creating too many logical object layers. Views and synonyms can often be used for security purposes. Be forewarned of the hidden costs of building your database structure from the point of view of the application.

The metadata cache can be examined in great detail using the V$ROWCACHE performance view. My Accounts schema high-activity code using DBMS_JOBS scheduled mass DML activity is apparent in the results of the following query. The Accounts schema uses sequences as primary keys so there is high sequence activity. Manual rollback segments are currently used in my database as is apparent as well, amongst other things.

SELECT COUNT "Entries", GETS "Requests", GETMISSES "Misses"
      ,MODIFICATIONS "DML Activity", parameter "Area"
FROM v$rowcache;

Entries  Requests  Misses  DML Activity  Area
-------  --------  ------  ------------  --------------
      0         0       0             0  dc_free_extents
      0         0       0             0  dc_used_extents
    169       683     169             0  dc_segments
      4     33577       4             0  dc_tablespaces
      2       193       2           193  dc_tablespace_quotas
      0         0       0             0  dc_files
     19    204738      19             0  dc_users
     29      5202      28            49  dc_rollback_segments
    292      1095     237            55  dc_objects
      6        12       6             0  dc_global_oids
      0         0       0             0  dc_constraints
    301   1075530     246            55  dc_object_ids
      7     31064       7         31064  dc_sequences
      5       332       5             0  dc_usernames
      0         0       0             0  dc_database_links
     79       226      79             0  dc_histogram_defs
      0         0       0             0  dc_table_scns
      0         0       0             0  dc_outlines
      1     40792       1             0  dc_profiles
      0         0       0             0  dc_encrypted_objects
      0         0       0             0  dc_encryption_
                                            profiles
      0         0       0             0  dc_qmc_cache_entries
      0         0       0             0  dc_users
      0         0       0             0  dc_histogram_data
      0         0       0             0  dc_histogram_data_
                                             values
      0         0       0             0  dc_partition_scns
     15       411      15             0  dc_user_grants
      0         0       0             0  dc_app_role

Pinning Objects in the Shared Pool

PL/SQL packages can be pinned in memory to place them into a contiguous area to prevent further parsing processing. Packages should be pinned at database startup to ensure that contiguous areas of memory are used and parts of packages are not spread all over the shared pool and thus defragmented. Packages can be pinned into memory using the DBMS_SHARED_POOL package using the KEEP procedure.

DBMS_SHARED_POOL.[UN]KEEP (object, type);

The type parameter is set as P, C, R, or Q for package, cursor, trigger or sequence, respectively.

15.2.3 Session Connection Cache

Every connected session is connected directly to a dedicated server process, or indirectly using a dispatcher to a shared server process. A server process provides access to the database server for a client connection. A proportion of shared server process memory requirements are passed to the large pool. The large pool will be discussed later in this chapter. Session connection cache for each connection can be automatically or manually managed for dedicated connections but only manually managed for shared server connections. General connection cache involves sort space, bitmap operation space, and space in memory for hash joins.

The session connection cache is the area of memory assigned to managing connections to the database. There are two methods of handling database connections.

  • Dedicated Server Connection. This type of connection maintains a connection to every client process even when client activity is idle. Dedicated server configuration is generally used for client server implementations or connections to middle-tier application or web servers where the burden of connection pooling and load balancing is placed onto the middle tier.

    Note

    Oracle Database 10 Grid Using the two parameters WORKAREA_SIZE_POLICY = AUTO and PGA_AGGREGATE_TARGET for automated session connection memory management has been recommended by Oracle Corporation since Oracle9i Database as better performing. Dedicated server connection configurations can perform well even at 1,000 concurrent connections.

  • Shared Server Connection. Server connection processes are shared between multiple database connections, usually implemented for highly active OLTP databases. Shared server architectures are totally unnecessary when a middle tier is included in the general system architecture. Shared servers connect to client connections through the management of network dispatcher processes.

    Note

    Oracle Database 10 Grid Oracle installation software and the Database Configuration Assistant defaults to dedicated server mode from Oracle9i Database onwards.

Various parameters are involved in controlling session-level connection memory. There are also two separate options. The first option is automated memory management and the second is manual memory management. Automated memory management can only be used with dedicated server database connections.

Automated Memory Management

  • WORKAREA_SIZE_POLICY = AUTO.

  • PGA_AGGREGATE TARGET = n. Total PGA memory available for dedicated server database connections. Statistics can be obtained from the V$PGASTAT performance view.

    SELECT * FROM v$pgastat;

NAME                                  VALUE    UNIT
--------                           --------   -----
aggregate PGA target parameter            0   bytes
aggregate PGA auto target                 0   bytes
global memory bound                       0   bytes
total PGA inuse 8950784 bytes
total PGA allocated                85715968   bytes
maximum PGA allocated              85715968   bytes
total freeable PGA memory                 0   bytes
PGA memory freed back to OS               0   bytes
total PGA used for auto workarea          0   bytes
maximum PGA used for auto workar          0   bytes
total PGA used for manual workar          0   bytes
maximum PGA used for manual work          0   bytes
over allocation count                     0
bytes processed                           0   bytes
extra bytes read/written                  0   bytes
cache hit percentage                      0   percent

Manual Memory Management

  • WORKAREA_SIZE_POLICY = MANUAL. This option switches off the use of the PGA_AGGREGATE_TARGET parameter.

  • SORT_AREA_SIZE = n. Determines the size of the sort buffer. When the buffer is exceeded for a connection sorting will be passed to temporary sort space on disk. Serious performance problems can occur in some situations. Firstly, if the sort buffer is too small and too many sorts are moved to disk heavy I/O will result. Secondly, too large a sort buffer could potentially use up too much memory since every database connection is allocated the amount of memory specified in the sort buffer as a maximum for sorting. If the SORT_AREA_SIZE is set to 1M then 2,000 users could require up to 2 Gb of RAM, assuming there are enough connections executing concurrent sorting. Thirdly, make sure a temporary sorting tablespace is created in the database and that each user is allocated that temporary tablespace. The default temporary sorting tablespace can be set in the CREATE DATABASE or ALTER DATABASE commands. If not set, the SYSTEM metadata tablespace will be used for sorting.

    Note

    Oracle Database 10 Grid On disk temporary space can be allocated as tablespace groups where a tablespace group can contain multiple locally managed temporary tablespaces. A tablespace group will allow spreading of SQL execution sorting across multiple temporary tablespaces, thereby potentially speeding up sorting operations. Processing is distributed. A user can be allocated a tablespace group as a temporary sort space as opposed to just a single temporary tablespace.

  • SORT_AREA_RETAINED_SIZE = less than 10% of SORT_ AREA_SIZE. This parameter helps to retain memory space for multiple sorts in the same query.

    Tip

    It is possible that the value of the SORT_AREA_RETAINED_ SIZE parameter should be set equal to the SORT_AREA_SIZE parameter, especially where large sorts are very common. Since many OLTP databases are often in reality OLTP/DSS databases this may always apply.

  • CREATE_BITMAP_AREA_SIZE = n. Used to create bitmap indexes in memory.

  • BITMAP_MERGE_AREA_SIZE = n. In memory bitmap merge operations.

  • HASH_AREA_SIZE = n. Used for hash joins in SQL join statements.

15.2.4 The Large Pool

The large pool is not a subset of the shared pool buffer but is used as an area of memory to contain what could possibly be termed shared pool buffer overflow in certain circumstances.

  • Some shared server session connection memory.

  • Backup processing.

  • Parallel execution.

Shared Servers and Virtual Circuits

A virtual circuit is a chunk of memory reserved for a shared database connection through a dispatcher process. A dispatcher process can support multiple concurrent client processes where each client connection is communicated with using a virtual circuit; the virtual circuit is used to pass requests and responses from and to the client.

15.2.5 The Redo Log Buffer

The redo log buffer is a cyclical buffer, which under certain circumstances is either fully or partially flushed to the currently active redo log file. The redo log buffer cache will be written to redo log files when it is 1/3 full; a COMMIT or ROLLBACK command is issued every 3 s or if it reaches 1 Mb.

Tip

A checkpoint is a process of flushing dirty buffers from database buffer cache to disk storage. The redo log buffer is not flushed to disk when a checkpoint occurs. This is the consensus of opinion from a number of database administrators and contrary to many Oracle software texts.

During a checkpoint the database writer (DBWR) will post the log writer (LGWR) to write redo from the buffer to the disk if the DBWR process has to write dirty blocks for which redo entries have not been written. This is essential for recoverability. Nothing will be written to datafiles without first being written to redo log files. The DBWR process will actually stop and wait for LGWR process to complete writing redo log entries. Redo log entries are always written to disk before database changes to ensure recoverability. Redo log files are best on the fastest disk storage available.

What can cause problems with respect to the redo log buffer?

  • The size of the redo log buffer must be appropriate for database activity.

    Tip

    It is unlikely that the size of the redo log buffer will ever need to be changed from the default value created by the Database Configuration Assistant. Quite often the log buffer is inappropriately sized because it has been tampered with.

  • The number and size of redo log files.

  • The frequency of log switching can affect the availability of redo log files for buffer flushing.

How can we tell if the redo log buffer is too small? Contrary to popular belief the redo log space requests entry in the V$SYSSTAT view records requests for the buffer to write out to a redo log file, not requests for space in the buffer. The redo entries statistic tells us how often redo entries are copied into the buffer. The redo writes statistic tells us how often buffer entries were written to redo log files. The redo buffer allocation retries event tells us about retries required when a process has attempted to allocate free space in the log buffer. If the log writer (LGWR) falls behind this will occur. However, this event can also occur when a log switch occurs. Therefore, the occurrence of this event can have more than one meaning. If this event occurs because log buffer space is unavailable then the log buffer may be too small. On the contrary, if it occurs because of a log switch then perhaps the log files are too small. Speeding up log writing by using faster disks should always be considered as well.

15.2.6 The Java Pool

The Java pool is set using the JAVA_POOL_SIZE parameter. Unless the Oracle JVM is created for a database this parameter can be set to zero (JAVA_POOL_SIZE = 0). It is a complete waste of memory otherwise. If the Oracle JVM is created in a database the Java pool buffer must be set.

Note

Oracle Database 10 Grid The JAVA_POOL_SIZE parameter is now dynamic and can be changed using the ALTER SYSTEM command and a binary parameter file (SPFILE).

15.2.7 Getting Advice on Buffers

There are various performance views providing advisory projections of potential performance as to sizing for various buffers. Let's start with the database buffer cache advisory view.

Database Buffer Cache Advice

The database buffer cache advisor performance view is called V$DB_CACHE_ADVICE. The following query shows the potential database buffer cache hit ratio decreasing as the cache size increases.

COL pool FORMAT a10;
SELECT (SELECT ROUND(value/1024/1024,0) FROM v$parameter
      WHERE name = 'db_cache_size') "Current Cache(Mb)"
,name "Pool", size_for_estimate "Projected Cache(Mb)"
,ROUND(100-estd_physical_read_factor,0) "Cache Hit Ratio%"
FROM v$db_cache_advice
WHERE block_size = (SELECT value FROM v$parameter
      WHERE name = 'db_block_size')
ORDER BY 1, 2;

Current     Pool       Projected   Cache Hit
Cache(Mb)              Cache(Mb)      Ratio%
---------   --------   ---------   ---------
       32   DEFAULT            4          92
       32   DEFAULT            8          93
       32   DEFAULT           12          96
       32   DEFAULT           16          99
       32   DEFAULT           20          99
       32   DEFAULT           24          99
       32   DEFAULT           28          99
       32   DEFAULT           32          99
       32   DEFAULT           36          99
       32   DEFAULT           40          99
       32   DEFAULT           44          99
       32   DEFAULT           48          99
       32   DEFAULT           52          99
       32   DEFAULT           56          99
       32   DEFAULT           60          99
       32   DEFAULT           64          99
       32   DEFAULT           68          99
       32   DEFAULT           72          99
       32   DEFAULT           76          99
       32   DEFAULT           80          99

For my Accounts schema I could decide to move some static and some very large tables into the keep and recycle buffer pools, respectively. Firstly, I change my parameters file on my database and bounce that database. I am not currently using an SPFILE binary parameter file for this database.

Tip

Many Oracle9i Database parameters can be changed online using the ALTER SYSTEM command and the binary SPFILE parameter file. Not all parameters can be changed online; check the ISSYS_MODIFIABLE column in the V$PARAMETER view. The binary parameter file allows for configuration changes without forcing a database restart. Be sure to make backup copies of parameter files.
Note

Oracle Database 10 Grid More parameters can be changed online.

My database parameter file has been changed and I have added the two keep and recycle pool parameters.

COL name FORMAT a24;
SELECT name, value FROM v$parameter WHERE name LIKE
   'db%cache%size%';
NAME                       VALUE
------------------------  -------
db_keep_cache_size        4194304
db_recycle_cache_size     8388608
db_2k_cache_size                0
db_4k_cache_size                0
db_8k_cache_size                0
db_16k_cache_size               0
db_32k_cache_size               0
db_cache_size            33554432

In Oracle8i Database the keep and recycle pools were subset MRU and LRU listings placed within the database buffer cache using the BUFFER_POOL storage parameter in the ALTER TABLE and ALTER INDEX commands. In Oracle9i Database keep and recycle pools are separate pools, as shown in the following by the change from 32M to around 44M for the entire database buffer cache.

SHOW SGA;

Total System Global Area   135338868   bytes
Fixed Size                 453492      bytes
Variable Size              88080384    bytes
Database Buffers           46137344    bytes
Redo Buffers               667648      bytes

In my Accounts schema I could distribute tables to different pools as shown in the following examples, placing the smallest tables in the keep pool and the largest tables in the recycle pool. Note that there are many other factors to consider. Assuming that static data sizes are known, the keep pool could be sized more precisely to actual data sizes. The Customer and Supplier tables are semi-static and relatively large, thus depending on current activity they could possibly benefit from being part of the keep pool.

ALTER TABLE type STORAGE(BUFFER_POOL KEEP);
ALTER TABLE subtype STORAGE(BUFFER_POOL KEEP);
ALTER TABLE period STORAGE(BUFFER_POOL KEEP);
ALTER TABLE posting STORAGE(BUFFER_POOL KEEP);
ALTER TABLE category STORAGE(BUFFER_POOL KEEP);
ALTER TABLE coa STORAGE(BUFFER_POOL KEEP);
ALTER TABLE generalledger STORAGE(BUFFER_POOL RECYCLE);
ALTER TABLE stockmovement STORAGE(BUFFER_POOL RECYCLE);

Since the tables shown previously are pushed into keep and recycle pools it would make sense to examine their respective indexes. Indexes can be shifted to different pools regardless of where their parent tables are placed. Since I have placed small static tables into the keep pool to account for Optimizer full table scans there is no point in placing their indexes in the keep pool. It would however be sensible to place Customer and Supplier indexes into the keep pool on that basis.

ALTER INDEX xak_cust_name STORAGE(BUFFER_POOL KEEP);
ALTER INDEX xak_cust_ticker STORAGE(BUFFER_POOL KEEP);
ALTER INDEX xak_supp_name STORAGE(BUFFER_POOL KEEP);
ALTER INDEX xak_supp_ticker STORAGE(BUFFER_POOL KEEP);
ALTER INDEX xpk_customer STORAGE(BUFFER_POOL KEEP);
ALTER INDEX xpk_supplier STORAGE(BUFFER_POOL KEEP);

In contradiction to what was previously mentioned, full table scans on small static tables bypass the database buffer cache altogether, making it pointless to place those tables in the keep buffer pool. So perhaps static table indexes should be placed in the keep pool, not the tables. However, large joins including these static tables are likely to require fast access to static tables and their indexes. The first assumption was correct.

Let's look at the V$DB_CACHE_ADVICE performance view again, restricting the output somewhat.

COL pool FORMAT a10;
SELECT (SELECT ROUND(value/1024/1024,0) FROM v$parameter
      WHERE name = 'db_cache_size') "Current Cache(Mb)"
,name "Pool", size_for_estimate "Projected Cache(Mb)"
,ROUND(100-estd_physical_read_factor,0) "Cache Hit Ratio%"
FROM v$db_cache_advice
WHERE block_size = (SELECT value FROM v$parameter
      WHERE name = 'db_block_size')
AND size_for_estimate IN (4,8,32)
ORDER BY 1, 2;

The keep pool is set at 5% of the default pool, which is 1.6 Mb. The V$DB_CACHE_ADVICE view does not offer an estimate less than 4M, thus there is no response for the keep pool.

Current               Projected        Cache
Cache(Mb)   Pool      Cache(Mb)   Hit Ratio%
---------   -------   ---------   ----------
       32   DEFAULT           4           97
       32   DEFAULT           8           98
       32   DEFAULT          32           99
       32   KEEP              4           99
       32   KEEP              8           99
       32   KEEP             32           99
       32   RECYCLE           4           99
       32   RECYCLE           8           99
       32   RECYCLE          32           99

Shared Pool Advice

The V$SHARED_POOL_ADVICE performance view provides potential information for varying shared pool sizes. Unfortunately as can be seen in the following query result there are no differences between estimates in the various columns. Perhaps this performance view will work a little more usefully in the next version of Oracle Database.

SELECT (SELECT ROUND(value/1024/1024,0) FROM v$parameter
      WHERE name = 'shared_pool_size') "Current Mb"
, shared_pool_size_for_estimate "Projected Mb"
, ROUND(shared_pool_size_factor*100) "%"
, ESTD_LC_SIZE "Library Mb"
, ESTD_LC_TIME_SAVED "Parse Savings"
,ESTD_LC_MEMORY_OBJECT_HITS "Hits"
FROM v$shared_pool_advice
ORDER BY 1;

Current   Projected    %    Library    Parse
Mb               Mb              Mb   Savings      Hits
-------   ---------   ---   -------   -------   -------
     64          32    50         7    121352   2824153
     64          40    63         7    121352   2824153
     64          48    75         7    121352   2824153
     64          56    88         7    121352   2824153
     64          64   100         7    121352   2824153
     64          72   113         7    121352   2824153
     64          80   125         7    121352   2824153
     64          88   138         7    121352   2824153
     64          96   150         7    121352   2824153
     64         104   163         7    121352   2824153
     64         112   175         7    121352   2824153
     64         120   188         7    121352   2824153
     64         128   200         7    121352   2824153

The V$LIBRARY_CACHE_MEMORY performance view provides a window into the Library cache part of the shared pool.

SELECT lc_namespace "Library"
,LC_INUSE_MEMORY_OBJECTS "Objects"
,LC_INUSE_MEMORY_SIZE "Objects Mb"
,LC_FREEABLE_MEMORY_OBJECTS "Freeable Objects"
,LC_FREEABLE_MEMORY_SIZE "Freeable Mb"
FROM v$library_cache_memory;

                          Objects   Freeable   Freeable
Library         Objects        Mb    Objects         Mb
-------------   -------   -------   --------   --------
BODY                  0         0         18          0
CLUSTER              13         0          4          0
INDEX                 8         0         31          0
JAVA DATA             0                    0          0
JAVA RESOURCE         0         0          0          0
JAVA SOURCE           0         0          0          0
OBJECT                0         0          0          0
OTHER/SYSTEM          0         0          6          0
PIPE                  0         0          0          0
SQL AREA             75         0        757          5
TABLE/PROCEDURE      61         0        521          1
TRIGGER               6         0         27          0

PGA Advice

The V$PGA_TARGET_ADVICE and V$PGA_TARGET_ADVICE_ HISTOGRAM performance views are predictive or advisory views when using automated workarea policy for database connection cache.

Java Pool Advice

Oracle Database 10 Grid Two news views are provided for assistance with and analysis of the Java pool: V$JAVA_POOL_ADVICE and V$JAVA_LIBRARY_ CACHE.


15.3 Tuning Oracle I/O Usage

Different types of applications require different approaches to tuning. OLTP databases need a quick response time for small transactions. Data warehouses or batch processing requires high throughput of large transactions. A number of factors are important with respect to tuning of I/O with Oracle Database. The two most important factors with respect to I/O are efficiency and recoverability. Do not sacrifice recoverability to gain a little extra efficiency unless downtime and partial data loss can be tolerated. For an OLTP database downtime is usually not acceptable. For most databases data loss is not acceptable.

  • Striping. Evenly distribute I/O by physically distributing files.

  • Seek Time. An adequate number of disks with adequate seek times.

  • Contention. High levels of contention due to high I/O request waits can be relieved using striping where files are spread across multiple disks.

  • Mirroring. Mirroring is a process of real-time maintenance of multiple copies of data. Mirroring can be expensive in I/O time.

  • Block Size and Multiple Block Reads. Block size can have a drastic effect on I/O but with Oracle9i Database and later, which allow multiple-sized block tablespaces, this issue is more manageable. The DB_FILE_MULTIBLOCK_READ_COUNT parameter can have a profound effect on access speed depending on application and database type, allowing many blocks to be read at once. Larger transactions benefit from a larger block size and reading of multiple blocks in a single read. The same can be detrimental to small transactions since full table scans are more likely and simply too much data is read. The DB_FILE_MULTIBLOCK_READ_COUNT parameter is discussed in other chapters.

The V$FILESTAT performance view can be somewhat useful in assessing I/O for a database. Notice how busy the SYSTEM, RBS (rollback), DATA, and INDX datafiles are. These activity levels and ratios are consistent with a DML intensive OLTP database, there is far more index activity (INDX) than table activity (DATA).

SELECT fs.phyrds "Reads", fs.phywrts "Writes"
      ,fs.avgiotim "Average I/O Time", df.name "Datafile"
FROM v$datafile df, v$filestat fs WHERE df.file# =
fs.file#;

                     Average
 Reads   Writes     I/O Time   Datafile
------   ------   ----------   --------
  6724     2423   -1.890E+09   C:\ORACLE\ORADATA\TEST\SYSTEM01.DBF
    29       27     76687938   C:\ORACLE\ORADATA\TEST\OEM01.DBF
    29       27   -558789658   C:\ORACLE\ORADATA\TEST\CWMLITE01.DBF
    29       27   -558789658   C:\ORACLE\ORADATA\TEST\DRSYS01.DBF
  3105   167903   1647296225   C:\ORACLE\ORADATA\TEST\RBS01.DBF
    29       27    712165534   C:\ORACLE\ORADATA\TEST\USERS01.DBF
    29       27   1270955192   C:\ORACLE\ORADATA\TEST\ODM01.DBF
    29       27   1347643130   C:\ORACLE\ORADATA\TEST\TOOLS01.DBF
 73836    81038    335849851   C:\ORACLE\ORADATA\TEST\DATA01.DBF
    34       27    239184070   C:\ORACLE\ORADATA\TEST\XDB01.DBF
541943   397992   -1.362E+09   C:\ORACLE\ORADATA\TEST\INDX01.DBF
    29       27   1944776757   C:\ORACLE\ORADATA\TEST\OBJ01.DBF
    29       27   -1.194E+09   C:\ORACLE\ORADATA\TEST\SMALLDATA.DBF

The next query looks at I/O activity for all datafiles in all table-spaces, including the temporary sort tablespace called TEMP.

COL ts FORMAT a10 HEADING "Tablespace";
COL reads FORMAT 999990;
COL writes FORMAT 999990;
COL br FORMAT 999990 HEADING "BlksRead";
COL bw FORMAT 999990 HEADING "BlksWrite";
COL rtime FORMAT 999990;
COL wtime FORMAT 999990;
SELECT ts.name AS ts, fs.phyrds "Reads", fs.phywrts "Writes"
      ,fs.phyblkrd AS br, fs.phyblkwrt AS bw
      ,fs.readtim "RTime", fs.writetim "WTime"
FROM v$tablespace ts, v$datafile df, v$filestat fs
WHERE ts.ts# = df.ts# AND df.file# = fs.file#
UNION
SELECT ts.name AS ts, ts.phyrds "Reads", ts.phywrts "Writes"
      ,ts.phyblkrd AS br, ts.phyblkwrt AS bw
      ,ts.readtim "RTime", ts.writetim "WTime"
FROM v$tablespace ts, v$tempfile tf, v$tempstat ts
WHERE ts.ts# = tf.ts# AND tf.file# = ts.file# ORDER BY 1;

This query does tell much. In general, reads appear much more numerous than writes, except for rollback segments and temporary sort space. This is normal for temporary sort space. Read values in rollback segments are indicative of transactions read from rollback to maintain data consistency since what is being read has already changed. This type of activity needs to be minimized at the application level if possible and if other performance factors are not compromised. For an OLTP database block to I/O rates should be as close as possible. Also read times should be much faster than write times, probably faster than in this case. This disparity could be application related and a result of read consistency, reading data from already changed data in rollback.

Tablespace   Reads   Writes   BlksRead   BlksWrite    RTime    WTime
----------   -----   ------   --------   ---------   ------   ------
DATA          6496     2708      20402        2708    27470    55010
INDX         26560    12080      76474       12080   172513   257221
OBJECTS          5        2          5           2       22        0
OEM              5        2          5           2       38        0
RBS            502     7622        502        7622     3598   121476
SMALLDATA        5        2          5           2       15        0
SYSTEM        2324      226       5889         226     2754     5283
TEMP        108686   124975     108686      124975    22295    17654
USERS            5        2          5           2       37        0

Let's get a slightly different picture and use the same query but produce relative rates.

COL ts FORMAT a10 HEADING "Tablespace";
SELECT ts.name AS ts
      ,ROUND(fs.phyrds/(fs.phywrts+1),1) "Read/Write"
      ,ROUND(fs.phyblkrd/(fs.phyrds+1),1) "Blocks/Read"
      ,ROUND(fs.phyblkwrt/(fs.phywrts+1),1) "Blocks/Write"
      ,ROUND(fs.readtim/(fs.writetim+1),1) "Read/Write Time"
FROM v$tablespace ts, v$datafile df, v$filestat fs
WHERE ts.ts# = df.ts# AND df.file# = fs.file#
UNION
SELECT ts.name AS ts
      ,ROUND(ts.phyrds/(ts.phywrts+1),1) "Read/Write"
      ,ROUND(ts.phyblkrd/(ts.phyrds+1),1) "Blocks/Read"
      ,ROUND(ts.phyblkwrt/(ts.phywrts+1),1) "Blocks/Write"
      ,ROUND(ts.readtim/(ts.writetim+1),1) "Read/Write Time"
FROM v$tablespace ts, v$tempfile tf, v$tempstat ts
WHERE ts.ts# = tf.ts# AND tf.file# = ts.file# ORDER BY 1;

In this query output we can see relative ratios. Note that these numbers are not necessarily meaningful because they are application dependent. In an OLTP database there should be more write activity than read activity and preferably fewer blocks per read, and definitely fewer blocks per write. A data warehouse would be exactly the opposite. More blocks per read imply full scans of one type or another. Full table scans in particular are not generally good for OLTP databases unless there is a mix of OLTP and DSS activity. Otherwise data model complexity causes full scans to occur frequently as a result of SQL code using large convoluted joins, amongst other possibilities.

Tablespace   Read/   Blocks/   Blocks/        Read/
             Write      Read     Write   Write Time
----------   -----   -------   -------   ----------
DATA           2.4       3.1         1           .5
INDX           2.2       2.9         1           .7
OBJECTS        1.7        .8        .7           22
OEM            1.7        .8        .7           38
RBS             .1         1         1            0
SMALLDATA      1.7        .8        .7           15
SYSTEM        10.2       2.5         1           .5
TEMP            .9       1 1         1           .3
USERS          1.7        .8        .7           37

The result might be a little clearer with 0 rounding.

Tablespace   Read/   Blocks/   Blocks/        Read/
             Write      Read     Write   Write Time
----------   -----   -------   -------   ----------
DATA             2         3         1            0
INDX             2         3         1            1
OBJECTS          2         1         1           22
OEM              2         1         1           38
RBS              0         1         1            0
SMALLDATA        2         1         1           15
SYSTEM          10         3         1            1
TEMP             1         1         1            1
USERS            2         1         1           37

15.3.1 RAID Arrays

The RAID acronym stands for Redundant Array of Inexpensive Disks; lots of small cheap disks. RAID arrays, in numerous levels of sophistication, and vast differences in cost, some exorbitant, can provide enormous increases in performance, loss recoverability and reliability. There are different types of RAID architectures. RAID types used with Oracle databases are generally RAID 0, RAID 1, RAID 0+1, and RAID 5.

  • RAID 0. RAID 0 is striping. Striping is a process of splitting files into little pieces and spreading those pieces over multiple disks. RAID 0 can provide fast random read and write access performance but nothing in the way of rapid recoverability and redundancy. RAID 0 is the most efficient method for OLTP databases but a little dangerous. RAID 0 could possibly be insured for recoverability and availability using clustering failovers or failover standby databases for automated, instant, up-to-the-moment replacement of service loss.

  • RAID 1. RAID 1 is mirroring. Mirroring is a simple process of multiple copies of files, making duplicate entries into multiple files every time a change is made to a file. RAID 1 can cause I/O bottleneck problems with respect to high-usage areas of disk storage, namely high-usage database objects such as frequently written tables. On the contrary, read access can be extremely fast since some highly sophisticated RAID arrays can read more than one mirror copy of a file at the same time, allowing for parallel reads. RAID 1 is potentially appropriate for sequential access for redo logs and index spaces in OLTP databases, and in data warehouses. RAID 1 is not appropriate for random access of table rows from indexes using ROWID pointers.

  • RAID 0+1. This option combines the best aspects of RAID 0 and RAID 1, providing both striping for read and write access performance plus mirroring recoverability and parallel read access. RAID 0+1 is not as fast as RAID 0 but faster than RAID 1.

  • RAID 5. RAID 5 provides a more simplistic version of mirroring where only parity rather than a complete duplicate is provided. RAID 5 can be particularly effective in very expensive RAID array architectures with special buffering RAM built onboard with the RAID array itself. RAID 5 is best used for sequential reading and not random write access. RAID 5 is not appropriate for OLTP databases other than to contain log files and perhaps indexes.

Once again further details on problem detection and tuning of I/O are discussed in later chapters. The next chapter will deal with tuning network usage.


Chapter 16: Tuning Network Usage

Overview

This chapter will cover Oracle network tuning. Oracle networking software is sometimes known as SQL*Net. Many times I have heard that the only way to tune SQL*Net is to change the SDU or Session Data Unit buffer size. This is not correct. There are many ways in which SQL*Net configuration can be adjusted, improving performance under varying circumstances. The easiest method of describing how, is to describe each facet of SQL*Net tuning, and to display by example. Let's begin with the Listener process and its various possible configurations.

Note

Oracle Database 10 Grid Managing network configuration files is simplified where manual configuration is no longer required. Additionally performance enhancements have been made including support for high-speed networks and better network bandwidth use between database and application servers.


16.1 The Listener

The Listener is a process residing on a database server or other machine that "listens" for database connection requests. When a request is received the Listener hands off or passes on the connection to a database server process. Following is a very simple Listener configuration file. This file is called listener.ora by default and is placed in the $ORACLE_HOME/network/admin directory.

LISTENER =
   (DESCRIPTION_LIST =
      (DESCRIPTION =
         (ADDRESS = (PROTOCOL = TCP) (HOST = <hostname>)
            (PORT = 1521))
      )
   )

SID_LIST_LISTENER =
   (SID_LIST =
      (SID_DESC =
         (GLOBAL_DBNAME = <SID>.<xyz.com>)
         (SID_NAME = <SID>)
         (ORACLE_HOME = /oracle/ora92)
      )
   )

What can be done to tune or at least improve the Listener under certain circumstances?

  • Listener Queue Size. Allows a larger number of Listener requests to be serviced by allowing them to wait in a queue.

  • Listener Logging and Tracing. Logging is defaulted on and tracing is defaulted off.

  • Multiple Listeners and Load Balancing. Randomized load balancing can be created between multiple Listeners, pointing at the same database.

16.1.1 Listener Queue Size

Very large quantities of network requests and traffic to the Listener can cause the Listener to use a queue in order to allow the Listener to keep up with requests. If there are too many requests then waiting requests to the Listener will be queued. The default length of the Listener queue is operating-system-specific and can be increased by setting the QUEUESIZE parameter in the Listener configuration file.

LISTENER =
   (DESCRIPTION_LIST =
      (DESCRIPTION =
         (ADDRESS =
            (PROTOCOL = TCP)
            (HOST = <hostname>)
            (PORT = 1521)
            (QUEUESIZE = 50)
         )
      )
   )

SID_LIST_LISTENER =
   (SID_LIST =
      (SID_DESC =
         (GLOBAL_DBNAME = <SID>.<xyz.com>)
         (SID_NAME = <SID>)
         (ORACLE_HOME = /oracle/ora92)
      )
   )

16.1.2 Switching Off Listener Logging and Tracing

By default a file called $ORACLE_HOME/network/log/listener.log file is created and constantly appended to. Tracing can also be switched on but is off by default. Logging and tracing of the Listener process can be switched off by setting the appropriate configuration parameters in the listener.ora file. Switch off Listener logging and especially tracing if this is switched on. Log and trace files can become extremely large and will affect performance. Unless really necessary do not log or trace the Listener process.

LISTENER =
   (DESCRIPTION_LIST =
      (DESCRIPTION =
      (ADDRESS = (PROTOCOL = TCP) (HOST = <hostname>)
         (PORT = 1521))
      )
   )

SID_LIST_LISTENER =
   (SID_LIST =
      (SID_DESC =
         (GLOBAL_DBNAME = <SID>.<xyz.com>)
         (SID_NAME = <SID>)
         (ORACLE_HOME = /oracle/ora92)
      )
   )

LOGGING_LISTENER = OFF
TRACE_LEVEL_LISTENER = OFF

16.1.3 Multiple Listeners and Load Balancing

Load balancing can help performance by providing multiple Listener connection points to a database server. If one Listener is busy another Listener process can be deferred to, thus reducing load on the first Listener. So let's add a Listener. The listener.ora configuration might now look something like that shown below. Note that the Listener called LISTENER2 uses a different port number to the Listener called LISTENER. Both of the two Listeners LISTENER and LISTENER2 allow connections to the same database.

LISTENER =
   (DESCRIPTION_LIST =
      (DESCRIPTION =
         (ADDRESS = (PROTOCOL = TCP) (HOST = <hostname>)
            (PORT = 1521))
      )
   )

LISTENER2 =
   (DESCRIPTION_LIST =
      (DESCRIPTION =
         (ADDRESS = (PROTOCOL = TCP) (HOST = <hostname>)
            (PORT = 1522))
      )
   )

SID_LIST_LISTENER =
   (SID_LIST =
      (SID_DESC =
         (GLOBAL_DBNAME = <SID>.<xyz.com>)
         (SID_NAME = <SID>)
         (ORACLE_HOME = /oracle/ora92)
      )
   )

SID_LIST_LISTENER2 =
   (SID_LIST =
      (SID_DESC =
         (GLOBAL_DBNAME = <SID>.<xyz.com>)
         (SID_NAME = <SID>)
         (ORACLE_HOME = /oracle/ora92)
      )
   )

So how do we implement load balancing between multiple Listener processes? The configuration file shown opposite is the client configuration file called the Transparent Network Substrate configuration file. This file is placed in the $ORACLE_HOME/network/ admin directory and is called tnsnames.ora. The tnsnames.ora file, which is a local naming client connection configuration file, can be placed on both Oracle server and client installations, specifying how a client process such as SQL*Plus can communicate with a Listener process. Note how there are two connection strings with ports 1521 and 1522, matching the different Listener address in the previously shown listener.ora file.

<TNSname> =
   (DESCRIPTION =
      (ADDRESS_LIST =
         (LOAD_BALANCE = YES)
         (ADDRESS = (PROTOCOL = TCP) (HOST = <hostname>)
            (PORT = 1521))
         (ADDRESS = (PROTOCOL = TCP) (HOST = <hostname>)
            (PORT = 1522))
      )
      (CONNECT_DATA = (SID = <SID>) (ORACLE_HOME =
         /oracle/ora81))
   )


16.2 Network Naming Methods

There are a number of different network naming methods Oracle Database can use. What is a naming method? A naming method is a way in which a node name on a network will be translated into a form understandable and routable by hardware and network software on that network. The only fact that can really be stated about tuning naming methods with respect to Oracle Database is as follows. The smaller number of steps required to pass messages across a network using the SQL*Net software then the faster database communication will be. It should be noted though that not much can be done to speed up SQL*Net in this respect. There are five different configurations available. Applying different configuration parameters within the different naming methods can help to increase Oracle SQL*Net performance processing.

  • Local Naming. Uses a configuration file called tnsnames.ora or Transparent Network Substrate. This method is the default method and the most commonly used. Using an IP-Address instead of a hostname will be noticeably faster but if network IP-Addresses change frequently, which they are want to do, this could cause a maintenance problem.

  • Directory Naming. Net service names and their mappings are stored in a lightweight directory access protocol (LDAP) server, much like a DNS server.

  • Oracle Names. Typically used for globally distributed networks involving the use of an Oracle Names server. The Oracle Names server is used to store service addresses and net service names. An Oracle Names server can reside on a machine separate to that of the database server, thus distributing processing.

    Note

    Oracle Database 10 Grid Oracle Names is desupported. Directory Naming can be used instead.

  • Host Naming. This is the simplest method of database communication using operating-system-based IP-Address to hostname mapping host files. Host files on Solaris Unix are placed in the /etc/system directory and on Windows in the c:\windows\ system32\drivers\etc directory.

  • External Naming. Third-party naming services.

  • Oracle Database 10 Grid Easy Connect. Allows a direct connection to a database server using a connection string as shown in this syntax diagram and the following example.

CONNECT user/password@host:port/service

This example connects me to my database server on my 2000server host, through port 1521, connecting to the database called TEST.

16.2.1 Local Naming

As far as tuning is concerned, local naming using the tnsnames.ora file is the naming method that will suffice for explanatory purposes.

Dedicated Versus Shared Servers

When a user connects to a database a server process is used to service the connection. The server process can either be a dedicated server process or a shared server process. For a dedicated server process the process is started up and retained for the duration of the connection. A shared server process, on the other hand, is shared indirectly between different connections through dispatcher processes. Shared server processes and dispatchers comprise what used to be called Oracle MTS or Multi-Threaded Server configuration.

Tip

Shared servers was called MTS or Multi-Threaded Server prior to Oracle9i Database.

Shared server processes and dispatchers will be covered later on in this chapter. Both dedicated and shared server connections are shown in Figure 16.1. This diagram is repeated for convenience and is a copy of Figure 11.1.

Click To expand
Figure 16.1: Dedicated Versus Shared Servers

A dedicated server process can be used to allocate a dedicated connection to a local name. A dedicated server process connection can be forced in a shared server process configuration, and is necessary for certain types of functionality such as using RMAN.

Tip

RMAN is an acronym for Recovery Manager.

Following is an example tnsnames.ora file containing TNS name connections for an OLTP database and an RMAN backup connection. RMAN requires a dedicated server connection.

<TNSname> =
   (DESCRIPTION =
      (ADDRESS_LIST =
         (ADDRESS = (PROTOCOL = TCP) (HOST = <hostname>)
            (PORT = 1521))
      )
      (CONNECT_DATA =
         (SID = <SID>)
         (ORACLE_HOME = /oracle/ora81)
         (SERVER = SHARED)
      )
   )

RMAN =
   (DESCRIPTION =
      (ADDRESS_LIST =
         (ADDRESS = (PROTOCOL = TCP) (HOST = <hostname>)
            (PORT = 1521))
      )
      (CONNECT_DATA = (SERVICE_NAME = RMAN) (SERVER =
         DEDICATED))
   )

The Session Data Unit Buffer (SDU)

SQL*Net uses a buffer to contain information ready for sending over the network. The buffer is flushed to the network and sent on request or when it is full. The larger the buffer is then the more information is sent over the network at once. SDU buffer size can range from 512 bytes up to 32 Kb. The default setting is operating system dependent. The following example shows a tnsnames.ora configuration file with its SDU buffer size altered. Sessions containing large transactions may benefit from a larger SDU buffer size.

<TNSname> =
   (DESCRIPTION =
      (ADDRESS_LIST =
         (ADDRESS = (PROTOCOL = TCP) (HOST = <hostname>)
            (PORT = 1521))
      )
      (SDU=1024)
      (CONNECT_DATA = (SID = <SID>) (ORACLE_HOME =
         /oracle/ora81)
      )
   )


16.3 Connection Profiles

A profile is a definitional configuration placed onto a client machine or a server where parameters within the profile definition determine how a connection to that server machine will behave. For instance, on a Unix machine, each Unix user has a profile. Essentially this profile declares variables global to the user (local variables) and any default behavior required. Unix user profiles would contain variables such as path names. Following is a stripped-down example Unix profile for a Unix Oracle database administrator user on an Oracle8i Release 2 (8.1.7) database server.

#This .cshrc file is executed in tcsh (/usr/local/bin/tcsh)
   on Solaris
setenv ORACLE_BASE /<mp1>/oracle
setenv ORACLE_HOME /<mp1>/oracle/product/8.1.7
setenv ORACLE_DOC /<mp1>/oracle/doc
setenv LD_LIBRARY_PATH /<mp1>/oracle/product/8.1.7/lib:
   /usr/openwin/lib
setenv JAVA_HOME /<mp1>/oracle/product/jre/1.1.8
setenv TNS_ADMIN /<mp1>/oracle/product/8.1.7/network/admin
setenv ORACLE_SID <SID>
setenv ORACLE_DBF1 /<mp1>/oracle/oradata/$ORACLE_SID
setenv ORACLE_DBF2 /<mp2>/oracle/oradata/$ORACLE_SID
setenv ORACLE_BACKUPS /<mp2>/backups
setenv ORACLE_SBIN /<mp1>/oracle/product/8.1.7/sbin
setenv ORACLE_ALERT /<mp1>/oracle/admin/$ORACLE_SID/bdump
setenv EDITOR vi

setenv PATH /bin:/usr/bin:/etc:/usr/ccs/bin:/usr/openwin/
   bin:/usr/ucb
setenv PATH ${PATH}:/usr/local/bin:/usr/sbin:/usr/X/bin:
   $JAVA_HOME
setenv PATH ${PATH}:/<mp1>/oracle/product/8.1.7/bin
setenv PATH ${PATH}:/<mp1>/oracle/product/8.1.7/sbin

set host='hostname'
alias sp 'set prompt="$user@$host:r":"$cwd>"
alias cd 'cd \!*;sp'
alias rm 'rm -i'
alias ll 'ls -la'
cd

alias dbs 'cd $ORACLE_HOME/dbs'
alias bin 'cd $ORACLE_HOME'
alias net8 'cd $TNS_ADMIN'
alias pfile 'cd $ORACLE_BASE/admin/$ORACLE_SID/pfile'
alias alert 'cd $ORACLE_ALERT'
alias sbin 'cd $ORACLE_HOME/sbin'
alias dbf1 'cd $ORACLE_BASE/oradata/$ORACLE_SID'
alias dbf2 'cd /<mp2>/oracle/oradata/$ORACLE_SID'
set filec
set history=100
umask 077

The $ORACLE_HOME/network/admin/sqlnet.ora contains profile information for connections to an Oracle database server. In the same way that a Unix profile is instantiated when a Unix user logs into Unix, the sqlnet.ora file instantiates a profile into a session through SQL*Net. SQL*Net profile parameters are numerous and generally cover client logging, Oracle Names configuration, SQL*Net security authentication, SQL*Net tracing (switch all this off), and even one or two Connection Manager oriented values.

It appears that more recent versions of Oracle Database are using SQL*Net profile parameters on client machines more and more. Expect more usage of SQL*Net profile parameters in the future.

As far as tuning SQL*Net profile parameters is concerned there are three parameters of immediate interest to tuning.

  • TCP.NODELAY=YES. Large amounts of data passed through SQL*Net can have the buffer flushed to the network immediately. Otherwise, there may be a delay before information is passed over the network.

  • USE_DEDICATED_SERVER=YES. Force all connections to the database server from the current client machine to connect to dedicated server connections, adding (SERVER = DEDICATED) to name connection descriptions.

  • SQLNET.EXPIRE_TIME=5. Timeout on client SQL*Net connections, allows automatic termination of idle SQL*Net connections to the database server in order to help free up needed resources.

  • Oracle Database 10 Grid Network Outage Detection. Various new profile parameters can be configured to limit network time used for any particular operation.

Using these profile parameters will not really make much difference to overall performance unless directed at resolving a specific performance problem. What happens when ad hoc SQL code users go to lunch? Connections not executing COMMIT or ROLLBACK commands, if they are required, could possibly use timeout capable connections.


16.4 Shared Servers

When using a dedicated server configuration connections are granted exclusively to a specific user request. Therefore, one dedicated server process plus a chunk of connection cache in memory is allocated to each connection. As a result dedicated servers cannot be scaled up to a large number of users. The solution to high resource usage is sharing of those resources using shared servers and dispatchers. A shared server process can be shared between multiple user connections where dispatcher processes dispatch requests and results between a shared server process and a client connection. In effect shared processes are never idle because they are shared or switched between different user connections.

Some functions, mostly database administration functions, are best served and sometimes only served by dedicated server connections. When using a dedicated server configuration process connection requests are routed by the Listener process. The Listener passes the client process an address for an available dedicated server process, which may have to be spawned. In the case of a shared server configuration the Listener will pass the client process the address of an available dispatcher process (spawned if none is available), which in turn sends the client process the address of a shared server process. The client process then communicates with the database server using a shared server process.

16.4.1 Configuration Parameters

Shared server configuration parameters are set in the database server configuration parameters file. Simple example parameter settings for using shared servers and dispatchers are as follows.

DISPATCHERS = "(PROTOCOL=TCP) (DISPATCHERS=1) (PORT=1522)"
SHARED_SERVERS = 2
CIRCUITS = 10
SHARED_SERVER_SESSIONS = 10
MAX_DISPATCHERS = 3
MAX_SHARED_SERVERS = 5
Note

Oracle Database 10 Grid Many parameters have been greatly simplified, particularly the shared server parameters. The SHARED_SERVERS parameter is the only parameter required to instantiate a shared server configuration.

SHARED_SERVERS and DISPATCHERS are created at database startup. CIRCUITS are virtual circuits for both in-and outbound network traffic. A virtual circuit is a chunk of memory which passes requests and responses between dispatcher and shared server processes. SHARED_SERVER_SESSIONS denotes user sessions per shared server. MAX_DISPATCHERS limits the number of dispatchers and connections per dispatcher. MAX_SHARED_SERVERS specifies the maximum number of shared servers allowed.

The DISPATCHERS parameter can get a lot more complicated. The DISPATCHERS parameter in the configuration parameter file is very much like a TNS name connection string. Effectively a dispatcher is a connection string because it performs a routing function between client processes and shared server processes.

DISPATCHERS = "(PROTOCOL=TCP) (DISPATCHERS=1) (PORT=1521)"

Now we can add further specifications to the DISPATCHERS parameter.

DISPATCHERS = "(DESCRIPTION=(ADDRESS=
      (PROTOCOL=TCP) (DISPATCHERS=1) (PORT=1521) (QUEUESIZE=50)))
      (SESSIONS=500) (CONNECTIONS=500) (MULTIPLEX=ON) (POOL=ON)
         (TICK=2)"

  • POOL=ON. This option enables connection pooling. When using connection pooling not only are shared servers shared but so are dispatchers. This increases scalability in terms of network connections and numbers of users.

  • TICK. Sets a timeout on a connection request and response, making an idle connection available for use by another client request or shared server response when the timeout is exceeded.

  • MULTIPLEX=ON. Oracle Connection Manager provides scalability by providing network node addresses plus routing directions to those network nodes. Oracle Connection Manager is more of a listening router rather than just a network names storage facility. Additionally Oracle Connection Manager can provide what is called multiplexing or funneling in addition to other features such as protocol transparency. Funneling is relevant to tuning. Connection concentration or funneling permits multiple connections between two processes. This can allow a shared server process to service more connection requests than by using connection pooling alone. The result is multiple client connection requests serviced by a single dispatcher process. Multiple Connection Managers can increase the number of concurrent client connections serviced astronomically.

    Note

    Oracle Database 10 Grid Oracle Connection Manager is improved.

There can also be multiple dispatchers using different ports.

DISPATCHERS = "(PROTOCOL=TCP) (DISPATCHERS=1) (PORT=1521)"
DISPATCHERS = "(PROTOCOL=TCP) (DISPATCHERS=1) (PORT=1522)"

More dispatchers will decrease user resource usage but decrease performance for each user due to all the switching. However, if connection numbers are high shared server processes are scalable if a middle tier is not utilized. On a Windows server set a single dispatcher for between every 200 to 1,000 connections, depending on application type.

16.4.2 Network Performance Views

Network performance views involve views for shared servers, dispatchers, virtual circuits, in addition to the usual event and wait event statistics views. These are the shared server parameter settings I have in my database.

SHARED_SERVERS = 2
CIRCUITS = 5
SHARED_SERVER_SESSIONS = 5
MAX_SHARED_SERVERS = 5
MAX_DISPATCHERS = 3

When executing queries within this section I had between 10 and 100 concurrent jobs running in my Accounts schema on my database.

Shared Servers

Three views which can be used to inspect shared server usage are the V$SHARED_SERVER, V$QUEUE, and V$SHARED_SERVER_ MONITOR performance views. The first query shows that only one of the shared server processes is being used.

COL name FORMAT a4;
SELECT name, messages, bytes, breaks, idle, busy, requests
FROM v$shared_server;

NAME   MESSAGES    BYTES   BREAKS     IDLE   BUSY   REQUESTS
----   --------   ------   ------   ------   ----   --------
S000        284    49131        0   810433   4385        141
S001          0        0        0   814722      0          0

This next query shows possible shared server process contention where COMMON examines shared server processes only, in 100th's of a second. The average wait per activity shown is less than 3 100th's of a second. Since only one shared server process is used this is not a problem.

SELECT wait/totalq FROM v$QUEUE WHERE type = 'COMMON';

WAIT/TOTALQ

 .269503546

The V$SHARED_SERVER_MONITOR view holds the highest values reached since database startup. My parameter values could be substantially reduced.

SELECT MAXIMUM_CONNECTIONS "Circuits"
      ,MAXIMUM_SESSIONS "Shared_Server_Sessions"
      ,SERVERS_STARTED "Started"
      ,SERVERS_TERMINATED "Terminated"
      ,SERVERS_HIGHWATER "Max_Shared_Servers"
FROM v$shared_server_monitor;

Circuits  Shared_Server_  Started  Terminated  Max_Shared_
                Sessions                           Servers
--------  --------------  -------  ----------  -----------
       3               3        0           0            2

Dispatchers

Use the V$DISPATCHER, V$QUEUE, and V$DISPATCHER_RATE performance views to monitor dispatcher activity. The first query shows dispatcher activity. There is little competition for this single dispatcher. Therefore, examining V$DISPATCHER_RATE output is pointless in this case.

COL name FORMAT a4;
COL status FORMAT a10;
SELECT name, status, messages, bytes, busy, idle, busy/idle
FROM v$dispatcher;

NAME   STATUS   MESSAGES   BYTES   BUSY     IDLE   BUSY/IDLE
----   ------   --------   -----   ----   ------   ---------
D000     WAIT        284   49131     25   852372   .00002933

As with shared servers this next query on the V$QUEUE performance view shows possible dispatcher process contention. The DISPATCHER value examines dispatcher processes only, in 100th's of a second. The average wait per activity is nonexistent.

SELECT wait/totalq FROM v$QUEUE WHERE type = 'DISPATCHER';

WAIT/TOTALQ
-----------
          0

Virtual Circuits

The V$CIRCUIT performance view provides a picture of shared server to dispatcher database connections. This query indicates no problems.

SELECT status, queue, bytes, breaks FROM v$circuit;

STATUS    QUEUE   BYTES   BREAKS
------   ------   -----   ------
NORMAL   NONE      9033        0
NORMAL   NONE      8675        0
NORMAL   SERVER   11484        0

Using Events

Events such as virtual circuit status and dispatcher timer can be used to detect potential problems with shared servers and dispatchers. Events of interest are as follows.

SELECT name "Event" FROM v$EVENT_NAME WHERE name LIKE
   '%dispatcher%'
OR name LIKE '%circuit%' OR name LIKE '%SQL*Net%';

Event
-----------------------------------------------
alter system set dispatcher
dispatcher shutdown
virtual circuit status
dispatcher timer
dispatcher listen timer
SQL*Net message to client
SQL*Net message to dblink
SQL*Net more data to client
SQL*Net more data to dblink
SQL*Net message from client
SQL*Net more data from client
SQL*Net message from dblink
SQL*Net more data from dblink
SQL*Net break/reset to client
SQL*Net break/reset to dblink

Some simple event statistics can be gathered with this query.

COL event FORMAT a30;
COL waits FORMAT 9999990;
COL timeouts FORMAT 99999990;
COL average FORMAT 99999990;
SELECT event "Event", time_waited "Total Time", total_waits
   "Waits", average_wait "Average", total_timeouts "Timeouts"
FROM V$SYSTEM_EVENT
WHERE event IN (
      SELECT name FROM v$EVENT_NAME WHERE name LIKE '%
         dispatcher%'
      OR name LIKE '%circuit%' OR name LIKE '%SQL*Net%'
) ORDER BY EVENT;

Total
Event                             Time  Waits  Average  Timeouts
-----------------------------  -------  -----  -------  --------
SQL*Net break/reset to client        3      8        0         0
SQL*Net message from client     605113    340     1780         0
SQL*Net message to client            0    344        0         0
SQL*Net more data to client          0      8        0         0
dispatcher timer                958324    630     1521       483
virtual circuit status         1871562    765     2446       618

That is all there is to know about basic Oracle network tuning. The next chapter will look into Oracle Partitioning and parallelism.


Chapter 17: Oracle Partitioning and Parallelism

Overview

This chapter will cover Oracle Partitioning and some parallel processing within those partitions. There are also various unique actions which can be performed with partitioning, such as moving and exchanging entire chunks of data at the speed of a Unix mv (move) command. Obviously the storage subsystem determines the speed of a move command. However, separate partitions can be plugged in and out of a partitioned object. We will start with a brief introduction as to what exactly can be done with partitioning, following on with some experimentation showing the performance benefits of Oracle Partitioning.


17.1 What is Oracle Partitioning?

Oracle Partitioning is the splitting of data sets usually into separate physical files using separate partition tablespaces. However, partitions can actually co-exist in the same physical file. Separating partitions into separate datafiles is not essential with certain types of RAID array architectures. Separate partitions are accessible in parallel, individually or in groups. Why is partitioning relevant to tuning? Partitioning can be used to break large tables into smaller subsets. Processing of smaller subsets of data separately and in parallel is potentially much faster than serial processing on very large data sets. Partitioning and parallelism generally only applies to very large databases on high end multiple CPU server platforms. Figure 17.1 shows a picture of how partitioning can be used to split a physical datafile containing a table into multiple physical datafiles mapped individually to separate partitions within that table.

Figure 17.1: Physical Partitioning

17.1.1 Why is Oracle Partitioning Beneficial?

Oracle Partitioning is beneficial usually in very large database environments and sometimes in smaller databases, depending on applications, for the following reasons:

  • Parallel processing by way of splitting of data into separately located physical files. Separating table and index datafiles onto separate disks in a nonpartitioned database is a form of partitioning. Since table and index physical spaces are often read in parallel it is highly beneficial to read their data from separate locations. Partitioning can enhance the benefits of splitting of datafiles.

  • Different partitions within the same table can have different physical storage structures. Different partitions within a table can even be both read-write and read-only, perhaps allowing separation of archived and current data.

  • Backup, recovery, and utility usage (SQL*Loader, Export, and Import) can utilize parallel processing and activities on individual partitions.

  • The Optimizer can access individual partitions when processing SQL code. This process is termed partition pruning since partitions can potentially be removed from the database read required by a query. Additionally the Optimizer can execute against multiple partitions using parallel processing, generally on high end multiple CPU server platforms, with datafiles spread across multiple disks.

    Tip

    Rows can also be accessed by partition name effectively allowing manual partition pruning.

  • Parallel processing with multiple CPUs and Oracle Partitioning applies to both DML and DDL commands.

17.1.2 How are Tables and Indexes Partitioned?

Partitions can be created on single or multiple columns of a table. A table can be divided up into separate partitions based on three methods. These methods are ranges of values, values in lists, and hashing algorithms on columns. Additionally partitions can be one of two composites of the subsequently mentioned partitioning methods.

A partition is divided on the basis of what is called a Partition Key. This key is internal to Oracle Database, effectively the data definition splitting table rows into separate partitions. For example, a range partition on a table could have rows separated into different partitions based on a date for each row, perhaps dividing financial data into quarters.

What are the specifics of the different partitioning methods?

  • Range Partition. Splits rows in a table based on ranges of values, such as splitting a table of transactions into periods or quarters derived from a transaction date.

  • List Partition. Splits rows based on lists of values, dividing rows into separate partitions based on matches between list entries and row column values.

  • Hash Partition. A hashing algorithm separates rows based on a column specification into a specified number of separate partitions.

  • Composite Partition. Partitions can contain subpartitions of two types. Firstly, a range partition can contain multiple hash subpartitions and secondly, a range partition can contain multiple list subpartitions.

Partitions can have indexes. How are indexes built for partitions? There are two types of partitioning indexes.

  • Local Index. These indexes have the same structure as their relative table partitions. Local indexes are preferred due to more automated maintenance.

  • Global Index. These indexes are created on partitioned tables but are not the same structure as the partitioning key.

    Note

    Oracle Database 10 Grid There is a new type of global partition index called a hash partitioned global index. It allows for an even spread of index values.

Let's now examine and experiment with different partitioning methods in my Accounts schema database. The machine used is once again my rather geriatric dual 300 MHz PII CPU box. Additionally this machine has different drives for separating different physical partition datafiles.

17.1.3 Oracle Partitioning Methods

I will use the Accounts schema to examine the usefulness of the different partitioning methods.

Partitioning by Range

The GeneralLedger table is most appropriately partitioned by date range. Partitioning by date may not make any difference to this table with respect to row inserts since rows are only added in the current period and thus the most recent partition. However, it could help performance for creating financial statements such as balance sheets. Since accounting reports of this nature are reported by date range periods it is more than apt to divide such a table based on a date range. Following is shown the division of data in the GeneralLedger table. These numbers are not exactly correct because the financial year in the Accounts data starts on March 1st, but for the purposes of this example this level of accuracy is good enough.

SELECT TO_CHAR(dte,'YYYY') "Year"
      ,COUNT(TO_CHAR(dte,'YYYY')) "Entries"
FROM generalledger GROUP BY TO_CHAR(dte,'YYYY') ORDER
BY 1;

Year    Entries
----    -------
1999     195888
2000     208624
2001     213660
2002     134568

New partitions can have separate tablespaces. How big is the GeneralLedger table?

COL Object FORMAT a24;
COL Type FORMAT a5;
SELECT segment_name "Object", segment_type "Type"
, ROUND(SUM(bytes)/1024/1024) "Mb"
, SUM(blocks) "Blocks"
FROM dba_extents
WHERE owner = 'ACCOUNTS'
AND segment_name = 'GENERALLEDGER'
GROUP BY segment_name, segment_type;

The following query result shows that the GeneralLedger table is sized at approximately 26 Mb.

Object           Type   Mb   Blocks
-------------   -----   --   ------
GENERALLEDGER   TABLE   26     3328

Data in the GeneralLedger table covers 4 years. There are 200,000 rows for each of the 4 years from 1999 through to 2002. I will create four partitions of 6 Mb each. That should be enough space to contain the rows in each partition.

Tip

The 6 Mb partition datafiles extended when rows were added because appropriate physical storage structure was not set for each separate partition. Partitions can have independent physical storage parameters. Physical storage tuning is covered in other chapters. This chapter focuses on partitioning.

I create four index partition tablespaces and grant unlimited quotas on all the tablespaces to the Accounts schema Oracle Database user. The years 2000, 2001, and 2002 are all on separate disk drives. Some scripts are included in Appendix A. Range partitions are created like this:

CREATE TABLE GLP
(
       generalledger_id    NUMBER NOT NULL
      ,coa#                CHAR(5) NOT NULL
      ,dr                  NUMBER(10,2) NOT NULL
      ,cr                  NUMBER(10,2) NOT NULL
      ,dte                 DATE NOT NULL
      ,CONSTRAINT XPK_GLP PRIMARY KEY (generalledger_id)
      ,CONSTRAINT FK_GLP_COA# FOREIGN KEY (coa#)
         REFERENCES COA
) TABLESPACE indx
PARTITION BY RANGE(dte)(

PARTITION DATAGLP1999 VALUES LESS THAN
      (TO_DATE('2000-03-01','YYYY-MM-DD')) TABLESPACE
       DATAGLP1999
,PARTITION DATAGLP2000 VALUES LESS THAN
      (TO_DATE('2001-03-01','YYYY-MM-DD')) TABLESPACE
       DATAGLP2000
,PARTITION DATAGLP2001 VALUES LESS THAN
      (TO_DATE('2002-03-01','YYYY-MM-DD')) TABLESPACE
       DATAGLP2001
,PARTITION DATAGLP2002 VALUES LESS THAN
      (MAXVALUE) TABLESPACE DATAGLP2002);

CREATE INDEX FKX_GLP_1 ON GLP(coa#) TABLESPACE indx;
CREATE INDEX LK_GLP_1 ON GLP (dte, coa#) LOCAL(
       PARTITION INDXGLP1999 TABLESPACE INDXGLP1999
      ,PARTITION INDXGLP2000 TABLESPACE INDXGLP2000
      ,PARTITION INDXGLP2001 TABLESPACE INDXGLP2001
      ,PARTITION INDXGLP2002 TABLESPACE INDXGLP2002);

Now add the data from the GeneralLedger table into the partitioned table called GLP.

INSERT INTO glp SELECT * FROM generalledger;
COMMIT;
ANALYZE TABLE glp COMPUTE STATISTICS;
ANALYZE INDEX fkx_glp_1 COMPUTE STATISTICS;
ANALYZE INDEX lk_glp_1 COMPUTE STATISTICS

Now let's try some queries. This first query retrieves all rows from the nonpartitioned GeneralLedger table as a query to measure against.

EXPLAIN PLAN SET statement_id='TEST' FOR SELECT * FROM
   generalledger;

Query                       Cost     Rows      Byte
-------------------------   ----   ------   -------
1. SELECT STATEMENT on 778 752740 1957124
2. TABLE ACCESS FULL on
     GENERALLEDGER 778 752740 1957124

The next query reads only the 1999 financial year partition using partition pruning, indicated by the drastic decrease in the number of rows read. The cost is therefore much lower. The pruning out of unwanted partitions allows only the 1999 partition to be read.

EXPLAIN PLAN SET statement_id='TEST' FOR
      SELECT * FROM glp WHERE dte < TO_DATE('2000-01-
         01','YYYY-MM-DD');

Note the values 1 and 1 at the tail of the Query column for the second line of the query plan. These columns are the PARTITION_START and PARTITION_STOP columns in the PLAN_TABLE, denoting the first and the last partitions read. In this case only the first partition was read.

Query                               Cost   Rows     Bytes
----------------------------        ----   ------   -------
1. SELECT STATEMENT on               203   195888   4505424
2.  TABLE ACCESS FULL on GLP  1  1   203   195888   4505424

This next query reads even fewer rows and the cost is lower still, reading only the partition for the financial year 2002. Note once again the two numbers tailing the second line of the query plan Query column; only the fourth partition is read to satisfy the query.

EXPLAIN PLAN SET statement_id='TEST' FOR
      SELECT * FROM glp
      WHERE dte BETWEEN TO_DATE('2002-03-01','YYYY-MM-DD')
      AND TO_DATE('2003-02-28','YYYY-MM-DD');

Query                               Cost   Rows     Bytes
----------------------------        ----   ------   -------
1. SELECT STATEMENT on               143   115657   2660111
2.  TABLE ACCESS FULL on GLP  4  4   143   115657   2660111

This proves the facts about pruning of partitions where SQL code accesses partitions containing only required data.

Tip

Rows can also be accessed by partition name effectively allowing manual partition pruning.

Partitioning by List

In the Accounts schema the Transactions table could be split into two partitions based on the type column. The Transactions table contains both sales invoices sent to customers and purchase invoices sent from suppliers. The type column contains the value "S" for a sale and "P" for a purchase. The size of the Transactions table is about 10 Mb. The ratio of purchases to sales is 80%. This dictates the size of the tablespaces for the two partitions. Let's create the partitioned version of the Transactions table. This time I will ignore Referential Integrity and indexes, simply creating the partitioned table. We are not trying to prove anything about indexing. We are merely trying to show that partitioning by itself can increase performance dramatically, especially for full table scans and if used in appropriate places.

CREATE TABLE tsp(
       transaction_id   NUMBER NOT NULL
      ,type             CHAR(1) NOT NULL
      ,customer_id      NUMBER NULL
      ,supplier_id      NUMBER NULL
      ,order_id         NUMBER NULL
      ,amount           NUMBER(10,2) NOT NULL
      ,dte              DATE NOT NULL
      ,drcoa#           CHAR(5) NOT NULL
      ,crcoa#           CHAR(5) NOT NULL)
PARTITION BY LIST (type)(
       PARTITION purchases VALUES ('P') TABLESPACE DATATSP
      ,PARTITION sales VALUES ('S') TABLESPACE DATATSS);

Add the rows to the partition table.

INSERT INTO tsp SELECT * FROM transactions;
COMMIT;
ANALYZE TABLE tsp COMPUTE STATISTICS;

Once again retrieve data from the nonpartitioned table as a measure of performance.

EXPLAIN PLAN SET statement_id='TEST' FOR SELECT * FROM
   transactions;
Query                                   Cost   Rows     Bytes
-------------------------------------   ----   ------   -------
1. SELECT STATEMENT on                   297   188185   7903770
2.  TABLE ACCESS FULL on TRANSACTIONS    297   188185   7903770

Now get all rows from the partitioned table. Fewer bytes are read with the same number of rows but the cost is very slightly higher, perhaps due to the second step in the query plan. The entire table with all partitions is being read anyway so there is no reason why reading a partitioned version of the table should be faster.

EXPLAIN PLAN SET statement_id='TEST' FOR SELECT * FROM tsp;

Query                                Cost     Rows   Bytes
----------------------------------   ----   ------   -------
1. SELECT STATEMENT on                298   188185   6774660
2.  PARTITION LIST ALL on 1 2
3.   TABLE ACCESS FULL on TSP 1 2     298   188185   6774660

Now let's go one step further and read all the rows from the partition, reading both partitions in parallel. The parallel read is suggested to the Optimizer by using the PARALLEL hint. The two separate partitions are stored in two separate tablespaces and the two datafiles mapped by the tablespace are stored on two separate disk drives. Both processors and both separate disks are assisting the performance of parallel execution. The cost is lower even though the number of rows is the same. There are only two partitions for this partitioned by list table, the PARTITION_START and PARTITION_STOP values tailing the Query column show both partitions read.

EXPLAIN PLAN SET statement_id='TEST' FOR
      SELECT /*+ PARALLEL(tsp, 2) */ * FROM tsp;

Query                           Cost   Rows
-----------------------------   ----   -------
1 SELECT STATEMENT on            149    188185
2  PARTITION LIST ALL on PARALLEL_
     COMBINED_WITH_PARENT 1 2
3   TABLE ACCESS FULL on TSP
      PARALLEL_TO_SERIAL 1 2      149   188185

Now read only the sales transactions.

EXPLAIN PLAN SET statement_id='TEST' FOR
      SELECT * FROM tsp WHERE type = 'S';

In this case only the second partition is read since sales transactions occupy the second partition. Consequently the cost is much lower and fewer rows are read.

Query                               Cost   Rows Bytes
---------------------------------   ----   ------   -------
1. SELECT STATEMENT on                54    33142   1193112
2.  TABLE ACCESS FULL on TSP 2 2      54    33142   1193112

And now read the sales transactions and suggest parallel execution on both processors using the PARALLEL hint once again. The cost is again lower and the same number of rows is accessed.

EXPLAIN PLAN SET statement_id='TEST' FOR
      SELECT /*+ PARALLEL(tsp, 2) */ * FROM tsp WHERE
         type = 'S';

Query Cost Rows
---------------------------   ----   -----
1. SELECT STATEMENT on          27   33142
2. TABLE ACCESS FULL on
TSP PARALLEL_TO_SERIAL 2 2      27   33142

Hash Partitions

A hash partition can be used when range or list partitioning is inappropriate. Most of the large tables in the Accounts schema have a potential range or list partition column in the form of a date or some kind of type column. Two tables not delimited in this fashion are the OrdersLine and TransactionsLine tables. When demonstrating how to use the Ordersline table firstly create a partitioned table.

CREATE TABLE olp(
       order_id                NUMBER NOT NULL
      ,seq#                    NUMBER NOT NULL
      ,amount                  NUMBER(10,2) NOT NULL
      ,stockmovement_id        NUMBER NOT NULL)
PARTITION BY HASH(order_id) PARTITIONS 3
      STORE IN(dataol1,dataol2,dataol3);

Now add the rows to the partitioned table.

INSERT INTO olp SELECT * FROM ordersline;
COMMIT;
ANALYZE TABLE olp COMPUTE STATISTICS;

Now query the nonpartitioned table.

EXPLAIN PLAN SET statement_id='TEST' FOR SELECT * FROM
      ordersline;

Query                                   Cost   Rows     Bytes
-------------------------------------   ----   ------   -------
1. SELECT STATEMENT on                   412   540827   9194059
2.  TABLE ACCESS FULL on ORDERSLINE      412   540827   9194059

And now query the partitioned table in parallel. The cost is halved indicating fewer rows and fewer bytes read.

EXPLAIN PLAN SET statement_id='TEST' FOR
      SELECT /*+ PARALLEL(olp, 2) */ * FROM olp;

Query                                   Cost   Rows
-------------------------------------   ----   -----
1 SELECT STATEMENT on                    207   40827
2  PARTITION HASH ALL on
     PARALLEL_COMBINED_WITH_PARENT 1 3
3   TABLE ACCESS FULL on OLP
      PARALLEL_TO_SERIAL 1 3             207   40827

Composite Partitions

Composite partitions can be of two forms. Those two forms are a range partition containing hash subpartitions or a range partition containing list subpartitions. Various tables in the Accounts schema could be partitioned using a range-hash partition. The Orders, Transactions, GeneralLedger, and StockMovement tables could be range-hash partitioned on the DTE column for the range partition and the primary key identifier column for the contained hash subpartitions.

CREATE TABLE glprh(
       generalledger_id NUMBER NOT NULL
      ,coa# CHAR(5) NOT NULL
      ,dr NUMBER(10,2) NOT NULL
      ,cr NUMBER(10,2) NOT NULL
      ,dte DATE NOT NULL)
PARTITION BY RANGE(dte) SUBPARTITION BY HASH
   (generalledger_id)
         SUBPARTITIONS 4 STORE IN (sp1, sp2, sp3, sp4)
(PARTITION DATAGLP1999 VALUES LESS THAN
        (TO_DATE('2000-01-01','YYYY-MM-DD')) TABLESPACE
          DATAGLP1999
,PARTITION DATAGLP2000 VALUES LESS THAN
        (TO_DATE('2001-01-01','YYYY-MM-DD')) TABLESPACE
          DATAGLP2000
,PARTITION DATAGLP2001 VALUES LESS THAN
        (TO_DATE('2002-01-01','YYYY-MM-DD')) TABLESPACE
          DATAGLP2001
,PARTITION DATAGLP2002 VALUES LESS THAN
        (MAXVALUE) TABLESPACE DATAGLP2002);

The GeneralLedger table in the Accounts schema could be rangelist partitioned on the DTE column for the range and the COA# column for the list subpartition. The Orders and Transactions tables are not functionally appropriate for range-list partitioning on their respective DTE and TYPE columns.

CREATE TABLE glprl(
       generalledger_id NUMBER NOT NULL
      ,coa# CHAR(5) NOT NULL
      ,dr NUMBER(10,2) NOT NULL
      ,cr NUMBER(10,2) NOT NULL
      ,dte DATE NOT NULL)
PARTITION BY RANGE(dte) SUBPARTITION BY LIST(coa#)
(PARTITION DATAGLP1999 VALUES LESS THAN
 (TO_DATE('2000-01-01','YYYY-MM-DD')) TABLESPACE
   DATAGLP1999
       (SUBPARTITION assets1999 VALUES('20001','20002')
      ,SUBPARTITION liabilities1999 VALUES('10001','10002')
      ,SUBPARTITION expenses1999 VALUES('50001','50002')
      ,SUBPARTITION incomes1999 VALUES('60001','60002'))
,PARTITION DATAGLP2000 VALUES LESS THAN
 (TO_DATE('2001-01-01','YYYY-MM-DD')) TABLESPACE
   DATAGLP2000
       (SUBPARTITION assets2000 VALUES('20001','20002')
      ,SUBPARTITION liabilities2000 VALUES('10001','10002')
      ,SUBPARTITION expenses2000 VALUES('50001','50002')
      ,SUBPARTITION incomes2000 VALUES('60001','60002'))
,PARTITION DATAGLP2001 VALUES LESS THAN
 (TO_DATE('2002-01-01','YYYY-MM-DD')) TABLESPACE
   DATAGLP2001
       (SUBPARTITION assets2001 VALUES('20001','20002')
      ,SUBPARTITION liabilities2001 VALUES('10001','10002')
      ,SUBPARTITION expenses2001 VALUES('50001','50002')
      ,SUBPARTITION incomes2001 VALUES('60001','60002'))
,PARTITION DATAGLP2002 VALUES LESS THAN
 (MAXVALUE) TABLESPACE DATAGLP2002
       (SUBPARTITION assets2002 VALUES('20001','20002')
      ,SUBPARTITION liabilities2002 VALUES('10001','10002')
      ,SUBPARTITION expenses2002 VALUES('50001','50002')
      ,SUBPARTITION incomes2002 VALUES('60001','60002')));


17.2 Tricks with Partitions

Why is this section entitled "Tricks with Partitions"? Partitions can have things done to them on an individual basis. For instance, without partitioning if one wanted to remove all of the rows for the year 1999 from the GeneralLedger table the entire GeneralLedger table would be have to full table scanned.

DELETE FROM GeneralLedger WHERE TO_CHAR(dte,'YYYY') =
   1999;

Using the partition table GLP, created as a date range partition on the GeneralLedger table, one would not even have to full table scan one-fifth of the rows: simply drop the partition. Dropping a single partition one-fifth the size of the table is much faster than removing 20% of the rows from the nonpartitioned table.

What are some other actions which can be performed specifically with partitions, beneficial to general database performance?

  • Add, Drop, and Truncate. Individual partitions can be added, dropped, or even truncated without affecting the rest of the partitions in a table.

  • Split and Merge. A partition can be split into two partitions or two partitions can be merged into a single partition. Once again there is no effect on rows outside of the partitions being split or merged.

  • Rename. Individual partitions can be renamed.

  • Move. A partition can be moved into a separate tablespace with the speed of a Unix mv (move) command, akin to renaming the file pointer for the datafile. Changing a file pointer is a simple tablespace header change. No costly SQL code is involved in transferring data between tablespaces.

  • Exchange. Partitions and subpartitions can be converted into tables and vice versa. Again no costly SQL code is involved in transferring data.

To reiterate, all of these listed actions will not affect the rest of the partitions in a partitioned table since they operate on specific partitions. Only specified partitions are read or accessed.

The important point to note about partitioning is that it is a tuning method in itself. There are obviously ways to tune partitions individually. For instance, different partitions and their related indexes can have individual partition physical storage attributes. Storage tuning is covered in other chapters of this book. Additionally there are actions which can be taken on individual partitions such as moving and exchanging. Equivalent actions on nonpartitioned tables would either be impossible or involve intense SQL activity. These special partition actions cannot be done nearly as efficiently with large nonpartitioned tables. Partitioning has been presented in this chapter as a tuning technique in itself.

This is all that needs to be discussed with respect to partitioning and tuning. This chapter completes the analysis of physical and configuration tuning for Oracle Database. The next chapter will start to examine how to uncover performance problems by examining use of ratios to help detect performance problems.


Chapter 18: Ratios: Possible Symptoms of Problems

Overview

There is much to be said for not using various ratios as tuning indicators, in fact for not using any of them. Many of these ratios can conceal some very large and ugly numbers since they divide one number by another. The denominator, or the number used to divide by, simply disappears. Ratios can be useful to indicate a possible problem or problems. Well in fact a low or high ratio could indicate a possible problem but it could also conceal a possible problem. Additionally a ratio could indicate no problem at all. Ratios are best used as possible symptoms of possible problems. "Hmmmm…"

Ratios have received a lot of bad press recently. Much of this is founded in truth. Ratios should not be completely ignored. I find even the database buffer cache hit ratio useful. However, I do not necessarily think it is a bad thing if it is low or high. So what! That could be because of application behavior. A possible mistake made with ratios in the past was that an unacceptable value was assumed to be tunable. A ratio is not tunable. However, it can indicate that something else could be tunable such as an ugly SQL statement in an application full of very well-tuned SQL code. If the database buffer cache is "doing something weird" like going up and down say between 60% and 90% every 5 s then something odd may be going on. Increasing the size of the database buffer cache will not help you to find that ugly SQL code statement, in fact nasty things might be further concealed. The whole point of tuning is finding things and solving them not attempting to conceal and bury them behind meaningless percentages.

The database buffer cache and the shared pool can actually suffer from tuning of their respective ratios not only because problems causing unpalatable ratios are not being solved but because extremely large buffers are much more difficult for Oracle Database to manage.

Tip

Use ratios as symptoms or indicators of potential problems. Tuning the ratios themselves by changing buffer sizes is often a complete waste of time! It also might make things worse. On the contrary, I disagree completely with Oracle database administrators who consider ratios to be completely useless. Ratios can give clues as to where to look in the Oracle Database Wait Event Interface when trying to solve a performance problem. The solution is perhaps to use both ratios and the Oracle Database Wait Event Interface.

The V$SYSSTAT performance view is the source of many ratios. What are some of these ratios?

  • Database Buffer Cache Hit Ratios. Cached database table and index blocks versus those read from disk.

  • Table Access Ratios. Full table scanning small static tables can help performance. Full table scanning large tables can completely devastate database performance. The best performance method for accessing tables is by index ROWID pointers and full scans on small tables.

  • Index Use Ratio. Compare index key reads to table reads.

  • Dictionary Cache Hit Ratio. Database metadata in the shared pool.

  • Library Cache Hit Ratios. Library cache parsing activity in the shared pool.

  • Disk Sort Ratio. How much is temporary disk sort space used?

  • Chained Rows Ratio. Proportion of rows fetched involving row chaining. Row chaining is overflow of blocks to other blocks since information cannot fit into a single block and can make data retrieval inefficient. It is extremely likely that chained blocks are spread across disk storage haphazardly.

  • Parse Ratios. Parsing is expensive. The less parsing the better.

  • Latch Hit Ratio. Latches protect memory buffers from concurrent use.


18.1 Database Buffer Cache Hit Ratio

This ratio measures a ratio between finding blocks in the buffer and on disk. It is a ratio of logical I/O to disk or physical I/O, commonly known as LIOs and PIOs, respectively.

This ratio should not be used as an indicator of database performance health. Much Oracle software documentation touts the database buffer cache hit ratio as being one of the most important tuning metrics. In my opinion and that of many others this statement is often a complete and absolute falsity. LIOs are not "free". LIOs are generally faster than PIOs but not nearly as much as most Oracle software documentation would lead us to believe. Buffers in memory are much more complex to manage than simple disk I/O due to latch contention, ordered free lists, high concurrency, fragmentation, and many other factors. In short, reading from a buffer can sometimes be much more complex than performing I/O.

If the database buffer cache hit ratio is high then it means that data is being found in memory. If the ratio is low then queries are executing a fair amount of I/O. It is very difficult to state with certainty what causes this ratio to be high or low. The problem is that most existing documentation assumes that a high hit ratio is good. In fact sometimes a not-so-high hit ratio is also an indicator of good performance health. For instance, many full table scans are sometimes necessary, both on large and small tables, depending on application and database type. Full table scans perform I/O and bypass the database buffer cache. This can artificially lower the ratio. However, very poorly tuned SQL code, particularly joins, can cause unwanted full table scans. This also lowers the ratio. In the opposite direction lack of use of SQL code bind variables can cause a very high hit ratio, and as a result the database buffer is often sized much too high.

In general, do not simply attempt to tune the database buffer cache hit ratio. Do not jump to the conclusion that the database buffer cache must always be made larger. Even consider reducing the database buffer cache if the hit ratio is high, potentially freeing up memory for other requirements. Rather attempt to isolate the cause, then decide if there is a problem, finally tuning a real problem and not a ratio.

Tip

Calculating the database buffer cache hit ratio is the same for both the Oracle8i Database DB_BLOCK_BUFFERS and the Oracle9i Database DB_CACHE SIZE parameters.

Database read buffer cache hit ratio =

1 - (physical reads/(db block gets + consistent gets))

These are the values I am interested in to calculate the read hit ratio.

SELECT value, name FROM V$SYSSTAT WHERE name IN
      ('physical reads', 'db block gets', 'consistent gets');

VALUE   NAME
-----   -----------------
 2031   physical reads
 3861   db block gets
85832   consistent gets

Database buffer cache hit ratio =

1 - (physical reads/(db block gets + consistent gets))

Here is a query to calculate the database buffer cache hit ratio.

SELECT 'Database Buffer Cache Hit Ratio ' "Ratio"
, ROUND((1 -
((SELECT SUM(value) FROM V$SYSSTAT WHERE name = 'physical
   reads')
/ ((SELECT SUM(value) FROM V$SYSSTAT WHERE name = 'db block
   gets')
+ (SELECT SUM(value) FROM V$SYSSTAT WHERE name = 'consistent
   gets')
))) * 100)||'%' "Percentage"
FROM DUAL;
Tip

All of the ratio queries in this chapter can be executed using joins but we would not be able to see immediate visual comparison of different ratios. There is a method to my madness in this respect.

This is the result of the query. This ratio is possibly healthy because it is not too high, but not necessarily. This value does not really tell me anything.

Ratio                                Percentage
--------------------------------     ----------
Database Buffer Cache Hit Ratio             95%

18.1.1 Default, Keep, and Recycle Pools

Inspecting database buffer cache hit ratios for the three different pools can be done as shown. The subquery is used for the ratio calculation to avoid divisions by zero, which is mathematically undefined.

COL pool FORMAT a10;
SELECT a.name "Pool", a.physical_reads, a.db_block_gets
      , a.consistent_gets
,(SELECT ROUND(
(1-(physical_reads/(db_block_gets + consistent_gets)))*100)
      FROM v$buffer_pool_statistics
      WHERE db_block_gets+consistent_gets ! = 0
      AND name = a.name) "Ratio"
FROM v$buffer_pool_statistics a;

The keep pool should be high. The recycle pool should probably be low.

Pool    PHYSICAL_READS   DB_BLOCK_GETS   CONSISTENT_GETS   Ratio
-----   --------------   -------------   ---------------   -----
KEEP                40          246570             12816     100
RECYCLE          75755         2874944            120748      97
DEFAULT         822495        17304843          10330119      97

Various tables and indexes were pushed into the keep and recycle pools in Chapter 15. Small static tables were pushed into the keep pool, excluding their indexes, assuming that the Optimizer would usually full table scan them. The Customer and Supplier tables are large semi-static tables and thus only their indexes were placed into the keep pool. The two largest tables, the GeneralLedger and StockMovement tables, were placed into the recycle pool. The results of 100%, 97%, and 97% on the keep, recycle, and default pools, respectively could tell a number of things but these ratios could also hide a lot of information. Let's cover a few points to drive home the point about potentially misleading information obtainable from database buffer ratios. Most importantly note that even though ratios are high for the keep and recycle pools, the physical and logical read numbers are low enough to be negligible.

Adjusting the Keep Pool for the Accounts Schema Changes in Chapter 15.

Small static tables are often full table scanned by the Optimizer. Since full table scans are read physically from disk there is no point putting them into the keep pool; they may never be retrieved from the buffer. Should their indexes be placed in the keep pool? Yes if those small tables are used in large joins. The Optimizer will actually use index reads, even for small static tables, in very well tuned SQL code joins. So I will push small static tables back into the default pool and place their indexes in the keep pool.

ALTER TABLE category STORAGE(BUFFER_POOL DEFAULT);
ALTER TABLE coa STORAGE(BUFFER_POOL DEFAULT);
ALTER TABLE customer STORAGE(BUFFER_POOL DEFAULT);
ALTER TABLE period STORAGE(BUFFER_POOL DEFAULT);
ALTER TABLE posting STORAGE(BUFFER_POOL DEFAULT);
ALTER TABLE subtype STORAGE(BUFFER_POOL DEFAULT);
ALTER TABLE supplier STORAGE(BUFFER_POOL DEFAULT);
ALTER TABLE type STORAGE(BUFFER_POOL DEFAULT);

ALTER INDEX xak_categ_text STORAGE(BUFFER_POOL KEEP);
ALTER INDEX xfk_coa_subtype STORAGE(BUFFER_POOL KEEP);
ALTER INDEX xfk_coa_type STORAGE(BUFFER_POOL KEEP);
ALTER INDEX xfk_post_crcoa STORAGE(BUFFER_POOL KEEP);
ALTER INDEX xfk_post_drcoa STORAGE(BUFFER_POOL KEEP);
ALTER INDEX xpk_category STORAGE(BUFFER_POOL KEEP);
ALTER INDEX xpk_coa STORAGE(BUFFER_POOL KEEP);
ALTER INDEX xpk_period STORAGE(BUFFER_POOL KEEP);
ALTER INDEX xpk_posting STORAGE(BUFFER_POOL KEEP);
ALTER INDEX xpk_subtype STORAGE(BUFFER_POOL KEEP);
ALTER INDEX xpk_type STORAGE(BUFFER_POOL KEEP);

The keep pool could be used to avoid removal from cache of static data by larger tables and indexes.

Adjusting the Recycle Pool for the Accounts Schema Changes in Chapter 15.

In my highly active Accounts schema the two largest tables, the GeneralLedger and StockMovement tables, are only ever inserted into. Referential Integrity activity checks on primary and foreign keys might make it useful to place the Referential Integrity indexes into the keep pool and simply ignore the tables, pushing them back into the default pool.

ALTER TABLE generalledger STORAGE(BUFFER_POOL DEFAULT);
ALTER TABLE stockmovement STORAGE(BUFFER_POOL DEFAULT);
ALTER INDEX xfk_coa# STORAGE(BUFFER_POOL KEEP);
ALTER INDEX xfk_sm_stock STORAGE(BUFFER_POOL KEEP);
ALTER INDEX xpk_generalledger STORAGE(BUFFER_POOL KEEP);
ALTER INDEX xpk_stockmovement STORAGE(BUFFER_POOL KEEP);

After a period of time the V$BUFFER_POOL_STATISTICS view contains values shown in the following query. We can see that use of all pools is the same as before. Use of the keep pool remains negligible. My active Accounts schema by the very nature of its activity would not make much use of the keep or recycle pools anyway. Only use specific database buffer pools when applications involve a mixture of OLTP and reporting activity, particularly where large properly tuned SQL code joins are concerned. And remember that full table scans bypass the database buffer cache. There is no point in pushing tables into the keep pool unless they are commonly read using index ROWID pointers.

Pool    PHYSICAL_READS   DB_BLOCK_GETS   CONSISTENT_GETS   Ratio
-----   --------------   -------------   ---------------   -----
KEEP               233          274857             13305     100
RECYCLE          77446         2958975            124690      97
DEFAULT         869020        18045781          10803769      97
Tip

Using the keep and recycle pools is subjective and unpredictable unless knowledge of application activity is well known. Some applications behave differently such as OLTP activity during the day and batch activity at night. Different buffer pool settings would be advisable for different activity cycles. Is it really worth using the keep and recycle pools? Perhaps not!


18.2 Table Access Ratios

Full scanning small static tables can be more efficient than reading both index and table. Full scans on large tables are often executed in data warehouses and for reporting. Full table scans are not necessarily a bad thing. However, poorly tuned SQL code can cause unwanted full table scans. Many applications inadvertently retrieve all table rows using views and subsequently filter view results. Querying a view will always execute the SQL code underlying the view in its fullest form, applying data retrieval SQL code filtering requirements only to the result of the view. So full table scans can be a very bad thing as well.

There are five ratios that might be useful. They are as follows:

  • short scan /(short scan + long scan)

  • short scan /(short scan + long scan + by ROWID)

  • long scan /(short scan + long scan + by ROWID)

  • by ROWID /(short scan + long scan + by ROWID)

  • (short_scan + by ROWID)/(short scan + long scan + by ROWID)

    • short scan. Implies full table scan of a small table. Good.

    • long scan. Implies full table scan of a large table. Bad!

    • by ROWID. Implies access to a table row via an index unique hit, a range scan or a full index scan. Good or bad depending on table size and usage.

The statistics we are interested in are as follows:

SELECT value, name FROM V$SYSSTAT WHERE name IN
      ('table fetch by rowid', 'table scans (short tables)'
      , 'table scans (long tables)');
Tip

For the statistic table scans (short tables) the table must have the CACHE option set for the table (ALTER TABLE table CACHE).

SELECT 'Short to Long Full Table Scans' "Ratio"
, ROUND(
(SELECT SUM(value) FROM V$SYSSTAT
WHERE name = 'table scans (short tables)')
/ (SELECT SUM(value) FROM V$SYSSTAT WHERE name IN
   ('table scans (short tables)', 'table scans (long
      tables)'))
* 100, 2)||'%' "Percentage"
FROM DUAL
UNION
SELECT 'Short Table Scans ' "Ratio"
, ROUND(
(SELECT SUM(value) FROM V$SYSSTAT
WHERE name = 'table scans (short tables)')
/ (SELECT SUM(value) FROM V$SYSSTAT WHERE name IN
   ('table scans (short tables)', 'table scans (long
      tables)'
, 'table fetch by rowid'))
* 100, 2)||'%' "Percentage"
FROM DUAL
UNION
SELECT 'Long Table Scans ' "Ratio"
, ROUND(
(SELECT SUM(value) FROM V$SYSSTAT
   WHERE name = 'table scans (long tables)')
/ (SELECT SUM(value) FROM V$SYSSTAT WHERE name
   IN ('table scans (short tables)', 'table scans (long
      tables)', 'table fetch by rowid'))
* 100, 2)||'%' "Percentage"
FROM DUAL
UNION
SELECT 'Table by Index ' "Ratio"
, ROUND(
(SELECT SUM(value) FROM V$SYSSTAT WHERE name = 'table fetch
   by rowid')
/ (SELECT SUM(value) FROM V$SYSSTAT WHERE name
    IN ('table scans (short tables)', 'table scans (long
      tables)', 'table fetch by rowid'))
* 100, 2)||'%' "Percentage"
FROM DUAL
UNION
SELECT 'Efficient Table Access ' "Ratio"
, ROUND(
(SELECT SUM(value) FROM V$SYSSTAT WHERE name
   IN ('table scans (short tables)','table fetch by rowid'))
/ (SELECT SUM(value) FROM V$SYSSTAT WHERE name
     IN ('table scans (short tables)', 'table scans (long
      tables)', 'table fetch by rowid'))
* 100, 2)||'%' "Percentage"
FROM DUAL;

As you can see from the results below table and index access in my Accounts schema could be fairly well tuned. Most of the full table scans on large tables are probably a result of ad hoc counting of all the rows in those tables. There is a lot of short table scanning which offsets the table by index access. Overall, using short full table scans as opposed to index access by ROWID pointers can give better efficiency.

Ratio                               Percentage
---------------------------------   ----------
Efficient Table Access                  99.97%
Long Table Scans                          .03%
Short Table Scans                       31.16%
Short to Long Full Table Scans          99.92%
Table by Index                          68.81%


18.3 Index Use Ratio

This is a simple ratio trying to assess the percentage of index usage as compared to table scans. This value is not precise. These are the statistics I am interested in.

SELECT value, name FROM V$SYSSTAT WHERE name IN
      ('table fetch by rowid', 'table scans
         (short tables)', 'table scans (long tables)')
OR name LIKE 'index fast full%' OR name = 'index fetch by
   key';
   VALUE    NAME
   -----    ---------------------------------------------
      75    table scans (short tables)
     116    table scans (long tables)
    3983    table fetch by rowid
       0    index fast full scans (full)
       0    index fast full scans (rowid ranges)
       0    index fast full scans (direct read)
    2770    index fetch by key

This query gives an idea of index usage percentages.

SELECT 'Index to Table Ratio ' "Ratio" , ROUND(
(SELECT SUM(value) FROM V$SYSSTAT
      WHERE name LIKE 'index fast full%'
      OR name = 'index fetch by key'
      OR name = 'table fetch by rowid')
/ (SELECT SUM(value) FROM V$SYSSTAT WHERE
   name IN
      ('table scans (short tables)', 'table scans (long
         tables)')
),0)||':1' "Result"
FROM DUAL;

This is the result. This number may or may not be good depending on applications. However, my highly active processing generally uses only Referential Integrity indexing, whose index hits may not be recorded. As a result this ratio could be very misleading. This is another example of a ratio which is not necessarily very useful.

Ratio                   Result
----------------------  ------
Index to Table Ratio      30:1


18.4 Dictionary Cache Hit Ratio

This ratio looks at buffer activity versus disk activity for database metadata. How well is metadata accessed in memory from the shared pool without having to resort to physical disk activity?

SELECT 'Dictionary Cache Hit Ratio ' "Ratio"
,ROUND((1 - (SUM(GETMISSES)/SUM(GETS))) * 100,2)||'%'
   "Percentage"
FROM V$ROWCACHE;

Ratio                         Percentage
---------------------------   ----------
Dictionary Cache Hit Ratio        99.86%


18.5 Library Cache Hit Ratios

These ratios show library cache pre-parsing and reloading activity.

SELECT 'Library Lock Requests' "Ratio"
 , ROUND(AVG(gethitratio) * 100, 2)
      ||'%' "Percentage" FROM V$LIBRARYCACHE
UNION
SELECT 'Library Pin Requests' "Ratio", ROUND(AVG
   (pinhitratio) * 100, 2)
       ||'%' "Percentage" FROM V$LIBRARYCACHE
UNION
SELECT 'Library I/O Reloads' "Ratio"
      , ROUND((SUM(reloads) / SUM(pins)) * 100, 2)
      ||'%' "Percentage" FROM V$LIBRARYCACHE
UNION
SELECT 'Library Reparses' "Ratio"
      , ROUND((SUM(reloads) / SUM(pins)) * 100, 2)
      ||'%' "Percentage" FROM V$LIBRARYCACHE;

Ratio                     Percentage
-----------------------   ----------
Library I/O Reloads       0%
Library Lock Requests     96.31%
Library Pin Requests      96.1%
Library Reparses          0%


18.6 Disk Sort Ratio

Increase the size of the SORT_AREA_SIZE parameter to prevent sorting to disk. When using shared servers a smaller sort buffer may ultimately preserve memory resources for other essential processing. Retain a chunk of sort space memory for idle connections by setting the SORT_AREA_RETAINED_SIZE parameter to a small percentage of the SORT_AREA_SIZE parameter, usually 10%. Do not always set SORT_AREA_RETAINED_SIZE equal to SORT_AREA_SIZE.

SELECT 'Sorts in Memory ' "Ratio"
, ROUND(
  (SELECT SUM(value) FROM V$SYSSTAT WHERE name = 'sorts
    (memory)')
/ (SELECT SUM(value) FROM V$SYSSTAT
    WHERE name IN ('sorts (memory)', 'sorts (disk)')) *
      100, 2)
||'%' "Percentage"
FROM DUAL;

A ratio of memory to disk sorts of less than 100% is not necessarily a problem.

Ratio                Percentage
------------------   ----------
Sorts in Memory      98.89%
Note

Oracle Database 10 Grid Using the two parameters WORKAREA_SIZE_POLICY = AUTO and PGA_AGGREGATE_TARGET for automated session connection memory management has been recommended by Oracle Corporation since Oracle9i Database as better performing. Dedicated server connection configurations can perform well even at 1,000 concurrent connections. Automated session connection memory management only functions for a dedicated server configuration.


18.7 Chained Rows Ratio

Chained rows in Oracle Database are as a result of row migration or row chaining. Row chaining occurs when a row is too large for a single block and thus the row is "chained" or spread across multiple blocks. Row migration occurs when updating increases the row size, prohibiting the whole row from fitting in a single block. Thus, the row is migrated or moved elsewhere leaving only a pointer in the original block. It is possible that a row migration can lead to a row chain as well if the row becomes large enough. A lot of row chaining in a database is a big problem. Row chains are removed by table reconstruction, recreating the table, or by using export and re-import. Row chaining and migration can be minimized by leaving more empty space in each block when first inserting rows: set PCTFREE to a higher value. PCTFREE leaves empty space in a block by reserving more space for future updates. The downside to increasing PCTFREE is that rows will be less compacted and there will be fewer rows per block on average. High row to block density leads to faster access time because there are more rows read per block. The approach is dependent on requirements and behavior required by applications.

SELECT 'Chained Rows ' "Ratio"
, ROUND(
(SELECT SUM(value) FROM V$SYSSTAT
WHERE name = 'table fetch continued row')
/ (SELECT SUM(value) FROM V$SYSSTAT
   WHERE name IN ('table scan rows gotten', 'table fetch
      by rowid'))
* 100, 3)||'%' "Percentage"
FROM DUAL;

Ratio            Percentage
--------------   ----------
Chained Rows     0%

The chain count in my Accounts schema is negligible.


18.8 Parse Ratios

Parsing involves syntax checking and building of an execution query plan for SQL code. Every time SQL code is parsed its cursor and query plan are stored in the shared pool. The stored shared cursors can be re-used by matching SQL code statements; the match must be an exact text string match. Parsing is expensive so always pay attention to parse-related ratios. The less re-parsing in the shared pool the better.

The most obvious solution is increasing the size of the shared pool. However, increasing the shared pool to a ridiculous amount can cause its own problems. One of the most common causes of heavy re-parsing is lack of bind variables in SQL code. Setting the CURSOR_SHARING configuration parameter to SIMILAR or FORCE can help alleviate this problem without recoding SQL code.

Also increasing the SESSION_CACHED_CURSORS parameter can help reduce re-parsing by caching more session-specific cursors per session.

SELECT 'Soft Parses ' "Ratio"
, ROUND(
((SELECT SUM(value) FROM V$SYSSTAT WHERE name = 'parse
   count (total)')
- (SELECT SUM(value) FROM V$SYSSTAT WHERE name = 'parse
   count (hard)'))
/ (SELECT SUM(value) FROM V$SYSSTAT WHERE name = 'execute
   count')
* 100, 2)||'%' "Percentage"
FROM DUAL
UNION
SELECT 'Hard Parses ' "Ratio"
, ROUND(
(SELECT SUM(value) FROM V$SYSSTAT WHERE name = 'parse count
   (hard)')
/ (SELECT SUM(value) FROM V$SYSSTAT WHERE name = 'execute
   count')
* 100, 2)||'%' "Percentage"
FROM DUAL
UNION
SELECT 'Parse Failures ' "Ratio"
, ROUND(
(SELECT SUM(value) FROM V$SYSSTAT
 WHERE name = 'parse count (failures)')
/ (SELECT SUM(value) FROM V$SYSSTAT WHERE name = 'parse
    count (total)')
* 100, 2)||'%' "Percentage"

FROM DUAL;

Ratio            Percentage
--------------   ----------
Hard Parses       .04%
Parse Failures   0%
Soft Parses     44.98%


18.9 Latch Hit Ratio

As already stated, latches protect memory buffers from concurrent usage. This is a healthy ratio for the Accounts schema active database at well over 99%. However, this ratio does not necessarily show much because not only are the numbers hidden, individual hit rates for specific latches are not shown.

SELECT 'Latch Hit Ratio ' "Ratio"
, ROUND(
(SELECT SUM(gets) - SUM(misses) FROM V$LATCH)
/ (SELECT SUM(gets) FROM V$LATCH)
* 100, 2)||'%' "Percentage"
FROM DUAL;

Ratio              Percentage
----------------   ----------
Latch Hit Ratio    99.96%

That's enough ratios for now. There are many others. Obviously all the examples above would be best written into a stored procedure. Executing all the ratios together gives the following result.

Ratio                               Percentage
---------------------------------   ----------
Chained Rows                         0%
Database Buffer Cache Hit Ratio     95%
Dictionary Cache Hit Ratio          99.86%
Efficient Table Access              99.97%
Hard Parses                          0.04%
Index to Table Ratio                30:1
Latch Hit Ratio                     99.96%
Library I/O Reloads                  0%
Library Lock Requests               96.31%
Library Pin Requests                96.1%
Library Reparses                     0%
Long Table Scans                     0.03%
Parse Failures                       0%
Short Table Scans                   31.16%
Short to Long Full Table Scans      99.92%
Soft Parses                         44.98%
Sorts in Memory                     98.89%
Table by Index                      68.81%

There are numerous potential ratios that can be used to attempt to assess symptoms of database health. Once again be aware that a ratio divides one number by another. This division of two numbers can hide the values in those numerator and denominator values, which can be independently significant. The next chapter will look at wait events. Many things can happen in the database potentially causing one task to wait for another to complete.


Chapter 19: Wait Events

Overview

A wait event is a database event indicating that something is waiting for something else. In other words, a process could be waiting for another process to release a resource that the first process requires.

A wait event occurs because something is being processed where other processes waiting for its completion are dependent on the result of the event causing the wait. When one process waits for another to complete a wait event is recorded in database statistics.

Tip

The TIMED_STATISTICS configuration parameter must be set to TRUE.

Much like ratio calculations wait events are sometimes best analyzed as symptoms of problems, such as contention. Wait event frequency points to symptoms of problems, rather than being direct solutions. Therefore, tune the problem causing the wait event. Do not necessarily simply attempt to reduce wait events without being sure that the real cause of the problem is addressed, although sometimes this will be the case. Understand the problem. Also note that frequent occurrence of some wait events is normal, depending on database type and applications.

There are numerous performance views involved in recording of wait event details. These are some of them:

  • V$EVENT_NAME. All possible events.

  • V$SYSTEM_EVENT. Total waits for all occurring database events, aggregated for all sessions.

  • V$SESSION_EVENT. The same information as V$SYSTEM_ EVENT except detailing events per session rather than aggregated for all sessions.

  • V$SESSION_WAIT. This view is session level and shows a real-time picture where wait events are listed as they are happening. Contains hooks through to physical objects and sessions, through sessions ultimately leading to SQL code, amongst many other things.

  • V$WAITSTAT. Shows contention or competition for access to different types of blocks such as datafile blocks, rollback, and sort space.

There is much more to wait events than presented in this chapter. This chapter and the next simply present wait events and latches. The Oracle Database Wait Event Interface will be covered in Chapter 22. The Oracle Database Wait Event Interface can be used to drill down through a multitude of performance views, in an attempt to isolate events causing performance bottlenecks.

Note

Oracle Database 10 Grid Numerous Wait Model changes are made including changes to various V$ performance views and a number of additional V$ performance views. These will be covered in Chapter 22.

Wait events can be divided into significant and idle events. A significant event is an event potentially indicating a problem such as physical or logical resource contention. An idle event describes an activity such as process polling and generally is not an indication of a problem. First of all let's get idle events out of the way so that we know what can generally be ignored.


19.1 Idle Events

An idle event is idle because nothing is happening. An idle event is a result of a process busy waiting for something else to respond. Idle events are generally insignificant as far as performance is concerned although they can sometimes show a distinct lack of activity, a lack of use of resources, or simply over configuration. High idle times in specific areas such as SQL*Net network usage is quite normal.

However, high idle times on SQL*Net configuration could indicate over-configuration of shared servers for instance or a client process waiting for submission of a command to the database. Over-configuration is not really a problem except that hardware resources are wasted and could be better utilized elsewhere.

All possible database events are listed in the V$EVENT_NAME view. This view contains a lot of entries. Events in this view are applicable to all Oracle software options such as Oracle Replication and Oracle Real Application Clusters. This book covers only Oracle Database tuning and does not delve into Oracle optional applications and architectures other than Oracle Partitioning. My concurrently active OLTP database has specific events in its simulation. These following events using the displayed query are generally idle and irrelevant for anything but information. I can isolate idle events with a query such as this.

SELECT name FROM v$event_name
WHERE name LIKE '%null%' OR name LIKE '%timer%'
OR name LIKE '%SQL*Net%' OR name LIKE '%rdbms ipc%'
OR name LIKE '%ispatcher%' OR name LIKE '%virtual circuit%'
OR name LIKE '%PX%' OR name LIKE '%pipe%' OR name LIKE
   '%message%'
OR name LIKE 'jobq%';
Note

Oracle Database 10 Grid The number of idle events is expanded.

There are a lot of rows returned as idle events so I will not display the results here. The query is placed into a function as shown below for ease of use in queries in the rest of this book against the V$SYSTEM_EVENT performance view. This is the function.

CREATE OR REPLACE FUNCTION IdleEvent(pEvent IN VARCHAR2
   DEFAULT NULL)
RETURN VARCHAR2 IS
      CURSOR cIdleEvents IS
            SELECT name FROM v$event_name
            WHERE name LIKE '%null%' OR name LIKE
               '%timer%'
            OR name LIKE '%SQL*Net%' OR name LIKE '%rdbms
               ipc%'
            OR name LIKE '%ispatcher%' OR name LIKE
               '%virtual circuit%'
            OR name LIKE '%PX%' OR name LIKE '%pipe%'

            OR name LIKE '%message%' OR name LIKE 'jobq%';
BEGIN
      FOR rIdleEvent in cIdleEvents LOOP
            IF pEvent = rIdleEvent.name THEN
                  RETURN NULL;
            END IF;
      END LOOP;
      RETURN pEvent;
EXCEPTION WHEN OTHERS THEN
      DBMS_OUTPUT.PUT_LINE(SQLERRM(SQLCODE));
END;
/
ALTER FUNCTION IdleEvent COMPILE;
/
Tip

If using STATSPACK the STATS$IDLE_EVENTS table can be used.

The only idle events that warrant examination are those to do with Oracle networking software. On rare occasions excessive occurrence of SQL*Net idle events could possibly indicate bottleneck issues with the network. However, as already stated, excessive network activity is often normal. Other chapters in this book cover Oracle network usage tuning.

Another idle event occurring often in my database is the jobq slave wait event. This wait event does not indicate a problem. It occurs because I am executing multiple jobs simultaneously on the same data to simulate a highly active concurrent OLTP database. The intention is to produce tunable problems, which it most certainly does. When the background processes that are used to service jobs find that there are no queued jobs to be processed they will enter into this wait state.

Let's examine significant database and wait events.


19.2 Significant Events

Firstly, let's simply pick out significant events from the database I have running.

COL event FORMAT a30;
COL waits FORMAT 9999990;
COL timeouts FORMAT 99999990;
COL average FORMAT 99999990;
SELECT event "Event", time_waited "Total Time", total_waits
   "Waits"
         ,average_wait "Average", total_timeouts "Timeouts"
FROM v$system_event
WHERE IdleEvent(event) IS NOT NULL
ORDER BY event;

Following is the list of significant performance affecting events from my database containing the highly concurrent active Accounts schema.

Tip

The most important factor is the total time spent waiting (Total Time) not the number of waits (Waits).

Event                       Total Time   Waits    Average   Timeouts
-------------------------   ----------   ------   -------   --------
LGWR wait for redo copy     29257        14557      2       5854
async disk IO               0            54         0       0
buffer busy waits           426039       131236     3       61
buffer deadlock             50977        8944       6       8944
control file heartbeat      411          1          411     1
control file parallel       7349         8888       1       0
   write
control file sequential     6335         3632       2       0
   read
db file parallel write      30626        5694       5       2815
db file scattered read      74492        28648      3       0
db file sequential read     467185       80872      6       0
db file single write        27           54         0       0
direct path read            58           108        1       0
direct path write           14           78         0       0
enqueue                     93255        9865       9       63
instance state change       0            2          0       0
latch free                  110683       29690      4       3254
library cache load lock     303          39         8       0
library cache pin           1894         186        10      0
log file parallel write     34751        21291      2       19359
log file sequential read    42           58         1       0
log file single write       22           58         0       0
log file switch             1189         79         15      0
   completion
log file sync               150289       15409      10      140
process startup             348          7          50      2
refresh controlfile         24           1          24      0
command
row cache lock              19           14         1       0
Tip

A number of new events shown below help to divide up latch and enqueue wait events.

latch activity
latch free
latch: In memory undo latch
latch: KCL gc element parent lat
latch: cache buffer handles
latch: cache buffers chains
latch: cache buffers lru chain
latch: checkpoint queue latch
latch: enqueue hash chains
latch: gcs resource hash
latch: ges resource hash list
latch: latch wait list
latch: library cache
latch: library cache lock
latch: library cache pin
latch: messages
latch: parallel query alloc buff
latch: redo allocation
latch: redo copy
latch: row cache objects
latch: session allocation
latch: shared pool
latch: undo global data
latch: virtual circuit queues

It makes perfect logical sense that when assessing the impact of a wait event it should be weighed against all other wait events. It is of course true that some wait events are completely debilitating to the database no matter what their percentage of occurrence. One example of this is the occurrence of buffer deadlock events in my previous V$SYSTEM_EVENT query listing. Deadlocks are always a serious problem but are fortunately very unusual. These deadlocks occur because I am trying to stress out my highly active concurrent database deliberately. I am using triggers and hot block table updates on summary columns. I get deadlocks. What a surprise.

Tip

Deadlocks are rare and are usually a result of coding design issues and not high load! However, database performance is often highly dependent on the way in which applications are built.

There are two main approaches when examining events:

  • Total Wait Time. Some types of events may not occur frequently but their wait times could be significant such that those total wait times exceed the total wait times of more frequently occurring events. Obviously this approach is relative between different events and their possible performance impact.

  • Polling Over a Period of Time. In this chapter I have used temporary tables and some very simple stored procedures to poll real-time views over a period of time. This type of statistics collection can be executed easily using STATSPACK. The reason for including this level of detail in this book is to promote understanding. Once a reasonable level of understanding is gained, using STATSPACK will be a matter of reading the manual for correct syntax. This book is intended as a performance tuning teaching guide and not a database administration reference manual.

Before we examine significant wait events in detail we need another query. I want to know what the total wait time for each event is, as a percentage measured against every other event. A query like this can give me a better idea, not a solution. Remember we are looking for symptoms of problems not solutions. Here is the query.

COL percentage FORMAT 9999999990;
SELECT event "Event", total_waits "Waits", time_waited
   "Total Time"
         ,TO_CHAR(
               (time_waited /
                     (SELECT SUM(time_waited) FROM
                        v$system_event
                     WHERE IdleEvent(event) IS NOT NULL)
         )*100, 990.99) "Percentage"
FROM v$system_event WHERE IdleEvent(event) IS NOT NULL
   ORDER BY event;

Here is the result. Generally I would want to examine in detail anything greater than 1%. More potentially catastrophic wait events such as deadlocks I would examine if greater than zero. Some of the more interesting events are highlighted.

Event                           Waits   Total Time   Percent
-----------------------------   ------   ---------   -------
LGWR wait for redo copy         14591    29377       0.37
async disk IO                   54       0           0.00
buffer busy waits               131301   426285      5.40
buffer deadlock                 8948     51000       0.65
control file heartbeat          1        411         0.01
control file parallel write     8892     7360        0.09
control file sequential read    3635     6374        0.08
db file parallel write          5702     30652       0.39
db file scattered read          28648    74492       0.94
db file sequential read         81015    468348      5.93
db file single write            54       27          0.00
direct path read                108      58          0.00
direct path write               78       14          0.00
enqueue                         9872     93266       1.18
instance state change           2        0           0.00
latch free                      29734    110921      1.41
library cache load lock         39       303         0.00
library cache pin               186      1894        0.02
log file parallel write         21323    34836       0.44
log file sequential read        58       42          0.00
log file single write           58       22          0.00
log file switch completion      79       1189        0.02
log file sync                   15437    150731      1.91
process startup                 7        348         0.00
refresh controlfile command     1        24          0.00
row cache lock                  14       19          0.00

What are some significant database events?

  • Buffer Busy Waits. Waiting for a database buffer cache block to be available; the block is being read in or changed.

  • Datafile Scattered and Sequential Reads. Reads of datafile physical blocks on disk, where scattered means potentially random (full physical scans) in scattered noncontiguous parts of the buffer and sequential means more focused (index reads).

  • Direct Path Read and Writes. Passing of sorting and hash joins to temporary sort space on disk.

  • Free Buffer Waits. No free or clean blocks available in the database buffer cache and waiting for writing to disk of database buffer cache dirty blocks.

  • Row Cache Lock Waits. Metadata cache waits on schema object definitions.

  • Library Cache Waits. The library cache contains parsed SQL code, PL/SQL blocks, and optimized query plans.

  • Redo Log Waits. Redo log wait events.

  • Rollback and Automated Undo Waits. Wait events can help determine sizes and numbers of rollback segments. Automatic undo is easier to manage but its reliability may be in question due its very recent introduction into Oracle Database. However, I have so far heard only good reports about automated undo.

  • Enqueue Waits. Locking in the database either of datafile tables and rows or latches busy in buffers.

  • Latch Free Waits. Waiting to access an area of a buffer; a latch required to access that buffer area is busy.

Now let's examine these significant events in turn.

19.2.1 Buffer Busy Waits

A buffer busy wait occurs when more than one process is attempting to access a single database buffer cache block at once. In other words, the block is not in memory but another process is already reading it into memory or the block is already in memory but currently being changed by another process.

Obviously it is a major problem if many DML statements are attempting to alter something in a block at the same time. Let's query the V$WAITSTAT view to see exactly which types of buffers are causing a problem, and examine a few others as well.

Tip

The V$WAITSTAT performance view shows a breakdown of buffer busy wait events by class of buffer.

SELECT * FROM v$waitstat:

CLASS                   COUNT      TIME
---------------------   ------    -------
data block              141860     474325
sort block                   0          0
save undo block              0          0
segment header             157        195
save undo header             0          0
free list                    0          0
extent map                   0          0
1st level bmb                0          0
2nd level bmb                0          0
3rd level bmb                0          0
bitmap block                 0          0
bitmap index block           0          0
file header block            2         34
unused                       0          0
system undo header           0          0
system undo block            0          0
undo header                614       1487
undo block                  27        103

  • Data Block. Indicates database buffer cache dirty writes or an inability to read from disk due to contention with other competing requests.

  • Sort Block. Indicates there is not enough sort space or a temporary sort is either nonexistent on disk or is not defaulted for the current user.

  • Segment Header. There is contention for the first block of a table or an index. A segment can be a table or index, amongst other database objects.

  • Free List. There is too much free space maintained for each block.

  • Extent Map. Extents may be poorly sized if there is too much recursion. Perhaps extent sizes are all different or they are too small. This is likely to be less common using locally managed tablespaces as opposed to dictionary-managed tablespaces.

    Tip

    Dictionary-managed tablespaces will be deprecated in a future version of Oracle Database.

  • Bitmap Block. Bitmap contention.

  • File Header Block. Datafile contention.

  • Undo Header and Block. For manual rollback there are not enough rollback segments, they are too small or there are inconsistent sizes across all rollback segments. If automated undo is used then perhaps change the UNDO_RETENTION parameter.

    Tip

    Manual rollback is deprecated and replaced by automatic undo (automatic rollback). However, many Oracle9i and even Oracle8i databases are still in use. Automated undo is recommended for use with Oracle9i Database.

It appears that we have problems with data blocks (tables and indexes), segment headers, and rollback. My database uses both manual rollback and automated undo at different points in time in order to demonstrate both. We can use the DBA_EXTENTS, V$SESSION_WAIT, and V$SESSION views to find objects causing problems. This query might help in this respect.

SELECT de.segment_name, sw.event
FROM dba_extents de, (
      SELECT sw.event, sw.p1 AS file_id, sw.p2 AS block_id
      FROM v$session_wait sw
      WHERE IdleEvent(event) IS NOT NULL
      AND sw.sid IN
            (SELECT sid FROM v$session WHERE username =
               'ACCOUNTS')
      AND sw.event != 'null event'
) sw
WHERE de.owner = 'ACCOUNTS'
AND de.file_id = sw.file_id AND de.block_id = sw.block_id;

This previous query could be extremely slow. Any queries on the DBA_EXTENTS view in my database are incredibly slow. My physical storage structure is poorly organized and my database is extremely busy. As a result I decided to do some manual statistics collection. What I did was to create a temporary table and insert the results of the subquery semi-join between the V$SESSION and V$SESSION_WAIT views. I also know the username I am looking at, namely the Accounts schema. Something like this:

Tip

This type of analysis is easy with STATSPACK and even easier using the Oracle Database Wait Event Interface, which will be covered in Chapter 22.

DROP TABLE tmp;
CREATE TABLE tmp (event VARCHAR2(64), file_id NUMBER,
   block_id NUMBER);
BEGIN
      FOR counter IN 1.1000 LOOP
            INSERT INTO tmp
              SELECT sw.event AS event
                ,sw.p1 AS file_id, sw.p2 AS block_id
              FROM v$session_wait sw
              WHERE IdleEvent(event) IS NOT NULL
              AND sw.sid IN
              , (SELECT sid FROM v$session WHERE username =
                 'ACCOUNTS')
              AND sw.event != 'null event';
             COMMIT;
      END LOOP;
END;
/

The execution of this anonymous procedure generated over a thousand rows into my temporary table. I then did this.

SELECT COUNT(event), event FROM tmp GROUP BY event;

COUNT(EVENT)    EVENT
------------    ---------------
          90    buffer busy waits
          12    buffer deadlock
         688    db file sequential read
          97    enqueue
          48    latch free
          91    log file sync

Now since we have 90 buffer busy waits we have a problem. Use a query like this to find out which tables and indexes are affected.

SELECT DISTINCT(segment_name) FROM dba_extents
WHERE (file_id, block_id) IN
  (SELECT file_id, block_id FROM tmp WHERE event = 'buffer
    busy waits')
AND owner = 'ACCOUNTS';

Then I joined the temporary table and the DBA_EXTENTS view and got some interesting results. The query returned over 3,000 rows. Ouch! Well I know there are problems in my database, serious problems. That is the way I intended it to be. In fact I know which tables are probably causing the problems with sequence number generated primary keys and locking of index blocks. The GeneralLedger and StockMovement tables are prime culprits for this type of activity because they are most heavily added to using triggers. "Yukkity-yuk!" The triggers do what I want them to do. They make my coding easier and they cause performance problems.

Causes of Buffer Busy Waits

Looking back at the query on V$WAITSTAT my Accounts schema highly active concurrent database is having problems with data blocks, segment headers, undo headers, undo blocks and file header blocks; just one or two problems.

  • Data Block. These errors usually occur when there are index problems resulting in too many full table scans, or mass data insertions adding multiple rows to the same block. My database uses sequence number primary keys and is doing a lot of insertion activity. The larger more active insertion tables should probably use reverse key indexes on the primary keys to avoid index hot block contention.

  • Segment Header. This occurs generally with lack of block concurrency. Better settings for FREELISTS and INITRANS might help. Once again tablespaces are all locally managed, allowing for better automated management of physical space than for tablespaces in my Accounts schema database. This database is simply very busy. Changing tablespaces to automatic segment space management would help by better managing freespace and concurrency.

    Tip

    Setting values for PCTUSED, FREELIST, and FREELIST_ GROUPS in database objects such as tables will be ignored if the containing tablespace is set to automatic segment space management. Automated segment space management is reputed to realize up to 35% performance improvement, specifically in Oracle RAC environments.

  • Undo Header. Rollback header problems hint at creating more rollback segments if using manual rollback.

  • Undo Block. This error is generally touted as indicating a need to make existing rollback segments larger. I do not agree with this approach. An OLTP database requires as many rollback segments as possible, small and sized the same. Perhaps extent size increments could even be substantially decreased assuming transactions are all small.

Decreasing Buffer Busy Waits

Let's take another look at all events.

COL percentage FORMAT 9999999990;
SELECT event "Event", total_waits "Waits", time_waited
   "Total
 ,TO_CHAR(
   (time_waited /
     (SELECT SUM(time_waited) FROM v$system_event
   WHERE IdleEvent(event) IS NOT NULL)
)*100, 990.99) "Percentage"

FROM v$system_event WHERE IdleEvent(event) IS NOT NULL
ORDER BY event;

Event                            Waits   Total Time   Percent
-----------------------------   ------   ----------   -------
LGWR wait for redo copy            100           84      0.00
buffer busy waits               114429      1721602      8.68
buffer deadlock                     31          116      0.00
control file heartbeat               1          410      0.00
control file parallel write       2568        40352      0.20
control file sequential read      1163        11180      0.06
db file parallel write            3618        42457      0.21
db file scattered read          114041      3148406     15.87
db file sequential read         718549     14837478     74.78
direct path read                    31            3      0.00
direct path write                    9            0      0.00
enqueue                            343        24617      0.12
latch free                         632         6341      0.03
library cache load lock              8           76      0.00
library cache pin                   86         1508      0.01
log file parallel write           5314         4155      0.02
log file sequential read            24           41      0.00
log file single write               24           17      0.00
log file sync                     1651         1810      0.01
process startup                      1           82      0.00
refresh controlfile command          1            6      0.00
row cache lock                       9          106      0.00

And let's look at buffer busy waits specifically.

COL event FORMAT a30;
COL percentage FORMAT 9999999990;
       SELECT event "Event" ,ROUND((time_waited /
      (SELECT SUM(time_waited) FROM v$system_event
WHERE IdleEvent(event) IS NOT NULL))*100, 0) "%"
FROM v$system_event WHERE event = 'buffer busy waits';

My current percentage of buffer busy waits is 9%.

Event                 %
------------------   ---
buffer busy waits     9

Now let's make a few changes to try to reduce those buffer busy wait events. Increase the database buffer cache size to reduce buffer contention.

ALTER SYSTEM SET db_cache_size = 64M;

Reduce multiple block read counts into the database buffer cache to reduce the amount read at once.

ALTER SYSTEM SET db_file_multiblock_read_count = 1;

Some of the larger tables are subject to heavy sequential primary key sequence insertion activity and are subject to potential concurrency problems. I do get segment header contention but no free list contention. Using reverse key indexes on these primary keys may reduce segment header contention.

ALTER INDEX XPK_GENERALLEDGER REBUILD REVERSE;
ALTER INDEX XPK_STOCKMOVEMENT REBUILD REVERSE;

Using automated segment space management may help with percentage used space and freelist space management. I use the CREATE TABLESPACE command to create 2K block size table-spaces for static tables and indexes in the Accounts schema. I create tablespaces with automated segment space management. As already stated using automated segment space management could help concurrency. Note that I also place the DATA2K and INDX2K new tablespaces on two new drives, separate to that of currently used tablespaces and separately to each other. Additionally I set the DB_2K_CACHE_SIZE parameter and move the static tables into the new tablespaces.

ALTER SYSTEM SET db_2k_cache_size = 8M;
CREATE TABLESPACE DATA2K
      DATAFILE 'e:\oracle\oradata\test\data2k01.dbf'
      SIZE 2M AUTOEXTEND ON EXTENT MANAGEMENT LOCAL
      SEGMENT SPACE MANAGEMENT AUTO;
CREATE TABLESPACE INDX2K
      DATAFILE 'f:\oracle\oradata\test\indx2k01.dbf'
      SIZE 2M AUTOEXTEND ON EXTENT MANAGEMENT LOCAL
      SEGMENT SPACE MANAGEMENT AUTO;
ALTER TABLE type MOVE TABLESPACE DATA2K;
ALTER TABLE subtype MOVE TABLESPACE DATA2K;
ALTER TABLE coa MOVE TABLESPACE DATA2K;
ALTER TABLE category MOVE TABLESPACE DATA2K;
ALTER TABLE stock MOVE TABLESPACE DATA2K;
ALTER INDEX XAK_CATEG_TEXT REBUILD ONLINE TABLESPACE INDX2K;
ALTER INDEX XPK_CATEGORY REBUILD ONLINE TABLESPACE INDX2K;
ALTER INDEX XFK_COA_SUBTYPE REBUILD ONLINE TABLESPACE INDX2K;
ALTER INDEX XFK_COA_TYPE REBUILD ONLINE TABLESPACE INDX2K;
ALTER INDEX XPK_COA REBUILD ONLINE TABLESPACE INDX2K;
ALTER INDEX XAK_STOCK_TEXT REBUILD ONLINE TABLESPACE INDX2K;
ALTER INDEX XFK_S_CATEGORY REBUILD ONLINE TABLESPACE INDX2K;
ALTER INDEX XPK_STOCK REBUILD ONLINE TABLESPACE INDX2K;
ALTER INDEX XPK_SUBTYPE REBUILD ONLINE TABLESPACE INDX2K;
ALTER INDEX XPK_TYPE REBUILD ONLINE TABLESPACE INDX2K;

Now let's take another look at buffer busy waits after allowing the database to execute my high concurrency processing for over an hour with the new tablespaces and parameter settings.

COL event FORMAT a30;
COL percentage FORMAT 9999999990;
SELECT event "Event" ,ROUND((time_waited /
      (SELECT SUM(time_waited) FROM v$system_event
      WHERE IdleEvent(event) IS NOT NULL))*100, 0) "%"
FROM v$system_event WHERE event = 'buffer busy waits';

The result is totally opposite to that expected. Additionally the percentage of buffer busy wait events is climbing over time. Drilling down into the Oracle Database Wait Event Interface might help to resolve this issue more easily, or perhaps even show that there is no problem. My buffer busy waits are now 45%. I expected less than 9%.

Event                    %
-------------------   -----
buffer busy waits       45

Let's take a look at all events again.

COL percentage FORMAT 9999999990;
SELECT event "Event", total_waits "Waits", time_waited
   "Total"
 ,TO_CHAR(
   (time_waited /
     (SELECT SUM(time_waited) FROM v$system_event
   WHERE IdleEvent(event) IS NOT NULL)
 )*100, 990.99) "Percentage"
FROM v$system_event WHERE IdleEvent(event) IS NOT NULL
   ORDER BY event;

Event                        Waits   Total Time   Percent
-------------------------   ------   ----------   -------
LGWR wait for redo copy         38           24      0.00
buffer busy waits           456308      4672151     45.07
buffer deadlock                 13           12      0.00
control file heartbeat           1          410      0.00
control file parallel
   write                      1429        14323      0.14
control file sequential
   read                        712         4260      0.04
db file parallel write        1610        16889      0.16
db file sequential read     409662      5630885     54.32
direct path read                37            0      0.00
direct path write               11            1      0.00
enqueue                        360        21290      0.21
instance state change            1            0      0.00
latch free                    1295          827      0.01
library cache load lock         26          270      0.00
library cache pin               68          919      0.01
log file parallel write       2944         2187      0.02
log file sequential
   read                         24           47      0.00
log file single write           24           17      0.00
log file sync                 1000         1553      0.01
process startup                  1           23      0.00
refresh controlfile
   command                       1            4      0.00
row cache lock                  16           11      0.00

In the previous query result the db file sequential read event is higher and db file scattered read events are nonexistent. It is possible that higher buffer busy wait events are caused by hot blocks on static tables in the new 2K-sized tablespaces, although this should be less likely because the block size is smaller.

Obviously all these changes did not help so I will change them all back. Be careful with any of these types of changes. It is possible that placing the much larger Customer and Supplier tables into the 2K block size tablespaces may help performance. The static data I did place into the 2K block size tablespaces are small enough tables as to probably be negligible. Additionally there is so much small transactional activity in my database that management of the extra table-spaces and the extra database buffer cache area caused more problems than were solved.

Tip

With respect to the futility of tuning of ratios and specifically the database buffer cache hit ratio, note how increasing the size of the database buffer cache and even splitting that cache into two had no useful effect whatsoever. Quite often a high database buffer cache hit ratio is as a result of a very much oversized database buffer cache. This is usually done in an attempt to "tune" the database buffer cache hit ratio. There is no sense in tuning the "untunable".

ALTER SYSTEM SET db_cache_size = 32M;
ALTER SYSTEM SET db_file_multiblock_read_count = 4;
ALTER INDEX XPK_GENERALLEDGER REBUILD NOREVERSE;
ALTER INDEX XPK_STOCKMOVEMENT REBUILD NOREVERSE;

ALTER TABLE type MOVE TABLESPACE DATA;
ALTER TABLE subtype MOVE TABLESPACE DATA;
ALTER TABLE coa MOVE TABLESPACE DATA;
ALTER TABLE category MOVE TABLESPACE DATA;
ALTER TABLE stock MOVE TABLESPACE DATA;

ALTER INDEX XAK_CATEG_TEXT REBUILD ONLINE TABLESPACE
   INDX;
ALTER INDEX XPK_CATEGORY REBUILD ONLINE TABLESPACE
   INDX;
ALTER INDEX XFK_COA_SUBTYPE REBUILD ONLINE TABLESPACE
   INDX;
ALTER INDEX XFK_COA_TYPE REBUILD ONLINE TABLESPACE INDX;
ALTER INDEX XPK_COA REBUILD ONLINE TABLESPACE INDX;
ALTER INDEX XAK_STOCK_TEXT REBUILD ONLINE TABLESPACE
   INDX;
ALTER INDEX XFK_S_CATEGORY REBUILD ONLINE TABLESPACE
   INDX;
ALTER INDEX XPK_STOCK REBUILD ONLINE TABLESPACE INDX;
ALTER INDEX XPK_SUBTYPE REBUILD ONLINE TABLESPACE INDX;
ALTER INDEX XPK_TYPE REBUILD ONLINE TABLESPACE INDX;

DROP TABLESPACE data2k INCLUDING CONTENTS;
DROP TABLESPACE indx2k INCLUDING CONTENTS;

ALTER SYSTEM SET db_2k_cache_size = 0;

19.2.2 Datafile Scattered and Sequential Reads

  • db file scattered read. The simplest explanation for this event is that it occurs during full physical scans, full table, or fast full index scans; there are large amounts of I/O, reading more than one block at once, effectively scattering blocks all over the database buffer cache, leading to fragmentation of the buffer.

  • db file sequential read. This event usually, but not always, occurs during a single block read of data from disk. The event is either an index to table ROWID pointer access or perhaps a very small, single block table read.

    Tip

    Other activities causing these events are usually negligible with respect to performance, especially in a highly active database.

Unfortunately there is a lot of conflicting documentation and opinion as to exactly what can be done to reduce the occurrence of these events, and even whether their frequency should be reduced. Mostly requirements between full and precise scanning are application dependent. In other words, OLTP databases with small transactions should have mostly sequential reads. Data warehouses should most often use scattered reads. This is not an absolute for every application and every database. In other words, scattered reads are not necessarily a sign of poor performance in an OLTP database.

Additionally high amounts of full physical scanning can indicate possible fragmentation or perhaps a multiple block scanning setting inappropriate for the application (the DB_FILE_MULTIBLOCK_ READ_COUNT parameter), amongst other possible causes.

Once again full physical scans are not necessarily a bad thing. For instance fast, very costly disk RAID arrays may sometimes read multiple blocks faster than single blocks and thus more full scans will result. Obviously this is relative to how much data is read and how much is required. In other words, full physical scans are still healthier in data warehouses or when reporting. The threshold for when full physical scans become less efficient may simply be higher with more expensive equipment.

Once again isolate specific culprit objects by joining V$SESSION, V$SESSION_WAIT, and DBA_EXTENTS. In general, use of random and sequential access is often application dependent. Properly tuned OLTP database SQL code will often execute more sequential access on indexes than otherwise. Significant scattered read access is often due to full table scans on smaller tables and index fast full scans, both of which can and do have a positive impact on performance. My Accounts schema is executing heavy sequential index access, appropriate to a small-sized transaction, highly active OLTP database.

My Accounts schema database is more or less properly tuned with respect to sequential and scattered reads. Current values are as shown.

COL event FORMAT a30;
COL percentage FORMAT 9999999990;
SELECT event "Event"
      ,ROUND(
            (time_waited /
                  (SELECT SUM(time_waited) FROM
                     v$system_event
                  WHERE IdleEvent(event) IS NOT NULL)
      )*100, 0) "%"
FROM v$system_event WHERE event IN
      ('db file sequential read','db file scattered
         read');

Multiple configuration changes have altered the statistics in my database somewhat as can been seen. These values are fairly good and consistent with my application requirements.

Event                         %
--------------------------   ----
db file sequential read       76
db file scattered read        11

Let's make a very simple change, encouraging the Optimizer to perform more full physical scans. Physical scans are beneficial to larger transactions such as when batch processing or in a data warehouse.

ALTER SYSTEM SET db_file_multiblock_read_count = 128;

Sequential reads have increased and scattered reads have decreased, once again not as expected. We could temporarily disable some key indexes and have the desired effect. However, it is interesting that full physical scan rates have actually been reduced and not increased.

Event                         %
--------------------------   ----
db file sequential read       84
db file scattered read         8

Both I/O read and full physical scan activity in my database is minimal due to the nature of my high concurrency coding. Decreasing the database buffer cache size at the same time might have the desired ill effect, increasing the likelihood of full table scans. Let's make the database buffer cache seriously small.

ALTER SYSTEM SET db_cache_size=4M;

And let's check the same events once again after a few minutes of execution. Once again there is little response and what there is is the opposite. If by now I have not pointed out the danger of randomly changing parameters according to published documentation then I am wasting my time. Most of the reason why I am not getting the expected results is because my application code simply does not need the configuration changes made in this chapter so far. My data model design is efficient and my application concurrency code has an absolutely minimal footprint on hardware resources. The value of data model and SQL code tuning cannot be stressed more vehemently!

Event                         %
--------------------------   ----
db file sequential read      85
db file scattered read        6

19.2.3 Direct Path Reads and Writes

  • direct path read.

  • direct path write.

    Tip

    Both these events involve asynchronous I/O, available in Windows but not always on Unix.

These two events occur when sorting passes to disk or hash joins are not executed entirely in memory. SORT_AREA_SIZE and HASH_AREA_SIZE parameters could be increased. Increasing these parameters may use up too much memory in a large concurrent shared server environment. Increasing these parameters could cause more problems than are resolved. In short it is better to tune the root cause of a problem, in this case likely to be SQL code, rather than allow for poorly tuned SQL code statements to co-exist with oversized buffer configuration values.

19.2.4 Free Buffer Waits

The free buffer waits event occurs when there are no free clean blocks available in the database buffer cache. In other words, there are too many dirty (changed) blocks in the database buffer cache, waiting to be written to disk. The reason why this event happens is simple to explain. There is too much going on. You could get a bigger machine, faster disks or even create more database writer processes. The truth is that free buffer wait events often occur when dirty buffers are not written out to disk fast enough. The reason is one or a combination of the previously mentioned factors.

Free buffer wait events can even sometimes occur when the database buffer cache is too small, where the percentage of dirty blocks is continuously high and waiting to be written to disk. If the database buffer cache is too small high DML activity and latch wait problems could cause the database writer processes to be unable to keep up.

Once again to reiterate on a previously mentioned premise, do not simply resize buffers up to enormous amounts of memory. The general purpose of buffer caches is as to behave as fast temporary transfer mechanisms or to enhance high sharing and concurrency capability. The database buffer cache is a fast temporary transfer mechanism catering to vast differences in speed between CPU processing speed and disk I/O performance. The shared pool allows for very high concurrency or sharing of previously obtained objects and executed code. Sizing the database buffer cache and the shared pool to fill all available RAM based on high buffer cache hit ratios is not a solution: it does not solve the underlying problems. Low and high cache hit ratios generally indicate problems on a deeper level, not resolved by simply bumping up those caches to ridiculous sizes.

Tip

Tuning of cache ratios is often a fruitless exercise but they might indicate that a problem exists, though not necessarily. Do not try to tune ratios.

19.2.5 Row Cache Lock Waits

Row cache lock waits are usually SYSTEM tablespace metadata update locks or latch waits. The Accounts schema does have a small number of row cache lock waits. The number is below 1% and probably negligible. The event is called a row cache lock wait.

19.2.6 Library Cache Waits

The library cache contains parsed SQL code, PL/SQL blocks, and optimized query plans. Three types of wait events occur in the library cache.

  • Library Cache Lock. Loading and locking an object into memory.

  • Library Cache Load Lock. Cannot obtain a lock to load an object into memory.

  • Library Cache Pin. An object is pinned into memory.

Both row cache lock and library cache wait events occur in the shared pool. In trying to create a problem I will make the following changes in the configuration parameter file and bounce my database. Not setting the SHARED_POOL_RESERVED_SIZE parameter will force Oracle Database to automatically set that parameter to 5% of the SHARED_POOL_SIZE parameter.

shared_pool_reserved_size = 0
shared_pool_size = 4M
Tip

These parameters can be changed online using the ALTER SYSTEM command. I did and my database effectively halted when decreasing in size. Obviously what is being used must be cleared. However, I also executed the ALTER SYSTEM FLUSH SHARED_POOL command and still got the same response.

COL event FORMAT a30;
COL percentage FORMAT 9999999990;
SELECT event "Event"
      ,ROUND(
            (time_waited /
                  (SELECT SUM(time_waited) FROM
                     v$system_event
                  WHERE IdleEvent(event) IS NOT NULL)
      )*100, 2) "%"
FROM v$system_event
WHERE event LIKE 'library cache%' OR event LIKE 'row
   cache%';

This time I get an expected result. Since my shared pool is too small I found a marked increase in both row cache and library cache areas. I also got a few very nasty ORA-00600 internal error codes in my alert log due to a serious shortage of shared pool space. Don't try this one at home! The point is made though. The smaller the shared pool is, the more shared pool type wait events will occur. On the contrary, do not size the shared pool up and up simply to cope with application SQL code and data model design issues, as you will eventually encounter scalability issues, probably at a point where you have enough paying customers to cause you a serious business problem, when you can least afford lack of scalability.

Event                           %
--------------------------   ------
row cache lock                  .38
library cache pin             16.72
library cache load lock        1.59

19.2.7 Redo Log Waits

Redo log waits are affected by the redo log buffer, redo log files, and archive log files. Many things affect the efficiency of logging in Oracle Database. Performance is more likely to be adversely affected by a log buffer which is too small rather than too large. The most profound effect on log performance is usually disk I/O speed and physical architecture, during log buffer flushing to disk and copying of redo log files to archive log files.

When examining redo log waits and problems look for the redo log buffer running out of space, too much time taken to write redo logs or copying of archives, and perhaps even too much redo log file switching.

  • Redo Entries. How often redo entries are copied into the buffer.

  • Redo Writes. How often buffer entries are written to redo log files.

  • Redo Buffer Allocation Retries. This event indicates failure to allocate space in the redo log buffer either because of slow log writer processing speed or a log switch. Thus the log buffer could be too small or redo logs could be too small. However, this event can also indicate poorly performing disk I/O where log files are stored.

  • Redo Log Space Requests. Records requests for the buffer to write out to a redo log file typically during a log switch.

    Tip

    The redo log space requests event has been misinterpreted in the past and does not indicate requests for space in the log buffer.

  • Redo Sync Writes. A COMMIT command will require the database writer process to write dirty blocks to disk. Any required redo log entries must be flushed from the log buffer to redo log files prior to database writer activity, for the sake of recoverability. High frequency of this event could indicate high COMMIT rates, which may or may not be a problem, but are usually caused by application requirements or poor SQL coding.

A redo buffer allocation retries event occurs if the log buffer is too small. However, I have never found it useful, even on the largest databases, to make a log buffer larger than 1 Mb and rarely above 512K. Log buffer size is entirely dependent on transaction size. I have seen log buffers in excess of 500 Mb and this caused other problems, prior to tuning of course. The redo log buffer does not have to be equivalent to redo log file sizes. Additionally writing a large redo log file to disk can take a lot of time. Redo log buffer and file events are found by querying the V$SYSSTAT view.

COL name FORMAT a32;
SELECT value, name FROM v$sysstat WHERE name LIKE '%redo%'
ORDER BY 1 DESC;

VALUE       NAME
--------    -----------------------
86290896    redo size
  927748    redo wastage
  293852    redo entries
  175867    redo blocks written
   25776    redo write time
   23132    redo synch time
    3920    redo writer latching time
    3569    redo writes
    1870    redo synch writes
      50    redo buffer allocation retries
       0    redo log space requests
       0    redo log space wait time
       0    redo log switch interrupts
       0    redo ordering marks

There is no sense in having a huge redo log buffer. The redo log buffer is cyclic in nature. This means that redo log entries can be written to one part of the buffer as other entries in another part of the buffer are flushed to disk. Contention can occur sometimes when the redo log buffer is too small. However, depending on database activity, configuration and application behavior, a large redo log buffer could possibly cause problems by having to flush large chunks of data to disk. Once again a memory cache buffer is a high-speed bridge between CPU and disk storage, not a temporary storage area. The purpose of buffers is twofold: firstly, to manage speed differences between CPU and disk storage and secondly, to take advantage of CPU speed over that of disk storage.

Tip

Most redo log waits are as a direct result of poor I/O response time or contention, rarely log buffer size unless the LOG_BUFFER parameter has been tampered with.

Do not always assume that the redo log buffer is the source of redo log and archive problems. One very easy method of assessing redo log problems is by checking time stamps for both redo logs and archive logs on disk. If the dates are all the same then redo log files might be too small. Obviously this task can be accomplished using various Oracle Database views. Personally I prefer to simply examine file timestamps in the operating system. Automated scripting for this type of functionality is also faster built at the operating-system level. Checking timestamps on archive logs can be done with the following query.

SELECT * FROM (
 SELECT TO_CHAR(completion_time,'MM/DD/YYYY HH24:MI:SS')
    "Timestamp"
,name "Archive Log" FROM v$archived_log ORDER BY 1 DESC)
WHERE ROWNUM < 10;

My timestamps are very close together and thus redo log file sizes in my database are probably too small. However, my database is not archived and there are no archive logs being written. As far as my database is concerned there is nothing to tune.

Timestamp               Archive Log
--------------------    --------------------------------------
04/02/2003 20:12:55     F:\ORACLE\ORADATA\TEST\ARCHIVE\
                           ARC_1350.ARC
04/02/2003 20:12:54     F:\ORACLE\ORADATA\TEST\ARCHIVE\
                           ARC_1349.ARC
04/02/2003 20:12:53     F:\ORACLE\ORADATA\TEST\ARCHIVE\
                           ARC_1347.ARC
04/02/2003 20:12:53     F:\ORACLE\ORADATA\TEST\ARCHIVE\
                           ARC_1348.ARC
04/02/2003 20:12:52     F:\ORACLE\ORADATA\TEST\ARCHIVE\
                           ARC_1346.ARC
04/02/2003 20:12:51     F:\ORACLE\ORADATA\TEST\ARCHIVE\
                           ARC_1345.ARC
04/02/2003 20:12:50     F:\ORACLE\ORADATA\TEST\ARCHIVE\
                           ARC_1343.ARC
04/02/2003 20:12:50     F:\ORACLE\ORADATA\TEST\ARCHIVE\
                           ARC_1344.ARC
04/02/2003 20:12:49     F:\ORACLE\ORADATA\TEST\ARCHIVE\
                           ARC_1342.ARC

If the database is periodically halting then archiving could be temporarily halting the database, waiting for redo log files to be copied to archive log files. A situation is possible where all redo files contain nonarchived redo log entries. The redo log files are cyclic just like the redo log buffer. If all redo log files are waiting for copying to archive log files then no logging activity can occur until at least one redo log file is archived. The database will wait for archiving to catch up. In this case perhaps create more redo log files or another archive process. The speed of disk storage and network communication to the disks containing redo log and archive log files could also be an issue. For instance, standby database log file transfers should have a good network connection.

Three other interesting facts about redo log files:

  • Duplexed Redo Logs. Multiple redo log file copies or duplexed redo log files are not an issue for performance, especially on a multiple CPU platform. Redo log files will be written in parallel. Redo log files are I/O-bound and not CPU-bound. This means that the I/O disk structure is so much more important to redo logs as to make CPU use more or less negligible.

  • Log Appending. Redo and archive log files are written sequentially by appending and are best placed onto sequential access disk storage RAID array structures.

  • Alternating Log Files. In extreme circumstances alternate redo log files can be split onto separate disk storage areas since they may be open at the same time where the first redo log file is being archived and the second flushed from the redo log buffer.

Tuning redo and archive details is not all about the redo log buffer. "Guesstimate" changes to the redo log buffer will probably not solve performance problems and could make them worse.

19.2.8 Rollback and Undo Waits

Rollback segment and automated undo wait statistics can be obtained from the V$WAITSTAT performance view as shown. My Accounts schema database uses both manual rollback segments and automated undo, optionally with either switched online at any one time.

SELECT class "Event", count, time FROM v$waitstat
      WHERE class LIKE '%undo%' ORDER BY class;

Event                 COUNT   TIME
-------------------   -----   ----
save undo block           0      0
save undo header          0      0
system undo block         0      0
system undo header        0      0
undo block               16     63
undo header             128    355

Manual Rollback Waits

Comparing rollback wait events and database buffer cache activity is a little vague at best. If there are a large amount of rollback or automated undo wait events in relation to general database activity a number of things could be inferred. The obvious answer is that an application is performing a lot of rollback activity. A high incidence of ROLLBACK command instigated rollback activity is probably abnormal. Application rollback activity is much more likely to be caused by heavy DML activity resulting in heavy rollback use obtaining consistent point-in-time snapshot views of data. This is normal. Increasing commit frequency at the application level can help to alleviate rollback use but can cause other problems as well.

Comparing rollback wait events to database buffer cache events should produce a very low ratio. Perhaps a high ratio implies that rollback space is being waited for. If the percentage of rollback space wait events to database buffer cache is high then perhaps there are either not enough rollback segments or those rollback segments are not large enough. This query shows a negligible ratio of well below 1% using manual rollback segments.

SELECT
(
      (SELECT SUM(count) FROM v$waitstat
      WHERE class IN ('undo header', 'undo block'))
      /
      (SELECT SUM(value) FROM v$sysstat
      WHERE name IN ('db block gets', 'consistent gets'))
)*100 "Rollback vs DBCache"
FROM DUAL;

Rollback vs DBCache
----------------------
         .004774797

High rollback activity may not necessarily indicate a problem. High rollback rates could be an application requirement. However, transaction failures would be an issue. Investigate applications.

Note

Oracle Database 10 Grid Manual rollback is deprecated.

Automated Undo Waits

Executing the same query using automated undo rollback, with the database under very heavy contention, gives slightly different, and perhaps slightly worse performance. The difference in performance is 0.3%. This is such a low value as to be completely negligible.

SELECT
(
      (SELECT SUM(count) FROM v$waitstat
      WHERE class IN ('undo header', 'undo block'))
      /
      (SELECT SUM(value) FROM v$sysstat
      WHERE name IN ('db block gets', 'consistent gets'))
)*100 "Rollback vs DBCache"
FROM DUAL;

Rollback vs DBCache
----------------------
         .007855806
Tip

It is quite possible that sometimes a database administrator could tune manual rollback segments to perform better than automated undo segments. This point is probably a matter of opinion amongst database administrators.
Note

Oracle Database 10 Grid Manual rollback is deprecated.

On the other hand, my database shows wait statistics considerably higher using automated undo as opposed to using manual rollback, as shown in the previous query of the V$WAITSTAT performance view. This could have something to do with my geriatric hardware.

SELECT class "Event", count, time FROM v$waitstat
WHERE class LIKE '%undo%' ORDER BY class;

Event                  COUNT   TIME
--------------------   -----   ----
save undo block            0      0
save undo header           0      0
system undo block          0      0
system undo header         3    124
undo block               206    748
undo header              179   2111

Automated undo has a parameter called UNDO_RETENTION. This parameter allows retention of committed rollback database changes for a period of time. The more potentially unnecessary data retained, the more out of control automated undo could become. Defaults using the Oracle Database Configuration Assistant appear to be 900 for an OLTP database creation and 10,800 for a data warehouse database creation. It might be best not to oversize this parameter. The V$UNDOSTAT performance view is used for automated undo and provides probably more useful information about automated undo than the V$ROLLSTAT view provides about manual rollback.

A lot of experimentation, particularly with manual rollback segments, is in place in other chapters in this book. In general, a highly concurrent small transaction load OLTP database will require as many small manual rollback segments as possible. In the case of automated undo a lower value for the UNDO_RETENTION parameter is appropriate to small transactions. The opposite is true for large transactions when executing batch processing or in a data warehouse.

19.2.9 Enqueue Waits

This type of wait is an event caused by a lock on a row or a buffer latch wait. The wait event occurs when another request is waiting for a lock held by another session process. To find locks we use the V$SESSION_WAIT and V$LOCK views. My database has an enqueue locking value of over 1% so it possibly an issue. Additionally there are deadlocks, exemplifying this point further.

Tip

Deadlocks are usually caused by poor application coding such as use of triggers. This is most certainly the case in my database.

Oracle Database has many different types of enqueue events. These are some of the more significant and common ones.

  • TM. DML enqueue sometimes due to lack of indexing on foreign keys or just lack of indexing in general.

  • TX. Transactional enqueue or exclusive lock on a row or a table. Hopefully a row and not a table.

  • SQ. Sequence numbers.

This query is executed against the V$LOCK performance view on my database in the Accounts schema, monitored over about 30 s. These numbers can be written off as normal.

SELECT type "Lock", COUNT(type) "Locks" FROM v$lock GROUP
   BY type;

Lo     Locks
----   ------
SQ         10
TX         80

Many older relational databases implemented what was called incremental locking. Incremental locking successively locked larger objects or amounts of data as the amount of resources to maintain current locking grew. Oracle Database does not escalate locking even if contention can appear to cause this type of behavior. Block-level contention can occur in Oracle Database when enough concurrent DML row locking activity occurs in a single block, which can be counteracted by changing block level concurrency storage parameter settings. Certain types of DML activity can cause different types of locking. In general, Oracle Database locking will occur in different ways as follows:

  • Row Share. Read share lock on a single row.

  • Row Exclusive. Write lock on a row excluding write access to other sessions.

  • Table Share. As with rows but for an entire table.

  • Table Exclusive. As with rows but for an entire table.

Removing locks from a database instance can be an administration issue when serious. Resolving locking issues in terms of tuning out the locks is more of an SQL code and data model issue and possibly the result of poor application coding and design.

Enqueue numbers can be altered using the ENQUEUE_ RESOURCES parameter. Resource limitations can be monitored using the V$RESOURCE_LIMITS performance view.

COL resource FORMAT a24;
SELECT RESOURCE_NAME "Resource", INITIAL_ALLOCATION "Initial"
,CURRENT_UTILIZATION "Current", MAX_UTILIZATION "Max"
,LIMIT_VALUE "Limit"
FROM v$resource_limit WHERE resource_name NOT LIKE('g%')
ORDER BY 1;

Resource                   Initial   Current   Max   Limit
----------------------   ---------   -------   ---   ---------
branches                       247         0     0   UNLIMITED
cmtcallbk                      247         0     0   UNLIMITED
dml_locks                      988        71   134   UNLIMITED
enqueue_locks                 2890        29    31   2890
enqueue_resources             1208        43    43   UNLIMITED
max_rollback_segments           50        17    17   50
max_shared_servers               5         2     2   5
parallel_max_servers            11         0     0   11
processes                      200        26    29   200
sessions                       225        32    36   225
sort_segment_locks       UNLIMITED         0     1   UNLIMITED
temporary_table_locks    UNLIMITED         0     0   UNLIMITED
transactions                   247        10    14   UNLIMITED

In my database enqueue wait events occur at a steady rate of around 1%.

COL event FORMAT a30;
COL percentage FORMAT 9999999990;
SELECT event "Event"
      ,ROUND(
            (time_waited /
                  (SELECT SUM(time_waited) FROM
                     v$system_event
                  WHERE IdleEvent(event) IS NOT NULL)
      )*100, 2) "%"
FROM v$system_event WHERE event = 'enqueue';

Event        %
-------   ----
enqueue   1.02

Let's try to produce some enqueue wait events. Firstly, I drop all my Accounts schema foreign key indexes using a command like the following, which generates DDL commands. I have scripts to regenerate all my foreign key indexes.

SELECT 'DROP INDEX'||index_name|| ';'
      FROM user_indexes WHERE index_name LIKE 'XFK_%';

Now let's check the types of locks occurring.

SELECT type "Lock", COUNT(type) "Locks" FROM v$lock GROUP
BY type;

There are now a lot of TM and quite a few TX enqueue lock events since I have removed all my foreign key indexes.

Lo    Locks
---   -----
CF        1
JQ       24
MR       10
RT        1
TM       52
TS        1
TX        9
XR        1

Now I am checking all wait events again.

COL percentage FORMAT 9999999990;
SELECT event "Event", total_waits "Waits", time_waited
   "Total"
,TO_CHAR(
  (time_waited /
    (SELECT SUM(time_waited) FROM v$system_event
   WHERE IdleEvent(event) IS NOT NULL)
 )*100, 990.99) "Percentage"
FROM v$system_event WHERE IdleEvent(event) IS NOT NULL
   ORDER BY event;

As expected, enqueue wait events have increased slightly. Notice how db file scattered read events are dramatically increased, most likely due to full physical table scanning of tables during Referential Integrity checks on foreign keys with no indexes, and otherwise where the Optimizer utilizes foreign key indexes in SQL code.

Event                          Waits     Total   Percent
---------------------------   ------   -------   -------
LGWR wait for redo copy           16        28      0.00
buffer busy waits              58396    238368      9.40
buffer deadlock                    7        38      0.00
control file heartbeat             1       410      0.02
control file parallel write      369      3467      0.14
control file sequential read     283      1104      0.04
db file parallel write            86       755      0.03
db file scattered read        102816   1774183     69.94
db file sequential read        32830    426158     16.80
direct path read                  31         2      0.00
direct path write                  9         0      0.00
enqueue                          304     80489      3.17
latch free                       683      9212      0.36
library cache load lock            5        23      0.00
library cache pin                118      1748      0.07
log file parallel write          376       257      0.01
log file sequential read          24        40      0.00
log file single write             24        17      0.00
log file sync                     74       284      0.01
refresh controlfile command        1         6      0.00
row cache lock                     3        46      0.00

19.2.10 Latch Free Waits

Latches are a special type of locking mechanism, protecting buffers in memory. Latches are used to lock blocks in buffers to ensure that multiple sessions or processes do not alter the same space in memory at the same time. Unlike a lock waiting for lock release, a latch can spin or sleep and then try to access the same buffer area once again. Latches protect buffers from corruption by changes from different processes at the same time.

My database shows a latch free wait value of less than 2%. This is insignificant due to the nature of the wait event. It is normal for a database to have to wait for access to parts of Oracle Database buffer caches. The buffer caches should be a great deal smaller than physical areas in the datafiles. Sizing buffers too small can cause buffer contention or competition for memory access due to not enough available space in memory for cache buffers. On the contrary, sizing buffers too large can possibly cause the same problems. Managing of data in buffers is much more complex than managing of data in disk storage.

Latch wait events can be found by querying the V$SESSION_ WAIT performance view: drill down using the various latch V$ performance views. Latches are a major topic all by themselves. Therefore, this is a good point to end this general discussion of events and wait events and pass the examination of latches onto the next chapter. The next chapter will look into latches in detail, plus some special wait events applicable to latches, and how to handle those events.


Chapter 20: Latches

Wait events were introduced in the previous chapter. A wait event is recorded when one thing is waiting for another. There are many different types of wait events. High latch free wait event rates indicate latch misses, showing possible problems with SGA memory buffers. High quantities of latch misses can seriously affect overall database performance; what is required is not being acquired.

20.1 What is a Latch?

Latches protect memory buffers much like locks protect table rows. When a latch is already held on an area of a buffer a latch free wait event occurs if another process attempts to access the same type of data and fails to do so. There are numerous types of latches for all the different types of memory buffer areas. Latches apply to items such as data in the database buffer cache, preventing multiple simultaneous changes to the same data.

Tip

A latch cannot be tuned. Latch problems are indicative of other problems and can help to focus where significant performance problems may be occurring.
Note

Oracle Database 10 Grid Oracle9i Database Release 2 (9.2) has 239 latches. Oracle Database 10g has 316 latches.

20.1.1 Latch Misses, Spins, and Sleeps

In the previous chapter we saw that a latch free wait event is generated when a latch is missed.

Note

Oracle Database 10 Grid There are many more latch wait events where there were only two in Oracle9i Database.

What is a missed latch? When a latch is requested and the section of a buffer requested is "latched" (held by another process) then a latch free wait event is generated. Therefore, the second request for a latch already obtained by another process will wait for the required latch to be freed up for use. Where a lock on a table row simply waits for release of that table row by constant retrying for a lock, a latch miss will generate one of two types of action. The first type of action will place the latch in immediate mode. Immediate mode will cause the process requesting the latch to take an alternative action. The second type of action will place the latch request into willing to wait mode. Willing to wait mode causes the latch to repeatedly retry and spin. When the latch has spun a number of times, it will sleep for a period of time, reawakening to retry and potentially spin, and perhaps even sleep again. The number times a latch will retry acquiring the latch depends on a configuration parameter called SPIN_COUNT.

Tip

The SPIN_COUNT parameter is no longer available in Oracle9i Database and Oracle Database 10g. SPIN_COUNT has become _SPIN_COUNT and is now a hidden configuration parameter. Hidden configuration parameters are hidden because Oracle Corporation prefers them not to be changed.

Spinning latches can consume enormous amounts of CPU time. When a latch spins it repeatedly requests a latch. This can affect performance drastically if many latches are spinning at the same time. The following query shows gets, misses, and a miss rate for all latches in my database. The redo copy latch is not an issue as it is a potentially immediate mode latch. Other miss rates are not too significant but there is still potential for problems.

COL latch FORMAT a28;
COL gets FORMAT 99999990 HEADING "Gets";
COL misses FORMAT 99990 HEADING "Misses";
COL sleeps FORMAT 99990 HEADING "Sleep";
COL immediate_gets FORMAT 99999990 HEADING "IGet";
COL immediate_misses FORMAT 9990 HEADING "IMiss";
SELECT name "Latch", gets, misses
   ,TO_CHAR(ROUND((misses/gets)*100,2),'99990.00') "Miss Rate"
   ,immediate_gets, immediate_misses
FROM v$latch WHERE gets!=0 AND misses! =0 ORDER BY 4 DESC;

                                       Miss
Latch                 Gets   Misses    Rate      IGet   IMiss
---------------   --------   ------   -----   -------   -----
redo copy             1620      243   15.00    240928    2138
cache buffers
   lru chain       8022562     6651    0.08      2119       0
multiblock
   read objects    6253965     2356    0.04        42       0
cache buffers
   chains         34445102     9074    0.03   4957516     847
sequence cache       55793        7    0.01         0       0
   library cache   2019837      177    0.01         0       0
session
   allocation       500846       15    0.00         0       0
library
   cache pin       1451385       15    0.00         0       0
library cache
   pin allocation   332613        6    0.00         0       0
shared pool         757897        6    0.00         0       0
undo global data     80337        1    0.00         0       0
dml lock
   allocation       441665        5    0.00         0       0
enqueues            286118        1    0.00         0       0
redo allocation     267146        8    0.00         0       0
simulator lru
   latch            564509       19    0.00        48       0

Let's take a slightly different perspective in my Accounts schema database once again using the V$LATCH performance view. This time we focus on the total waiting time for each latch, more important than the number of misses.

COL latch FORMAT a28;
COL misses FORMAT 99990 HEADING "Miss";
COL sleeps FORMAT 99990 HEADING "Sleep";
SELECT name "Latch", ROUND(wait_time/1000) "WaitTime"
   ,ROUND((wait_time
    /(SELECT SUM(wait_time) FROM v$latch))*100,0) "Percent"
   ,misses, sleeps
FROM v$latch WHERE wait_time! =0 ORDER BY 3 DESC;

The picture is somewhat different. Library cache events have become more significant, the shared pool latch has become the least significant and the cache buffers chains latch has disappeared due to zero waiting time. The session allocation latch is interesting. This latch tells us about database connection activity. Continual connection and disconnection from the database can cause problems. There are many other latches used in Oracle Database, not shown here. This query shows significant latch activity in my database.

Latch                      WaitTime   Percent   Miss   Sleep
------------------------   --------   -------   ----   -----
cache buffers lru chain       10068        49   6653    7113
redo copy                      3693        18    243     297
library cache                  3520        17    177     207
multiblock read objects        2434        12   2357    2390
session allocation              569         3     15      16
library cache pin               108         1     15      18
enqueues                         10         0      1       1
library cache pin
allocation                       21         0      6       7
simulator lru latch              14         0     19      20
redo allocation                  35         0      8       9
dml lock allocation              50         0      5       7
undo global data                  0         0      1       1
sequence cache                   41         0      7       7
shared pool                      31         0      6       6

This introduces potential latch issues in the Accounts schema. Let's take a small step sideways and briefly examine latch performance views.

20.1.2 Latch Performance Views

There are a number of Oracle Database performance views for examining latches and latch statistics, the most important of these being the V$LATCH and the V$LATCHHOLDER performance views.

  • V$LATCH. Aggregation of all latch statistics.

  • V$LATCHNAME. All possible latches in an Oracle database.

  • V$LATCHHOLDER. Contains real-time current latch information about sessions and processes holding latches. This view can be used in conjunction with other views such as V$SESSION, V$SESSION_WAIT, and V$PROCESS to examine latch problems as they occur.

  • V$LATCH_MISSES. Details about latch misses.

  • V$LATCH_PARENT and V$LATCH_CHILDREN. Child and parent details of latches.

20.1.3 Latches in Real Time

Now let's take a look at my database containing the active Accounts schema in real time. I use an anonymous PL/SQL procedure to poll latches in the database many times at the session level, as those latches occur, in order to get a useful result over a period of time.

Tip

This type of processing can be done easily with Statspack and even more easily with the Oracle Database Wait Event Interface. The purpose here is to explain the details.

DROP TABLE statistics_latches;
CREATE TABLE statistics_latches(event varchar2(64)
      ,latch varchar2(64), address NUMBER, latch# NUMBER
      ,sleeps NUMBER);
BEGIN
 FOR counter IN 1..100000 LOOP
  INSERT INTO statistics_latches
  -
  --P1 is the latch address, P2 is the latch number, P3 is
    the number
  --of sleeps
  --
  SELECT sw.event AS event, l.name AS latch, sw.p1 AS address,
    sw.p2 AS latch#, sw.p3 AS sleep
  FROM v$session_wait sw, v$latchname l
  WHERE sw.event = 'latch free' AND sw.p2 = l.latch#;
  COMMIT;
 END LOOP;
END;
/

Now running a query against the table created we can see that the database buffer cache and the library cache have potential latch free wait problems, amongst other potential latch performance possibilities.

SELECT latch "Latch", COUNT(latch) "Latch Waits",
   SUM(sleeps) "Sleeps"
FROM statistics_latches GROUP BY latch;

Latch                       Latch Waits   Sleep
-------------------------   -----------   -----
cache buffers chains                 63       0
cache buffers lru chain              26       7
dml lock allocation                   1       0
enqueue hash chains                  11       0
library cache                         1       0
multiblock read objects               2       0
redo allocation                       1       0
redo copy                             6       0

Now we know a little about latches and potential latch problems in the Accounts schema. Let's examine what the most significant latches are in general for an Oracle installation, indicating potential latch problems and resulting performance issues.

I could make changes to configuration similar to those made in the previous chapter, attempting to both increase and decrease the occurrence of particular latches. At this point these actions are not necessary.


20.2 The Most Significant Latches

The most significant latches occur with respect to use of the most highly used memory buffer structures in the shared global area (SGA). The SGA includes the database buffers and the shared pool. The shared pool contains various subset buffer caches, structures such as the library cache, the metadata cache, and parsed SQL code.

Tip

Concurrency requirements for the shared pool are much higher than for the database buffer cache. The database buffer cache is simply a speed bridge between datafiles and data access. The shared pool attempts to minimize processing repetition, which can be substantial. Missing a latch in the shared pool is a much more significant event than missing a latch in the database buffer cache. The concept of sharing is much more important with respect to the shared pool.

As already stated a latch is similar to a lock on a row in a table except that a latch is used to allocate a hold on a section of a buffer in memory. What are the different types of latches and to which buffers do they apply?

  • Database Buffer Cache Latches. Cache buffers lru chain, cache buffers chains, cache buffer handles, multiblock read objects, cache protection latch.

  • Shared Pool, Library, and MetaData Cache Latches. Shared pool, sequence cache, library cache, library cache pin, library cache pin allocation, library cache load lock, row cache objects

  • Redo Log Buffer Latches. Redo allocation, redo copy, redo writing.

  • Network and Database Connection Latches. Virtual circuit buffers, virtual circuit queues, virtual circuits, session allocation, session switching, session idle bit, session queue latch.

20.2.1 The Database Buffer Cache

Two latches are significant in the database buffer cache.

  • Cache Buffers lru Chain. This latch is acquired for one of two reasons. Firstly, when copying a new block into the cache: the block must be locked in cache to prevent any changes to it during the I/O process from disk to buffer. Secondly, when writing a dirty block to disk: the block cannot be changed or read into the cache again until I/O from buffer to disk is complete. Excessive waits on this latch could be a result of swapping between cache and disk in order to cope with constant requirements for different blocks.

    Potential culprits:

    • Excessive I/O activity from full scans.

    • The database buffer cache could be too small.

    • Inappropriate, superfluous, or highly degraded index structures.

    • Poorly tuned SQL code.

    Potential solutions:

    • Reduce full scanning by tuning SQL code properly.

    • Reduce the DB_FILE_MULTIBLOCK_READ_COUNT parameter to discourage full scans.

    • Create multiple cache sizes and tablespaces (separate buffer pools in Oracle8i Database) allowing separation in two ways: use the ALTER TABLE table [NO]CACHE command or the KEEP and RECYCLE pools.

      • Separate highly active and less used data.

      • Separate full scans from single-block reads (SQL code dealing with small numbers of rows).

    • Increase database buffer cache size. This is always the easy option but rarely the correct one.

    • More database writer processes.

    Tip

    A parameter called DB_BLOCK_LRU_LATCHES was used in the past. This parameter is now automated and hidden in Oracle9i Database and beyond, probably with good reason. Changing a parameter such as this is an attempt to tune a latch. A latch wait is the result of a performance problem, and not the problem itself. Thus tuning a latch is not a solution.

In OLTP databases high I/O is usually caused by poorly tuned SQL code, often causing excessive full physical scans. Full table scans and overuse of fast full index scans will cause heavy I/O activity, both of which read information directly from disk and bypass the database buffer cache altogether.

Tip

Data is read directly from disk not the cache for full physical scans. The database buffer cache is not read but bypassed, even though data read from the disk is loaded into the database buffer cache.

Any required blocks already in the buffer exacerbate the problem because they are read from disk and copied into the buffer even though they already exist in the buffer; latches are required anyway, increasing block contention because the same data block is read. Poorly tuned SQL code often results in repeated full table scanning of the same table, continuously read from disk, copied into the database buffer cache, but never read from the database buffer cache itself. Just imagine! The database buffer cache entries exist but are effectively never reused. In an extreme situation performing the same full table scans over and over again makes the database buffer cache more or less useless if all query plans only ever use full table scans. This is a very unlikely extreme but the database buffer cache might as well be set to zero.

Tip

Once again full physical scans such as full table scans and fast full index scans read directly from disk and bypass the database buffer cache during the read process. The data is still loaded into the database buffer cache but it simply does not have the option of being read from the cache. This can have the additional undesirable effect of removing frequently used blocks from cache.

  • Caches Buffers Chains. This latch is acquired when a block is accessed and pinned. A block is pinned because something is changing it. This latch prevents another process from changing the block at the same time, essential to data integrity. A lot of DML activity with very high I/O rates in SQL code can cause waits on this latch to occur too often. Tune the SQL code. Otherwise this latch is commonly caused by what are called hot blocks. A hot block is a block in the buffer cache subject to change by too many sessions simultaneously. Hot block issues are often a consequence of data model design flaws. Some types of data model culprits are as follows:

    • Static or semi-static lookup data tables subject to change. These types of updates should be done in batch mode and avoided during times of peak activity. Summary column updates in parent tables are common causes.

    • System reference tables perhaps similar in function to Oracle Database sequences but storing centrally located information for all data. The Oracle Database configuration parameter file is a perfect example of a system reference file except it is stored in memory and not in a database table. Many applications use tables of this nature. Updates by applications to tables of this nature can cause serious hot block issues. These types of tables were common in single-user databases in the last century and are completely incompatible with any type of multi-user database engine.

    • Primary keys using sequence generators as index values where index leaf blocks are accessed at high concurrency rates. Reverse primary key indexes can help on tables with very high insertion rates, attempting to store consecutive sequence values into the same index leaf block. Appended log or audit trail tables with sequence number generated primary key indexes are typical culprits.

    What can be done about hot blocks?

    • Proper data model and database object design is the best approach but can be expensive.

    • Block concurrency levels can be raised using the INITRANS parameter, amongst other methods. Block structural changes are covered in Chapter 14.

    • Use reverse key indexes on sequential primary keys using sequences for tables with extremely high concurrent insertion rates.

      Tip

      Changing block structure will only affect new blocks not already-created blocks. Changing existing block storage structure requires table, tablespace, or even database recreation. Using automated segment space management can help with block-level concurrency issues. Setting values for PCTUSED, FREELIST, and FREELIST_GROUPS in database objects such as tables will be ignored if the containing tablespace is set to automatic segment space management. One of the reviewers for this book commented that automatic segment space management realizes up to 35% performance improvement, specifically in Oracle RAC environments.

Hot blocks can be caused by excessively high I/O but are more likely the result of data model design flaws or indexing issues.

20.2.2 The Shared Pool

Shared pool latches can be divided into three areas: the shared pool, the library cache, and the metadata or dictionary cache.

  • Shared Pool. This latch covers the shared pool as a whole, including library and metadata caches, and usually occurs when the shared pool buffer is sized too large. It can occur when the shared pool is too small. A very large shared pool requires excessive maintenance using long freelists.

  • Sequence Cache. Waits on this latch implies that cached sequence values are experiencing lack of buffer space. There is very high sequence usage.

Library Cache Latches

Library cache latches generally involve SQL code problems.

Tip

Tune SQL code properly. Properly tuned SQL code will alleviate and resolve so many other problems. Proper SQL code tuning simply cannot be ignored or avoided.

  • Library Cache. This latch is acquired to add and parse a new SQL code statement into the library cache, for future sharing. If an application requires frequent re-parsing then it is possible that SQL code is not matching existing SQL code statements already in the library cache. This can be as a result of lack of bind variable use or lack of consistency of SQL coding. SQL code must match exactly for reuse, down to the case of characters and even the number of spaces between an equals sign and column values in a WHERE clause join. Formatting should be standardized because it has to be precise.

    Tip

    Setting the CURSOR_SHARING parameter to FORCE or SIMILAR can help decrease re-parsing. However, the accuracy of the Optimizer in relation to statistics might be decreased in large databases.
    Note

    Oracle Database 10 Grid Case sensitivity of SQL code is less important to performance due to Optimizer improvements.

  • Library Cache Load Lock. This latch is a library cache or database object lock catering for concurrency.

  • Library Cache Pin. A pin is acquired after a library cache load lock latch is acquired. A wait on this latch occurs when an SQL code statement already in the library cache is executed again. A re-executed SQL statement implies that an SQL statement is re-parsed because it is not matched. Once again the cause is probably no bind variable use and inconsistent SQL coding standards.

Once again, to reiterate, the biggest cause of latch problems with the library cache, perhaps the shared pool and even an entire Oracle installation in general, is often the result of poorly coded and tuned SQL code. There are various methods of resolving SQL code issues:

  • Increasing shared pool size can help to reduce re-parsing of SQL code. On the contrary, excessive shared pool size can cause other problems.

  • Bind variables, bind variables, bind variables, and then bind variables again! The CURSOR_SHARING parameter can be set to SIMILAR or FORCE. There are possible side effects using the CURSOR_SHARING parameter as well. Let's make sure we understand exactly what bind variable use is. This SQL statement uses a bind variable:

    SELECT * FROM customer WHERE name = :name;

    This SQL statement does not use a bind variable but a literal value:

    SELECT * FROM customer WHERE name = 'Joe';

    This SQL statement will be re-parsed because the string 'Joe' is not the same as the string 'Jim':

    SELECT * FROM customer WHERE name = 'Jim';

If both of the SQL code statements finding Joe and Jim were submitted to the database using a bind variable, as in the first example, then the SQL code SELECT statement would have been parsed once. This seems trivial? Imagine a scenario to get an idea of scale, exemplifying sharing and the shared use of the shared pool library cache, from a realistic perspective.

Let's say you have 10,000 users all executing a statement like this, at exactly the same time:

SELECT * FROM customer where name = :name;

All 10,000 executions will not be re-parsed because a bind variable is used. Now imagine replacing the bind variable with every different one of the 10,000 customer's names. You have 10,000 re-parses and thus the shared pool might need to be 10,000 times bigger. Now imagine a full application with many tables, indexes, a multitude of SQL statements using complex joins, and so on. You can imagine what happens to the shared pool.

Tip

Data warehouse databases are different where queries and amounts of data are very large. It is sometimes prudent not to use bind variables. Why? Because using bind variables tends to lower Optimizer accuracy in relation to statistics. Statistics are very important when querying very large data sets. Once again setting the CURSOR_SHARING parameter to SIMILAR or FORCE will cause the Optimizer to lose query plan accuracy due to less precise use of statistics.

It is so very important to create properly tuned SQL code from the start.

Metadata Cache Latches

Latch waits on the row cache objects latch implies that there is contention for cache loaded data dictionary metadata definitions. In other words, logical objects such as table or index definitions are not fitting into cache. This type of situation is usually a result of design issues, for instance caused by applications where each end user has a separate schema or perhaps applications with multiple layers of views. Both of these factors are often utilized for ease of applications programming but can be most unfortunate for performance if scalability becomes an important requirement. Most successful businesses require scalability if growth is the intention.

20.2.3 The Redo Log Buffer

The redo log buffer uses three latches.

  • Redo Allocation. Acquired to allow for space in the buffer prior to writing to the buffer.

  • Redo Copy. Acquired to make actual buffer entries. Latch free waits on the redo copy latch are usually high since multiple latches are checked before spinning. Additionally parallel CPU platforms allow for multiple redo copy latches and thus parallel copies into the redo log buffer.

  • Redo Writing. Writing from the log buffer to a redo log file verifying that the log writer process is inactive before allowing the write to occur.

20.2.4 Network and Database Connection Latches

Most of these types of latches are trivial. Sometimes virtual circuit latch problems can indicate issues with shared server and dispatcher configuration parameters and perhaps even shared or large pool configuration issues.

That is enough information about latches. Oracle Database has well over 200 different latches.

Note

Oracle Database 10 Grid Oracle9i Release 2 (9.2) has 239 latches. Oracle Database 10g 316 latches.

In any particular environment such as an OLTP or data warehouse database, different functionality and behavior is likely to make different latch problems more likely. The next chapter will briefly examine various tools used for physical and configuration tuning of Oracle installations.


Chapter 21: Tools and Utilities

There are a multitude of tools available to both monitor and tune Oracle installations. This chapter will simply introduce some of those tools and categorize what each can do. It is best to explain some of the various tools graphically. Let's begin with Oracle Enterprise Manager. In Chapter 10 of Part II we looked at the SQL code tuning aspects of Oracle Enterprise Manager; this chapter attempts to focus on the physical tuning aspects.

21.1 Oracle Enterprise Manager

Oracle Enterprise Manager in relation to physical tuning can be divided into two parts. These two parts contain diagnostic and tuning tools. Diagnosis usually implies monitoring for potential problems and sometimes repairing them; tuning and analysis can even automatically repair possible performance problems.

21.1.1 Diagnostics Pack

With respect to physical tuning the interesting parts of the diagnostics pack are as follows:

  • Event Monitoring. Allows definition, automated detection of and potential automated "FixIt" jobs, which can automatically correct problems.

  • Lock Monitoring. Allows monitoring and resolution of locks in a database.

  • TopSessions. Detects and displays the top best- or worst-performing sessions ordered by a large number of different possible sorting parameters.

  • TopSQL. Detects and displays the top best- and worst-performing SQL statements, again ordered by a large number of different possible sorting parameters.

  • Performance Manager. Performance Manager wraps everything together such as details contained in the TopSessions and TopSQL tools, with extra functionality and GUI usability.

    Tip

    The ultimate of Oracle Enterprise Manager is a GUI window interface into the Oracle Database Wait Event Interface.

Event Monitoring

Event monitoring covers areas such as performance, resource usage, how space is used, amongst many other areas. There are an absolute of plethora of events which can be included for automated detection and potential "FixIt" action to resolve problems. Figure 21.1 shows a very small subsection of the fully user definable event detection tool. The event detection construction tool is accessible from the Event menu in the Oracle Enterprise Manager Console.

Click To expand
Figure 21.1: Defining Events for Detection and Resolution
Note

Oracle Database 10 Grid Oracle Enterprise Manager repository, events and jobs are automatically configured.

Lock Monitoring

Figure 21.2 shows lists of locks on a very busy database. Note the drilldown menu with various options including the option to kill off a session. Sometimes it is necessary to kill a session causing potentially damaging problems.

Click To expand
Figure 21.2: Monitoring and Managing Locking

TopSessions

The TopSessions tool allows ordered visualization of statistical performance information where the best or worst sessions can be displayed first. Figure 21.3 shows the first 10 sessions executing the most physical reads.

Click To expand
Figure 21.3: TopSessions Monitoring in a Specified Order

TopSQL

TopSQL is similar to the TopSessions tool where the TopSQL tool detects and displays the best- or worst-performing SQL statements in a large number of different possible orders. An example of TopSQL is shown in Figure 21.4.

Click To expand
Figure 21.4: TopSQL Statements
Note

Oracle Database 10 Grid TopSQL is much improved in relation to more usable reporting capabilities.

Performance Manager

The Performance Manager is a very comprehensive tool. It allows an intensive GUI picture of an environment with every possible monitoring and performance metric imaginable. The Performance Manager appears complex but it is comprehensive and very easily usable.

Figure 21.5 shows the Performance Overview screen. This screen is accessible from the Oracle Enterprise Manager Console on the Tools menu. Select the Diagnostic Pack submenu. The Performance Overview interface gives an overall picture of database performance and is highly adaptable depending on requirements and what you want to monitor.

Click To expand
Figure 21.5: The Performance Overview Tool
Note

Oracle Database 10 Grid Performance Overview charts are much improved allowing monitoring and diagnostics checking from a central point using an HTML interface.

The Performance Manager allows simple drill-down further into the separate subset areas of those shown in Performance Overview interface, to the point and beyond of drilling down into the Oracle Database Wait Event Interface. The Oracle Database Wait Event Interface will be covered in detail in the next chapter. Figure 21.6 shows a small part of the possible types of information which can be monitored with the Performance Manager.

Click To expand
Figure 21.6: The Performance Manager Main Screen

Figure 21.7 shows a drilldown into latch get/miss rates.

Click To expand
Figure 21.7: Latch Get/Miss Rates Drilldown

Finally Figure 21.8 shows a detailed analysis of latches.

Click To expand
Figure 21.8: Latch Analysis

21.1.2 Tuning Pack

As far as physical tuning is concerned only the Tablespace Map and Reorg Wizard tools are of interest. Other parts of the Oracle Enterprise Manager tuning pack were covered in Chapter 10 of Part II of this book.

Tablespace Map and the Reorg Wizard

The tablespace map presents a graphical representation of physical data distribution within tablespaces. After executing a tablespace analysis in the INDX tablespace of my database, two indexes are recommended for reorganization, as shown in Figure 21.9.

Click To expand
Figure 21.9: After Execution of Tablespace Analysis

As can be seen in the drop-down menu in Figure 21.9 various options are available:

  • Tablespace Analysis. Analyze a tablespace for problems such as fragmentation.

  • Reorganize Tablespace. Execute the Reorg Wizard.

  • Reorganize Selected Segments. Execute reorganization on selected indexes (segments) only.

  • Coalesce Free Extents. Coalescence attempts to merge empty adjacent physical areas into single reusable chunks of disk storage space. In the past, I have not found coalescence to be particularly useful.

Figure 21.10 simply shows the same screen as in Figure 21.9 except with the Tablespace Analysis Report tab selected. The table- space analysis report describes exactly why the table analysis process considers these indexes the cause for reorganization, namely fragmentation.

Click To expand
Figure 21.10: What Should be Reorganized and Why

The result of execution of the Reorg Wizard and tablespace coalescence on the INDX tablespace is shown in Figure 21.11. The actual reorganization process is created as an instantly executed or future scheduled job, to be executed inside the job control scheduling system, visible in the Oracle Enterprise Manager Console job scheduling screen.

Click To expand
Figure 21.11: After Reorganization and Coalescence on the INDX Tablespace

That's enough about Oracle Enterprise Manager. The best use of Oracle Enterprise Manager for physical and configuration tuning is diagnosis and analysis of potential bottlenecks using the GUI drill- down access paths into the Oracle Database Wait Event Interface. The Oracle Database Wait Event Interface will be covered in detail in the next chapter. There are many other non-Oracle Corporation tools for monitoring and tuning Oracle databases. Spotlight is one of these tools.


21.2 Spotlight

Spotlight on Oracle is an excellent tool for real-time monitoring and reactive, not predictive, tuning of busy production databases.

With respect to physical tuning Spotlight can do much and its GUI is of excellent caliber. Spotlight can be used for SQL code tuning as well. However, it is best used as a monitoring and physical tuning advisory tool.

Figure 21.12 shows the primary screen in Spotlight on Oracle, used for monitoring a busy Oracle production database. Different colors represent different levels of alert: green being OK, red being serious, or just panic!

Click To expand
Figure 21.12: Spotlight on Oracle Main Monitoring Screen

There are an enormous number of subset screens in Spotlight on Oracle, for every aspect of Oracle Database tuning imaginable. Figure 21.13 shows a screen displaying real time usage of the Shared Global Area (SGA) for a database.

Click To expand
Figure 21.13: Spotlight SGA View

Figure 21.14 shows a mixture of various views from Spotlight with both textual and instantly recognizable graphical displays.

Click To expand
Figure 21.14: Various Spotlight Views

Spotlight is a very comprehensive tuning and monitoring tool for Oracle Database, especially in the area of real time production database monitoring. As already stated many non-Oracle Corporation produced Oracle database monitoring and tuning tools are available. Spotlight is one of the most comprehensive and useful of those tools I have used in relation to its price.


21.3 Operating System Tools

21.3.1 Windows Performance Monitor

The Windows Performance Monitor is useful for monitoring operating system-level performance. The GUI can be started up on a Windows 2K server in the Administrative Tools icon on the Control Panel. This tool allows graphical display of all aspects of hardware and operating system-level activity. Figure 21.15 shows a picture of a dual-CPU server displaying both processors, memory, and network activity for a relatively idle database server.

Click To expand
Figure 21.15: The Windows Performance Monitor

Figure 21.16 on the other hand shows the Windows Performance Monitor GUI for a highly active single-CPU system. The Windows Task Manager snapshot is added to show how busy this system was when these screenshots were taken.

Click To expand
Figure 21.16: This is a Very Busy Server

There are a multitude of command line and GUI tools used for operating system monitoring and tuning in other operating systems such as Unix or Linux.

21.3.2 Unix Utilities

There are a plethora of tools used in all flavors of Unix and Linux operating systems, both command line utilities and GUI-based monitoring tools. These tools are some of the command line monitoring tools used on Solaris.

  • CPU Usage. sar, vmstat, mpstat, iostat.

  • Disk I/O Activity. sar, iostat.

  • Memory Usage. sar, vmstat.

  • Network Usage. netstat.

Since I am not an experienced systems administrator I see no point in attempting to explain the detailed syntax and use of these tools.


21.4 Other Utilities and Tools

21.4.1 Import, Export, and SQL*Loader

The Export and SQL*Loader utilities can be executed in DIRECT mode. DIRECT mode implies that the SQL engine, executing INSERT statements, is completely bypassed and data is appended directly to datafiles on disk. DIRECT mode is much faster than passing through the SQL engine. Import and Export have a parameter called BUFFER. The BUFFER parameter limits chunks of rows processed. In very busy environments using the BUFFER parameter can limit the impact on other database services.

Tip

Do not use the Import and Export Utilities for Backup and Recovery. Generally only use Export to copy individual static tables you are absolutely sure will not change. Additionally using a previously FULL Export backup file to Import and recreate a lost database is incredibly inefficient. I have seen a 200 Gb database reconstruction using a FULL database Export take an entire weekend to complete. This particular database was apparently destroyed by a junior DBA executing a script to drop all users in cascade mode, by mistake of course. Oops!
Note

Oracle Database 10 Grid Data Pump Import and Data Pump Export have much better performance than the Import and Export utilities. Data Pump allows parallel execution, is self-tuning, and does not require compression into a single extent.

SQL*Loader is an excellent tool for loading data into a database. SQL*Loader is very easy to use and can map data files in text format directly into database tables. Use SQL*Loader as an alternative option to that of loading tables using large quantities of INSERT statements, it can give performance increases in the thousands. Use SQL*Loader as it is very easy to learn to use.

Note

Oracle Database 10 Grid SQL*Loader can now cache dates reducing conversions on duplicated date values.

21.4.2 Resource Management and Profiling

Resources can be managed and effectively shared in Oracle Database to create round-robin-type queues. Hardware resources such as CPU time can be evenly distributed across multiple profiles.

Profiles contain groups of users or functionality sets. Different activities can be given priority at different times of day. Perhaps OLTP activity can be favored during the daytime and backup or batch processing activities favored after-hours. This of course assumes that you do not have to maintain a constant level of 24 × 7 database availability. There is a small amount of resource overhead with respect to Oracle Database internal implementation and execution of resource management.

21.4.3 Recovery Manager (RMAN)

RMAN or Recovery Manager is a slightly more power-user-oriented backup utility when compared to using backup mode tablespace datafile copies. However, RMAN simplifies backup and recovery processing and DBA involvement. Additionally RMAN can be executed in parallel.

21.4.4 STATSPACK

People have written and published entire books about STATSPACK. STATSPACK is a comprehensive statistics monitoring and tuning analysis tool, the next generation of the UTLBSTAT.sql and UTLESTAT.sql tuning scripts. In addition STATSPACK can be used to store statistical information in the database in a special repository for later comparison and analysis between different statistics set collections. Therefore, when a performance problem occurs a current snapshot can be compared against a previously obtained baseline snapshot, allowing for easy comparison and thus rapid diagnosis of the problem.

Tip

STATSPACK is useful for analysis and detection of bottlenecks but using the Oracle Database Wait Event Interface and the Capacity Planner in Oracle Enterprise Manager is better. The Oracle Database Wait Event Interface will be covered in the next chapter, along with some use of STATSPACK.

STATSPACK is very easy to install, configure, and to use. The problem with it is that it produces enormous quantities of what could amount to superfluous information. Wading through all the mounds of information produced by STATSPACK could be somewhat daunting to the tuning novice or someone who is trying to solve a problem in a hurry. Large amounts of information can even sometimes hide small things from an expert. The expression "trying to find a needle in a haystack" comes to mind.

STATSPACK is more useful for general performance tuning and analysis as opposed to attempting to find specific bottlenecks. The Oracle Database Wait Event Interface in Oracle Enterprise Manager is much more capable than STATSPACK in rapidly isolating problems.

To use STATSPACK a specific tablespace must be created:

CREATE TABLESPACE perfstat DATAFILE
      'ORACLE_HOME/<SID>/perfstat01.dbf'
      SIZE 25M AUTOEXTEND ON NEXT 1M MAXSIZE UNLIMITED
      EXTENT MANAGEMENT LOCAL SEGMENT SPACE MANAGEMENT
         AUTO;

Do not create a user because the scripts will crash! Additionally you might want to set the ORACLE_SID variable if your database server has multiple databases. The following script will create STATSPACK goodies.

@ORACLE_HOME/rdbms/admin/spcreate.sql;

If the installation completely freaks out, the following script will drop everything created by SPCREATE.SQL so that you can start all over again.

@ORACLE_HOME/rdbms/admin/spdrop.sql;

Once installed take a snapshot of the database by executing these commands in SQL*Plus.

CONNECT perfstat/perfstat[@tnsname];
EXEC STATSPACK.SNAP;

The DBMS_JOBS package can be used to automate STATSPACK SNAP procedure executions on a periodical basis, as in the example script shown overleaf, which executes every 5 min. Different snapshot levels can be used between 0 and 10. 0 is not enough, 10 far too much, 5 is the default and 6 provides query execution plans. Since the large majority of performance problems are caused by poorly written SQL code I would recommend starting at snapshot level 6.

Snapshot levels are as follows:

  • 0: simple performance statistics.

  • 5: include SQL statements, which is the default level.

  • 6: include SQL plans.

  • 7: include segment statistics.

  • 10: include parent and child latches.

    Tip

    Running STATSPACK will affect performance so do not leave it running constantly.

DECLARE
 jobno NUMBER;
 i INTEGER DEFAULT 1;
BEGIN
 DBMS_JOB.SUBMIT(jobno,' STATSPACK.SNAP(I_SNAP_LEVEL=>6);'
       ,SYSDATE,'SYSDATE+1/288');
COMMIT;
END;
/

Remove all jobs using an anonymous procedure such as this.

DECLARE
      CURSOR cJobs IS SELECT job FROM user_jobs;
BEGIN
      FOR rJob IN cJobs LOOP
            DBMS_JOB.REMOVE(rJob.job);
      END LOOP;
END;
/
COMMIT;

Run a STATSPACK report using the following script. The script will prompt for two snapshots to execute between, comparing sets of statistics with each other.

@ORACLE_HOME/rdbms/admin/spreport.sql;

An SQL report can be executed on one SQL statement, searching for a bottleneck, where the hash value for the SQL code statement is found in the STATSPACK instance report.

@ORACLE_HOME/rdbms/admin/sprepsql.sql;

There are a number of other "SP*.SQL" STATSPACK scripts used for various functions. Two of those scripts are SPPURGE.SQL and SPTRUNCATE.SQL. Purging allows removal of a range of snapshots. If the database is bounced then snapshots cannot be taken across the database restart. Truncate simply removes all STATSPACK data.

STATSPACK can also have threshold value settings such that characteristics below threshold values will be ignored. Default STATSPACK parameter settings including threshold values are stored in the STATS$STATSPACK_PARAMETER table.

This is enough information about physical tuning tools and use of utilities. The next chapter will look at physical and configuration tuning from the perspective of finding bottlenecks using the Oracle Database Wait Event Interface.


Chapter 22: Tuning with the Wait Event Interface and STATSPACK

Overview

Thus far in Part III we have looked at physical and configuration tuning in a number of ways. Firstly, we looked at initial installation and database creation. Secondly, we examined how to create a proper physical structure. Thirdly, we examined ratios, wait events and latches, with some very brief example drilling down into the Oracle Database Wait Event Interface. Finally, in the previous chapter we briefly discussed some of the tools which can be used to aid in physical and configuration tuning, including what each tool can be used for. In this chapter we will dig a little deeper into some very specific performance views containing database object facts and statistics. These details can be utilized to attempt to find what are commonly termed bottlenecks. After we have described a small portion of what are an extremely confusing plethora of performance views we will take the easy route and use Oracle Enterprise Manager to drill down into the Oracle Database Wait Event Interface. As a final step an example report using the performance monitoring and tuning tool called STATSPACK will be examined.


22.1 What is a Bottleneck?

A bottleneck is quite literally an area of part of the hardware or software architecture where too much processing is being funneled through a specific area. Much more is being done in one specific place than anywhere else.

What are the causes of bottlenecks? Causes are numerous. More often than not, bottlenecks are a result of poorly designed SQL code or a poorly structured data model. Rarely is there a hardware issue. One of the most commonly occurring hardware performance issues is inappropriate disk structure for redo logs and archive logs. Hardware issues are generally beyond the scope of this book and fall under the guise of systems administration. This book is all about tuning of Oracle databases and not about tuning at the operating system level. Therefore, any hardware or operating system aspects are not particularly relevant to this text.


22.2 Detecting Potential Bottlenecks

So what can be done to detect potential bottlenecks? If you have read this entire book by now you probably realize that there are multiple methods which can be used to tune Oracle installations. I tend to think of tuning methods as being approached based on the skills set of database administrators. Database administrators generally come from two main schools of thought. The first is development and the second is systems administration, systems administration being by far the more prominent. Developers will take a top-down approach to tuning and systems administrators will take a bottom-up approach to tuning. What does this mean? Developers will approach tuning from the application, leading down into the database. Systems administrators will approach a problem from the operating system. Therefore, developers are more likely to move directly into the data model and particularly application SQL code. Systems administrators will more than likely look at the hardware and the operating system, leading up into the database. The result is that developers are more likely to tune from the perspective of the first two parts in this book while systems administrators will veer towards the third part of this book. This leads us to the purpose of this chapter, which is to explain how to find performance problems by using Oracle Database internal structures, namely database statistics in the Oracle Database Wait Event Interface, using performance views, Oracle Enterprise Manager and STATSPACK.

Part II of this book gradually built up to tuning of SQL code, eventually describing the "how" and "what with" of tuning SQL code. Part III has presented various fundamentals of physical and configuration tuning as already mentioned. This final chapter of tuning in Part III is thus a method of tuning most likely to be used by systems administrators but very useful for everyone. Therefore, there has to be some complexity to promote understanding followed by easy solutions. This chapter as a result covers what is often called the Oracle Database Wait Event Interface, or quite simply put, a bunch of performance views allowing drilling into wait events and all sorts of statistics, allowing for detection of potential bottlenecks. The objective is obviously to tune the bottlenecks.

Therefore, using the Oracle Database Wait Event Interface is simply another tuning method or tool and can often provide database administrators with enough ammunition to convince developers where SQL code can be altered to drastically improve performance. Obviously there are numerous steps database administrators can take to improve performance without involving developers or having to persuade them to alter application code. However, there is a point where application-based SQL code simply must be changed if better performance is required.

The other aspect of using the Oracle Database Wait Event Interface for tuning Oracle installations is that it tends to be a reactive rather than a proactive tuning method. This means that wait event tuning is used to drill down into problems as they occur and is thus akin to putting out fires rather than preventing them. This is often the only available tuning method due to high time and cost limitations placed on the development process. Perhaps more planning and expense may appear to be required if tuning is done proactively. Proactive tuning means writing better SQL code and designing the database model for performance in the first place. However, this is often not possible. Therefore, reactive tuning is usually the only available approach.


22.3 What is the Wait Event Interface?

The Wait Event Interface is a group of Oracle Database performance views allowing drilling down into various areas of activity occurring in an Oracle Database installation. These activities are either summarized from the most recent database startup or maintained for the life of each session. The simplest way to present those views and their inter-related structure is to present them in steps. It should also be noted that drilling into wait events includes waits not only on events but deeper layers such as latches and others.

Note

Oracle Database 10 Grid Numerous Wait Event Model changes are made including changes to various V$ performance views along with a number of additional V$ performance views.

Firstly, Figure 22.1 shows a brief picture of various tools which can be used to drill down into wait events. These tools can be used to allow easy access to underlying Oracle Database performance views. In this book, I want to start by explaining the lowest level of application of the wait event interface using the performance views. Simply displaying how to use various tools or describing their output does not help to provide better understanding. When it comes to examples Oracle Enterprise Manager will be used to drill down into wait events as they occur. STATSPACK will be used to compare two different points in time for a database.

Click To expand
Figure 22.1: Tools for Drilling into the Oracle Database Wait Event Interface

I like to divide the Wait Event Interface performance views into three sections: the system aggregation layer, the session layer followed by the third layer, and beyond. The first I call the "system aggregation" layer because in its simplest form it contains a record of statistics collected by what I like to call the "session layer". The session layer contains statistics about sessions during the life of those sessions. There is a deeper layer of the Wait Event Interface involving hooks through to many other performance views. I like to call this deeper layer the "crazy about V$ views" layer or "the third layer and beyond". This third layer will not be examined in detail but described briefly and then displayed by example using Oracle Enterprise Manager. There is too much complexity in the third layer to go into every detail. It is not worth explaining every nitty-gritty piece, possibly boring my readers to tears.

22.3.1 The System Aggregation Layer

A brief overview of the system aggregation layer of the Wait Event Interface is shown in Figure 22.2. The V$EVENT_NAME view gives a list of all possible events. The V$SYSTEM_EVENT view contains an aggregation of existing wait events and total wait times for each. V$SYSTEM_EVENT view is cleared on database restart.

Click To expand
Figure 22.2: The System Aggregation Layer of the Oracle Database Wait Event Interface
Tip

The TIMED_STATISTICS parameter must be set to TRUE for V$SYSTEM_EVENT to contain information. Setting the TIMED_STATISTICS parameter has minimum performance impact.

The V$SYSTEM_EVENT view can be joined to the V$EVENT_ NAME view using the EVENT and NAME columns, respectively, as shown in Figure 22.2.

The V$EVENT_NAME view has information about hooks to drill down into the third layer, mentioned above, of potential bottleneck problems using the parameter columns: PARAMETER1, PARAMETER2, and PARAMETER3. These parameter columns contain information as to what event parameters in third-layer views such as the V$LATCH view will be pointed to from the V$SESSION_WAIT view.

Tip

V$EVENT_NAME view parameter columns do not contain actual values, only descriptions of what values are. The parameter values are contained in the session layer in the V$SESSION_WAIT performance view.

Some of the values in the V$EVENT_NAME view parameter columns are as shown. There are a lot of them.

SELECT DISTINCT(parameter1) FROM v$event_name
      WHERE parameter1 LIKE '%#%';

PARAMETER1
--------------------------------------------
block#
branch#
buffer#
by thread#
circuit#
copy latch #
end-point#
event #
file#
log#
segment#
session#
thread#
undo seg#|slot#
undo segment#

SELECT DISTINCT(parameter2) FROM v$event_name
      WHERE parameter2 LIKE '%#%';

PARAMETER2
--------------------------------------------
#bytes
block#
chain#
our thread#
process#
wrap#

SELECT DISTINCT(parameter3) FROM v$event_name
      WHERE parameter3 LIKE '%#%';

PARAMETER3
--------------------------------------------
process#
set-id#

For instance, the combination of file# in PARAMETER1 and block# in PARAMETER2 can be used to drill down into extents to find an offending segment, a table in others words. The diagram shown in Figure 22.2 is appropriately adjusted in Figure 22.3.

Click To expand
Figure 22.3: Isolating Segments using Event Parameters

This query shows how to join the various views for existing wait events and will not be executed in this chapter. A demonstration later on using Oracle Enterprise Manager will suffice.

SELECT de.segment_name, sw.event
FROM dba_extents de, (
      SELECT sw.event, sw.p1 AS file_id, sw.p2 AS
        block_id
      FROM v$session_wait sw WHERE IdleEvent(event) IS NOT
        NULL
      AND sw.sid IN
            (SELECT sid FROM v$session WHERE username =
              'ACCOUNTS')
      AND sw.event != 'null event') sw
WHERE de.owner = 'ACCOUNTS'
AND de.file_id = sw.file_id AND de.block_id =
  sw.block_id;

Examining the DBA_EXTENTS view in detail from this perspective we can see the different segment types.

SELECT DISTINCT(segment_type) FROM dba_extents;

SEGMENT_TYPE
--------------------------------------------
CACHE
CLUSTER
INDEX
INDEX PARTITION
LOBINDEX
LOBSEGMENT
NESTED TABLE
ROLLBACK
TABLE
TABLE PARTITION
TYPE2 UNDO

From another perspective we can see table and index names.

SELECT DISTINCT(segment_name) FROM dba_extents
WHERE segment_type IN ('TABLE','INDEX') AND
   owner = 'ACCOUNTS';

SEGMENT_NAME
--------------------------------------------
AK_GL_DTE
AK_SP_ZIP
AUDITSIM
CASHBOOK
CASHBOOKLINE
CATEGORY
COA
CUSTOMER
GENERALLEDGER
ORDERS
ORDERSLINE
PERIOD
PERIODSUM
PLAN_TABLE
POSTING
…

So we should be able to see from the various queries already shown that potential wait event issues can be isolated to specific objects in the database. Let's look at some details. Firstly, let's list the top currently occurring wait events. Throughout this final chapter of Part III on physical performance tuning, I use queries used in previous chapters or slightly modified versions thereof, only in this chapter with a more specific objective in mind. Previous chapters in Part III mostly describe architecture and configuration. This chapter attempts to find and solve existing problems.

COL event FORMAT a32;
COL percentage FORMAT 9999999990;
SELECT * FROM(
SELECT event "Event", total_waits "Waits", time_waited
   "Total Time"
         ,TO_CHAR(
               (time_waited /
                     (SELECT SUM(time_waited) FROM
                        v$system_event
                     WHERE IdleEvent(event) IS NOT NULL)
         )*100, 990.99) "Percentage"
FROM v$system_event WHERE IdleEvent(event) IS NOT NULL
ORDER BY 3 DESC) WHERE ROWNUM <= 10;

These are my top 10 wait events based on the total amount of time the system spends waiting for each event. The only thing to decide at this point is what is a significant event. Which event in this list is the most significant and can have the most profound effect on performance? Some knowledge of applications might be required at this stage because high incidence of some wait events could be normal and thus acceptable, depending on applications.

Event                            Waits    Total Time   Percent
------------------------------   ------   ----------   -------
db file sequential read          623840   7542616        71.59
buffer busy waits                259455   2330819        22.12
enqueue                            1944    566381         5.38
latch free                         3996     49586         0.47
control file parallel write        4956     21016         0.20
db file parallel write             1102      7472         0.07
library cache pin                   112      6666         0.06
control file sequential read       2156      6371         0.06
log file parallel write            1559      1621         0.02
library cache load lock               8       714         0.01

The real problem with the preceding query is that the information obtained is a little too vague at this point. The db file sequential read event could indicate hot block issues but could also be normal. The same applies to buffer busy waits. The enqueue, latch free and wait events in the library cache could have more significant impact on performance.

22.3.2 The Session Layer

The session layer uses various performance views such as V$SESSION_ EVENT and V$SESSION_WAIT. These session views only contain information for a session as long as that session exists and are shown in Figure 22.4.

Click To expand
Figure 22.4: Session-Level Event and Wait Views
Tip

The V$SYSTEM_EVENT view contains an accumulated aggregation or summation of values in the V$SESSION_EVENT view, aggregated on termination of a session.

It is important to stress that using session-level performance views only finds wait events as they occur. If sessions are application- controlled or connections are shared then using session-level wait event detection may be awkward to decipher without deeper analysis.

Tip

Snapshots may have to be taken over specified time periods such as is possible using STATSPACK.

Since continual connecting and disconnecting, common to client– server systems, usually does not occur in OLTP database environments, sessions are often shared between connections in one form or another. The net result is that the specific nature of session-level wait event tracking can become somewhat blurred and possibly less useful, and perhaps even more akin to that of system-level event tracking. Session-level wait event detection may often be more appropriate to tracking down bad SQL code executed in ad hoc environments. Ad hoc SQL code is rare in mission critical OLTP-type databases. Most OLTP databases are controlled by applications.

The V$SESSION_EVENT performance view provides wait events for sessions for the entire life of the session. The V$SESSION_WAIT performance view provides the same information but as the wait events are occurring, also with access to deeper layers using parameter pointer values as shown in Figure 22.4. The V$SESSION_ EVENT view is effectively an accumulation of wait events for sessions during the life of those sessions. Thus V$SESSION_EVENT could be used to isolate sessions causing specific problems. Here are the worst 10 events by session.

COL event FORMAT a32;
SELECT * FROM (
SELECT sid, event, total_waits "Waits", time_waited "Total
   Time"
FROM v$session_event WHERE IdleEvent(event) IS NOT NULL
ORDER BY 4 DESC) WHERE ROWNUM <= 10;

SID                         EVENT   Waits   Total Time
---   ---------------------------   -----   ----------
 50       db file sequential read    8826        91390
  4       db file sequential read    3204        33929
 17                       enqueue      94        28399
 19                       enqueue      94        28398
 25                       enqueue      93        28255
 35       db file sequential read    6791        28112
 40             buffer busy waits    3857        21593
 45             buffer busy waits    3811        21338
  3   control file parallel write    2383        20054
 37       db file sequential read    1493        17461

This query shows events causing the highest wait times, more significant than those occurring most frequently. The most frequently occurring events would be found by summing the number of waits.

COL event FORMAT a32;
SELECT * FROM (
SELECT event, SUM(time_waited) "Total Time"
FROM v$session_event WHERE IdleEvent(event) IS NOT NULL
GROUP BY event ORDER BY 2 DESC) WHERE ROWNUM <= 10;

EVENT                          Total Time
----------------------------   ----------
db file sequential read            489236
enqueue                            360328
buffer busy waits                  292884
control file parallel write         20653
db file parallel write              10501
control file sequential read         6169
latch free 3959
log file parallel write              2306
LGWR wait for redo copy               165
log file sequential read               52

Examining the V$SESSION_WAIT performance view will not only show wait events as they occur but allow us to use parameter value pointers to find specific details such as database objects as shown in the DBA_EXTENTS view query previously in this chapter. An accumulation on the V$SESSION_WAIT view is useful to get a better overall picture as shown by using the anonymous procedure following.

Tip

This type of cumulative processing can be performed easily using a tool such as STATSPACK. At this stage the objective is to explain and teach the basics rather than the syntax of a tool such as STATSPACK.

In this procedure let's try to find offending segments by using the file# and block# pointers as found in the V$EVENT_NAME view.

DROP TABLE tmp;
CREATE TABLE tmp
 (sid NUMBER, event VARCHAR2(64), file_id NUMBER, block_id
   NUMBER,
 wait_time NUMBER, seconds_in_wait NUMBER);
BEGIN
 FOR counter IN 1..1000 LOOP
  INSERT INTO tmp
   SELECT sw.sid, sw.event AS event, sw.p1 AS file_id,
    sw.p2 AS block_id, sw.wait_time, sw.seconds_in_wait
   FROM v$session_wait sw WHERE IdleEvent(sw.event) IS NOT
     NULL
   AND sw.event IN (SELECT name FROM v$event_name
   WHERE parameter1 = 'file#' AND parameter2 = 'block#');
   COMMIT;
END LOOP;
END;
/

COL event FORMAT a32;
SELECT * FROM (
SELECT sid, event, SUM(seconds_in_wait) "Current Wait Time"
FROM tmp
WHERE IdleEvent(event) IS NOT NULL
GROUP BY sid, event ORDER BY 2 DESC) WHERE ROWNUM <= 10;

EVENT                          Current Wait Time
----------------------------   -----------------
db file sequential read                     2670
buffer busy waits                            617
db file sequential read                      163
buffer busy waits                             43
control file sequential read                   0

Now let's take a step back and try to find the worst-performing sessions in the V$SESSION_EVENT view.

COL event FORMAT a32;
SELECT * FROM (
SELECT sid, SUM(time_waited) "Total Time"
FROM v$session_event WHERE IdleEvent(event) IS NOT NULL
GROUP BY sid ORDER BY 2 DESC) WHERE ROWNUM <= 10;

SID   Total Time
---   ----------
 50       124491
 17       54034
 19       54027
 25       52969
 40       52958
 45       52933
  4       41486
 35       40264
 41       34010
 29       33403

That query does not really tell me much. What is session identifier (SID) 50? We can find more information about SID 50 using the V$SESSION performance view. The real value of the session-level part of the Wait Event Interface, other than searching for offending database objects in the DBA_EXTENTS view, is the way in which the session identifier can be connected to sessions in the V$SESSION view and ultimately to other views including SQL code and Optimizer query execution plans. Figure 22.5 shows the drill-down path of session-level executed SQL code and query plans. One of the primary factors causing database performance problems is poorly built SQL code.

Click To expand
Figure 22.5: Hooking Wait Events to Sessions
Tip

Sometimes data model problems can be isolated as a result of finding poorly performing SQL code.

So we could find the usernames (schemas) of offending sessions by joining the V$SESSION_EVENT or V$SESSION_WAIT views with the V$SESSION performance view.

COL event FORMAT a32;
SELECT * FROM (
SELECT s.username, SUM(se.time_waited) "Total Time"
FROM v$session_event se, v$session s
WHERE IdleEvent(se.event) IS NOT NULL
GROUP BY s.username ORDER BY 2 DESC) WHERE ROWNUM <= 10;

USERNAME   Total Time
--------   ----------
ACCOUNTS     69680468
             24120162
SYS           5360036

Now let's add some of the various SQL code views as shown in Figure 22.6.

Click To expand
Figure 22.6: Find SQL Code for Sessions

The following query simply joins sessions and SQL code statements to find SQL code which is currently executing the most often.

SELECT * FROM (SELECT s.username, a.sql_text, a.executions
FROM v$session s, v$sqlarea a
WHERE s.sql_address = a.address AND s.sql_hash_value =
   a.hash_value
ORDER BY 3 DESC) WHERE ROWNUM <= 10;

USERNAME   SQL_TEXT
--------   -----------------------------------------------------
           SELECT to_number(to_char(SYSTIMESTAMP,'FF3'))+1
              from dual
           SELECT to_number(to_char(SYSTIMESTAMP,'FF3'))+1
              from dual
ACCOUNTS   INSERT into transactions values
              (transactions_seq.nextval,'S'
ACCOUNTS   INSERT into transactions values
              (transactions_seq.nextval,'S'
ACCOUNTS   INSERT into transactions values
              (transactions_seq.nextval,'S'
ACCOUNTS   INSERT into transactions values
              (transactions_seq.nextval,'S'
ACCOUNTS   INSERT into transactions values
              (transactions_seq.nextval,'S'
ACCOUNTS   SELECT min(order_id),max(order_id) from orders
ACCOUNTS   SELECT min(order_id),max(order_id) from orders
ACCOUNTS   SELECT min(order_id),max(order_id) from orders

Similar join queries can be used to access both entire SQL code statements from the V$SQLTEXT view and stored query execution plans from the V$SQL_PLAN view, joined using the ADDRESS and HASH_VALUE columns.

22.3.3 The Third Layer and Beyond

From examination of the system aggregation and session layers you should have noticed two general paths of drill down which can be followed. Firstly, drilling into the parameters allows access to details about events and why they are causing issues. Secondly, SQL code can be accessed directly using address and hash values. These two wait event drill down access paths can be joined together because the SQL views are linked to a session and a session is linked to session wait events in the V$SESSION_WAIT performance view.

Let's take a look at a specific example of drilling down into the third layer before we go on to showing how Oracle Enterprise Manager does all this stuff! Let's take a look at latch free wait events. Figure 22.7 shows a pseudo-type structure for latch V$ performance views. There are numerous views but we will simply look at the topmost part of the third layer, for latches only.

Click To expand
Figure 22.7: Drilling Down into Latches

For latch free wait events the LATCH# column on the V$LATCHNAME view is contained in the PARAMETER2 column of the V$SESSION_WAIT view, allowing direct or indirect drill down access into all the various latch performance views. Let's take a look at current latch wait problems in order to demonstrate parameter level drill down into the third layer of the Oracle Database Wait Event Interface.

First check the parameter values.

SELECT parameter1||' '||parameter2||' '||parameter3
FROM v$event_name WHERE name = 'latch free';

PARAMETER1||''||PARAMETER2||''||PARAMETER3
--------------------------------------------
address number tries

PARAMETER2 contains a latch number, the equivalent of the LATCH# column in the V$LATCH performance view. The following query will not really tell us much but it joins the V$LATCH and V$SESSION_WAIT performance views.

SELECT sw.sid, sw.event, sw.wait_time, sw.seconds_in_wait
      ,l.name, l.gets, l.misses, l.sleeps
FROM v$session_wait sw, v$latch l WHERE sw.p2 = l.latch#;

We could quite obviously drill into the latches further as can be seen in Figure 22.7, linking through to sessions, processes, SQL code, query execution plans, and so on. The point to make now is as follows. There is a much easier way of drilling down into the Oracle Database Wait Event Interface, using Oracle Enterprise Manager. Similar functionality is available in Spotlight and STATSPACK. However, I think that Oracle Enterprise Manager is a better tool for Wait Event Interface drill down. That has not always been the case. Before we go on to using Oracle Enterprise Manager to drill down into the Oracle Database Wait Event Interface we should digress a little.


22.4 Oracle Database Wait Event Interface Improvements

Oracle Database 10 Grid These V$ performance views have been added to:

  • V$EVENT_NAME. Class information is added allowing classification and links to other V$ performance views containing class columns. A class is a classification or category for a wait event or statistic as can be seen in the following query.

    SQL> SELECT DISTINCT(class) FROM v$event_name;

CLASS
--------------------------------------------
Administrative
Application
Archival Process Scheduling
Cluster
Commit
Concurrency
Configuration
Idle
Managed Recovery Processing Scheduling
Network
Other
Scheduler
System I/O
User I/O

  • V$SESSION. Has a lot more information in it including class and parameter details for direct hooks to the third layer and beyond of the Oracle Database Wait Event Interface.

These V$ performance views are new to Oracle Database 10g:

  • V$SYSTEM_WAIT_CLASS. Aggregation layer wait times for classes.

  • V$SESSION_WAIT_CLASS. As above but in the session layer.

  • V$EVENT_HISTOGRAM. Waits and wait times event histogram.

  • V$FILE_HISTOGRAM and V$TEMP_HISTOGRAM. Single block read histograms for data files and temporary files.


22.5 Oracle Enterprise Manager and the Wait Event Interface

Oracle Enterprise Manager is an excellent tool for drilling down visually into the Oracle Database Wait Event Interface. Spotlight is best for real-time monitoring of a busy production database since it places all the important monitoring information in one place. However, Oracle Enterprise Manager is more visually palatable and easier to use as a drill down tool. We now have a basic understanding of the underlying V$ performance views allowing access into Oracle Database Wait Event Interface statistics.

On that basis I have abandoned any deeper-layer drill-down into the Oracle Database Wait Event Interface using the Oracle Database V$ performance views because Oracle Enterprise Manager is so much easier to use in this respect. Why make it difficult?

Some of the visual aspects of Oracle Enterprise Manager have already been covered in two previous chapters of this book. It is assumed that you know how to execute the console and log into a Management Server to allow access and drill-down into the Oracle Database Wait Event Interface for a particular database.

Tip

Use of the Management Server is not an absolute requirement but it does sometimes provide better functionality.

What I am going to do is to connect to my very busy database in the console (logged in through a Management Server) and execute the Performance Overview option in the Diagnostics Pack under the Tools menu. Then I will see if there are any problems with my database. If there are I will drill down into those problem areas attempting to locate something I can tune.

Looking at the Database Memory portion in the Database Health Overview chart I notice something odd. The database buffer cache hit ratio is going up and down, vacillating between 60% and 90% as shown in Figure 22.8. Depending on the application this may not be a problem. However, since my application is mostly doing mostly very small transactions it seems a little odd.

Figure 22.8: Database Health Overview Chart– Database Memory Portion

I am certainly not going to make my database buffer cache larger. Another clue is noticeable latch free wait events as well as scattered reads being much more prominent than sequential reads when the red flag is up as seen in Figure 22.9.

Click To expand
Figure 22.9: Database Health Overview Chart–Database Memory Portion

A large transaction is probably forcing highly used blocks out of the buffer; consequently those highly used blocks will probably be loaded back into the buffer. Let's drill down. Looking at Figure 22.10, I drill down into the "Top Objects by Physical Reads" and the "TopSQL (Physical Reads)". These two drill-downs tell me a lot. Note a number of points.

Click To expand
Figure 22.10: Top Objects and TopSQL

  • The Cashbook table is at the top of both database objects and SQL in terms of data reads.

  • The number of executions for the top SQL statement is relatively low.

  • The physical reads for the Cashbook table is double as compared with the next object down on the list, the Orders table primary key index.

  • The really obvious part is that the Cashbook table is shown as being heavily active. An index on the Cashbook table is not being used.

Examining Figure 22.10 a little further, do I know what to look for yet? Yes I do. The Cashbook table is probably being full table scanned when it should not be. Why? The SQL code statement on the Cashbook table has a WHERE clause against a foreign key index. I checked foreign key indexes on the Cashbook table. I found that the foreign key index on the Cashbook table TRANSACTION_ID column foreign key relation to the Transactions table was missing. The chances are I dropped the index in error sometime during the course of writing this book. Or it was never created. Let's create that index.

CREATE INDEX XFK_CB_TRANS ON cashbook(transaction_id)
      TABLESPACE INDX ONLINE;

I then drilled down into the TopSQL and Top Objects tools again and found that the Cashbook table was not even in the top 10 for both tools as was evident in Figure 22.10. The problem was solved.

Any type of wait event can be drilled down into to the lowest level of detail with a multitude of iterations of each tool using the Oracle Database Wait Event Interface, as implemented in the Oracle Enterprise Manager GUI. I would not recommend using the Oracle Database Wait Event Interface in any other way.


22.6 Using STATSPACK

In the previous chapter we took a brief look at how to set up STATSPACK, how to execute snapshots at different periods of time and how to execute STATSPACK reports. As already stated in the previous chapter STATSPACK is useful for analysis and detection of bottlenecks. In my opinion Oracle Enterprise Manager is better at providing more focused and easy-to-use drill-down into the Oracle Database Wait Event Interface. The Oracle Enterprise Manager Capacity Planner tool can be used to gather and compare historical sets of statistics much like STATSPACK can.

Tip

STATSPACK is currently the official Oracle Corporation performance monitoring and tuning tool.

The easiest method of briefly explaining STATSPACK and its use is to display an example output report and describe the different sections step by step as textual inserts into the report itself. The example used in this section is a range of 10 level 6 snapshots with the database in a consistent state across all snapshots.

Tip

With the TIMED_STATISTICS parameter set to TRUE events are listed in order of wait times as opposed to the number of waits. Wait times are much more significant than number of waits.

22.6.1 An Example STATSPACK Report

The first section shows basic configuration and database instance information and is really a header for the STATSPACK report.

STATSPACK report for

DB Name      DB Id   Instance  Inst Num    Release  Cluster  Host
-------  ----------  --------  --------  ---------  -------  ----------
TEST     1798396642  test             1  9.2.0.1.0       NO  2000SERVER

             Snap Id      Snap Time      Sessions  Curs/Sess  Comment
             -------  -----------------  --------  ---------  -------
Begin Snap:      191  08-Aug-03 16:15:40       21        7.5
  End Snap:      200  08-Aug-03 16:31:04       21        7.1
   Elapsed:       1          5.40 (mins)

Cache Sizes (end)
-----------------
               Buffer Cache:   32M    Std Block Size:      8K
           Shared Pool Size:   64M        Log Buffer:    512K

The load profile section gives an overall health check and load information for the database server. The three most important things are hard parses, which should be as low as possible, executions per second and transactions per second. Other values in this example report are all typical of an OLTP database, which the Accounts schema is attempting to simulate.

Load Profile
------------
                   Per Second   Per Transaction
                   ----------   ---------------
      Redo size:    69,010.88          8,372.64
  Logical reads:     1,945.87            236.08
  Block changes:       489.67             59.41
 Physical reads:       126.89             15.39
Physical writes:        20.40              2.47
     User calls:         8.54              1.04
         Parses:        60.15              7.30
    Hard parses:         0.02              0.00
          Sorts:        27.41              3.33
         Logons:         2.76              0.34
       Executes:       120.65             14.64
   Transactions:         8.24
% Blocks changed per Read: 25.16    Recursive Call %: 98.68
Rollback per transaction %: 0.33       Rows per Sort: 14.94

The instance efficiency section simply shows various ratios. Remember that ratios can be symptoms or indications of possible problems elsewhere. Do not attempt to tune ratios. The only possible problems in this case could be significant re-parsing and perhaps slightly low memory usage.

Instance Efficiency Percentages (Target 100%)
---------------------------------------------
         Buffer Nowait    %:  99.69       Redo NoWait %:   100.00
         Buffer Hit       %:  93.65    In-memory Sort %:   99.85
         Library Hit      %:  99.99        Soft Parse %:   99.97
         Execute to Parse %:  50.15         Latch Hit %:   100.00
Parse CPU to Parse Elapsd %:  66.64     % Non-Parse CPU:   97.37

Shared Pool Statistics       Begin     End
                             -----   -----
           Memory Usage %:   32.76   33.78
  % SQL with executions>1:   73.04   88.26
% Memory for SQL w/exec>1:   54.99   89.13

The top five timed wait events are wait events which can potentially most significantly affect performance. In this example report sequential reads are much higher than scattered reads. This is normal for a small-sized transactional OTLP-type database. Buffer busy wait events are probably occurring due to conflict between many small and a few large transactions or due to lack of concurrency. The Accounts schema database has a small amount of reporting functionality executing on it as well as containing heavy small transactional load.

Top 5 Timed Events
-------------------                            % Total
Event                      Waits   Time (s)   Ela Time
-----------------------   ------   --------   --------
db file sequential read   79,028   7,261         80.74
db file scattered read     3,325     424          4.71
buffer busy waits          5,451     366          4.06
CPU time                             286          3.18
direct path read           1,513     205          2.28

Now we get a list of all wait events in order of descending wait time. Once again the most significant wait time by far is for sequential reads. Sequential reads often indicate single-block reads. High single-block read wait times indicate concurrency issues at the block level, perhaps hot blocks; in the Accounts schema this very likely.

Wait Events for DB: TEST Instance: test Snaps: 191–200
-> s – second
-> cs – centisecond – 100th of a second
-> ms – millisecond – 1000th of a second
-> us – microsecond – 1000000th of a second
-> ordered by wait time desc, waits desc (idle events last)

                                                                 Avg
                                                   Total Wait    wait   Waits/
Event                           Waits   Timeouts     Time (s)    (ms)      txn
----------------------------   ------   --------   ----------   -----   ------
db file sequential read        79,028          0        7,261      92     10.4
db file scattered read          3,325          0          424     127      0.4
buffer busy waits               5,451          0          366      67      0.7
direct path read                1,513          0          205     135      0.2
db file parallel write            820        408          105     128      0.1
enqueue                           447          2          103     230      0.1
control file sequential read      610          0           66     108      0.1
log file sync                   2,775          0           63      23      0.4
log file parallel write         7,017      4,957           57       8      0.9
control file parallel write       298          0           29      96      0.0
latch free                        321         59           17      54      0.0
log file switch completion          9          1            8     873      0.0
direct path write                  86          0            4      51      0.0
LGWR wait for redo copy            92         12            1       6      0.0
buffer deadlock                     6          6            0      21      0.0
log file sequential read            2          0            0      11      0.0
log file single write               2          0            0       9      0.0
SQL*Net break/reset to clien        4          0            0       0      0.0
SQL*Net more data to client         3          0            0       0      0.0
wakeup time manager                29         29          858   29601      0.0
SQL*Net message from client       154          0          242    1569      0.0
SQL*Net more data from clien       22          0            0       0      0.0
SQL*Net message to client         154          0            0       0      0.0

The list of all wait events is followed by background processing wait events.

Background Wait Events for DB: TEST Instance: test Snaps:
  191–200
-> ordered by wait time desc, waits desc (idle events last)

                                                                 Avg
                                                   Total Wait    wait   Waits/
Event                           Waits   Timeouts     Time (s)    (ms)      txn
----------------------------   ------   --------   ----------   -----   ------
db file parallel write            822        409          105     128      0.1
log file parallel write         7,021      4,959           57       8      0.9
control file parallel write       298          0           29      96      0.0
control file sequential read      136          0           18     129      0.0
db file scattered read             71          0           12     166      0.0
latch free                         24          0            4     162      0.0
db file sequential read            28          0            4     133      0.0
LGWR wait for redo copy            92         12            1       6      0.0
direct path write                  10          0            0      11      0.0
direct path read                   10          0            0       8      0.0
log file sequential read            2          0            0      11      0.0
log file single write               2          0            0       9      0.0
buffer busy waits                   1          0            0       9      0.0
rdbms ipc reply                     4          0            0       2      0.0
rdbms ipc message              13,706      8,089        5,372     392      1.8
smon timer                          3          2          782  ######      0.0

The next sections contain parsed SQL code ordered in various different ways. The first section orders SQL statements in descending database buffer gets order where buffer gets exceed 10,000. It is my personal opinion that sequencing SQL code statements based on database buffer cache hits is not particularly useful to tuning in general since it advocates "fiddling" with the size of the database buffer cache and shared pool.

Tip

Hash values can be used to access details of specific SQL code statements.

SQL ordered by Gets for DB: TEST Instance: test Snaps:
  191–200
-> End Buffer Gets Threshold: 10000
-> Note that resources reported for PL/SQL includes the
  resources used by all SQL statements called within the
  PL/SQL code. As individual SQL statements are also
  reported, it is possible and valid for the summed total %
  to exceed 100
Tip

The number of executions is nearly always significant regardless of how much of a type of activity an SQL code statement is causing. A poorly tuned SQL code statement, even a simple statement, executed much more often than any other SQL code is always a problem.

                         Gets per               CPU  Elapsd
Buffer Gets  Executions      Exec  %Total  Time (s)  Time (s)  Hash Value
-----------  ----------  --------  ------  --------  --------  ----------
  1,646,666       2,548     646.3    91.6    254.07   8266.96  1852168585
DECLARE job BINARY_INTEGER := :job; next_date DATE := :mydate;
broken BOOLEAN := FALSE; BEGIN SIMULATE; :mydate := next_date; I
F broken THEN :b := 1; ELSE :b := 0; END IF; END;

    235,786         468     503.8    13.1    132.43    871.74  2026502321
SELECT min(order_id),max(order_id) from orders

    136,975         251     545.7     7.6      6.43   1642.38  3111014165
SELECT tl.* from transactions t join transactionsline tl on(tl.t
ransaction_id = t.transaction_id) where t.supplier_id = :b1

    132,626       8,416      15.8     7.4      4.81     70.13  2781724698
INSERT into transactionsline values(:b4,:b3,:b2,:b1)

    132,557       8,415      15.8     7.4      5.06     62.82  1787838546
Tip

All but the first five SQL code statements have been deleted from this section of this STATSPACK report in order to avoid large amounts of superfluous text, an equally large quantity of paper and a certain propensity to confusion on the part of the reader.

Ordering SQL code statements by descending physical reads, actual I/O activity, is much more useful than using descending buffer gets order. I/O activity can be a performance hog but it can also be necessary depending on application and database type. For instance, full physical scans are preferable in data warehouses.

SQL ordered by Reads for DB: TEST Instance: test Snaps:
   191–200
-> End Disk Reads Threshold: 1000
Tip

The number of executions is nearly always significant regardless of how much of a type of activity an SQL code statement is causing. A poorly tuned SQL code statement, even a simple statement, executed much more often than any other SQL code is always a problem.

Physical                Reads               CPU   Elapsd
   Reads Executions  per Exec  %Total  Time (s)  Time (s)  Hash Value
--------  ---------  --------  ------  --------  --------  ----------
 111,203      2,548      43.6    94.8    254.07   8266.96  1852168585
DECLARE job BINARY_INTEGER := :job; next_date DATE := :mydate;
broken BOOLEAN := FALSE; BEGIN SIMULATE; :mydate := next_date; I
F broken THEN :b := 1; ELSE :b := 0; END IF; END;

  32,461        468      69.4    27.7    132.43    871.74  2026502321
SELECT min(order_id),max(order_id) from orders

  18,974        357      53.1    16.2      5.43   1622.03  3751280151
SELECT ol.* from orders o join ordersline ol on(ol.order_id = o.
order_id) where o.supplier_id = :b1

  18,891        251      75.3    16.1      6.43   1642.38  3111014165
SELECT tl.* from transactions t join transactionsline tl on(tl.t
ransaction_id = t.transaction_id) where t.supplier_id = :b1

   7,602        250      30.4     6.5      4.65    647.74  92829708
SELECT tl.* from transactions t join transactionsline tl on(tl.t
ransaction_id = t.transaction_id) where t.customer_id = :b1
Tip

All but the first five SQL code statements have been deleted from this section of this STATSPACK report in order to avoid large amounts of superfluous text, an equally large quantity of paper and a certain propensity to confusion on the part of the reader.

In the next STATSPACK report snippet SQL code statements have been ordered in descending order of execution. Ordering by number of executions is generally the most useful sorting option. The most useful method for detecting problems for highly active OLTP databases is usually a combination of executions and physical reads.

SQL ordered by Executions for DB: TEST Instance: test
  Snaps: 191–200
-> End Executions Threshold: 100

Rows Rows CPU per Elap per
Executions  Processed  per Exec  Exec (s)  Exec (s)  Hash Value
----------  ---------  --------  --------  --------  ----------
    13,313     13,313       1.0      0.00      0.00  2489014428
SELECT to_number(to_char(SYSTIMESTAMP,'FF3'))+1 from dual

     8,416      8,416       1.0      0.00      0.01  2781724698
INSERT into transactionsline values(:b4,:b3,:b2,:b1)

     8,416      8,416       1.0      0.00      0.00  2822097328
INSERT into cashbookline values(:b3,:b2,:b1)

     8,415      8,415       1.0      0.00      0.01  1787838546
INSERT into ordersline values(:b4,:b3,:b2,:b1)

     8,413      8,413       1.0      0.00      0.01  52940216
INSERT into stockmovement values(stockmovement_seq.nextval,:b4,(
:b3*–1),:b2,:b1) returning stockmovement_id INTO :b0
Tip

All but the first five SQL code statements have been deleted from this section of this STATSPACK report in order to avoid large amounts of superfluous text, an equally large quantity of paper and a certain propensity to confusion on the part of the reader.

The number of parse calls for an SQL code statement is relevant because it denotes how often SQL code statements are re-parsed. Re-parsing is inefficient because it costs significant CPU processing time.

SQL ordered by Parse Calls for DB: TEST Instance: test
  Snaps: 191–200
-> End Parse Calls Threshold: 1000

                        % Total
Parse Calls  Executions  Parses  Hash Value
-----------  ----------  ------  ----------
      3,612       3,612    6.50  3615375148
COMMIT

      2,571       2,558    4.63  493392946
update sys.job$ set this_date= :1 where job= :2

      2,570       2,570    4.62  990457800
select u1.user#, u2.user#, u3.user#, failures, flag,
  interval#,
    what, nlsenv, env, field1 from sys.job$ j, sys.user$
      u1, sys .user$ u2, sys.user$ u3 where job=:1 and
      (next_date < sysdate or :2 != 0) and lowner = u1.name
    and powner = u2.name and cowner = u3.name

      2,570       2,557    4.62  3128358781
update sys.job$ set failures= 0, this_date = null, flag= :1,
  last_da te = :2, next_date = greatest(:3, sysdate),
  total=total+(sysdate-nvl(this_date,sysdate)) where job=:4

      2,558       2,558    4.60  3013728279
select privilege#,level from sysauth$ connect by
  grantee# = prior privilege# and privilege#>0 start with
   (grantee# = :1 or grantee# = 1) and privilege#>0
Tip

All but the first five SQL code statements have been deleted from this section of this STATSPACK report in order to avoid large amounts of superfluous text, an equally large quantity of paper and a certain propensity to confusion on the part of the reader.

The next section lists out general instance activity statistics. There are a lot of different statistics to peruse. A large amount of information can be gathered from the instance activity statistics. Looking at the statistics we can see CPU usage, database writer process (DBWR) activity, buffers, enqueue locking, indexing, tables, I/O, logging, session activity, rollback and undo, transactional activity, amongst many other indicators. This is simply a list of all the instance activity statistics contained in a single section of the STATSPACK report.

Two statistics in the STATSPACK report snippet shown below, called process last nonidle time and session connect time, are off the scale. The former statistic refers to the last time a process was executed and the latter to session connection time. Both these statistics are irrelevant based on the way in which my Accounts schema has OLTP activity executed against it.

Instance Activity Stats for DB: TEST Instance: test Snaps: 191–200

Statistic                                  Total      per Second     per Trans
----------------------------------  ------------  --------------  ------------
CPU used by this session                  28,578            30.9           3.8
CPU used when call started                28,578            30.9           3.8
CR blocks created                            644             0.7           0.1
DBWR buffers scanned                     108,190           117.1          14.2
DBWR checkpoint buffers written            1,040             1.1           0.1
DBWR checkpoints                               1             0.0           0.0
DBWR free buffers found                   94,700           102.5          12.4
DBWR lru scans                               429             0.5           0.1
DBWR make free requests                      429             0.5           0.1
DBWR revisited being-written buff              0             0.0           0.0
DBWR summed scan depth                   108,190           117.1          14.2
DBWR transaction table writes                 51             0.1           0.0
DBWR undo block writes                     3,800             4.1           0.5
SQL*Net roundtrips to/from client            151             0.2           0.0
active txn count during cleanout           6,271             6.8           0.8
background checkpoints completed               1             0.0           0.0
background checkpoints started                 1             0.0           0.0
background timeouts                        1,064             1.2           0.1
branch node splits                             3             0.0           0.0
buffer is not pinned count               902,275           976.5         118.5
buffer is pinned count                   130,645           141.4          17.2
bytes received via SQL*Net from c        138,570           150.0          18.2
bytes sent via SQL*Net to client          71,041            76.9           9.3
calls to get snapshot scn: kcmgss        194,194           210.2          25.5
calls to kcmgas                           11,457            12.4           1.5
calls to kcmgcs                              648             0.7           0.1
change write time                            955             1.0           0.1
cleanout - number of ktugct calls          2,753             3.0           0.4
cleanouts and rollbacks – consist            184             0.2           0.0
cleanouts only - consistent read             141             0.2           0.0
cluster key scan block gets                  241             0.3           0.0
cluster key scans                            214             0.2           0.0
commit cleanout failures: block l            113             0.1           0.0
commit cleanout failures: buffer               9             0.0           0.0
commit cleanout failures: callbac             57             0.1           0.0
commit cleanout failures: cannot               6             0.0           0.0
commit cleanouts                          46,281            50.1           6.1
commit cleanouts successfully com         46,096            49.9           6.1
commit txn count during cleanout           5,998             6.5           0.8
consistent changes                         2,968             3.2           0.4
consistent gets                        1,101,924         1,192.6         144.7
consistent gets – examination            108,479           117.4          14.2
current blocks converted for CR               26             0.0           0.0
cursor authentications                        82             0.1           0.0
data blocks consistent reads – un          2,363             2.6           0.3
db block changes                         452,457           489.7          59.4
db block gets                            696,070           753.3          91.4
deferred (CURRENT) block cleanout         10,492            11.4           1.4
dirty buffers inspected                    1,266             1.4           0.2
enqueue conversions                        4,213             4.6           0.6
enqueue deadlocks                              1             0.0           0.0
enqueue releases                          41,603            45.0           5.5
enqueue requests                          41,610            45.0           5.5
enqueue timeouts                              58             0.1           0.0
enqueue waits                                423             0.5           0.1
exchange deadlocks                             6             0.0           0.0
execute count                            111,483           120.7          14.6
free buffer inspected                      1,311             1.4           0.2
free buffer requested                    120,649           130.6          15.8
hot buffers moved to head of LRU          34,312            37.1           4.5
immediate (CR) block cleanout
   app                                       325             0.4           0.0
immediate (CURRENT) block
   cleanou                                 5,150             5.6           0.7
index fast full scans (full)                 463             0.5           0.1
index fetch by key                        25,431            27.5           3.3
index scans kdiixs1                      116,315           125.9          15.3
leaf node 90-10 splits                       251             0.3           0.0
leaf node splits                             547             0.6           0.1
logons cumulative                          2,552             2.8           0.3
messages received                          7,614             8.2           1.0
messages sent                              7,614             8.2           1.0
no buffer to keep pinned count                 0             0.0           0.0
no work - consistent read gets           858,628           929.3         112.7
opened cursors cumulative                 54,152            58.6           7.1
parse count (failures)                         1             0.0           0.0
parse count (hard)                            18             0.0           0.0
parse count (total)                       55,575            60.2           7.3
parse time cpu                               753             0.8           0.1
parse time elapsed                         1,130             1.2           0.2
physical reads                           117,242           126.9          15.4
physical reads direct                      3,116             3.4           0.4
physical writes                           18,848            20.4           2.5
physical writes direct                     3,125             3.4           0.4
physical writes non                       18,137            19.6           2.4
   checkpoint
pinned buffers inspected                      44             0.1           0.0
prefetched blocks                         31,706            34.3           4.2
process last non-idle
   time                        2,692,317,558,691  ##############  ############
recovery blocks read                           0             0.0           0.0
recursive calls                          590,362           638.9          77.5
recursive cpu usage                       28,269            30.6           3.7
redo blocks written                      132,490           143.4          17.4
redo buffer allocation                         7             0.0           0.0
   retries
redo entries                             229,257           248.1          30.1
redo log space requests                       10             0.0           0.0
redo log space wait time                     786             0.9           0.1
redo size                             63,766,056        69,010.9       8,372.6
redo synch time                            6,466             7.0           0.9
redo synch writes                          2,663             2.9           0.4
redo wastage                           1,924,580         2,082.9         252.7
redo write time                            8,476             9.2           1.1
redo writer latching time                     52             0.1           0.0
redo writes                                7,004             7.6           0.9
rollback changes - undo                      518             0.6           0.1
   records a
rollbacks only -                             529             0.6           0.1
   consistent read
rows fetched via callback                 17,855            19.3           2.3
session connect time           2,692,317,558,691  ##############  ############
session logical reads                  1,797,981         1,945.9         236.1
session pga memory max                   100,372           108.6          13.2
session uga memory                    29,372,040        31,787.9       3,856.6
session uga memory max               268,212,120       290,272.9      35,216.9
shared hash latch                         76,480            82.8          10.0
   upgrades - no w
shared hash latch                              1             0.0           0.0
   upgrades – wait
sorts (disk)                                  38             0.0           0.0
sorts (memory)                            25,287            27.4           3.3
sorts (rows)                             378,384           409.5          49.7
summed dirty queue length                 15,041            16.3           2.0
switch current to new buffer                 997             1.1           0.1
table fetch by rowed                     321,386           347.8          42.2
table fetch continued row                      0             0.0           0.0
table scan blocks gotten                  53,011            57.4           7.0
table scan rows gotten                 9,862,509        10,673.7       1,295.0
table scans (long tables)                      2             0.0           0.0
table scans (short tables)                24,593            26.6           3.2
transaction rollbacks                        123             0.1           0.0
transaction tables consistent rea             13             0.0           0.0
transaction tables consistent rea            604             0.7           0.1
user calls                                 7,890             8.5           1.0
user commits                               7,591             8.2           1.0
user rollbacks                                25             0.0           0.0
write clones created in backgroun              1             0.0           0.0
write clones created in foregroun             41             0.0           0.0

The next two sections show tablespace and datafile physical read and write I/O statistics. In general, user application tables are contained within the DATA tablespace and related indexes in the INDX tablespace. Table reads should probably be higher than index reads since tables are read completely (full table scans) or as a result of index ROWID pointer searches and scans. Index writes are higher than table writes, probably indicating many indexes for each table and high insertion activity. In fact high row insertion activity is probably causing contention and thus the significant buffer wait events already seen. Other values in the following section indicate temporary sort space on disk using the TEMP tablespace is possibly significant. Perhaps the sort area buffers could be increased in size, namely the SORT_AREA_SIZE and SORT_AREA_RETAINED SIZE parameters. Rollback (automated undo) reads is likely caused by consistent transaction requirements. Write values for rollback space are much higher than that of read values. However, read values probably remain significant. However, in the Accounts schema this type of rollback consistent requirement is normal for application and database type.

Tablespace IO Stats for DB: TEST Instance: test Snaps:
   191–200
–>ordered by IOs (Reads + Writes) desc

Tablespace
-----------------------

                       Av      Av       Av               Av     Buffer  Av Buf
             Reads  Reads/s  Rd(ms)  Blks/Rd  Writes  Writes/s   Waits  Wt(ms)
------------------  -------  ------  -------  ------  --------  ------  ------
DATA
            42,865       46   101.1      1.0   3,280         4   1,287    66.2
INDX
            36,831       40    82.4      1.8   7,106         8   4,145    68.3
TEMP
             2,849        3   133.9      1.1   3,115         3       0     0.0
UNDOTBS1
               470        1   106.3      1.0   3,870         4       3     3.3
PERFSTAT
             1,721        2   130.7      1.0   1,367         1       0     0.0
SYSTEM
               453        0   131.1      2.6      95         0      75    43.2
OBJECTS
                 1        0   170.0      1.0       1         0       0     0.0
TOOLS
                 1        0   260.0      1.0       1         0       0     0.0
USERS
                 1        0   260.0      1.0       1         0       0     0.0
           -----------------------------------------------------------

With respect to datafiles I/O activity appears to be spread well between the two datafiles for the DATA tablespace but not as well for the two datafiles in the INDX tablespace.

File IO Stats for DB: TEST Instance: test Snaps: 191–200
–>ordered by Tablespace, File

Tablespace               Filename
-----------------------  ----------------------------------
                   Av      Av       Av               Av     Buffer  Av Buf
         Reads  Reads/s  Rd(ms)  Blks/Rd  Writes  Writes/s   Waits  Wt(ms)
--------------  -------  ------  -------  ------  --------  ------  ------
DATA                      E:\ORACLE\ORADATA\TEST\DATA01.DBF
        21,460       23   103.4      1.0   1,606         2     549    65.1
                          E:\ORACLE\ORADATA\TEST\DATA02.DBF
        21,405       23    98.8      1.1   1,674         2     738    67.0
INDX                      E:\ORACLE\ORADATA\TEST\INDX01.DBF
        20,039       22    86.2      1.7   4,046         4   2,029    65.7
                          E:\ORACLE\ORADATA\TEST\INDX02.DBF
        16,792       18    77.9      1.9   3,060         3   2,116    70.8
OBJECTS                   E:\ORACLE\ORADATA\TEST\OBJECTS01.DBF
             1        0   170.0      1.0       1         0       0
PERFSTAT                  E:\ORACLE\ORADATA\TEST\PERFSTAT01.DBF
         1,721        2   130.7      1.0   1,367         1       0
SYSTEM                    E:\ORACLE\ORADATA\TEST\SYSTEM01.DBF
           453        0   131.1      2.6      95         0      75    43.2
TEMP                      E:\ORACLE\ORADATA\TEST\TEMP01.DBF
         2,849        3   133.9      1.1   3,115         3       0
TOOLS                     E:\ORACLE\ORADATA\TEST\TOOLS01.DBF
             1        0   260.0      1.0       1         0       0
UNDOTBS1                  E:\ORACLE\ORADATA\TEST\UNDOTBS01.DBF
           470        1   106.3      1.0   3,870         4       3     3.3
USERS                     E:\ORACLE\ORADATA\TEST\USERS01.DBF
             1        0   260.0      1.0       1         0       0
        ----------------------------------------------------------

Next we get database buffer cache statistics. Buffer busy waits are significant.

Buffer Pool Statistics for DB: TEST Instance: test Snaps:
   191–200
–> Standard block size Pools D: default, K: keep, R: recycle
–> Default Pools for other block sizes: 2k, 4k, 8k, 16k, 32k

                                                   Free    Write  Buffer
 Number of Cache     Buffer  Physical  Physical  Buffer Complete    Busy
P Buffers  Hit %       Gets     Reads    Writes   Waits    Waits   Waits
---------  -----  ---------  --------  --------  ------  -------  ------
D   4,000   95.7  2,667,765   113,687    15,693       0        0   5,473
    ------------------------------------------------------------

These are recovery requirement estimates.

Instance Recovery Stats for DB: TEST Instance: test Snaps:
   191–200
–> B: Begin snapshot, E: End snapshot

Targt Estd                          Log File     Log Ckpt      Log Ckpt

  MTTR  MTTR  Recovery   Actual      Target      Size     Timeout    Interval
   (s)   (s)  Estd Ios  Redo Blks  Redo Blks  Redo Blks  Redo Blks  Redo Blks
- ----  ----  --------  ---------  ---------  ---------  ---------  ---------
B   35    27       496      25653     184320     184320     249736
E   35    27       614      25742     184320     184320     221989
         ---------------------------------------------------------

This is the database buffer cache advisory. Buffer advisories are covered in a previous chapter. I do not find advisories particularly useful, tending to encourage frequent upward resizing of the database buffer cache and shared pool. Resizing buffer pools upwards does not solve problems but perhaps sometimes alleviates symptoms of problems and can even "help" to obscure significant performance problems.

Buffer Pool Advisory for DB: TEST Instance: test End Snap:
   200
-> Only rows with estimated physical reads >0 are
   displayed
-> ordered by Block Size, Buffers For Estimate

       Size for   Size  Buffers for  Est Physical       Estimated
P  Estimate (M)  Factr     Estimate   Read Factor  Physical Reads
-  ------------  -----  -----------  ------------  --------------
D             4     .1          500          4.83       8,118,743
D             8     .3        1,000          3.10       5,218,381
D            12     .4        1,500          1.99       3,340,086
D            16     .5        2,000          1.55       2,609,060
D            20     .6        2,500          1.35       2,275,264
D            24     .8        3,000          1.21       2,038,548
D            28     .9        3,500          1.10       1,851,173
D            32    1.0        4,000          1.00       1,681,112
D            36    1.1        4,500          0.91       1,533,069
D            40    1.3        5,000          0.83       1,401,561
D            44    1.4        5,500          0.76       1,281,701
D            48    1.5        6,000          0.70       1,171,290
D            52    1.6        6,500          0.64       1,070,867
D            56    1.8        7,000          0.58         979,671
D            60    1.9        7,500          0.53         892,039
D            64    2.0        8,000          0.48         808,529
D            68    2.1        8,500          0.44         737,110
D            72    2.3        9,000          0.40         673,978
D            76    2.4        9,500          0.37         618,692
D            80    2.5       10,000          0.34         569,011

Data block waits indicate hot block issues, common in highly active small transactional OLTP databases. Possible solutions are already well covered in this book.

Buffer wait Statistics for DB: TEST Instance: test Snaps:
   191–200
–> ordered by wait time desc, waits desc

                   Tot Wait        Avg
Class       Waits  Time (s)  Time (ms)
----------  -----  --------  ---------
data block  5,488       371         68
undo header     3         0          3

Enqueue activity is generally consistent with details covered already in this book. There is no need to reiterate.

Enqueue activity for DB: TEST Instance: test Snaps:
   191–200
–> Enqueue stats gathered prior to 9i should not be
compared with 9i data
–> ordered by Wait Time desc, Waits desc

                         Failed            Avg Wt     Wait
Eq  Requests  Succ Gets    Gets  Waits  Time (ms)  Time(s)
--  --------  ---------  ------  -----  ---------  -------
TX    10,107     10,105       0    362     295.01      107
SQ       992        992       0     60      39.73        2
HW        69         69       0      1        .00        0

Rollback segment statistics show some wrapping with possibly significant consistent transaction read requirements as already stated before. In high small transactional activity OLTP databases this behavior is probably unavoidable. For automated undo the UNDO_RETENTION parameter could be altered; for manual rollback segments create more, smaller rollback segments. Obviously tuning rollback space does not solve the real problem but merely attempts to spread rollback more evenly and thus avoid rollback contention. The real problem is that the database is very busy. Possible solutions to this type of performance issue are better handling of concurrency at the block level by raising INITRANS values, using reverse key indexing or perhaps even clustering. All these items and more have been covered in this book already.

Rollback Segment Stats for DB: TEST Instance: test Snaps:
   191–200
–> A high value for "Pct Waits" suggests more rollback
   segments may be required
–> RBS stats may not be accurate between begin and end
   snaps when using Auto Undo managment, as RBS may be
   dynamically created and dropped as needed

        Trans Table    Pct  Undo Bytes
RBS No         Gets  Waits     Written  Wraps  Shrinks  Extends
------  -----------  -----  ----------  -----  -------  -------
     0         12.0   0.00           0      0        0        0
     1      2,275.0   0.00   2,240,820      4        0        0
     2      2,318.0   0.00   2,000,692      4        0        0
     3      2,080.0   0.00   2,109,416      4        0        0
     4      1,958.0   0.05   2,608,464      5        0        0
     5      2,182.0   0.00   1,984,204      2        0        0
     6      2,426.0   0.00   2,248,984      2        0        0
     7      1,912.0   0.00   2,053,060      4        0        0
     8      2,546.0   0.00   2,975,356      4        0        0
     9      2,345.0   0.00   2,350,076      3        0        0
    10      2,541.0   0.00   2,671,298      6        1        1
            ---------------------------------------------------

Rollback Segment Storage for DB: TEST Instance: test Snaps:
   191–200
-> Optimal Size should be larger than Avg Active

RBS No  Segment Size  Avg Active  Optimal Size  Maximum Size
------  ------------  ----------  ------------  ------------
     0       385,024           0                     385,024
     1    35,840,000     860,252                  35,840,000
     2    34,725,888     866,432                  34,725,888
     3    35,774,464     850,308                  35,774,464
     4    33,677,312     884,223                  33,677,312
     5    36,823,040   1,035,943                  36,823,040
     6    43,114,496   4,062,790                  43,114,496
     7    35,774,464     849,535                  35,774,464
     8    35,774,464     959,362                  35,774,464
     9    35,774,464     987,376                  35,774,464
    10    32,759,808     697,641                  36,036,608
        ----------------------------------------------------

Undo Segment Summary for DB: TEST Instance: test Snaps:
   191-200
-> Undo segment block stats:
-> uS - unexpired Stolen, uR - unexpired Released, uU -
   unexpired reUsed
-> eS - expired Stolen, eR - expired Released, eU - expired
   reUsed

Undo    Undo      Num  Max Qry    Max Tx  Snapshot  Out of    uS/uR/uU/
 TS#  Blocks    Trans  Len (s)  Concurcy   Too Old   Space     eS/eR/eU
----  ------  -------  -------  --------  --------  ------  -----------
   1   5,291  452,288      196         3         0       0  0/0/0/0/0/0

Undo Segment Stats for DB: TEST Instance: test Snaps:
   191-200
-> ordered by Time desc

                Undo      Num  Max Qry  Max Tx     Snap  Out of    uS/uR/uU/
End Time      Blocks    Trans  Len (s)   Concy  Too Old   Space     eS/eR/eU
------------  ------  -------  -------  ------  -------  ------  -----------
08-Aug 16:36   2,378  157,059      196       2        0       0  0/0/0/0/0/0
08-Aug 16:26   2,334  150,680      169       2        0       0  0/0/0/0/0/0
08-Aug 16:16     579  144,549       28       3        0       0  0/0/0/0/0/0
         -----------------------------------------------------------

Next is latch activity. There appear to be significant issues with the database buffer cache and the library cache in the shared pool. These values are once again significant with details covered previously in this book. The database buffer cache and shared pool are deliberately set to low values in my database containing the Accounts schema, in order to create tunable problems.

Latch Activity for DB: TEST Instance: test Snaps: 191–200
–>"Get Requests", "Pct Get Miss" and "Avg Slps/Miss" are
   statistics for willing-to-wait latch get requests
–>"NoWait Requests", "Pct NoWait Miss" are for no-wait
   latch get requests
–>"Pct Misses" for both should be very close to 0.0

                                       Pct   Avg  Wait              Pct
                               Get     Get  Slps  Time    NoWait NoWait
Latch                       Requests  Miss  Miss   (s)  Requests   Miss
-------------------------  ---------  ----  ----  ----  --------  -----
Consistent RBA                 7,028   0.0           0         0
FIB s.o chain latch                4   0.0           0         0
FOB s.o list latch                57   0.0           0         0
SQL memory manager worka         603   0.0           0         0
active checkpoint queue          901   0.0           0         0
archive control                   12   0.0           0         0
cache buffer handles           1,556   0.0           0         0
cache buffers chains       4,761,717   0.0   1.0     0   154,473    0.0
cache buffers lru chain      153,530   0.0   1.0     0        84    0.0
channel handle pool latc           2   0.0           0         0
channel operations paren         587   0.0           0         0
checkpoint queue latch        66,578   0.0           0    15,155    0.0
child cursor hash table          293   0.0           0         0
dml lock allocation           43,873   0.0   1.0     0         0
dummy allocation               5,120   0.0           0         0
enqueue hash chains           88,018   0.0   6.0    15         0
enqueues                      23,702   0.0   1.0     0         0
event group latch                  1   0.0           0         0
file number translation        6,045   0.0           0         0
hash table column usage           10   0.0           0        30    0.0
job workq parent latch             0                 0     3,122    0.0
job_queue_processes para         375   0.0           0         0
ktm global data                    5   0.0           0         0
kwqit: protect wakeup ti          29   0.0           0         0
lgwr LWN SCN                   7,029   0.0           0         0
library cache              1,022,754   0.0   1.1     1         0
library cache load lock           12   0.0           0         0
library cache pin            630,683   0.0   1.0     0         0
library cache pin alloca     323,891   0.0   1.1     0         0
list of block allocation       8,313   0.0           0         0
loader state object free         170   0.0           0         0
longop free list parent           19   0.0           0        19    0.0
messages                      35,912   0.0   1.0     0         0
mostly latch-free SCN          7,031   0.0           0         0
multiblock read objects       12,287   0.0           0         2    0.0
ncodef allocation latch           23   0.0           0         0
post/wait queue               14,181   0.0           0   2,790 0.0
process allocation                 1   0.0           0         1    0.0
process group creation             2   0.0           0         0
redo allocation              242,912   0.0   1.1     0         0
redo copy                         60   1.7   2.0     0   228,845    0.1
redo writing                  22,607   0.0           0         0
row cache enqueue latch       29,914   0.0           0         0
row cache objects             36,319   0.0           0         0
sequence cache                57,781   0.0   1.0     0         0
session allocation            68,320   0.0           0         0
session idle bit              18,430   0.0           0         0
session switching                 23   0.0           0         0
session timer                    485   0.0           0         0
shared pool                  333,910   0.0   2.0     0         0
sim partition latch                0                 0        67    0.0
simulator hash latch         122,962   0.0           0         0
simulator lru latch           12,525   0.0           0       118    0.0
sort extent pool                 254   0.0           0         0
transaction allocation         6,507   0.0           0         0
transaction branch alloc          23   0.0           0         0
undo global data              36,910   0.0   1.0     0         0
user lock                      5,096   0.0           0         0

Latch sleeps are a result of missed latch acquisition. Missed latches are problematic for performance. Once again significant issues are apparent in the database buffer cache and the library cache in the shared pool.

ZZZZZZZZZZZZZ721

Latch Sleep breakdown for DB: TEST Instance: test Snaps:
   191–200
–> ordered by misses desc

                                  Get                       Spin &
Latch Name                   Requests  Misses  Sleeps  Sleeps 1->4
--------------------------  ---------  ------  ------  -----------
library cache               1,022,754     115     121  0/109/6/0/0
cache buffers chains        4,761,717       58     58    0/0/0/0/0
cache buffers lru chain       153,530       30     31   0/29/1/0/0
redo allocation               242,912       17     18   0/16/1/0/0
enqueue hash chains            88,018        9     54    0/3/0/6/0
library cache pin allocati     323,891       9     10    0/8/1/0/0
library cache pin              630,683       8      8    0/8/0/0/0
dml lock allocation             43,873       7      7    0/7/0/0/0
sequence cache                  57,781       4      4    0/4/0/0/0
enqueues                        23,702       3      3    0/3/0/0/0
undo global data                36,910       2      2    0/2/0/0/0
messages                        35,912       1      1    0/1/0/0/0
redo copy                           60       1      2    0/0/1/0/0
shared pool                    333,910       1      2    0/0/1/0/0

The next section shows very deep level sources of missed latches. From my perspective this information is too excessive to be useful considering other tuning tools available.

Latch Miss Sources for DB: TEST Instance: test Snaps:
  191–200
–> only latches with sleeps are shown
–> ordered by name, sleeps desc

NoWait Waiter
Latch Name               Where                       Misses  Sleeps  Sleeps
-----------------------  --------------------------  ------  ------  ------
cache buffers chains     kcbgtcr: kslbegin excl           0      21      24
cache buffers chains     kcbrls: kslbegin                 0      13       9
cache buffers chains     kcbgcur: kslbegin                0       7      11
cache buffers chains     kcbchg: kslbegin: call CR        0       3       0
cache buffers chains     kcbget: exchange rls             0       3       0
cache buffers chains     kcbzwb                           0       3       2
cache buffers chains     kcbget: exchange                 0       2       0
cache buffers chains     kcbgtcr: fast path               0       2       3
cache buffers chains     kcbget: pin buffer               0       2       8
cache buffers chains     kcbchg: kslbegin: bufs not       0       1       0
cache buffers chains     kcbnlc                           0       1       0
cache buffers lru chain  kcbzgb: wait                     0      27      29
cache buffers lru chain  kcbbiop: lru scan                0       2       0
cache buffers lru chain  kcbbxsv: move to being wri       0       1       0
cache buffers lru chain  kcbzgb: posted for free bu       0       1       1
dml lock allocation      ktaiam                           0       4       3
dml lock allocation      ktaidm                           0       3       4
enqueue hash chains      ksqcmi: kslgpl                   0      46       0
enqueue hash chains      ksqrcl                           0       5      10
enqueue hash chains      ksqgtl3                          0       2      35
enqueue hash chains      ksqcmi: get hash chain lat       0       1       9
enqueues                 ksqgel: create enqueue           0       2       0
enqueues                 ksqdel                           0       1       2
library cache            kglupc: child                    0      26      71
library cache            kglpndl: child: before pro       0      21       3
library cache            kglpndl: child: after proc       0      13       0
library cache            kgllkdl: child: cleanup          0      10       2
library cache            kglpin: child: heap process      0       8       1
library cache            kglpnc: child                    0       8       2
library cache            kglhdgc: child:                  0       7       0
library cache            kglobpn: child:                  0       7       1
library cache            kglget: child: KGLDSBRD          0       6      13
library cache            kglhdgn: child:                  0       5       4
library cache            kglic                            0       5       5
library cache            kglget: child: KGLDSBYD          0       3       0
library cache            kgldte: child 0                  0       1      19
library cache            kglpin                           0       1       0
library cache pin        kglpnal: child: alloc spac       0       4       2
library cache pin        kglpnc: child                    0       4       0
library cache pin        kglpndl                          0       2       0
library cache pin        kglupc                           0       1       9
library cache pin        alloca kgllkdl                   0       6       5
library cache pin        alloca kglpnal                   0       4       5
messages                 ksaamb: after wakeup             0       1       0
redo allocation          kcrfwr                           0      16      11
redo allocation          kcrfwi: before write             0       1       2
redo allocation          kcrfwi: more space               0       1       5
redo copy                kcrfwr: nowait                   0       2       0
sequence cache           kdnnxt: cached seq               0       3       0
sequence cache           kdnss                            0       1       4
shared pool              kghalo                           0       2       0
undo global data         ktubnd:child                     0       1       1
undo global data         ktudba: KSLBEGIN                 0       1       0
            -------------------------------------------------------

This is another example of excessively deep level analysis by STATSPACK analyzing the row cache in the shared pool, specifically the data dictionary activity cache from the V$ROWCACHE performance view. The only interesting factor here is the occurrence of histograms. It appears that the ANALYZE command used to compute statistics generates histograms for all indexed columns, by default in Oracle9i Release 2 (9.2) on NT/2000, when computing statistics on tables. At least it does on my database server. The occur- rence of the high value for dc_histogram_defs is the result.

Dictionary Cache Stats for DB: TEST Instance: test Snaps:
   191–200
–>"Pct Misses" should be very low (< 2% in most cases)
–>"Cache Usage" is the number of cache entries being used
–>"Pct SGA" is the ratio of usage to allocated size for
   that cache

                           Get   Pct  Scan   Pct   Mod  Final
Cache                 Requests  Miss  Reqs  Miss  Reqs  Usage
--------------------  --------  ----  ----  ----  ----  -----
dc_histogram_defs           98  80.6     0           0    194
dc_object_ids               76   2.6     0           0    293
dc_objects                 212   0.0     0           0    447
dc_profiles              2,541   0.0     0           0      1
dc_rollback_segments        67   0.0     0           0     12
dc_segments                136   0.0     0           0    206
dc_sequences               931   0.0     0         931      8
dc_tablespace_quotas         5   0.0     0           5      4
dc_tablespaces             663   0.0     0           0      6
dc_user_grants              17   0.0     0           0     12
dc_usernames                25   0.0     0           0      4
dc_users                12,796   0.0     0           0     14
                  ------------------------------------

This section shows library cache activity based on object type. There is a lot more SQL activity than anything else and a small number of reloads.

Library Cache Activity for DB: TEST Instance: test Snaps:
   191–200
–>"Pct Misses" should be very low

                      Get   Pct       Pin   Pct           Invali-
Namespace        Requests  Miss  Requests  Miss  Reloads  dations
---------------  --------  ----  --------  ----  -------  -------
BODY                5,092   0.0     5,092   0.0        0        0
CLUSTER                 2   0.0         3   0.0        0        0
INDEX                 116   0.0       116   0.0        0        0
SQL AREA           19,071   0.0   237,733   0.0       22        0
TABLE/PROCEDURE     4,237   0.0    55,925   0.0        0        0
TRIGGER             7,547   0.0     7,547   0.0        0        0
                  ----------------------------------------

Once again this is a buffer area advisory, potentially encouraging excessive increased re-sizing of the shared pool, avoiding solving and obscuring underlying performance problems.

Shared Pool Advisory for DB: TEST Instance: test End Snap: 200
–> Note there is often a 1:Many correlation between a
   single logical object in the Library Cache, and the
   physical number of memory objects associated with it.
   Therefore comparing the number of Lib Cache objects
   (e.g. in v$librarycache), with the number of Lib Cache
   Memory Objects is invalid

                                                         Estd
Shared Pool     SP       Estd       Estd    Estd Lib  LC Time
   Size for   Size  Lib Cache  Lib Cache  Cache Time    Saved  Estd Lib Cache
  Estim (M)  Factr   Size (M)    Mem Obj   Saved (s)    Factr    Mem Obj Hits
-----------  -----  ---------  ---------  ----------  -------  --------------
         32     .5         11      2,813      35,757      1.0       2,269,393
         40     .6         11      2,813      35,757      1.0       2,269,393
         48     .8         11      2,813      35,757      1.0       2,269,393
         56     .9         11      2,813      35,757      1.0       2,269,393
         64    1.0         11      2,813      35,757      1.0       2,269,393
         72    1.1         11      2,813      35,757      1.0       2,269,393
         80    1.3         11      2,813      35,757      1.0       2,269,393
         88    1.4         11      2,813      35,757      1.0       2,269,393
         96    1.5         11      2,813      35,757      1.0       2,269,393
        104    1.6         11      2,813      35,757      1.0       2,269,393
        112    1.8         11      2,813      35,757      1.0       2,269,393
        120    1.9         11      2,813      35,757      1.0       2,269,393
        128    2.0         11      2,813      35,757      1.0       2,269,393
        ---------------------------------------------------------------

This is a summary of SGA (system global area) sizes.

SGA Memory Summary for DB: TEST Instance: test Snaps:
   191–200

SGA regions       Size in Bytes
----------------  -------------
Database Buffers     33,554,432
Fixed Size              453,452
Redo Buffers            667,648
Variable Size        92,274,688
                  -------------
Sum                 126,950,220
      -------------------------

The next section contains a breakdown of the SGA.

SGA breakdown difference for DB: TEST Instance: test Snaps:
191–200

Pool    Name                       Begin value   End value  % Diff
------  -------------------------  -----------   ---------  ------
Large   free memory                  8,388,608   8,388,608    0.00
Shared  1M buffer                    2,098,176   2,098,176    0.00
Shared  Checkpoint queue               282,304     282,304    0.00
Shared  FileIdentificatonBlock         323,292     323,292    0.00
Shared  FileOpenBlock                  695,504     695,504    0.00
Shared  KGK heap                         3,756       3,756    0.00
Shared  KGLS heap                    1,766,080   1,788,664    1.28
Shared  KQR M PO                       451,100     492,572    9.19
Shared  KQR S PO                        58,940      58,940    0.00
Shared  KQR S SO                         3,072       3,072    0.00
Shared  KSXR large reply queue         166,104     166,104    0.00
Shared  KSXR pending messages que      841,036     841,036    0.00
Shared  KSXR receive buffers         1,033,000   1,033,000    0.00
Shared  MTTR advisory                   12,692      12,692    0.00
Shared  PL/SQL DIANA                 1,580,060   1,580,060    0.00
Shared  PL/SQL MPCODE                  497,840     507,844    2.01
Shared  PLS non-lib hp                   2,068       2,068    0.00
Shared  character set object           274,508     274,508    0.00
Shared  dictionary cache             1,610,880   1,610,880    0.00
Shared  enqueue                        171,860     171,860    0.00
Shared  event statistics per sess    1,718,360   1,718,360    0.00
Shared  fixed allocation callback          180         180    0.00
Shared  free memory                 56,403,656  55,548,136   -1.52
Shared  joxs heap init                   4,220       4,220    0.00
Shared  kgl simulator                  608,780     613,016    0.70
Shared  ksm_file2sga region            148,652     148,652    0.00
Shared  library cache                3,598,836   3,665,656    1.86
Shared  message pool freequeue         834,752     834,752    0.00
Shared  miscellaneous                4,250,564   4,250,564    0.00
Shared  parameters                      31,412      33,500    6.65
Shared  processes                      144,000     144,000    0.00
Shared  sessions                       410,720     410,720    0.00
Shared  sim memory hea                  25,400      25,400    0.00
Shared  sql area                     3,823,264   4,531,524   18.53
Shared  table definiti                   3,684       3,740    1.52
Shared  trigger defini                   6,732       6,732    0.00
Shared  trigger inform                     496         496    0.00
Shared  trigger source                     100         100    0.00
        buffer_cache                33,554,432  33,554,432    0.00
        fixed_sga                      453,452     453,452    0.00
        log_buffer                     656,384     656,384    0.00
          ------------------------------------------------

And finally we get a listing of nondefault setting database configuration parameters.

init.ora Parameters for DB: TEST Instance: test Snaps:
   191–200

                                                                   End value
Parameter Name                  Begin value                        (if different)
------------------------------  ---------------------------------  --------------
aq_tm_processes                 1
background_dump_dest            E:\oracle\admin\test\bdump
compatible                      9.2.0.0.0
control_files                   E:\oracle\oradata\test\control01.
core_dump_dest                  E:\oracle\admin\test\cdump
db_block_size                   8192
db_cache_size                   33554432
db_domain
db_file_multiblock_read_count   16
db_name test
fast_start_mttr_target          300
hash_join_enabled               TRUE
instance_name                   test
java_pool_size                  0
job_queue_processes             10
large_pool_size                 8388608
open_cursors                    300
pga_aggregate_target            0
processes                       150
query_rewrite_enabled           FALSE
remote_login_passwordfile       EXCLUSIVE
shared_pool_size                67108864
sort_area_size                  32768
star_transformation_enabled     FALSE
timed_statistics                TRUE
undo_management                 AUTO
undo_retention                  10800
undo_tablespace                 UNDOTBS1
user_dump_dest                  E:\oracle\admin\test\udump
         ------------------------------------------------------
End of Report

As you can now see STATSPACK produces a large amount of information in a single report. This section on using STATSPACK is all a single STATSPACK report. Finding a very specific problem amongst all that detail is probably more difficult than using drill- down into the Oracle Database Wait Event Interface, within the Oracle Enterprise Manager GUI Performance Overview tool. STATSPACK is best to get an overall picture of underlying physical structure. Using the Wait Event Interface is perhaps easier to isolate specific issues after having gained a general perspective using STATSPACK.

This chapter covering the Oracle Database Wait Event Interface and STATSPACK completes the third part of this book on physical and configuration tuning for Oracle Database, and concludes this book. I hope you have enjoyed reading it as much as I have enjoyed writing it.


Appendix A: Sample Databases

Database Schemas

Only schema entity relationship diagrams are included. There is little relevance with respect to tuning in including the schema creation scripting.

Click To expand
Figure A.1: Employees Schema ERD Version One
Click To expand
Figure A.2: Employees Schema ERD Version Two
Click To expand
Figure A.3: Accounts Schema ERD Denormalized Version

Inclusion of the highly Normalized version of the Accounts schema in Figure A.4 is not strictly necessary but useful to show the vast difference between an over-Normalized and a fairly well Denormalized schema entity relationship diagram as shown in Figure A.3.

Click To expand
Figure A.4: Accounts Schema ERD Normalized Version


Active Concurrent Database Scripting

These scripts do not include trigger code. Use of triggers is easy but contradictory to performance tuning.

Internet OLTP Database Simulation

–every 1 second
declare
      jobno number;
      i integer default 1;
begin
      for j in 1..10 loop
      dbms_job.submit(jobno,'SIMULATE;',SYSDATE,
         'SYSDATE+1/86400');
      end loop;
      commit;
end;
/

declare
      cursor cJobs is select job from user_jobs;
begin
      for rJob in cJobs loop
            dbms_job.remove(rJob.job);
      end loop;
end;
/
commit;

create or replace function random(n IN NUMBER DEFAULT 1)
   return integer is
      random integer;
begin
      select to_number(to_char(SYSTIMESTAMP,'FF3'))+1 into
         random from dual;
      random := (random/1000)*n;
      if random = 0 then random := 1; end if;
      return random;
exception when others then
      dbms_output.put_line('FUNC: random '||SQLERRM
         (SQLCODE));
end;
/
alter function random compile;

create or replace function getTime(ptime float default 0)
   return float is
begin
   return to_number(to_char(systimestamp,'SSSSS.FF')) –
      ptime;
exception when others then
      dbms_output.put_line('FUNC: getTime '
         ||SQLERRM(SQLCODE));
end;
/
alter function getTime compile;

create or replace procedure getPrice
(
       pstock_id IN integer default 0
      ,psupplier_id OUT integer
      ,pprice OUT float
) as
begin
      –find cheapest price for stock_id
      select a.supplier_id, a.price into psupplier_id, pprice
      from (
            select supplier_id, min(price) as price
            from stocksource
            where stock_id = pstock_id
            group by supplier_id
            order by price
      ) a where rownum = 1;
exception when others then
      dbms_output.put_line('PROC: getPrice '
         ||SQLERRM(SQLCODE));
end;
/
alter procedure getprice compile;

create or replace procedure InsertSANDP (pdte IN date
   DEFAULT SYSDATE) as
      vcustomer_id integer;
      vsupplier_id integer;
      vstock_id integer;
      vorder_id integer;
      vstockmovement_id integer;
      vtransaction_id integer;
      vcheque_id integer;
      vqty integer;
      vlines integer;
      vqtyonhand integer;
      vprice float;
      vamount float;
      vseq# integer;
      vmarkup float default 1.01;
      vtime float;
begin
      vtime := to_number(to_char(systimestamp,'SSSSS.FF'));

      select max(customer_id) into vcustomer_id from
        customer; vcustomer_id := random(vcustomer_id);
      select max(stock_id) into vstock_id from stock;
      vstock_id := random(vstock_id);
      vqty := random(20); vlines := vqty;
      select qtyonhand into vqtyonhand from stock
         where stock_id = vstock_id;

      if vqty - vqtyonhand > 0 then

            vqty := vqty - vqtyonhand; —vqty - vqtyonhand
              always +ve
            getPrice(vstock_id,vsupplier_id,vprice);
            vamount := vqty * vprice;

            insert into orders values(orders_seq.nextval
            ,null,vsupplier_id,'P',(vamount*–1),pdte)
                  returning order_id into vorder_id;

            insert into transactions
              values(transactions_seq.nextval
            ,'P',null,vsupplier_id,vorder_id,(vamount*–1),pdte
                  ,(select drcoa# from posting
                  where posting='PurchaseInvoice')
                 ,(select crcoa# from posting
                 where posting='PurchaseInvoice'))
                 returning transaction_id into
                   vtransaction_id;

      insert into cashbook values(cheque_seq.nextval
            ,(vamount*–1),pdte,'Purchase Invoice
            '||to_char(vtransaction_id)
                  ,(select drcoa# from posting
                  where posting='PurchaseInvoice_CashBook')
                  ,(select crcoa# from posting
                  where posting='PurchaseInvoice_CashBook')
                  ,vtransaction_id)
                  returning cheque_id into vcheque_id;

      for vseq# in 1..vlines loop

            insert into stockmovement values(stockmovement_
               seq.nextval
            ,vstock_id,vqty,vprice,pdte)
            returning stockmovement_id into vstockmovement_id;

            insert into ordersline values(vorder_id,vseq#
            ,(vprice*–1),vstockmovement_id);

            insert into transactionsline
            values(vtransaction_id,vseq#
            ,(vprice*–1),vstockmovement_id);

            insert into cashbookline values(vcheque_id,
               vseq#,(vprice*–1));

      end loop;

      dbms_output.put_line('PROC: InsertSANDP
            Purchase Complete '||to_char(getTime (vtime)));

end if;

      –make sale, gen transaction and pay for it for
         vqty – vqtyonhand
      getPrice(vstock_id,vsupplier_id,vprice);
      vprice := vprice * vmarkup;
      vamount := vqty * vprice;

      insert into orders values(orders_seq.nextval
      ,vcustomer_id,null,'S',vamount,pdte)
            returning order_id into vorder_id;

      insert into transactions values(transactions_seq.nextval
            ,'S',vcustomer_id,null,vorder_id,vamount,pdte
            ,(select drcoa# from posting where
               posting='SalesInvoice')
            ,(select crcoa# from posting where
               posting='SalesInvoice'))
            returning transaction_id into vtransaction_id;

      insert into cashbook values(cheque_seq.nextval
            ,vamount,pdte,'Sales Invoice '||to_char(vtrans
               action_id)
            ,(select drcoa# from posting
            where posting='SalesInvoice_CashBook')
            ,(select crcoa# from posting
            where posting='SalesInvoice_CashBook')
            ,vtransaction_id)
            returning cheque_id into vcheque_id;

      for vseq# in 1..vlines loop

            insert into stockmovement values(stockmovement_
               seq.nextval
                  ,vstock_id,(vqty*–1),vprice,pdte)
                  returning stockmovement_id into
                     vstockmovement_id;

            insert into ordersline values(vorder_id
            ,vseq#,vprice,vstockmovement_id);

            insert into transactionsline values
               (vtransaction_id
            ,vseq#,vprice,vstockmovement_id);

            insert into cashbookline values(vcheque_id,
               vseq#,vprice);

      end loop;

      dbms_output.put_line('PROC: InsertSANDP Sale
         Complete
      '||to_char(getTime(vtime)));
      commit;
exception when others then
      dbms_output.put_line('PROC: InsertSANDP '||SQLERRM
         (SQLCODE));
      rollback;
end;
/
alter procedure InsertSANDP compile;

create or replace procedure UpdateSORP (pdte IN date
   DEFAULT SYSDATE) as
      vminorder_id integer;
      vmaxorder_id integer;
      vorder_id integer;
      vtransaction_id integer;
      vcheque_id integer;
      voption integer;
      vmod float;
      vtime float;
begin
      –not supposed to execute a stockmovement (too
         difficult)
      vtime := to_number(to_char(systimestamp,'SSSSS.FF'));
            select min(order_id),max(order_id) into
            vminorder_id,vmaxorder_id from orders;
      vorder_id := random(vmaxorder_id
         vminorder_id)+vminorder_id;
      select order_id into vorder_id from orders
            where order_id = vorder_id;
      voption := random(2);
      if voption <= 1 then vmod := 1.1;
      else vmod := 0.9; end if;

      update ordersline set amount = amount*vmod
            where order_id = vorder_id;
      update orders set dte = pdte, amount =
            (select sum(amount) from Ordersline
            where order_id = vorder_id group by order_id)
      where order_id = vorder_id;

      select transaction_id into vtransaction_id from
         transactions where order_id = vorder_id;
      update transactionsline set amount = amount*vmod
      where transaction_id = vtransaction_id;
      update transactions set dte = pdte, amount =
            (select sum(amount) from transactionsline
            where transaction_id = vtransaction_id
            group by transaction_id)
      where transaction_id = vtransaction_id;

      select cheque_id into vcheque_id from cashbook
      where transaction_id = vtransaction_id;
      update cashbookline set amount = amount*vmod
      where cheque_id = vcheque_id;
      update cashbook set dte = pdte, amount =
            (select sum(amount) from cashbookline
            where cheque_id = vcheque_id group by cheque_id)
      where cheque_id = vcheque_id;

      dbms_output.put_line('PROC: UpdateSORP
         Complete '||to_char(getTime(vtime)));
      commit;

      exception when others then
            dbms_output.put_line('PROC: UpdateSORP '||SQLERRM
               (SQLCODE));rollback;
      end;
/
alter procedure UpdateSORP compile;

create or replace procedure DeleteSORP(pdte IN DATE DEFAULT
   SYSDATE) as
      vminorder_id integer;
      vmaxorder_id integer;
      vorder_id integer;
      vtransaction_id integer;
      vstockmovement_id integer;
      vstock_id integer;
      vcheque_id integer;
      vqty integer;
      vprice float;
      vid integer;
      vtime float;
      type tStockMovement is ref cursor return
         StockMovement%rowtype;
      cStockMovements tStockMovement;
      rStockMovement StockMovement%rowtype;
begin

      vtime := to_number(to_char(systimestamp,'SSSSS.FF'));

      select min(order_id),max(order_id) into
         vminorder_id,vmaxorder_id from orders;
      vorder_id := random(vmaxorder_id-vminorder_id)+ vminorder_id;
         select order_id into vorder_id from orders
      where order_id = vorder_id;
      select transaction_id into vtransaction_id from
         transactions where order_id = vorder_id;
      select cheque_id into vcheque_id from cashbook
      where transaction_id = vtransaction_id;

      begin
            open cStockMovements for
                  select stockmovement_seq.nextval, sm.stock_id
                  ,(sm.qty*–1) as qty, sm.price, pdte
                  from stockmovement sm
                  where exists(
                        select stockmovement_id from
                           transactionsline
                        where stockmovement_id =
                           sm.stockmovement_id and
                           transaction_id = vtransaction_id);
            loop
                     fetch cStockMovements into
                        rStockMovement;
                     exit when cStockMovements%NOTFOUND;
                     insert into stockmovement values
                     (rStockMovement.stockmovement_id
                     ,rStockMovement.stock_id,rStockMovement.
                        qty
                     ,rStockMovement.price,rStockMovement.dte);
            end loop;
            close cStockMovements;
      exception when others then
            close cStockMovements;
      end;

      delete from cashbookline where cheque_id =
         vcheque_id;
      delete from cashbook where cheque_id = vcheque_id;
      delete from transactionsline
            where transaction_id = vtransaction_id;
      delete from ordersline where order_id = vorder_id;
      delete from transactions where transaction_id =
         vtransaction_id;
      delete from orders where order_id = vorder_id;

      dbms_output.put_line('PROC: DeleteSORP Complete
         '||to_char(getTime(vtime)));
      commit;

exception when others then
      dbms_output.put_line('PROC: DeleteSORP
         '||SQLERRM(SQLCODE));
      rollback;

end;
/
alter procedure DeleteSORP compile;

create or replace procedure simulate as
      voption integer;
      vrange integer;
      vdte date default '01-MAR-03';
begin
      voption := random(10); vdte := vdte + random(366);
      dbms_output.put_line('PROC: Simulate
         '||to_char(vdte)||' '||to_char(voption));
      if voption in (1,2,3,4,5,6,7) then InsertSANDP(vdte);
      elsif voption in (8,9) then UpdateSORP(vdte);
      elsif voption in (10) then DeleteSORP(vdte);
      end if;
exception when others then
      dbms_output.put_line('PROC: Simulate '
         ||SQLERRM(SQLCODE));
end;
/
alter procedure simulate compile;

Internet OLTP and Reporting Simulation

create or replace procedure Reports (popt IN integer) as
      vcustomer_id integer;
      vsupplier_id integer;
      vorder_id integer;
      vtransaction_id integer;
      vstock_id integer;
      vtime float;
      vcnt integer;

      type tOrders is ref cursor return Orders%rowtype;
      cOrders tOrders;
      rOrders Orders%rowtype;

      type tOrdersLine is ref cursor return OrdersLine%row
         type;
      cOrdersLine tOrdersLine;
      rOrdersLine OrdersLine%rowtype;

      type tTransactions is ref cursor return
         Transactions%rowtype;
      cTransactions tTransactions;
      rTransactions Transactions%rowtype;

      type tTransactionsLine is ref cursor return
            TransactionsLine%rowtype;
      cTransactionsLine tTransactionsLine;
      rTransactionsLine TransactionsLine%rowtype;

      type tCashbook is ref cursor return Cashbook%rowtype;
      cCashbook tCashbook;
      rCashbook Cashbook%rowtype;

      type tCashbookLine is ref cursor return
         CashbookLine%rowtype;
      cCashbookLine tCashbookLine;
      rCashbookLine CashbookLine%rowtype;

      type tCOA is ref cursor return COA%rowtype;
      cCOA tCOA;
      rCOA COA%rowtype;

      type tStockMovement is ref cursor return
         StockMovement%rowtype;
      cStockMovement tStockMovement;
      rStockMovement StockMovement%rowtype;

      voption integer;

begin

      vtime := to_number(to_char(systimestamp,'SSSSS.FF'));

      voption := rand(8);

      if voption = 1 then
            —print all orders
      select max(customer_id) into vcustomer_id from
         customer;
      vcustomer_id := rand(vcustomer_id);
      open cOrders for select * from orders
      where customer_id = vcustomer_id;
      loop
            fetch cOrders into rOrders;
            exit when cOrders%notfound;
      end loop;
      select max(supplier_id) into vsupplier_id from
         supplier;
      vsupplier_id := rand(vsupplier_id);
      open cOrders for select * from orders
      where supplier_id = vsupplier_id;
      loop
            fetch cOrders into rOrders;
            exit when cOrders%notfound;
      end loop;

elsif voption = 2 then
      —print individual orders
      select max(customer_id) into vcustomer_id from
         customer;
      vcustomer_id := rand(vcustomer_id);
      select max(order_id) into vorder_id from
         orders
      where customer_id = vcustomer_id;
      vorder_id := rand(vorder_id);
      open cOrdersLine for select ol.* from orders o
            join ordersline ol on(ol.order_id =
               o.order_id)
            where o.customer_id = vcustomer_id;
      loop
            fetch cOrdersLine into rOrdersLine;
            exit when cOrdersLine%notfound;
      end loop;
      select max(supplier_id) into vsupplier_id from
         supplier;
      vsupplier_id := rand(vsupplier_id);
      select max(order_id) into vorder_id from orders
      where supplier_id = vsupplier_id;
      vorder_id := rand(vorder_id);
      open cOrdersLine for select ol.* from
         orders o
            join ordersline ol on(ol.order_id =
               o.order_id)
            where o.supplier_id = vsupplier_id;
      loop
            fetch cOrdersLine into rOrdersLine;
            exit when cOrdersLine%notfound;
      end loop;

elsif voption = 3 then
      —print all transactions
      select max(customer_id) into vcustomer_id from
         customer;
      vcustomer_id := rand(vcustomer_id);
      open cTransactions for select * from transactions
      where customer_id = vcustomer_id;
      loop
            fetch cTransactions into rTransactions;
            exit when cTransactions%notfound;
      end loop;
      select max(supplier_id) into vsupplier_id from
         supplier;
      vsupplier_id := rand(vsupplier_id);
      open cTransactions for select * from transactions
            where supplier_id = vsupplier_id;
      loop
            fetch cTransactions into rTransactions;
            exit when cTransactions%notfound;
      end loop;

elsif voption = 4 then
      —print individual transactions
      select max(customer_id) into vcustomer_id from
         customer;
      vcustomer_id := rand(vcustomer_id);
      select max(transaction_id) into vtransaction_id
            from transactions where customer_id =
               vcustomer_id;
      vtransaction_id := rand(vtransaction_id);
      open cTransactionsLine for select tl.* from
         transactions t
            join transactionsline tl
                  on(tl.transaction_id =
                     t.transaction_id)
            where t.customer_id = vcustomer_id;
      loop
            fetch cTransactionsLine into
               rTransactionsLine;
            exit when cTransactionsLine%notfound;
      end loop;
      select max(supplier_id) into vsupplier_id from
         supplier;
      vsupplier_id := rand(vsupplier_id);
      select max(transaction_id) into vtransaction_id
            from transactions where supplier_id =
            vsupplier_id;
      vtransaction_id := rand(vtransaction_id);
      open cTransactionsLine for select tl.* from
         transactions t
            join transactionsline tl
                  on(tl.transaction_id =
                     t.transaction_id)
            where t.supplier_id = vsupplier_id;
      loop
            fetch cTransactionsLine into
               rTransactionsLine;
            exit when cTransactionsLine%notfound;
      end loop;

elsif voption = 5 then
      —print all invoices
      select max(customer_id) into vcustomer_id from
         customer;
      vcustomer_id := rand(vcustomer_id);
      select max(transaction_id) into vtransaction_id
            from transactions where customer_id =
               vcustomer_id;
      vtransaction_id := rand(vtransaction_id);
      open cCashbook for select cb.* from
         transactions t
            join cashbook cb
                  on(cb.transaction_id =
                     t.transaction_id)
            where t.customer_id = vcustomer_id;
      loop
            fetch cCashbook into rCashbook;
            exit when cCashbook%notfound;
      end loop;
      select max(supplier_id) into vsupplier_id from
         supplier;
      vsupplier_id := rand(vsupplier_id);
      select max(transaction_id) into
         vtransaction_id
      from transactions where supplier_id =
         vsupplier_id;
      vtransaction_id := rand(vtransaction_id);
      open cCashbook for select cb.* from
         transactions t
            join cashbook cb
                  on(cb.transaction_id =
                     t.transaction_id)
            where t.supplier_id = vsupplier_id;
      loop
            fetch cCashbook into rCashbook;
            exit when cCashbook%notfound;
      end loop;

elsif voption = 6 then
      —print individual invoices
      select max(customer_id) into vcustomer_id from
         customer;
      vcustomer_id := rand(vcustomer_id);
      select max(transaction_id) into vtransaction_id
      from transactions where customer_id =
         vcustomer_id;
      vtransaction_id := rand(vtransaction_id);
      open cCashbookLine for select cbl.* from
         transactions t
               join transactionsline tl
                     on(tl.transaction_id =
                        t.transaction_id)
                     join cashbook cb
                           on(cb.transaction_id =
                              t.transaction_id)
                           join cashbookline cbl
                           on(cbl.cheque_id =
                              cb.cheque_id)
            where t.customer_id = vcustomer_id;
      loop
      fetch cCashbookLine into rCashbookLine;
            exit when cCashbookLine%notfound;
            end loop;
      select max(supplier_id) into vsupplier_id from
         supplier;
      vsupplier_id := rand(vsupplier_id);
      select max(transaction_id) into vtransaction_id
      from transactions where supplier_id =
         vsupplier_id;
      vtransaction_id := rand(vtransaction_id);
      open cCashbookLine for select cbl.* from
         transactions t
               join transactionsline tl
                     on(tl.transaction_id =
                        t.transaction_id)
                     join cashbook cb
                           on(cb.transaction_id =
                              t.transaction_id)
                           join cashbookline cbl
                                 on(cbl.cheque_id =
                                    cb.cheque_id)
            where t.supplier_id = vsupplier_id;
      loop
            fetch cCashbookLine into rCashbookLine;
            exit when cCashbookLine%notfound;
      end loop;

elsif voption = 7 then
      —coa
      open cCOA for select c.* from coa c, type t,
         subtype s
            where c.type = t.type and c.subtype =
               s.subtype
            order by c.coa#, t.type, s.subtype;
      loop
            fetch cCOA into rCOA;
            exit when cCOA%notfound;
      end loop;

elsif voption = 8 then
      —stock
      select max(stock_id) into vstock_id from stock;
      vstock_id := rand(vstock_id);
      open cStockMovement for select sm.* from
         category c
            join stock s on(s.category_id =
            c.category_id)
            join stockmovement sm on(sm.stock_id =
               s.stock_id)
      where s.stock_id = vstock_id;
      loop
            fetch cStockMovement into rStockMovement;
            exit when cStockMovement%notfound;
      end loop;

elsif voption in (9,10) then
      —sorting
      select count(*) into vcnt from
            (select * from customer order by address);
      select count(*) into vcnt from
            (select drcoa#, sum(amount)
            from transactions group by drcoa#);
      —selections from the generalledger and
         stockmovement
      —tables are avoided here due to reverse index
         primary keys
end if;

dbms_output.put_line('PROC: Reports Complete '
      ||to_char(getTime(vtime)));
      simAudit('PROCEDURE','Reports',getTime(vtime),
         popt);

exception when others then
      dbms_output.put_line('PROC: Reports '||SQLERRM(SQL-
      CODE));
      rollback;
end;
/
alter procedure Reports compile;

create or replace procedure simulate as
      voption integer;
      vrange integer;
      vdte date default '01-MAR-03';
begin
      voption := rand(20); vdte := vdte + rand(366);
      dbms_output.put_line('PROC: Simulate ('||to_char
         (voption)||')'
               ||to_char(vdte)||' '||to_char(voption));
      if voption in (1,2,3,4,5,6,7,8,9,10) then
         Reports(voption);
      elsif voption in (11,12,13,14,15) then
      InsertSANDP(vdte,voption);
      elsif voption in (16,17,18) then UpdateSORP(vdte,
         voption);
      elsif voption in (19,20) then DeleteSORP(vdte,voption);
      end if;
exception when others then
      dbms_output.put_line('PROC: Simulate '||SQLERRM
         (SQLCODE));
end;
/
alter procedure simulate compile;
Tip

Pinning all these procedures into the shared pool using the DBMS_SHARED_POOL package KEEP procedure would be sensible. I have not done so because I am attempting to simulate a tunable environment.


Updating Summary Fields

create or replace procedure summaries as
      vtime float;
begin
      vtime := to_number(to_char(systimestamp,'SSSSS.FF'));

      —customer.balance
      update customer set balance =
      (
            select a.amount from
            (
                  select customer_id, sum(t.amount
                     cb.amount) as amount
                  from transactions t, cashbook cb
                  where t.transaction_id =
                     cb.transaction_id
                  group by customer_id
            ) a
            where a.customer_id = customer.customer_id
      );
      commit;

      —customer.lastdate
      update customer set lastdate =
      (
            select a.dte from
            (
                  select customer_id, max(dte) as dte
                  from transactions
                  group by customer_id
            ) a
            where a.customer_id = customer.customer_id
      );
      commit;

      —supplier.balance
      update supplier set balance =
      (
            select a.amount from
            (
                  select supplier_id, sum(t.amount
                     cb.amount) as amount
                  from transactions t, cashbook cb
                  where t.transaction_id =
                     cb.transaction_id
                  group by supplier_id
            ) a
            where a.supplier_id = supplier.supplier_id
      );
      commit;

      —supplier.lastdate
      update supplier set lastdate =
      (
            select a.dte from
            (
                  select supplier_id, max(dte) as dte
                  from transactions
                  group by supplier_id
            ) a
            where a.supplier_id = supplier.supplier_id
      );
      commit;

      —coa.lastdate
      update coa set lastdate =
      (
            select a.dte from
            (
                  select coa#, max(dte) as dte
                  from generalledger
                  group by coa#
            ) a
            where a.coa# = coa.coa#
      );
      commit;

      —periodsum
      insert into tmp_periodsum(year,period,coa#,amount)
            select to_number(to_char(dte,'IYYY')) as year
                  ,to_number(to_char(dte,'MM')) as period
                  ,coa# as coa
                  ,abs(sum(dr-cr)) as amount
            from generalledger
            group by to_number(to_char(dte,'IYYY'))
                  ,to_number(to_char(dte,'MM')), coa#;
      delete from periodsum;
      insert into periodsum
                  select * from tmp_periodsum where period
                     in(1,2);
      delete from tmp_periodsum where period in(1,2);
      update periodsum set year=year–1,period=period+10;
      update tmp_periodsum set period=period–2;
      insert into periodsum select * from tmp_periodsum;
      commit;

      —coa.balance
      update coa set balance =
      (
            select amount from periodsum
            where year=2003 and period=1 and coa#=coa.coa#
      );
      commit;

      update coa set ytd =
      (
            select sum(amount) from periodsum
            where year=2003 and coa#=coa.coa#
      );
      commit;

      —stock.lastdate
      update stock set lastdate =
      (
            select a.dte from
            (
                  select stock_id, max(dte) as dte
                  from stockmovement
                  group by stock_id
            ) a
            where a.stock_id = stock.stock_id
      );
      commit;

      —stock.qtyonhand
      update stock set qtyonhand =
      (
            select a.qty from
            (
                  select stock_id, sum(qty) as qty
                  from stockmovement
                  group by stock_id
            ) a
            where a.stock_id = stock.stock_id
      );
      commit;

      —stock.totalvalue
      update stock set totalvalue =
      (
            select a.amount from
            (
                  select stock_id, sum(qty*price) as amount
                  from stockmovement
                  group by stock_id
            ) a
            where a.stock_id = stock.stock_id
      );
      commit;

      dbms_output.put_line('PROC: UpdateSummaries Complete '
            ||to_char(getTime(vtime)));
      simAudit('PROCEDURE','UpdateSummaries',getTime(vtime),
         0);
exception when others then
      dbms_output.put_line('PROC: UpdateSummaries
         '||SQLERRM(SQLCODE));
      rollback;
end;
/
alter procedure summaries compile;
/


Partitioning Tablespace Creation

These commands create tablespaces for separate partitions created on the Accounts schema GeneralLedger table, used by partitioning examples in Chapter 17. Other examples in Chapter 17 creating partitions for other tables use similar tablespace creation scripts. Not all scripts are included here.

create tablespace DATAGLP1999 logging datafile
'c:\oracle\oradata\dual\dataglP1999.dbf' size 6M autoextend
on next 1M maxsize unlimited extent management local uniform
size 1M;

create tablespace DATAGLP2000 logging datafile
'c:\oracle\oradata\dual\dataglP2000.dbf' size 6M autoextend
on next 1M maxsize unlimited extent management local uniform
size 1M;

create tablespace DATAGLP2001 logging datafile
'd:\oracle\oradata\dual\dataglP2001.dbf' size 6M autoextend
on next 1M maxsize unlimited extent management local uniform
size 1M;
create tablespace DATAGLP2002 logging datafile
'e:\oracle\oradata\dual\dataglP2002.dbf' size 6M autoextend
on next 1M maxsize unlimited extent management local uniform
size 1M;

create tablespace INDXGLP1999 logging datafile
'c:\oracle\oradata\dual\indxglP1999.dbf' size 2M autoextend
on next 1M maxsize unlimited extent management local uniform
size 1M;

create tablespace INDXGLP2000 logging datafile
'c:\oracle\oradata\dual\indxglP2000.dbf' size 2M autoextend
on next 1M maxsize unlimited extent management local uniform
size 1M;

create tablespace INDXGLP2001 logging datafile
'd:\oracle\oradata\dual\indxglP2001.dbf' size 2M autoextend
on next 1M maxsize unlimited extent management local uniform
size 1M;

create tablespace INDXGLP2002 logging datafile
'e:\oracle\oradata\dual\indxglP2002.dbf' size 2M autoextend
on next 1M maxsize unlimited extent management local uniform
size 1M;

alter user accounts quota unlimited on DATAGLP1999
      quota unlimited on DATAGLP2000 quota unlimited
         on DATAGLP2001
      quota unlimited on DATAGLP2002;

alter user accounts quota unlimited on INDXGLP1999
      quota unlimited on INDXGLP2000 quota unlimited
         on INDXGLP2001
      quota unlimited on INDXGLP2002;


Appendix B: Sample Scripts

Interpreting EXPLAIN PLAN

All of these scripts SELECT rows from the PLAN_TABLE containing Optimizer query plans generated by the EXPLAIN PLAN command.

explain plan set statement_id='TEST' for <SQL Statement;
–a simple version
COL Cost FORMAT 9990;
COL Rows FORMAT 999990;
COL Bytes FORMAT 99999990;
COL Query FORMAT a64;
SELECT cost "Cost", cardinality "Rows", bytes "Bytes"
      ,operation||' '||options||' on '||object_name "Query"
FROM plan_table ORDER BY id;

–for serial queries
COL Query FORMAT a48;
COL Pos FORMAT 990;
COL Cost FORMAT 990;
COL Rows FORMAT 999990;
COL Bytes FORMAT 99999990;
COL Sort FORMAT 99999990;
COL IO FORMAT 9999990;
COL CPU FORMAT 99999990;
SELECT TRIM(LEVEL)||'. '||LPAD (' ', LEVEL – 1)||operation
      ||' '||options||' on '||object_name "Query"
      ,cost "Cost", cardinality "Rows", bytes "Bytes"
      ,decode(level,1,0,position) "Pos", temp_space "Sort"
      ,io_cost "IO", cpu_cost "CPU"
FROM plan_table WHERE statement_id = 'TEST'
CONNECT BY prior id = parent_id AND prior statement_id =
   statement_id
START WITH id = 0 AND statement_id = 'TEST' ORDER BY id;
delete from plan_table where statement_id='TEST';
commit;

–a parallel query version
COL Query FORMAT a60;
COL Pos FORMAT 990;
COL Cost FORMAT 990;
COL Rows FORMAT 999990;
COL Bytes FORMAT 99999990;
COL Sort FORMAT 99999990;
COL IO FORMAT 9999990;
COL CPU FORMAT 99999990;
SELECT TRIM(LEVEL)||'. '||LPAD (' ', LEVEL – 1)||operation||
      ' '||options||' on '||object_name||' '||other_tag||'
      '||partition_start||' '||partition_stop "Query"
      ,cost "Cost", cardinality "Rows", bytes "Bytes"
      ,decode(level,1,0,position) "Pos"
      ,temp_space "Sort", io_cost "IO", cpu_cost "CPU"
FROM plan_table WHERE statement_id = 'TEST'
CONNECT BY prior id = parent_id AND prior statement_id =
   statement_id
START WITH id = 0 AND statement_id = 'TEST' ORDER BY id;
delete from plan_table where statement_id='TEST';
commit;

Using the DBMS_XPLAN package.

–serial
SELECT PLAN_TABLE_OUTPUT FROM
TABLE(DBMS_XPLAN.DISPLAY('plan_table',null,'serial'));

–parallel
SELECT * FROM TABLE(DBMS_XPLAN.DISPLAY());


Statistics Generation

Oracle recommends using the DBMS_STATS package to generate statistics with the latest version of Oracle. A similar type of script can be created using the DBMS_STATS package.

set termout off echo off feed off trimspool on head off
   pages 0;
spool c:\tmp\statistics.sql;
select 'analyze '||object_type||' '||object_name||'
      compute statistics;'
from user_objects where object_type in ('TABLE','INDEX')
order by object_type,object_name;
spool off;
set termout on;
@@c:\tmp\statistics.sql;
exit;


Count Rows

set termout off echo off feed off trimspool on head off
   pages 0;
spool C:\temp\counts.log;
set column table_name format a16;
select 'SELECT '''||table_name||','',
   TO_CHAR(count(*),''999,999,990'')
from '||table_name||';' from user_tables;
spool off;
set termout on;
@@C:\temp\counts.log;
set termout on echo on feed on trimspool off head
   on pages 40;


Constraints

set wrap off linesize 132 pages 80;
column key format a10;
column pos format 990;
column col format a10;
column cons format a20;
column tab format a20;
column own format a10;
select decode(t.constraint_type,'P','Primary','R','Foreign'
      ,'U','Alternate','Unknown') "Key"
      ,t.table_name "Tab", t.constraint_name "Cons"
      ,c.column_name "Col", c.position "Pos"
from user_constraints t, user_cons_columns c
where t.constraint_type in ('P','R','U')
and t.table_name = c.table_name
and t.constraint_name = c.constraint_name
order by t.table_name, t.constraint_type, c.position;


Indexes

set wrap off linesize 132 pages 80;
column pos format 990;
column col format a10;
column ind format a25;
column tab format a25;
column typ format a20;
column tbs format a25;
select t.table_name "Tab"
      ,decode(t.index_type,'NORMAL','BTree','BITMAP','Bitmap'
            ,'FUNCTION-BASED NORMAL','Function-Based BTree'
            ,t.index_type) "Typ"
      ,t.index_name "Ind", c.column_name "Col",
         c.column_position "Pos"
      ,t.tablespace_name "Tbs"
from user_indexes t, user_ind_columns c
where t.table_name = c.table_name
and t.index_name = c.index_name
and t.index_type not in ('IOT – TOP','LOB')
order by t.table_name, t.index_name, c.column_position;


Space in the Database

–database size
SELECT 'Database Size' "*****"
      ,round(sum(round(sum(nvl(fs.bytes/1024/1024,0)))) /
        sum(round(sum(nvl(fs.bytes/1024/1024,0))) + round
        (df.bytes/1024/1024 – sum(nvl(fs.bytes/1024/1024,0))))
        * 100, 0) "%Free"
      ,round(sum(round(df.bytes/1024/1024 –
        sum(nvl(fs.bytes/1024/1024,0)))) / sum(round(sum
        (nvl(fs.bytes/1024/1024,0))) + round(df.bytes/1024/
        1024 – sum(nvl(fs.bytes/1024/1024,0)))) * 100, 0)
        "%Used"
      ,sum(round(sum(nvl(fs.bytes/1024/1024,0)))) "Mb Free"
      ,sum(round(df.bytes/1024/1024
      – sum(nvl(fs.bytes/1024/1024,0)))) "Mb Used"
      ,sum(round(sum(nvl(fs.bytes/1024/1024,0))) + round
        (df.bytes/1024/1024
      – sum(nvl(fs.bytes/1024/1024,0)))) "Size"
FROM dba_free_space fs, dba_data_files df
WHERE fs.file_id(+) = df.file_id
GROUP BY df.tablespace_name, df.file_id, df.bytes,
   df.autoextensible
ORDER BY df.file_id;

–tablespace size
COL Tablespace FORMAT a16;
SELECT df.tablespace_name "Tablespace"
      ,round((sum(nvl(fs.bytes,0))/ (df.bytes)) * 100)
         "%Free"
      ,round(((df.bytes – sum(nvl(fs.bytes,0)))
      / (df.bytes) ) * 100) "%Used"
      ,round(sum(nvl(fs.bytes/1024/1024,0))) "Mb Free"
      ,round(df.bytes/1024/1024
      – sum(nvl(fs.bytes/1024/1024,0))) "Mb Used"
FROM dba_free_space fs, dba_data_files df
WHERE fs.file_id(+) = df.file_id
GROUP BY df.tablespace_name, df.file_id, df.bytes,
   df.autoextensible
ORDER BY df.file_id;

–extent size
COL Object FORMAT a24;
COL Type FORMAT a5;
SELECT segment_name "Object", segment_type "Type"
      ,ROUND(SUM(bytes)/1024/1024) "Mb", ROUND
         (SUM(bytes)/1024) "Kb"
      ,SUM(bytes) "Bytes", SUM(blocks) "Blocks"
FROM dba_extents
WHERE owner = 'ACCOUNTS' AND segment_type IN ('TABLE','INDEX')
GROUP BY segment_name, segment_type
ORDER BY segment_name, segment_type DESC;

–segment size
COL Object FORMAT a24;
COL Type FORMAT a5;
SELECT segment_name "Object", segment_type "Type"
      ,ROUND(bytes/1024/1024) "Mb", ROUND(bytes/1024) "Kb"
      ,bytes "Bytes", blocks "Blocks"
FROM dba_segments WHERE owner = 'ACCOUNTS'
AND segment_type IN ('TABLE','INDEX')
ORDER BY segment_name, segment_type DESC;


TKPROF Trace Files Interpretation Scripts

A Perl Wrapper Script

Tip

Please do not use this script without testing.

$source = shift; #a trace path and file name

open(trace,$source) || die "Cannot open file $source\n";
   @lines = <trace; close (trace); $len = @lines;
@p = split("\\.",$source); $output = $p[0].".out";
open(output,">$output");

$x = 0; $i = 0; while ($i < $len)
{
      if (($lines[$i] =~ /^.*SELECT.*$/i)
      || ($lines[$i] =~ /^.*INSERT.*$/i)
      || ($lines[$i] =~ /^.*UPDATE.*$/i)
      || ($lines[$i] =~ /^.*DELETE.*$/i))
            { $x = $i; }

      if ($lines[$i] =~ /^.*TABLE ACCESS FULL DUAL.*$/i)
            { $i++; next; }

      if (($lines[$i] =~ /^.*SORT MERGE.*$/i)
      || ($lines[$i] =~ /^.*TABLE ACCESS FULL.*$/i))
      {
            $y = $i;
            while ($lines[$i] ne "\n") { $y = $i; $i++; }
            print output "--------------------------------
--------------------------------\n";
            for ($j = $x; $j <= $y; $j++)
            {
                  if ($lines[$j] ne "\n")
                  {
                        if (
                              ($lines[$j] =~ /^call.*count.*$/)
                              || ($lines[$j] =~ /^Parse.*$/)
                              || ($lines[$j] =~ /^Execute .*$/)
                              || ($lines[$j] =~ /^Fetch.*$/)
                              || ($lines[$j] =~ /^total.*$/)
                              || ($lines[$j] =~ /^—-.*$/)
                              || ($lines[$j] =~ /^Misses.*$/)
                              || ($lines[$j] =~ /^Optimizer.*$/)
                              || ($lines[$j] =~ /^Parsing.*$/)
                        ){ next; }
                        print output "$lines[$j]";
                  }
            }
      }
      $i++;
}
close(output);


The TKPROF Interpretation Script

Tip

Please do not use this script without testing.

#!/bin/ksh

#execute profile – for example .cshrc

explain=$1 #set the explain variable to a non-null value if
   required.

if [ –d /tmp/tkprof ]; then sleep 0; else mkdir
   /tmp/tkprof; fi

if [ –z "$explain" ]; then

      ls –lat $ORACLE_TRACE/$ORACLE_SID*.trc\
      | awk '{print $9}'\
      | awk –F/ '{print $8}'\
      | awk –F. '{\
            printf("tkprof %s/%s.trc\
            /tmp/tkprof/%s.prf\n",oracle_trace,$1,$1)\
      }' oracle_trace=$ORACLE_TRACE
else
      ls –lat $ORACLE_TRACE/$ORACLE_SID*.trc\
      | awk '{print $9}'\
      | awk –F/ '{print $8}'\
      | awk –F. '{\
            printf("tkprof %s/%s.trc /tmp/tkprof/%s.prf\
            explain=<owner>/<password>
               table=<owner>.plan_table\
            sys=no record=/tmp/tkprof/%s.rec\n",
               oracle_trace,$1,$1,$1)\
      }' oracle_trace=$ORACLE_TRACE

fi

ls –lat /tmp/tkprof/$ORACLE_SID*.prf\
| awk '{ printf("perl tkprof.pl %s\n",$9) }'
exit 0


Appendix C: Sources of Information

  • The author of this book can be contacted on the following email addresses:

    • oracledbaexpert@earthlink.net

    • gjtpowell@earthlink.net

  • Oracle Technet at http://technet.oracle.com is an excellent source for entire Oracle reference documentation sets.

  • Metalink at http://metalink.oracle.com is also excellent and a source of current information from support calls, questions and answers placed by both Oracle users and Oracle support staff. The information on this site is well worth the Oracle licensing fees required.

  • Search for a term such as "free buffer waits" in search engines such http://www.yahoo.com. Be aware that not all information will be current and might be incorrect. Verify on Oracle Technet. If no results are found using Yahoo try the full detailed listings on http://www.google.com.

  • Try http://www.amazon.com and http://www.barnesandnoble.com where many Oracle titles can be found.


Previous Section
< Day Day Up >
Next Section

Index

A

Accounts schema:
count of rows, 126
ERD version,
Denormalized, 698
Normalized, 699
Oracle Expert analysis, 380–83
and partitioning methods, 548–56
and SQL subqueries, 175–78
tuning, 594–95


Previous Section
< Day Day Up >
Next Section

Index

B

Bitmap indexes, 210–19
composite indexes, 213–15
defined, 210
faster than BTree indexes, 211–13
and overflow, 217–19
BTree indexes, 206–9, 220–36
compressed,
composite indexes, 224–28
indexes and DML activity, 228–30
function-based indexes, 230–33
lost index space, 222–23
NULLS and indexes, 233–35
in Oracle database, 207
overflow and rebuilding, 221–22
reverse key indexes, 223–24
summary of indexing, 235–36


Previous Section
< Day Day Up >
Next Section

Index

C

Configuration tuning, 16–17
Constraints, use of, 48–55, 721
COUNT function, 126, 143–45, 277, 721
Creating a database, 405–26 see also Installing Oracle database
Database Configuration Assistant, 406–13
current trends, 412–13
dedicated or shared servers, 409
default structure, 413
memory buffer parameters, 410
optional features, 407
redo logs, 412
sort buffer, 411
tablespaces, 411
manual creation, 413–26
automated storage management, 425–26
catalog and SPFILE, 424–25
create the database, 415–17
directory structure, 414–15
tablespace and rollback, 417–23


Previous Section
< Day Day Up >
Next Section

Index

D

Data access methods, 240–41, 256–301 see also Indexing
cluster and hash scans, 281–82
full table scans, 256–66
reading deleted rows, 262–64
using PARALLEL, 264–66
index scans, 268–81
bitmap join, 281
COUNT function, 277
DISTINCT clause, 276–77
fast full scan, 275–76
full scan, 272–75
index join, 280
IS NOT NULL comparison, 277–78
parallel scan, 278–80
range scan, 269–71
skip scan, 271–72
unique scan, 269
joining tables, 282–301
cartesian join, 298
hash join, 289–90
join order execution, 283
mutable join nesting, 293–97
nested loop join, 285–89
outer join, 299–301
semi join, 297–98
sort merge join, 290–93
types of join, 283–85
ROWID scans, 267
sample table scans, 266–67
Data definition language see DDL
Data manipulation language see DML
Data model, definition, xii–xiii
Database Configuration Assistant see Creating a database
Database samples, 695–718
active concurrent database scripting, 695–712
internet OLTP and reporting simulation, 706–13
internet OLTP database simulation, 695–706
partitioning tablespace creation, 716–17
schemas, 695, 696, 697, 698, 699
updating summary fields, 713–16
DDL (data definition language), definition, 99–100
Denormalization of a database, 61–80
adding composite keys, 67–68
definition, 64
mutable and complex joins, 65–68
by reversing normal forms, 70–74
employees schema, 73
2nd normal form, 74
3rd normal form, 72
4th normal form, 71
useful tricks, 74–79
application caching, 79
copying columns between entities, 75–76
placing summary columns into parent entities, 76–77
separating active and inactive data, 77–78
using views, 78–79
using Oracle database objects, 79–80
DML (data manipulation language), definition, 99–100, 228–30


Previous Section
< Day Day Up >
Next Section

Index

E

Employees schema, 73, 81–82, 88
ERD version,
one, 696
two, 697
Enterprise Manager Diagnostics Pack, 625–34
Enterprise Manager Tuning Pack, 365–84
Index Tuning Wizard, 365–72
select schemas, 369
select the application type, 368
wizard recommendations, 370
Oracle Expert, 380–82
Optimizer query plans, 381–82
recommendations for the Accounts schema, 383
SQL Access Advisor, 382
SQL Analyze, 372–80
analysis stage, 373
comparing different versions, 380
index recommendations, 377
query plan, 376
query plan assessment, 378
SQL statement assessment, 374
SQL text display, 375
virtual indexing, hints, and tuning, 379
and Wait Event Interface, 665–69
EXPLAIN PLAN:
sample script, 719–20
use to detect problem queries, 16, 328–35


Previous Section
< Day Day Up >
Next Section

Index

F

File structure layers, 427–52
file system layer, 430–37
control files, 434
tuning datafiles, 432–34
tuning redo logs and archive logs, 434–37
types of files, 431–32
logical layer, 438–52
auto extend, 442
automatic undo, 449–50
avoiding datafile header contention, 446
BIGFILE tablespaces, 445–46
block size, 442–43
dictionary managed tablespaces, 439–41
extent management, 443–44
locally managed tablespaces, 441–42
logging, 443
manual rollback, 450–52
segment space management, 444–45
temporary sort space, 446–49
networking layer, 437
Oracle Instance, 427–30
memory cache buffers, 428
processes, 428–30


Previous Section
< Day Day Up >
Next Section

Index

H

Hardware resource usage tuning, 489–526
CPU usage, 489–99
high network activity, 498–99
poor index usage, 492
poorly tuned SQL code, 491–92
rollback and undo, 493–96
row locks and latch waits, 497–98
swapping and paging, 490–91
temporary sort space, 497
I/O usage, 520–26
efficiency and recoverability, 520–23
RAID arrays, 524–25
memory usage, 499–519
advice on buffers, 515–20
automated memory management, 511
database buffer cache, 502–3, 515–18
Java pool, 514, 520
large pool, 512–13
library cache, 504–7
manual memory management, 511–12
metadata or dictionary cache, 508–9
pinning objects in the shared pool, 509
redo log buffer, 513–14
session connection cache, 509–10
shared pool, 503–4, 518–20
shared servers and virtual circuits, 513
system global area, 499–501
Hardware upgrades, xv
History of data modelling, 89–96
evolution, 90, 94
Oracle database, 94–96
roots of SQL, 96
types of models, 90–93
hierarchical, 90–91
network, 90–91
object, 92–93
object-relational, 92–93
relational, 90–92


Previous Section
< Day Day Up >
Next Section

Index

I

Indexing, 199–235, 721–22
alternate or secondary indexing, 202–3
Index Tuning Wizard, 365–72
select schemas, 369
select the application type, 368
wizard recommendations, 370
sample scripts, 721–22
types of index in Oracle database, 203–20
bitmap indexes (see Bitmap indexes)
BTree index (see BTree indexes)
clusters, 219–20
read only index, 209
syntax, 205–6
using referential integrity, 201–2
what and how to index, 199–200
when not to use indexes, 200–201
Installing Oracle database, 387–405
archiving, 401
auditing and tracing, 399–401
basic network files, 389
configuring the client, 390–91
configuring the listener, 389–90
block size, 392
check pointing, 401
control files, 392
job scheduling, 403–4
memory buffers and I/O behavior, 393–98
connection memory requirements, 395–96
database buffer cache, 393–94
Java pool, 397
large pool, 396–97
radio log buffer cache, 397–98
shared pool, 394–95
networking, 404–5
parameter file, 391–92
resource management, 403
rollback and undo, 402–3
SQL code Optimization, 398–99
using different editions, 388


Previous Section
< Day Day Up >
Next Section

Index

J

Joins in SQL, 161–72
anti joins, 171–72
cartesian products, 170
efficient joins, 165–66
equi joins and range joins, 168–69
inefficient joins, 169–70
intersections, 166–68
join formats, 161–65
mutable and complex joins, 172
outer joins, 170–71
self joins, 168
tuning, 172


Previous Section
< Day Day Up >
Next Section

Index

L

Latches, 611–24
defined, 611
latch misses, spins, and sleeps, 611–14
latch performance views, 614–15
latches in real time, 615–16
most significant latches, 616–24
database buffer cache, 617–20
library cache latches, 621–23
metadata cache latches, 623
network and database connection latches, 624
redo log buffer, 623
shared pool, 620–21
Low level physical tuning, 467–88
block and extent tuning, 471
block concurrency, 483–84
choosing block size, 471–73
extent level storage parameters, 485–88
different cache recently used lists, 487–88
managing concurrency, 487
minimizing rollback resizing, 487
minimum and maximum extents, 486
setting extent sizes, 485–86
variable extent sizes, 486–87
high water mark, 467–68
physical block structure, 473–82
parts of a block, 474–76
PCTFREE settings, 482
space management, 476–82
row chaining and row migration, 469–70
space used, 468–69
types of objects, 470–71


Previous Section
< Day Day Up >
Next Section

Index

M

MERGE statement in SQL, 115–17


Previous Section
< Day Day Up >
Next Section

Index

N

Network usage tuning, 527–44
connection profiles, 535–37
listener, 527–31
multiple listeners and load balancing, 529–31
queue size, 528–29
switching off logging and tracing, 529
naming methods, 531–35
dedicated versus shared servers, 532–34
session data unit buffer, 534–35
shared servers, 537–44
configuration parameters, 538–40
dispatchers, 541
network performance views, 540–41
using events, 542–43
virtual circuits, 542
Normalization of a database, 27–41
1st normal form, 29–31
2nd normal form, 31–34
3rd normal form, 34–36
4th normal form, 36–38
5th normal form, 38–40
and tuning, 43–44


Previous Section
< Day Day Up >
Next Section

Index

O

Object tuning, 453–66 see also Indexing
caching, 453–54
deallocating unused space, 459
dropping columns, 458–59
fragmentation and coalescing of indexes, 462
index-organized tables and clusters, 462–63
index parallelism, 461–62
logging, 454
monitoring indexes, 459–61
recycle bin, 465
sequences, 463
storing LOBs separately, 457–58
synonyms and views, 464–65
table parallelism, 454–57
Online Transaction Processing (OLTP), definition, xiii
Optimization, 239–56, 313–26
cost-based, 246–47
defined, 239–40
outlines, 244
overriding using hints, 313–26
change index scans, 319–22
change joins, 322–24
change queries and subqueries, 325
change table scans, 318–19
classifying hints, 316
influence the Optimizer, 317–18
other hints, 325–26
parallel SQL, 324–25
rule-based, 243–45
setting the mode, 242–43
and statistics, 247–55
ANALYZE command, 250–52
generation, 248–50, 720–21
histograms, 253–55
use of hints, 244–45
Oracle:
and BTree index, 207
and Denormalization, 79–80
Enterprise Manager,
Diagnostics Pack, 625–34
Tuning Pack, 365–84, 388
and Wait Event Interface, 665–69
Expert, 380–82
Optimizer query plans, 381–82
recommendations for the Accounts schema, 383
Oracle Database 10 Grid database, 22–23, 387, 388
history, 94–96
9 i database, 387
installation (see Installing Oracle database)
Instance, 427–30
making SQL efficient, 237–327
Metalink, 727
networking software (see SQL*Net)
partitioning, 545–55
software, xi, 22
Spotlight on Oracle, 634–36, 637, 638
Technet, 727
tuning requirements, 1–24
types of indexes, 203–20
Wait Event Interface tuning, 647–94


Previous Section
< Day Day Up >
Next Section

Index

P

Partitioning and parallelism, 545–58
benefits, 546–47
composite partitions, 555–56
definition, 545
hash partitions, 554–55
partitioning by list, 552–54
partitioning by range, 548–51
physical partitioning, 546
tables and indexes, 547–48
tricks with partitions, 556–58
Physical database and configuration tuning, definition, xv
Physical tuning, 17
PL/SQL (Programming Language for SQL), 188–91
Problem queries see Tools to detect problem queries
Programimg Language for SQL see PL/SQL


Previous Section
< Day Day Up >
Next Section

Index

R

Ratios as symptoms of problems, 559–74
buffer cache hit ratio, 561–65
default, keep, and recycle pools, 563–65
chained rows ratio, 570–71
dictionary cache hit ratio, 569
disc sort ratio, 569–70
index use ratio, 568
latch hit ratios, 572–73
library cache hit ratios, 569
parse ratios, 571–72
ratios and tuning, 559–60
table access ratios, 565–67
Recovery Manager (RMAN), 642
Referential integrity, 13–14, 44–59 see also SQL; Tuning the data model
coding business rules in the database, 55–59
using stored procedures, 58–59
using triggers, 56–57
definition, 41
how to implement, 46–48
in indexing, 201–2
and tuning, 13–14, 44–58
using constraints, 48–55
avoiding locking on foreign keys, 54
efficient keys, 49–52
hierarchies of relational entities, 49, 51
indexing foreign keys, 52–53
sacrificing Referential Integrity for performance, 53–55
whether to use, 44–46
Relational database model forms, 81–88
object database model, 83–85
object versus relational model, 85
run time objects, 86
object-relational database model, 86–87
objects overlaid onto relations, 87–88
purist's employees schema, 81–82, 88


Previous Section
< Day Day Up >
Next Section

Index

S

Sample databases, xvi
Sample scripts:
constraints, 721
count rows, 721
EXPLAIN PLAN, 719–20
indexes, 721–22
space in the database, 722–23
statistics generation, 720–21
trace file scripts, 723–25
Shared Global Area, 635, 638
Sorting in SQL, 301–5
Spotlight on Oracle, 634–36, 637, 638
SQL (structured query language), 97–199, 237–326, 365 see also Hardware resource usage tuning; Indexing; Referential integrity; Tuning the data model
Access Advisor, 382
accessing data in the database see Data access methods
code,
definition, xiii–xiv, 99–100
tuning, xiv, 8–9, 14–16
comparison conditions, 102–4, 154–61
equi, anti, and range, 155–56
groups, 161
like pattern matching, 156–57
set membership, 157–60
concatenation, 305–8
DELETE replaced by TRUNCATE, 195
functions, 113–14, 143–52
COUNT function, 143–45
datatype conversions, 146–49
DECODE function, 145–46
in the GROUP BY clause, 151–52
in the ORDER BY clause, 150–51
in the SELECT statement, 149–50
in the WHERE clause, 150
INSERT statement, 114–15
INTERSECT operator, 313
joins, 161–72
anti-joins, 171–72
cartesian products, 170
efficient joins, 165–66
equi joins and range joins, 168–69
inefficient joins, 169–70
intersections, 166–68
join formats, 161–65
mutable and complex joins, 172
outer joins, 170–71
self joins, 168
tuning, 172
IN list operator, 309–10
MERGE statement, 115–17
MINUS operator, 312–13
object and relational conflicts, 195–97
large binary objects, 195–96
object-relational collections, 196–97
optimization see Optimization
parallel queries, 119–20
parser, 237–39
PL/SQL, use of, 188–91
RETURNING INTO clause, 191
use of JAVA, 193–95
when to use with cursors, 192–93
pseudocolumns, 113, 152–54
ROWID pointers, 153
ROWNUM, 154
sequences, 152–54
SELECT statement, 100–114, 122–43
comparison conditions, 102–4
count of rows, 126
DISTINCT, use of, 139
DUAL table, 112
GROUP BY clause for sorting, 138–39
grouping result sets, 137–38
HAVING clause, 139–41
joining tables, 104–7
logical operators, 102
NULLS, 112–13
ORDER BY clause for sorting, 133–35
overriding WHERE with ORDER BY, 135–137
ROLLUP, CUBE, and GROUPING SETS, 141–42
FOR UPDATE clause, 142
WHERE clause for filtering, 126–33
set operators, 110–11
sorting, 301–5
aggregate sort, 305
GROUP BY sort, 304
ORDER BY sort, 303
sort merge join sort, 304–5
types of sort, 301–2
unique sort, 302–3
subqueries, 107–9, 172–82
FROM clause subquery nesting, 178–82
correlated versus noncorrelated, 173
IN versus EXISTS, 173
nested, 173–74
remove tables using EXISTS, 175–78
replacing joins with subqueries, 174–75
synonyms, use of, 182
temporary tables, 188
Trace, 336–40
finding trace files, 338
session level tracing, 338
setting up, 337–38
using, 339–40
transaction control, 117–19
COMMIT versus ROLLBACK, 117–18
UNION operator, 310–12
views, use of, 182–88
SQL*Loader, 639–41
SQL*Net:
buffer unit, 534
tuning, 527
STATSPACK:
and bottlenecks, 642–43
example report, 669–93
snapshots, 643–44
uses, 642–45
Structured query language see SQL
Subqueries in SQL, 107–9, 172–82


Previous Section
< Day Day Up >
Next Section

Index

T

TKPROF, 340–49, 723–25
Tools and utilities, 625–46
Enterprise Manager Diagnostics Pack, 625–33 (see also Enterprise Manager Tuning Pack)
event monitoring, 626
latch get/miss rates, 630, 632, 633
lock monitoring, 626, 628
performance manager, 628–30, 632
reorg wizard, 631–34, 635, 636
tablespace map, 631–34
TopSessions, 626–27, 629
TopSQL, 627, 630
Export utility, 639
Recovery Manager (RMAN), 642
Spotlight on Oracle, 634–36, 637, 638, 639
Shared Global Area, 635, 638
SQL*Loader, 639–41
STATSPACK (see STATSPACK)
Unix utilities, 638
Windows Performance Monitor, 636–37, 640
Tools to detect problem queries, 327–64
autotrace, 349–50
EXPLAIN PLAN, 328–35
command syntax, 331–35
operations, 329–30
PLAN_TABLE, 335
producing the best plan, 331
search for problems, 330
what is not provided, 335–36
performance views, 350–64
examining cached query plans, 362–63
examining SQL code, 352–53
finding cached SQL code, 351–52
using V$SQL, 359–62
using V$SQLAREA, 354–59
SQL Trace, 336–40
finding trace files, 338
session level tracing, 338
setting up, 337–38
using, 339–40
TKPROF, 340–49, 723–25
interpretation of output, 342–49
interpretation script, 724–25
perl wrapper script, 723–24
syntax, 340–41
using, 341–42
TRCSESS, 349
Tuning the data model, xii–xiii, 1–88 see also Denormalization; Normalization; Referential integrity; SQL
from development to production, 11–17
configuration tuning, 16–17
data model tuning, 12–14
physical tuning, 17
query plan from EXPLAIN PLAN, 16
SQL code tuning, 14–16
steps in tuning, 11–12
using static generic entities, 14, 45–46
environment, 1–11
bottlenecks, 7
configuration, 7
data model tuning, 9–11
duplicating production databases, 3–4
personnel, 2
physical space usage, 7–8
SQL code tuning, 8–9
staging and testing, 2–3
tools, 2
what is required, 1
what to tune in production, 5–6
when to stop tuning in production, 6–7
when to tune, 4–5
optimizing with alternate indexes, 59–61
relational database model, 43–80
tuning pack (see Enterprise Manager Tuning Pack)


Previous Section
< Day Day Up >
Next Section

Index

U

Undoing Normalization see Denormalization
Unix utilities, 638


Previous Section
< Day Day Up >
Next Section

Index

W

Wait Event Interface tuning, 647–94
bottlenecks, 647–49
Enterprise Manager, 665–69
health overview chart, 667
Top Objects and TopSQL, 668
improvements, 664–65
interface defined, 649–51
session layer, 656–62
event and wait event views, 657
hooking events to sessions, 661
SQL code for sessions, 662
system aggregation layer, 651–56
isolating segments, 654
third layer, 663–64
drilling down into latches, 663
tools used, 650
using STATSPACK, 669–93
Wait events, 575–609
definition, 575–76
idle events, 576–78
significant events, 578–609
automated undo waits, 603–5
buffer busy waits, 583–86
causes of buffer busy waits, 586–87
datafile scattered and sequential reads, 592–95
decreasing buffer busy waits, 587–92
direct path reads and writes, 595
enqueue waits, 605–8
examples of events, 578–83
free buffer waits, 595–96
latch free waits, 608–9
library cache waits, 597–98
manual rollback waits, 602–3
redo log waits, 598–602
row cache lock waits, 596
Windows Performance Monitor, 636–37


Previous Section
< Day Day Up >

List of Figures

Introduction

Figure 1: The Steps in Tuning an Oracle Installation
Figure 2: Using Static Generic Entities
Figure 3: A Simple Query Plan from EXPLAIN PLAN

Chapter 1: The Relational Database Model

Figure 1.1: 1st Normal Form
Figure 1.2: 1st Normal Form Rows for Figure 1.1
Figure 1.3: 2nd Normal Form
Figure 1.4: 2nd Normal Form Rows for Figure 1.3
Figure 1.5: 3rd Normal Form
Figure 1.6: 3rd Normal Form Rows for Figure 1.5
Figure 1.7: A More Concise Form of the Rows in Figure 1.6
Figure 1.8: 4th Normal Form
Figure 1.9: 4th Normal Form Rows for Figure 1.8
Figure 1.10: 5th Normal Form
Figure 1.11: 5th Normal Form Rows for Figure 1.10

Chapter 2: Tuning the Relational Database Model

Figure 2.1: Static Generic Entities
Figure 2.2: A Hierarchy of Relational Entities
Figure 2.3: A Hierarchy of Relational Entities Using Integer Keys
Figure 2.4: A Hierarchy of Relational Entities Using Unique Integer Identifier Keys
Figure 2.5: Avoiding Locking on Foreign Keys
Figure 2.6: An Object-Like Many-to-Many Join Resolution Entity
Figure 2.7: A Traditional Relational Version of Figure 2.6
Figure 2.8: 4th Normal Form Denormalization
Figure 2.9: 3rd Normal Form Denormalization
Figure 2.10: The Employees Schema
Figure 2.11: 2nd Normal Form Denormalization
Figure 2.12: Duplicating Individual Columns Between Entities
Figure 2.13: Summary Columns in Parent Entities

Chapter 3: Different Forms of the Relational Database Model

Figure 3.1: A Purist's Form of the Employees Schema
Figure 3.2: The Object versus the Relational Model
Figure 3.3: The Run Time Objects
Figure 3.4: The Employees Schema

Chapter 4: A Brief History of Data Modeling

Figure 4.1: The History and Evolution of Data Modeling
Figure 4.2: A Hierarchical Model for Company Employees
Figure 4.3: A Network Model for Company Employees
Figure 4.4: A Relational Model for Company Employees
Figure 4.5: The Object versus the Relational Model
Figure 4.6: The Evolution of the Relational Databases
Figure 4.7: The History of Oracle Database

Chapter 5: What is SQL?

Figure 5.1: DML Versus DDL
Figure 5.2: The Employees Schema
Figure 5.3: Separating the Employee Table into Manager and Employee
Figure 5.4: The MERGE Command
Figure 5.5: COMMIT versus ROLLBACK
Figure 5.6: Locking Rows Before Transaction Completion

Chapter 6: The Basics of Efficient SQL

Figure 6.1: Row Counts of Accounts Schema Tables

Chapter 7: Common Sense Indexing

Figure 7.1: An Oracle Database BTree Index
Figure 7.2: A Bitmap Index
Figure 7.3: A Skewed BTree Index
Figure 7.4: A Reverse Key Index
Figure 7.5: A Compressed Composite-Column Index
Figure 7.6: A Function-Based Index

Chapter 8: Making SQL Efficient in Oracle Database

Figure 8.1: An Example Histogram
Figure 8.2: A Skewed BTree Index
Figure 8.3: Passing a ROWID Pointer from an Index to a Table

Chapter 10: Tuning SQL with Oracle Enterprise Manager

Figure 10.1: The Index Tuning Wizard
Figure 10.2: Select the Application Type
Figure 10.3: Select Schemas to Analyze
Figure 10.4: The Index Tuning Wizard Makes Suggestions
Figure 10.5: SQL Analyze and the Analysis Stage
Figure 10.6: SQL Analyze Single SQL Statement Assessment and Testing
Figure 10.7: SQL Analyze SQL Text Display
Figure 10.8: SQL Analyze Query Plan
Figure 10.9: SQL Analyze Index Recommendations
Figure 10.10: SQL Analyze Query Plan Assessment
Figure 10.11: SQL Analyze Virtual Indexing, Hints, and Tuning
Figure 10.12: Comparing Different Versions of an SQL Statement
Figure 10.13: An ALL_ROWS Optimizer Query Plan
Figure 10.14: A FIRST_ROWS Optimizer Query Plan
Figure 10.15: Oracle Expert Recommendations for the Accounts Schema

Chapter 11: Installing Oracle and Creating a Database

Figure 11.1: Dedicated and Shared Servers Architecture
Figure 11.2: Choosing the Appropriate Database Template
Figure 11.3: Options and Examples when Creating a Database
Figure 11.4: More Optional Database Features
Figure 11.5: Choosing Dedicated or Shared Servers
Figure 11.6: Oracle Database Memory Buffer Parameters
Figure 11.7: The Sort Buffer
Figure 11.8: Creation of Tablespaces and Datafiles
Figure 11.9: Creation of Redo Logs
Figure 11.10: Physical Datafile Default Structure

Chapter 12: Tuning Oracle Database File Structures

Figure 12.1: Oracle Instance Memory Cache Buffers
Figure 12.2: Oracle Instance Process and Memory Layers
Figure 12.3: The Oracle Database and the Oracle Instance
Figure 12.4: Relationships Between Oracle Database Files
Figure 12.5: Oracle Database Physical and Logical Structure

Chapter 13: Object Tuning

Figure 13.1: Nonparallel Versus Parallel Processing

Chapter 14: Low-Level Physical Tuning

Figure 14.1: Block Fixed Header
Figure 14.2: Block Table Directory
Figure 14.3: Block Row Directory
Figure 14.4: Block Free Space
Figure 14.5: Block Row Data
Figure 14.6: Accounts Schema Current Row and Block Numbers
Figure 14.7: Changes to Block Structure Space Usage
Figure 14.8: Changes to Block Structure Concurrency

Chapter 16: Tuning Network Usage

Figure 16.1: Dedicated Versus Shared Servers

Chapter 17: Oracle Partitioning and Parallelism

Figure 17.1: Physical Partitioning

Chapter 21: Tools and Utilities

Figure 21.1: Defining Events for Detection and Resolution
Figure 21.2: Monitoring and Managing Locking
Figure 21.3: TopSessions Monitoring in a Specified Order
Figure 21.4: TopSQL Statements
Figure 21.5: The Performance Overview Tool
Figure 21.6: The Performance Manager Main Screen
Figure 21.7: Latch Get/Miss Rates Drilldown
Figure 21.8: Latch Analysis
Figure 21.9: After Execution of Tablespace Analysis
Figure 21.10: What Should be Reorganized and Why
Figure 21.11: After Reorganization and Coalescence on the INDX Tablespace
Figure 21.12: Spotlight on Oracle Main Monitoring Screen
Figure 21.13: Spotlight SGA View
Figure 21.14: Various Spotlight Views
Figure 21.15: The Windows Performance Monitor
Figure 21.16: This is a Very Busy Server

Chapter 22: Tuning with the Wait Event Interface and STATSPACK

Figure 22.1: Tools for Drilling into the Oracle Database Wait Event Interface
Figure 22.2: The System Aggregation Layer of the Oracle Database Wait Event Interface
Figure 22.3: Isolating Segments using Event Parameters
Figure 22.4: Session-Level Event and Wait Views
Figure 22.5: Hooking Wait Events to Sessions
Figure 22.6: Find SQL Code for Sessions
Figure 22.7: Drilling Down into Latches
Figure 22.8: Database Health Overview Chart– Database Memory Portion
Figure 22.9: Database Health Overview Chart–Database Memory Portion
Figure 22.10: Top Objects and TopSQL

Appendix A: Sample Databases

Figure A.1: Employees Schema ERD Version One
Figure A.2: Employees Schema ERD Version Two
Figure A.3: Accounts Schema ERD Denormalized Version
Figure A.4: Accounts Schema ERD Normalized Version