如何用chatGTP function calling 写sql query,附代码!

当与数据库工作时,查询特定数据是至关重要的。如果您可以简单地提出一个自然语言问题来获取所需的数据,而不是手动编写SQL查询,那会怎样呢?有了ChatGPT的函数调用功能,我们可以做到这一点!

让我们看看如何建立一个使用ChatGPT为我们生成SQL查询的系统,以Chinook样本数据库为例。

1. 连接到数据库

首先,我们需要建立与SQLite数据库的连接。

import sqlite3

conn = sqlite3.connect("data/Chinook.db")
print("成功打开数据库")

2. 提取数据库架构

为了做出明智的查询,ChatGPT需要理解我们数据库的结构。我们可以通过创建实用函数来提取表名、列名和整体数据库信息来实现这一点。

def get_table_names(conn):
    table_names = []
    tables = conn.execute("SELECT name FROM sqlite_master WHERE type='table';")
    for table in tables.fetchall():
        table_names.append(table[0])
    return table_names

def get_column_names(conn, table_name):
    column_names = []
    columns = conn.execute(f"PRAGMA table_info('{table_name}');").fetchall()
    for col in columns:
        column_names.append(col[1])
    return column_names

def get_database_info(conn):
    table_dicts = []
    for table_name in get_table_names(conn):
        columns_names = get_column_names(conn, table_name)
        table_dicts.append({"table_name": table_name, "column_names": columns_names})
    return table_dicts

使用这些函数,您现在可以为数据库生成一个架构表示。

3. 为ChatGPT定义函数规范

有了数据库架构,我们可以为ChatGPT定义一个函数规范。这将为模型提供有关我们数据库结构的上下文,并告诉它如何生成SQL查询。

database_schema_dict = get_database_info(conn)
database_schema_string = "\n".join(
    [
        f"Table: {table['table_name']}\nColumns: {', '.join(table['column_names'])}"
        for table in database_schema_dict
    ]
)

functions = [
    {
        "name": "ask_database",
        "description": "使用此功能回答用户关于音乐的问题。输入应该是一个完整的SQL查询。",
        "parameters": {
            "type": "object",
            "properties": {
                "query": {
                    "type": "string",
                    "description": f"使用此数据库架构提取信息的SQL查询:{database_schema_string}。查询应以纯文本返回,而不是JSON。"
                }
            },
            "required": ["query"],
        },
    }
]

4. 执行SQL查询

接下来,我们需要一个函数来执行针对我们数据库的生成的SQL查询。

def ask_database(conn, query):
    try:
        results = str(conn.execute(query).fetchall())
    except Exception as e:
        results = f"查询失败,错误为:{e}"
    return results

5. ChatGPT交互和SQL查询执行

现在,使用ChatGPT API,我们可以与模型互动。当我们从模型得到一个函数调用的响应时,我们可以执行SQL查询并返回结果。

messages = []
messages.append({"role": "system", "content": "通过生成针对Chinook音乐数据库的SQL查询来回答用户问题。"})
messages.append({"role": "user", "content": "你好,哪5位艺术家的歌曲数量最多?"})

chat_response = chat_completion_request(messages, functions)
assistant_message = chat_response.json()["choices"][0]["message"]
messages.append(assistant_message)

if assistant_message.get("function_call"):
    results = execute_function_call(assistant_message)
    messages.append({"role": "function", "name": assistant_message["function_call"]["name"], "content": results})

结论

通过将ChatGPT与数据库交互相结合,我们展示了如何使用自然语言问题生成SQL查询。这种方法允许更直观地检索数据,尤其是对于可能不熟悉SQL语法的用户。但是,请始终确保验证并清洁生成的查询,特别是在生产环境中使用时,以保持数据完整性和安全性。

代码

原文文末见github code ink

AI好书推荐

AI日新月异,但是万丈高楼拔地起,离不开良好的基础。您是否有兴趣了解人工智能的原理和实践? 不要再观望! 我们关于 AI 原则和实践的书是任何想要深入了解 AI 世界的人的完美资源。 由该领域的领先专家撰写,这本综合指南涵盖了从机器学习的基础知识到构建智能系统的高级技术的所有内容。 无论您是初学者还是经验丰富的 AI 从业者,本书都能满足您的需求。 那为什么还要等呢?

人工智能原理与实践 全面涵盖人工智能和数据科学各个重要体系经典

北大出版社,人工智能原理与实践 人工智能和数据科学从入门到精通 详解机器学习深度学习算法原理

  • 0
    点赞
  • 0
    收藏
    觉得还不错? 一键收藏
  • 0
    评论
Contents Overview 1 Lesson 1: Concepts – Locks and Lock Manager 3 Lesson 2: Concepts – Batch and Transaction 31 Lesson 3: Concepts – Locks and Applications 51 Lesson 4: Information Collection and Analysis 63 Lesson 5: Concepts – Formulating and Implementing Resolution 81 Module 4: Troubleshooting Locking and Blocking Overview At the end of this module, you will be able to:  Discuss how lock manager uses lock mode, lock resources, and lock compatibility to achieve transaction isolation.  Describe the various transaction types and how transactions differ from batches.  Describe how to troubleshoot blocking and locking issues.  Analyze the output of blocking scripts and Microsoft® SQL Server™ Profiler to troubleshoot locking and blocking issues.  Formulate hypothesis to resolve locking and blocking issues. Lesson 1: Concepts – Locks and Lock Manager This lesson outlines some of the common causes that contribute to the perception of a slow server. What You Will Learn After completing this lesson, you will be able to:  Describe locking architecture used by SQL Server.  Identify the various lock modes used by SQL Server.  Discuss lock compatibility and concurrent access.  Identify different types of lock resources.  Discuss dynamic locking and lock escalation.  Differentiate locks, latches, and other SQL Server internal “locking” mechanism such as spinlocks and other synchronization objects. Recommended Reading  Chapter 14 “Locking”, Inside SQL Server 2000 by Kalen Delaney  SOX000821700049 – SQL 7.0 How to interpret lock resource Ids  SOX000925700237 – TITLE: Lock escalation in SQL 7.0  SOX001109700040 – INF: Queries with PREFETCH in the plan hold lock until the end of transaction Locking Concepts Delivery Tip Prior to delivering this material, test the class to see if they fully understand the different isolation levels. If the class is not confident in their understanding, review appendix A04_Locking and its accompanying PowerPoint® file. Transactions in SQL Server provide the ACID properties: Atomicity A transaction either commits or aborts. If a transaction commits, all of its effects remain. If it aborts, all of its effects are undone. It is an “all or nothing” operation. Consistency An application should maintain the consistency of a database. For example, if you defer constraint checking, it is your responsibility to ensure that the database is consistent. Isolation Concurrent transactions are isolated from the updates of other incomplete transactions. These updates do not constitute a consistent state. This property is often called serializability. For example, a second transaction traversing the doubly linked list mentioned above would see the list before or after the insert, but it will see only complete changes. Durability After a transaction commits, its effects will persist even if there are system failures. Consistency and isolation are the most important in describing SQL Server’s locking model. It is up to the application to define what consistency means, and isolation in some form is needed to achieve consistent results. SQL Server uses locking to achieve isolation. Definition of Dependency: A set of transactions can run concurrently if their outputs are disjoint from the union of one another’s input and output sets. For example, if T1 writes some object that is in T2’s input or output set, there is a dependency between T1 and T2. Bad Dependencies These include lost updates, dirty reads, non-repeatable reads, and phantoms. ANSI SQL Isolation Levels An isolation level determines the degree to which data is isolated for use by one process and guarded against interference from other processes. Prior to SQL Server 7.0, REPEATABLE READ and SERIALIZABLE isolation levels were synonymous. There was no way to prevent non-repeatable reads while not preventing phantoms. By default, SQL Server 2000 operates at an isolation level of READ COMMITTED. To make use of either more or less strict isolation levels in applications, locking can be customized for an entire session by setting the isolation level of the session with the SET TRANSACTION ISOLATION LEVEL statement. To determine the transaction isolation level currently set, use the DBCC USEROPTIONS statement, for example: USE pubs GO SET TRANSACTION ISOLATION LEVEL REPEATABLE READ GO DBCC USEROPTIONS GO Multigranular Locking Multigranular Locking In our example, if one transaction (T1) holds an exclusive lock at the table level, and another transaction (T2) holds an exclusive lock at the row level, each of the transactions believe they have exclusive access to the resource. In this scenario, since T1 believes it locks the entire table, it might inadvertently make changes to the same row that T2 thought it has locked exclusively. In a multigranular locking environment, there must be a way to effectively overcome this scenario. Intent lock is the answer to this problem. Intent Lock Intent Lock is the term used to mean placing a marker in a higher-level lock queue. The type of intent lock can also be called the multigranular lock mode. An intent lock indicates that SQL Server wants to acquire a shared (S) lock or exclusive (X) lock on some of the resources lower down in the hierarchy. For example, a shared intent lock placed at the table level means that a transaction intends on placing shared (S) locks on pages or rows within that table. Setting an intent lock at the table level prevents another transaction from subsequently acquiring an exclusive (X) lock on the table containing that page. Intent locks improve performance because SQL Server examines intent locks only at the table level to determine whether a transaction can safely acquire a lock on that table. This removes the requirement to examine every row or page lock on the table to determine whether a transaction can lock the entire table. Lock Mode The code shown in the slide represents how the lock mode is stored internally. You can see these codes by querying the master.dbo.spt_values table: SELECT * FROM master.dbo.spt_values WHERE type = N'L' However, the req_mode column of master.dbo.syslockinfo has lock mode code that is one less than the code values shown here. For example, value of req_mode = 3 represents the Shared lock mode rather than the Schema Modification lock mode. Lock Compatibility These locks can apply at any coarser level of granularity. If a row is locked, SQL Server will apply intent locks at both the page and the table level. If a page is locked, SQL Server will apply an intent lock at the table level. SIX locks imply that we have shared access to a resource and we have also placed X locks at a lower level in the hierarchy. SQL Server never asks for SIX locks directly, they are always the result of a conversion. For example, suppose a transaction scanned a page using an S lock and then subsequently decided to perform a row level update. The row would obtain an X lock, but now the page would require an IX lock. The resultant mode on the page would be SIX. Another type of table lock is a schema stability lock (Sch-S) and is compatible with all table locks except the schema modification lock (Sch-M). The schema modification lock (Sch-M) is incompatible with all table locks. Locking Resources Delivery Tip Note the differences between Key and Key Range locks. Key Range locks will be covered in a couple of slides. SQL Server can lock these resources: Item Description DB A database. File A database file Index An entire index of a table. Table An entire table, including all data and indexes. Extent A contiguous group of data pages or index pages. Page An 8-KB data page or index page. Key Row lock within an index. Key-range A key-range. Used to lock ranges between records in a table to prevent phantom insertions or deletions into a set of records. Ensures serializable transactions. RID A Row Identifier. Used to individually lock a single row within a table. Application A lock resource defined by an application. The lock manager knows nothing about the resource format. It simply compares the 'strings' representing the lock resources to determine whether it has found a match. If a match is found, it knows that resource is already locked. Some of the resources have “sub-resources.” The followings are sub-resources displayed by the sp_lock output: Database Lock Sub-Resources: Full Database Lock (default) [BULK-OP-DB] – Bulk Operation Lock for Database [BULK-OP-LOG] – Bulk Operation Lock for Log Table Lock Sub-Resources: Full Table Lock (default) [UPD-STATS] – Update statistics Lock [COMPILE] – Compile Lock Index Lock sub-Resources: Full Index Lock (default) [INDEX_ID] – Index ID Lock [INDEX_NAME] – Index Name Lock [BULK_ALLOC] – Bulk Allocation Lock [DEFRAG] – Defragmentation Lock For more information, see also… SOX000821700049 SQL 7.0 How to interpret lock resource Ids Lock Resource Block The resource type has the following resource block format: Resource Type (Code) Content DB (2) Data 1: sub-resource; Data 2: 0; Data 3: 0 File (3) Data 1: File ID; Data 2: 0; Data 3: 0 Index (4) Data 1: Object ID; Data 2: sub-resource; Data 3: Index ID Table (5) Data 1: Object ID; Data 2: sub-resource; Data 3: 0. Page (6) Data 1: Page Number; Data 3: 0. Key (7) Data 1: Object ID; Data 2: Index ID; Data 3: Hashed Key Extent (8) Data 1: Extent ID; Data 3: 0. RID (9) Data 1: RID; Data 3: 0. Application (10) Data 1: Application resource name The rsc_bin column of master..syslockinfo contains the resource block in hexadecimal format. For an example of how to decode value from this column using the information above, let us assume we have the following value: 0x000705001F83D775010002014F0BEC4E With byte swapping within each field, this can be decoded as: Byte 0: Flag – 0x00 Byte 1: Resource Type – 0x07 (Key) Byte 2-3: DBID – 0x0005 Byte 4-7: ObjectID – 0x 75D7831F (1977058079) Byte 8-9: IndexID – 0x0001 Byte 10-16: Hash Key value – 0x 02014F0BEC4E For more information about how to decode this value, see also… Inside SQL Server 2000, pages 803 and 806. Key Range Locking Key Range Locking To support SERIALIZABLE transaction semantics, SQL Server needs to lock sets of rows specified by a predicate, such as WHERE salary BETWEEN 30000 AND 50000 SQL Server needs to lock data that does not exist! If no rows satisfy the WHERE condition the first time the range is scanned, no rows should be returned on any subsequent scans. Key range locks are similar to row locks on index keys (whether clustered or not). The locks are placed on individual keys rather than at the node level. The hash value consists of all the key components and the locator. So, for a nonclustered index over a heap, where columns c1 and c2 where indexed, the hash would contain contributions from c1, c2 and the RID. A key range lock applied to a particular key means that all keys between the value locked and the next value would be locked for all data modification. Key range locks can lock a slightly larger range than that implied by the WHERE clause. Suppose the following select was executed in a transaction with isolation level SERIALIZABLE: SELECT * FROM members WHERE first_name between ‘Al’ and ‘Carl’ If 'Al', 'Bob', and 'Dave' are index keys in the table, the first two of these would acquire key range locks. Although this would prevent anyone from inserting either 'Alex' or 'Ben', it would also prevent someone from inserting 'Dan', which is not within the range of the WHERE clause. Prior to SQL Server 7.0, page locking was used to prevent phantoms by locking the entire set of pages on which the phantom would exist. This can be too conservative. Key Range locking lets SQL Server lock only a much more restrictive area of the table. Impact Key-range locking ensures that these scenarios are SERIALIZABLE:  Range scan query  Singleton fetch of nonexistent row  Delete operation  Insert operation However, the following conditions must be satisfied before key-range locking can occur:  The transaction-isolation level must be set to SERIALIZABLE.  The operation performed on the data must use an index range access. Range locking is activated only when query processing (such as the optimizer) chooses an index path to access the data. Key Range Lock Mode Again, the req_mode column of master.dbo.syslockinfo has lock mode code that is one less than the code values shown here. Dynamic Locking When modifying individual rows, SQL Server typically would take row locks to maximize concurrency (for example, OLTP, order-entry application). When scanning larger volumes of data, it would be more appropriate to take page or table locks to minimize the cost of acquiring locks (for example, DSS, data warehouse, reporting). Locking Decision The decision about which unit to lock is made dynamically, taking many factors into account, including other activity on the system. For example, if there are multiple transactions currently accessing a table, SQL Server will tend to favor row locking more so than it otherwise would. It may mean the difference between scanning the table now and paying a bit more in locking cost, or having to wait to acquire a more coarse lock. A preliminary locking decision is made during query optimization, but that decision can be adjusted when the query is actually executed. Lock Escalation When the lock count for the transaction exceeds and is a multiple of ESCALATION_THRESHOLD (1250), the Lock Manager attempts to escalate. For example, when a transaction acquired 1250 locks, lock manager will try to escalate. The number of locks held may continue to increase after the escalation attempt (for example, because new tables are accessed, or the previous lock escalation attempts failed due to incompatible locks held by another spid). If the lock count for this transaction reaches 2500 (1250 * 2), Lock Manager will attempt escalation again. The Lock Manager looks at the lock memory it is using and if it is more than 40 percent of SQL Server’s allocated buffer pool memory, it tries to find a scan (SDES) where no escalation has already been performed. It then repeats the search operation until all scans have been escalated or until the memory used drops under the MEMORY_LOAD_ESCALATION_THRESHOLD (40%) value. If lock escalation is not possible or fails to significantly reduce lock memory footprint, SQL Server can continue to acquire locks until the total lock memory reaches 60 percent of the buffer pool (MAX_LOCK_RESOURCE_MEMORY_PERCENTAGE=60). Lock escalation may be also done when a single scan (SDES) holds more than LOCK_ESCALATION_THRESHOLD (765) locks. There is no lock escalation on temporary tables or system tables. Trace Flag 1211 disables lock escalation. Important Do not relay this to the customer without careful consideration. Lock escalation is a necessary feature, not something to be avoided completely. Trace flags are global and disabling lock escalation could lead to out of memory situations, extremely poor performing queries, or other problems. Lock escalation tracing can be seen using the Profiler or with the general locking trace flag, -T1200. However, Trace Flag 1200 shows all lock activity so it should not be usable on a production system. For more information, see also… SOX000925700237 “TITLE: SQL 7.0 Lock escalation in SQL 7.0” Lock Timeout Application Lock Timeout An application can set lock timeout for a session with the SET option: SET LOCK_TIMEOUT N where N is a number of milliseconds. A value of -1 means that there will be no timeout, which is equivalent to the version 6.5 behavior. A value of 0 means that there will be no waiting; if a process finds a resource locked, it will generate error message 1222 and continue with the next statement. The current value of LOCK_TIMEOUT is stored in the global variable @@lock_timeout. Note After a lock timeout any transaction containing the statement, is rolled back or canceled by SQL Server 2000 (bug#352640 was filed). This behavior is different from that of SQL Server 7.0. With SQL Server 7.0, the application must have an error handler that can trap error 1222 and if an application does not trap the error, it can proceed unaware that an individual statement within a transaction has been canceled, and errors can occur because statements later in the transaction may depend on the statement that was never executed. Bug#352640 is fixed in hotfix build 8.00.266 whereby a lock timeout will only Internal Lock Timeout At time, internal operations within SQL Server will attempt to acquire locks via lock manager. Typically, these lock requests are issued with “no waiting.” For example, the ghost record processing might try to clean up rows on a particular page, and before it can do that, it needs to lock the page. Thus, the ghost record manager will request a page lock with no wait so that if it cannot lock the page, it will just move on to other pages; it can always come back to this page later. If you look at SQL Profiler Lock: Timeout events, internal lock timeout typically have a duration value of zero. Lock Duration Lock Mode and Transaction Isolation Level For REPEATABLE READ transaction isolation level, update locks are held until data is read and processed, unless promoted to exclusive locks. "Data is processed" means that we have decided whether the row in question matched the search criteria; if not then the update lock is released, otherwise, we get an exclusive lock and make the modification. Consider the following query: use northwind go dbcc traceon(3604, 1200, 1211) -- turn on lock tracing -- and disable escalation go set transaction isolation level repeatable read begin tran update dbo.[order details] set discount = convert (real, discount) where discount = 0.0 exec sp_lock Update locks are promoted to exclusive locks when there is a match; otherwise, the update lock is released. The sp_lock output verifies that the SPID does not hold any update locks or shared locks at the end of the query. Lock escalation is turned off so that exclusive table lock is not held at the end. Warning Do not use trace flag 1200 in a production environment because it produces a lot of output and slows down the server. Trace flag 1211 should not be used unless you have done extensive study to make sure it helps with performance. These trace flags are used here for illustration and learning purposes only. Lock Ownership Most of the locking discussion in this lesson relates to locks owned by “transactions.” In addition to transaction, cursor and session can be owners of locks and they both affect how long locks are held. For every row that is fetched, when SCROLL_LOCKS option is used, regardless of the state of a transaction, a cursor lock is held until the next row is fetched or when the cursor is closed. Locks owned by session are outside the scope of a transaction. The duration of these locks are bounded by the connection and the process will continue to hold these locks until the process disconnects. A typical lock owned by session is the database (DB) lock. Locking – Read Committed Scan Under read committed isolation level, when database pages are scanned, shared locks are held when the page is read and processed. The shared locks are released “behind” the scan and allow other transactions to update rows. It is important to note that the shared lock currently acquired will not be released until shared lock for the next page is successfully acquired (this is commonly know as “crabbing”). If the same pages are scanned again, rows may be modified or deleted by other transactions. Locking – Repeatable Read Scan Under repeatable read isolation level, when database pages are scanned, shared locks are held when the page is read and processed. SQL Server continues to hold these shared locks, thus preventing other transactions to update rows. If the same pages are scanned again, previously scanned rows will not change but new rows may be added by other transactions. Locking – Serializable Read Scan Under serializable read isolation level, when database pages are scanned, shared locks are held not only on rows but also on scanned key range. SQL Server continues to hold these shared locks until the end of transaction. Because key range locks are held, not only will this prevent other transactions from modifying the rows, no new rows can be inserted. Prefetch and Isolation Level Prefetch and Locking Behavior The prefetch feature is available for use with SQL Server 7.0 and SQL Server 2000. When searching for data using a nonclustered index, the index is searched for a particular value. When that value is found, the index points to the disk address. The traditional approach would be to immediately issue an I/O for that row, given the disk address. The result is one synchronous I/O per row and, at most, one disk at a time working to evaluate the query. This does not take advantage of striped disk sets. The prefetch feature takes a different approach. It continues looking for more record pointers in the nonclustered index. When it has collected a number of them, it provides the storage engine with prefetch hints. These hints tell the storage engine that the query processor will need these particular records soon. The storage engine can now issue several I/Os simultaneously, taking advantage of striped disk sets to execute multiple operations simultaneously. For example, if the engine is scanning a nonclustered index to determine which rows qualify but will eventually need to visit the data page as well to access columns that are not in the index, it may decide to submit asynchronous page read requests for a group of qualifying rows. The prefetch data pages are then revisited later to avoid waiting for each individual page read to complete in a serial fashion. This data access path requires that a lock be held between the prefetch request and the row lookup to stabilize the row on the page so it is not to be moved by a page split or clustered key update. For our example, the isolation level of the query is escalated to REPEATABLE READ, overriding the transaction isolation level. With SQL Server 7.0 and SQL Server 2000, portions of a transaction can execute at a different transaction isolation level than the entire transaction itself. This is implemented as lock classes. Lock classes are used to control lock lifetime when portions of a transaction need to execute at a stricter isolation level than the underlying transaction. Unfortunately, in SQL Server 7.0 and SQL Server 2000, the lock class is created at the topmost operator of the query and hence released only at the end of the query. Currently there is no support to release the lock (lock class) after the row has been discarded or fetched by the filter or join operator. This is because isolation level can be set at the query level via a lock class, but no lower. Because of this, locks acquired during the query will not be released until the query completes. If prefetch is occurring you may see a single SPID that holds hundreds of Shared KEY or PAG locks even though the connection’s isolation level is READ COMMITTED. Isolation level can be determined from DBCC PSS output. For details about this behavior see “SOX001109700040 INF: Queries with PREFETCH in the plan hold lock until the end of transaction”. Other Locking Mechanism Lock manager does not manage latches and spinlocks. Latches Latches are internal mechanisms used to protect pages while doing operations such as placing a row physically on a page, compressing space on a page, or retrieving rows from a page. Latches can roughly be divided into I/O latches and non-I/O latches. If you see a high number of non-I/O related latches, SQL Server is usually doing a large number of hash or sort operations in tempdb. You can monitor latch activities via DBCC SQLPERF(‘WAITSTATS’) command. Spinlock A spinlock is an internal data structure that is used to protect vital information that is shared within SQL Server. On a multi-processor machine, when SQL Server tries to access a particular resource protected by a spinlock, it must first acquire the spinlock. If it fails, it executes a loop that will check to see if the lock is available and if not, decrements a counter. If the counter reaches zero, it yields the processor to another thread and goes into a “sleep” (wait) state for a pre-determined amount of time. When it wakes, hopefully, the lock is free and available. If not, the loop starts again and it is terminated only when the lock is acquired. The reason for implementing a spinlock is that it is probably less costly to “spin” for a short time rather than yielding the processor. Yielding the processor will force an expensive context switch where:  The old thread’s state must be saved  The new thread’s state must be reloaded  The data stored in the L1 and L2 cache are useless to the processor On a single-processor computer, the loop is not useful because no other thread can be running and thus, no one can release the spinlock for the currently executing thread to acquire. In this situation, the thread yields the processor immediately. Lesson 2: Concepts – Batch and Transaction This lesson outlines some of the common causes that contribute to the perception of a slow server. What You Will Learn After completing this lesson, you will be able to:  Review batch processing and error checking.  Review explicit, implicit and autocommit transactions and transaction nesting level.  Discuss how commit and rollback transaction done in stored procedure and trigger affects transaction nesting level.  Discuss various transaction isolation level and their impact on locking.  Discuss the difference between aborting a statement, a transaction, and a batch.  Describe how @@error, @@transcount, and @@rowcount can be used for error checking and handling. Recommended Reading  Charter 12 “Transactions and Triggers”, Inside SQL Server 2000 by Kalen Delaney Batch Definition SQL Profiler Statements and Batches To help further your understanding of what is a batch and what is a statement, you can use SQL Profiler to study the definition of batch and statement.  Try This: Using SQL Profiler to Analyze Batch 1. Log on to a server with Query Analyzer 2. Startup the SQL Profiler against the same server 3. Start a trace using the “StandardSQLProfiler” template 4. Execute the following using Query Analyzer: SELECT @@VERSION SELECT @@SPID The ‘SQL:BatchCompleted’ event is captured by the trace. It shows both the statements as a single batch. 5. Now execute the following using Query Analyzer {call sp_who()} What shows up? The ‘RPC:Completed’ with the sp_who information. RPC is simply another entry point to the SQL Server to call stored procedures with native data types. This allows one to avoid parsing. The ‘RPC:Completed’ event should be considered the same as a batch for the purposes of this discussion. Stop the current trace and start a new trace using the “SQLProfilerTSQL_SPs” template. Issue the same command as outlines in step 5 above. Looking at the output, not only can you see the batch markers but each statement as executed within the batch. Autocommit, Explicit, and Implicit Transaction Autocommit Transaction Mode (Default) Autocommit mode is the default transaction management mode of SQL Server. Every Transact-SQL statement, whether it is a standalone statement or part of a batch, is committed or rolled back when it completes. If a statement completes successfully, it is committed; if it encounters any error, it is rolled back. A SQL Server connection operates in autocommit mode whenever this default mode has not been overridden by either explicit or implicit transactions. Autocommit mode is also the default mode for ADO, OLE DB, ODBC, and DB-Library. A SQL Server connection operates in autocommit mode until a BEGIN TRANSACTION statement starts an explicit transaction, or implicit transaction mode is set on. When the explicit transaction is committed or rolled back, or when implicit transaction mode is turned off, SQL Server returns to autocommit mode. Explicit Transaction Mode An explicit transaction is a transaction that starts with a BEGIN TRANSACTION statement. An explicit transaction can contain one or more statements and must be terminated by either a COMMIT TRANSACTION or a ROLLBACK TRANSACTION statement. Implicit Transaction Mode SQL Server can automatically or, more precisely, implicitly start a transaction for you if a SET IMPLICIT_TRANSACTIONS ON statement is run or if the implicit transaction option is turned on globally by running sp_configure ‘user options’ 2. (Actually, the bit mask 0x2 must be turned on for the user option so you might have to perform an ‘OR’ operation with the existing user option value.) See SQL Server 2000 Books Online on how to turn on implicit transaction under ODBC and OLE DB (acdata.chm::/ac_8_md_06_2g6r.htm). Transaction Nesting Explicit transactions can be nested. Committing inner transactions is ignored by SQL Server other than to decrements @@TRANCOUNT. The transaction is either committed or rolled back based on the action taken at the end of the outermost transaction. If the outer transaction is committed, the inner nested transactions are also committed. If the outer transaction is rolled back, then all inner transactions are also rolled back, regardless of whether the inner transactions were individually committed. Each call to COMMIT TRANSACTION applies to the last executed BEGIN TRANSACTION. If the BEGIN TRANSACTION statements are nested, then a COMMIT statement applies only to the last nested transaction, which is the innermost transaction. Even if a COMMIT TRANSACTION transaction_name statement within a nested transaction refers to the transaction name of the outer transaction, the commit applies only to the innermost transaction. If a ROLLBACK TRANSACTION statement without a transaction_name parameter is executed at any level of a set of nested transaction, it rolls back all the nested transactions, including the outermost transaction. The @@TRANCOUNT function records the current transaction nesting level. Each BEGIN TRANSACTION statement increments @@TRANCOUNT by one. Each COMMIT TRANSACTION statement decrements @@TRANCOUNT by one. A ROLLBACK TRANSACTION statement that does not have a transaction name rolls back all nested transactions and decrements @@TRANCOUNT to 0. A ROLLBACK TRANSACTION that uses the transaction name of the outermost transaction in a set of nested transactions rolls back all the nested transactions and decrements @@TRANCOUNT to 0. When you are unsure if you are already in a transaction, SELECT @@TRANCOUNT to determine whether it is 1 or more. If @@TRANCOUNT is 0 you are not in a transaction. You can also find the transaction nesting level by checking the sysprocess.open_tran column. See SQL Server 2000 Books Online topic “Nesting Transactions” (acdata.chm::/ac_8_md_06_66nq.htm) for more information. Statement, Transaction, and Batch Abort One batch can have many statements and one transaction can have multiple statements, also. One transaction can span multiple batches and one batch can have multiple transactions. Statement Abort Currently executing statement is aborted. This can be a bit confusing when you start talking about statements in a trigger or stored procedure. Let us look closely at the following trigger: CREATE TRIGGER TRG8134 ON TBL8134 AFTER INSERT AS BEGIN SELECT 1/0 SELECT 'Next command in trigger' END To fire the INSERT trigger, the batch could be as simple as ‘INSERT INTO TBL8134 VALUES(1)’. However, the trigger contains two statements that must be executed as part of the batch to satisfy the clients insert request. When the ‘SELECT 1/0’ causes the divide by zero error, a statement abort is issued for the ‘SELECT 1/0’ statement. Batch and Transaction Abort On SQL Server 2000 (and SQL Server 7.0) whenever a non-informational error is encountered in a trigger, the statement abort is promoted to a batch and transactional abort. Thus, in the example the statement abort for ‘select 1/0’ promotion results in an entire batch abort. No further statements in the trigger or batch will be executed and a rollback is issued. On SQL Server 6.5, the statement aborts immediately and results in a transaction abort. However, the rest of the statements within the trigger are executed. This trigger could return ‘Next command in trigger’ as a result set. Once the trigger completes the batch abort promotion takes effect. Conversely, submitting a similar set of statements in a standalone batch can result in different behavior. SELECT 1/0 SELECT 'Next command in batch' Not considering the set option possibilities, a divide by zero error generally results in a statement abort. Since it is not in a trigger, the promotion to a batch abort is avoided and subsequent SELECT statement can execute. The programmer should add an “if @@ERROR” check immediately after the ‘select 1/0’ to T-SQL execution to control the flow correctly. Aborting and Set Options ARITHABORT If SET ARITHABORT is ON, these error conditions cause the query or batch to terminate. If the errors occur in a transaction, the transaction is rolled back. If SET ARITHABORT is OFF and one of these errors occurs, a warning message is displayed, and NULL is assigned to the result of the arithmetic operation. When an INSERT, DELETE, or UPDATE statement encounters an arithmetic error (overflow, divide-by-zero, or a domain error) during expression evaluation when SET ARITHABORT is OFF, SQL Server inserts or updates a NULL value. If the target column is not nullable, the insert or update action fails and the user receives an error. XACT_ABORT When SET XACT_ABORT is ON, if a Transact-SQL statement raises a run-time error, the entire transaction is terminated and rolled back. When OFF, only the Transact-SQL statement that raised the error is rolled back and the transaction continues processing. Compile errors, such as syntax errors, are not affected by SET XACT_ABORT. For example: CREATE TABLE t1 (a int PRIMARY KEY) CREATE TABLE t2 (a int REFERENCES t1(a)) GO INSERT INTO t1 VALUES (1) INSERT INTO t1 VALUES (3) INSERT INTO t1 VALUES (4) INSERT INTO t1 VALUES (6) GO SET XACT_ABORT OFF GO BEGIN TRAN INSERT INTO t2 VALUES (1) INSERT INTO t2 VALUES (2) /* Foreign key error */ INSERT INTO t2 VALUES (3) COMMIT TRAN SELECT 'Continue running batch 1...' GO SET XACT_ABORT ON GO BEGIN TRAN INSERT INTO t2 VALUES (4) INSERT INTO t2 VALUES (5) /* Foreign key error */ INSERT INTO t2 VALUES (6) COMMIT TRAN SELECT 'Continue running batch 2...' GO /* Select shows only keys 1 and 3 added. Key 2 insert failed and was rolled back, but XACT_ABORT was OFF and rest of transaction succeeded. Key 5 insert error with XACT_ABORT ON caused all of the second transaction to roll back. Also note that 'Continue running batch 2...' is not Returned to indicate that the batch is aborted. */ SELECT * FROM t2 GO DROP TABLE t2 DROP TABLE t1 GO Compile and Run-time Errors Compile Errors Compile errors are encountered during syntax checks, security checks, and other general operations to prepare the batch for execution. These errors can prevent the optimization of the query and thus lead to immediate abort. The statement is not run and the batch is aborted. The transaction state is generally left untouched. For example, assume there are four statements in a particular batch. If the third statement has a syntax error, none of the statements in the batch is executed. Optimization Errors Optimization errors would include rare situations where the statement encounters a problem when attempting to build an optimal execution plan. Example: “too many tables referenced in the query” error is reported because a “work table” was added to the plan. Runtime Errors Runtime errors are those that are encountered during the execution of the query. Consider the following batch: SELECT * FROM pubs.dbo.titles UPDATE pubs.dbo.authors SET au_lname = au_lname SELECT * FROM foo UPDATE pubs.dbo.authors SET au_lname = au_lname If you run the above statements in a batch, the first two statements will be executed, the third statement will fail because table foo does not exist, and the batch will terminate. Deferred Name Resolution is the feature that allows this batch to start executing before resolving the object foo. This feature allows SQL Server to delay object resolution and place a “placeholder” in the query’s execution. The object referenced by the placeholder is resolved until the query is executed. In our example, the execution of the statement “SELECT * FROM foo” will trigger another compile process to resolve the name again. This time, error message 208 is returned. Error: 208, Level 16, State 1, Line 1 Invalid object name 'foo'. Message 208 can be encountered as a runtime or compile error depending on whether the Deferred Name Resolution feature is available. In SQL Server 6.5 this would be considered a compile error and on SQL Server 2000 (and SQL Server7.0) as a runtime error due to Deferred Name Resolution. In the following example, if a trigger referenced authors2, the error is detected as SQL Server attempts to execute the trigger. However, under SQL Server 6.5 the create trigger statement fails because authors2 does not exist at compile time. When errors are encountered in a trigger, generally, the statement, batch, and transaction are aborted. You should be able to observe this by running the following script in pubs database: Create table tblTest(iID int) go create trigger trgInsert on tblTest for INSERT as begin select * from authors select * from authors2 select * from titles end go begin tran select 'Before' insert into tblTest values(1) select 'After' go select @@TRANCOUNT go When run in a batch, the statement and the batch are aborted but the transaction remains active. The follow script illustrates this: begin tran select 'Before' select * from authors2 select 'After' go select @@TRANCOUNT go One other factor in a compile versus runtime error is implicit data type conversions. If you were to run the following statements on SQL Server 6.5 and SQL Server 2000 (and SQL Server 7.0): create table tblData(dtData datetime) go select 1 insert into tblData values(12/13/99) go On SQL Server 6.5, you get an error before execution of the batch begins so no statements are executed and the batch is aborted. Error: 206, Level 16, State 2, Line 2 Operand type clash: int is incompatible with datetime On SQL Server 2000, you get the default value (1900-01-01 00:00:00.000) inserted into the table. SQL Server 2000 implicit data type conversion treats this as integer division. The integer division of 12/13/99 is 0, so the default date and time value is inserted, no error returned. To correct the problem on either version is to wrap the date string with quotes. See Bug #56118 (sqlbug_70) for more details about this situation. Another example of a runtime error is a 605 message. Error: 605 Attempt to fetch logical page %S_PGID in database '%.*ls' belongs to object '%.*ls', not to object '%.*ls'. A 605 error is always a runtime error. However, depending on the transaction isolation level, (e.g. using the NOLOCK lock hint), established by the SPID the handling of the error can vary. Specifically, a 605 error is considered an ACCESS error. Errors associated with buffer and page access are found in the 600 series of errors. When the error is encountered, the isolation level of the SPID is examined to determine proper handling based on information or fatal error level. Transaction Error Checking Not all errors cause transactions to automatically rollback. Although it is difficult to determine exactly which errors will rollback transactions and which errors will not, the main idea here is that programmers must perform error checking and handle errors appropriately. Error Handling Raiserror Details Raiserror seems to be a source of confusion but is really rather simple. Raiserror with severity levels of 20 or higher will terminate the connection. Of course, when the connection is terminated a full rollback of any open transaction will immediately be instantiated by the SQL Server (except distributed transaction with DTC involved). Severity levels lower than 20 will simply result in the error message being returned to the client. They do not affect the transaction scope of the connection. Consider the following batch: use pubs begin tran update authors set au_lname = 'smith' raiserror ('This is bad', 19, 1) with log select @@trancount With severity set at 19, the 'select @@trancount' will be executed after the raiserror statement and will return a value of 1. If severity is changed to 20, then the select statement will not run and the connection is broken. Important Error handling must occur not only in T-SQL batches and stored procedures, but also in application program code. Transactions and Triggers (1 of 2) Basic behavior assumes the implicit transactions setting is set to OFF. This behavior makes it possible to identify business logic errors in a trigger, raise an error, rollback the action, and add an audit table entry. Logically, the insert to the audit table cannot take place before the ROLLBACK action and you would not want to build in the audit table insert into every applications error handler that violated the business rule of the trigger. For more information, see also… SQL Server 2000 Books Online topic “Rollbacks in stored procedure and triggers“ (acdata.chm::/ac_8_md_06_4qcz.htm) IMPLICIT_TRANSACTIONS ON Behavior The behavior of firing other triggers on the same table can be tricky. Say you added a trigger that checks the CODE field. Read only versions of the rows contain the code ‘RO’ and read/write versions use ‘RW.’ Whenever someone tries to delete a row with a code ‘RO’ the trigger issues the rollback and logs an audit table entry. However, you also have a second trigger that is responsible for cascading delete operations. One client could issue the delete without implicit transactions on and only the current trigger would execute and then terminate the batch. However, a second client with implicit transactions on could issue the same delete and the secondary trigger would fire. You end up with a situation in which the cascading delete operations can take place (are committed) but the initial row remains in the table because of the rollback operation. None of the delete operations should be allowed but because the transaction scope was restarted because of the implicit transactions setting, they did. Transactions and Triggers (2 of 2) It is extremely difficult to determine the execution state of a trigger when using explicit rollback statements in combination with implicit transactions. The RETURN statement is not allowed to return a value. The only way I have found to set the @@ERROR is using a ‘raiserror’ as the last execution statement in the last trigger to execute. If you modify the example, this following RAISERROR statement will set @@ERROR to 50000: CREATE TRIGGER trgTest on tblTest for INSERT AS BEGIN ROLLBACK INSERT INTO tblAudit VALUES (1) RAISERROR('This is bad', 14,1) END However, this value does not carry over to a secondary trigger for the same table. If you raise an error at the end of the first trigger and then look at @@ERROR in the secondary trigger the @@ERROR remains 0. Carrying Forward an Active/Open Transaction It is possible to exit from a trigger and carry forward an open transaction by issuing a BEGIN TRAN or by setting implicit transaction on and doing INSERT, UPDATE, or DELETE. Warning It is never recommended that a trigger call BEGIN TRANSACTION. By doing this you increment the transaction count. Invalid code logic, not calling commit transaction, can lead to a situation where the transaction count remains elevated upon exit of the trigger. Transaction Count The behavior is better explained by understanding how the server works. It does not matter whether you are in a transaction, when a modification takes place the transaction count is incremented. So, in the simplest form, during the processing of an insert the transaction count is 1. On completion of the insert, the server will commit (and thus decrement the transaction count). If the commit identifies the transaction count has returned to 0, the actual commit processing is completed. Issuing a commit when the transaction count is greater than 1 simply decrements the nested transaction counter. Thus, when we enter a trigger, the transaction count is 1. At the completion of the trigger, the transaction count will be 0 due to the commit issued at the end of the modification statement (insert). In our example, if the connection was already in a transaction and called the second INSERT, since implicit transaction is ON, the transaction count in the trigger will be 2 as long as the ROLLBACK is not executed. At the end of the insert, the commit is again issued to decrement the transaction reference count to 1. However, the value does not return to 0 so the transaction remains open/active. Subsequent triggers are only fired if the transaction count at the end of the trigger remains greater than or equal to 1. The key to continuation of secondary triggers and the batch is the transaction count at the end of a trigger execution. If the trigger that performs a rollback has done an explicit begin transaction or uses implicit transactions, subsequent triggers and the batch will continue. If the transaction count is not 1 or greater, subsequent triggers and the batch will not execute. Warning Forcing the transaction count after issuing a rollback is dangerous because you can easily loose track of your transaction nesting level. When performing an explicit rollback in a trigger, you should immediately issue a return statement to maintain consistent behavior between a connection with and without implicit transaction settings. This will force the trigger(s) and batch to terminate immediately. One of the methods of dealing with this issue is to run ‘SET IMPLICIT_TRANSACTIONS OFF’ as the first statement of any trigger. Other methods may entails checking @@TRANCOUNT at the end of the trigger and continue to COMMIT the transaction as long as @@TRANCOUNT is greater than 1. Examples The following examples are based on this table: create table tbl50000Insert (iID int NOT NULL) go Note If more than one trigger is used, to guarantee the trigger firing sequence, the sp_settriggerorder command should be used. This command is omitted in these examples to simplify the complexity of the statements. First Example In the first example, the second trigger was never fired and the batch, starting with the insert statement, was aborted. Thus, the print statement was never issued. print('Trigger issues rollback - cancels batch') go create trigger trg50000Insert on tbl50000Insert for INSERT as begin select 'Inserted', * from inserted rollback tran select 'End of trigger', @@TRANCOUNT as 'TRANCOUNT' end go create trigger trg50000Insert2 on tbl50000Insert for INSERT as begin select 'In Trigger2' select 'Trigger 2 Inserted', * from inserted end go insert into tbl50000Insert values(1) print('---------------------- In same batch') select * from tbl50000Insert go -- Cleanup drop trigger trg50000Insert drop trigger trg50000Insert2 go delete from tbl50000Insert Second Example The next example shows that since a new transaction is started, the second trigger will be fired and the print statement in the batch will be executed. Note that the insert is rolled back. print('Trigger issues rollback - increases tran count to continue batch') go create trigger trg50000Insert on tbl50000Insert for INSERT as begin select 'Inserted', * from inserted rollback tran begin tran end go create trigger trg50000Insert2 on tbl50000Insert for INSERT as begin select 'In Trigger2' select 'Trigger 2 Inserted', * from inserted end go insert into tbl50000Insert values(2) print('---------------------- In same batch') select * from tbl50000Insert go -- Cleanup drop trigger trg50000Insert drop trigger trg50000Insert2 go delete from tbl50000Insert Third Example In the third example, the raiserror statement is used to set the @@ERROR value and the BEGIN TRAN statement is used in the trigger to allow the batch to continue to run. print('Trigger issues rollback - uses raiserror to set @@ERROR') go create trigger trg50000Insert on tbl50000Insert for INSERT as begin select 'Inserted', * from inserted rollback tran begin tran -- Increase @@trancount to allow -- batch to continue select @@trancount as ‘Trancount’ raiserror('This is from the trigger', 14,1) end go insert into tbl50000Insert values(3) select @@ERROR as 'ERROR', @@TRANCOUNT as 'Trancount' go -- Cleanup drop trigger trg50000Insert go delete from tbl50000Insert Fourth Example For the fourth example, a second trigger is added to illustrate the fact that @@ERROR value set in the first trigger will not be seen in the second trigger nor will it show up in the batch after the second trigger is fired. print('Trigger issues rollback - uses raiserror to set @@ERROR, not seen in second trigger and cleared in batch') go create trigger trg50000Insert on tbl50000Insert for INSERT as begin select 'Inserted', * from inserted rollback begin tran -- Increase @@trancount to -- allow batch to continue select @@TRANCOUNT as 'Trancount' raiserror('This is from the trigger', 14,1) end go create trigger trg50000Insert2 on tbl50000Insert for INSERT as begin select @@ERROR as 'ERROR', @@TRANCOUNT as 'Trancount' end go insert into tbl50000Insert values(4) select @@ERROR as 'ERROR', @@TRANCOUNT as 'Trancount' go -- Cleanup drop trigger trg50000Insert drop trigger trg50000Insert2 go delete from tbl50000Insert Lesson 3: Concepts – Locks and Applications This lesson outlines some of the common causes that contribute to the perception of a slow server. What You Will Learn After completing this lesson, you will be able to:  Explain how lock hints are used and their impact.  Discuss the effect on locking when an application uses Microsoft Transaction Server.  Identify the different kinds of deadlocks including distributed deadlock. Recommended Reading  Charter 14 “Locking”, Inside SQL Server 2000 by Kalen Delaney  Charter 16 “Query Tuning”, Inside SQL Server 2000 by Kalen Delaney Q239753 – Deadlock Situation Not Detected by SQL Server Q288752 – Blocked SPID Not Participating in Deadlock May Incorrectly be Chosen as victim Locking Hints UPDLOCK If update locks are used instead of shared locks while reading a table, the locks are held until the end of the statement or transaction. UPDLOCK has the advantage of allowing you to read data (without blocking other readers) and update it later with the assurance that the data has not changed since you last read it. READPAST READPAST is an optimizer hint for use with SELECT statements. When this hint is used, SQL Server will read past locked rows. For example, assume table T1 contains a single integer column with the values of 1, 2, 3, 4, and 5. If transaction A changes the value of 3 to 8 but has not yet committed, a SELECT * FROM T1 (READPAST) yields values 1, 2, 4, 5. Tip READPAST only applies to transactions operating at READ COMMITTED isolation and only reads past row-level locks. This lock hint can be used to implement a work queue on a SQL Server table. For example, assume there are many external work requests being thrown into a table and they should be serviced in approximate insertion order but they do not have to be completely FIFO. If you have 4 worker threads consuming work items from the queue they could each pick up a record using read past locking and then delete the entry from the queue and commit when they're done. If they fail, they could rollback, leaving the entry on the queue for the next worker thread to pick up. Caution The READPAST hint is not compatible with HOLDLOCK.  Try This: Using Locking Hints 1. Open a Query Window and connect to the pubs database. 2. Execute the following statements (--Conn 1 is optional to help you keep track of each connection): BEGIN TRANSACTION -- Conn 1 UPDATE titles SET price = price * 0.9 WHERE title_id = 'BU1032' 3. Open a second connection and execute the following statements: SELECT @@lock_timeout -- Conn 2 GO SELECT * FROM titles SELECT * FROM authors 4. Open a third connection and execute the following statements: SET LOCK_TIMEOUT 0 -- Conn 3 SELECT * FROM titles SELECT * FROM authors 5. Open a fourth connection and execute the following statement: SELECT * FROM titles (READPAST) -- Conn 4 WHERE title_ID < 'C' SELECT * FROM authors How many records were returned? 3 6. Open a fifth connection and execute the following statement: SELECT * FROM titles (NOLOCK) -- Conn 5 WHERE title_ID 0 the lock manager also checks for deadlocks every time a SPID gets blocked. So a single deadlock will trigger 20 seconds of more immediate deadlock detection, but if no additional deadlocks occur in that 20 seconds, the lock manager no longer checks for deadlocks at each block and detection again only happens every 5 seconds. Although normally not needed, you may use trace flag -T1205 to trace the deadlock detection process. Note Please note the distinction between application lock and other locks’ deadlock detection. For application lock, we do not rollback the transaction of the deadlock victim but simply return a -3 to sp_getapplock, which the application needs to handle itself. Deadlock Resolution How is a deadlock resolved? SQL Server picks one of the connections as a deadlock victim. The victim is chosen based on either which is the least expensive transaction (calculated using the number and size of the log records) to roll back or in which process “SET DEADLOCK_PRIORITY LOW” is specified. The victim’s transaction is rolled back, held locks are released, and SQL Server sends error 1205 to the victim’s client application to notify it that it was chosen as a victim. The other process can then obtain access to the resource it was waiting on and continue. Error 1205: Your transaction (process ID #%d) was deadlocked with another process and has been chosen as the deadlock victim. Rerun your transaction. Symptoms of deadlocking Error 1205 usually is not written to the SQL Server errorlog. Unfortunately, you cannot use sp_altermessage to cause 1205 to be written to the errorlog. If the client application does not capture and display error 1205, some of the symptoms of deadlock occurring are:  Clients complain of mysteriously canceled queries when using certain features of an application.  May be accompanied by excessive blocking. Lock contention increases the chances that a deadlock will occur. Triggers and Deadlock Triggers promote the deadlock priority of the SPID for the life of the trigger execution when the DEADLOCK PRIORITY is not set to low. When a statement in a trigger causes a deadlock to occur, the SPID executing the trigger is given preferential treatment and will not become the victim. Warning Bug 235794 is filed against SQL Server 2000 where a blocked SPID that is not a participant of a deadlock may incorrectly be chosen as a deadlock victim if the SPID is blocked by one of the deadlock participants and the SPID has the least amount of transaction logging. See KB article Q288752: “Blocked Spid Not Participating in Deadlock May Incorrectly be Chosen as victim” for more information. Distributed Deadlock – Scenario 1 Distributed Deadlocks The term distributed deadlock is ambiguous. There are many types of distributed deadlocks. Scenario 1 Client application opens connection A, begins a transaction, acquires some locks, opens connection B, connection B gets blocked by A but the application is designed to not commit A’s transaction until B completes. Note SQL Server has no way of knowing that connection A is somehow dependent on B – they are two distinct connections with two distinct transactions. This situation is discussed in scenario #4 in “Q224453 INF: Understanding and Resolving SQL Server 7.0 Blocking Problems”. Distributed Deadlock – Scenario 2 Scenario 2 Distributed deadlock involving bound connections. Two connections can be bound into a single transaction context with sp_getbindtoken/sp_bindsession or via DTC. Spid 60 enlists in a transaction with spid 61. A third spid 62 is blocked by spid 60, but spid 61 is blocked by spid 62. Because they are doing work in the same transaction, spid 60 cannot commit until spid 61 finishes his work, but spid 61 is blocked by 62 who is blocked by 60. This scenario is described in article “Q239753 - Deadlock Situation Not Detected by SQL Server.” Note SQL Server 6.5 and 7.0 do not detect this deadlock. The SQL Server 2000 deadlock detection algorithm has been enhanced to detect this type of distributed deadlock. The diagram in the slide illustrates this situation. Resources locked by a spid are below that spid (in a box). Arrows indicate blocking and are drawn from the blocked spid to the resource that the spid requires. A circle represents a transaction; spids in the same transaction are shown in the same circle. Distributed Deadlock – Scenario 3 Scenario 3 Distributed deadlock involving linked servers or server-to-server RPC. Spid 60 on Server 1 executes a stored procedure on Server 2 via linked server. This stored procedure does a loopback linked server query against a table on Server 1, and this connection is blocked by a lock held by Spid 60. Note No version of SQL Server is currently designed to detect this distributed deadlock. Lesson 4: Information Collection and Analysis This lesson outlines some of the common causes that contribute to the perception of a slow server. What You Will Learn After completing this lesson, you will be able to:  Identify specific information needed for troubleshooting issues.  Locate and collect information needed for troubleshooting issues.  Analyze output of DBCC Inputbuffer, DBCC PSS, and DBCC Page commands.  Review information collected from master.dbo.sysprocesses table.  Review information collected from master.dbo.syslockinfo table.  Review output of sp_who, sp_who2, sp_lock.  Analyze Profiler log for query usage pattern.  Review output of trace flags to help troubleshoot deadlocks. Recommended Reading Q244455 - INF: Definition of Sysprocesses Waittype and Lastwaittype Fields Q244456 - INF: Description of DBCC PSS Command for SQL Server 7.0 Q271509 - INF: How to Monitor SQL Server 2000 Blocking Q251004 - How to Monitor SQL Server 7.0 Blocking Q224453 - Understanding and Resolving SQL Server 7.0 Blocking Problem Q282749 – BUG: Deadlock information reported with SQL Server 2000 Profiler Locking and Blocking  Try This: Examine Blocked Processes 1. Open a Query Window and connect to the pubs database. Execute the following statements: BEGIN TRAN -- connection 1 UPDATE titles SET price = price + 1 2. Open another connection and execute the following statement: SELECT * FROM titles-- connection 2 3. Open a third connection and execute sp_who; note the process id (spid) of the blocked process. (Connection 3) 4. In the same connection, execute the following: SELECT spid, cmd, waittype FROM master..sysprocesses WHERE waittype 0 -- connection 3 5. Do not close any of the connections! What was the wait type of the blocked process?  Try This: Look at locks held Assumes all your connections are still open from the previous exercise. • Execute sp_lock -- Connection 3 What locks is the process from the previous example holding? Make sure you run ROLLBACK TRAN in Connection 1 to clean up your transaction. Collecting Information See Module 2 for more about how to gather this information using various tools. Recognizing Blocking Problems How to Recognize Blocking Problems  Users complain about poor performance at a certain time of day, or after a certain number of users connect.  SELECT * FROM sysprocesses or sp_who2 shows non-zero values in the blocked or BlkBy column.  More severe blocking incidents will have long blocking chains or large sysprocesses.waittime values for blocked spids.  Possibl
[PHP] ;;;;;;;;;;;;;;;;;;; ; About php.ini ; ;;;;;;;;;;;;;;;;;;; ; PHP's initialization file, generally called php.ini, is responsible for ; configuring many of the aspects of PHP's behavior. ; PHP attempts to find and load this configuration from a number of locations. ; The following is a summary of its search order: ; 1. SAPI module specific location. ; 2. The PHPRC environment variable. (As of PHP 5.2.0) ; 3. A number of predefined registry keys on Windows (As of PHP 5.2.0) ; 4. Current working directory (except CLI) ; 5. The web server's directory (for SAPI modules), or directory of PHP ; (otherwise in Windows) ; 6. The directory from the --with-config-file-path compile time option, or the ; Windows directory (C:\windows or C:\winnt) ; See the PHP docs for more specific information. ; http://php.net/configuration.file ; The syntax of the file is extremely simple. Whitespace and lines ; beginning with a semicolon are silently ignored (as you probably guessed). ; Section headers (e.g. [Foo]) are also silently ignored, even though ; they might mean something in the future. ; Directives following the section heading [PATH=/www/mysite] only ; apply to PHP files in the /www/mysite directory. Directives ; following the section heading [HOST=www.example.com] only apply to ; PHP files served from www.example.com. Directives set in these ; special sections cannot be overridden by user-defined INI files or ; at runtime. Currently, [PATH=] and [HOST=] sections only work under ; CGI/FastCGI. ; http://php.net/ini.sections ; Directives are specified using the following syntax: ; directive = value ; Directive names are *case sensitive* - foo=bar is different from FOO=bar. ; Directives are variables used to configure PHP or PHP extensions. ; There is no name validation. If PHP can't find an expected ; directive because it is not set or is mistyped, a default value will be used. ; The value can be a string, a number, a PHP constant (e.g. E_ALL or M_PI), one ; of the INI constants (On, Off, True, False, Yes, No and None) or an expression ; (e.g. E_ALL & ~E_NOTICE), a quoted string ("bar"), or a reference to a ; previously set variable or directive (e.g. ${foo}) ; Expressions in the INI file are limited to bitwise operators and parentheses: ; | bitwise OR ; ^ bitwise XOR ; & bitwise AND ; ~ bitwise NOT ; ! boolean NOT ; Boolean flags can be turned on using the values 1, On, True or Yes. ; They can be turned off using the values 0, Off, False or No. ; An empty string can be denoted by simply not writing anything after the equal ; sign, or by using the None keyword: ; foo = ; sets foo to an empty string ; foo = None ; sets foo to an empty string ; foo = "None" ; sets foo to the string 'None' ; If you use constants in your value, and these constants belong to a ; dynamically loaded extension (either a PHP extension or a Zend extension), ; you may only use these constants *after* the line that loads the extension. ;;;;;;;;;;;;;;;;;;; ; About this file ; ;;;;;;;;;;;;;;;;;;; ; PHP comes packaged with two INI files. One that is recommended to be used ; in production environments and one that is recommended to be used in ; development environments. ; php.ini-production contains settings which hold security, performance and ; best practices at its core. But please be aware, these settings may break ; compatibility with older or less security conscience applications. We ; recommending using the production ini in production and testing environments. ; php.ini-development is very similar to its production variant, except it's ; much more verbose when it comes to errors. We recommending using the ; development version only in development environments as errors shown to ; application users can inadvertently leak otherwise secure information. ; This is php.ini-development INI file. ;;;;;;;;;;;;;;;;;;; ; Quick Reference ; ;;;;;;;;;;;;;;;;;;; ; The following are all the settings which are different in either the production ; or development versions of the INIs with respect to PHP's default behavior. ; Please see the actual settings later in the document for more details as to why ; we recommend these changes in PHP's behavior. ; display_errors ; Default Value: On ; Development Value: On ; Production Value: Off ; display_startup_errors ; Default Value: Off ; Development Value: On ; Production Value: Off ; error_reporting ; Default Value: E_ALL & ~E_NOTICE & ~E_STRICT & ~E_DEPRECATED ; Development Value: E_ALL ; Production Value: E_ALL & ~E_DEPRECATED & ~E_STRICT ; html_errors ; Default Value: On ; Development Value: On ; Production value: On ; log_errors ; Default Value: Off ; Development Value: On ; Production Value: On ; max_input_time ; Default Value: -1 (Unlimited) ; Development Value: 60 (60 seconds) ; Production Value: 60 (60 seconds) ; output_buffering ; Default Value: Off ; Development Value: 4096 ; Production Value: 4096 ; register_argc_argv ; Default Value: On ; Development Value: Off ; Production Value: Off ; request_order ; Default Value: None ; Development Value: "GP" ; Production Value: "GP" ; session.gc_divisor ; Default Value: 100 ; Development Value: 1000 ; Production Value: 1000 ; session.hash_bits_per_character ; Default Value: 4 ; Development Value: 5 ; Production Value: 5 ; short_open_tag ; Default Value: On ; Development Value: Off ; Production Value: Off ; track_errors ; Default Value: Off ; Development Value: On ; Production Value: Off ; url_rewriter.tags ; Default Value: "a=href,area=href,frame=src,form=,fieldset=" ; Development Value: "a=href,area=href,frame=src,input=src,form=fakeentry" ; Production Value: "a=href,area=href,frame=src,input=src,form=fakeentry" ; variables_order ; Default Value: "EGPCS" ; Development Value: "GPCS" ; Production Value: "GPCS" ;;;;;;;;;;;;;;;;;;;; ; php.ini Options ; ;;;;;;;;;;;;;;;;;;;; ; Name for user-defined php.ini (.htaccess) files. Default is ".user.ini" ;user_ini.filename = ".user.ini" ; To disable this feature set this option to empty value ;user_ini.filename = ; TTL for user-defined php.ini files (time-to-live) in seconds. Default is 300 seconds (5 minutes) ;user_ini.cache_ttl = 300 ;;;;;;;;;;;;;;;;;;;; ; Language Options ; ;;;;;;;;;;;;;;;;;;;; ; Enable the PHP scripting language engine under Apache. ; http://php.net/engine engine = On ; This directive determines whether or not PHP will recognize code between ; <? and ?> tags as PHP source which should be processed as such. It is ; generally recommended that <?php and ?> should be used and that this feature ; should be disabled, as enabling it may result in issues when generating XML ; documents, however this remains supported for backward compatibility reasons. ; Note that this directive does not control the <?= shorthand tag, which can be ; used regardless of this directive. ; Default Value: On ; Development Value: Off ; Production Value: Off ; http://php.net/short-open-tag short_open_tag = Off ; Allow ASP-style <% %> tags. ; http://php.net/asp-tags asp_tags = Off ; The number of significant digits displayed in floating point numbers. ; http://php.net/precision precision = 14 ; Output buffering is a mechanism for controlling how much output data ; (excluding headers and cookies) PHP should keep internally before pushing that ; data to the client. If your application's output exceeds this setting, PHP ; will send that data in chunks of roughly the size you specify. ; Turning on this setting and managing its maximum buffer size can yield some ; interesting side-effects depending on your application and web server. ; You may be able to send headers and cookies after you've already sent output ; through print or echo. You also may see performance benefits if your server is ; emitting less packets due to buffered output versus PHP streaming the output ; as it gets it. On production servers, 4096 bytes is a good setting for performance ; reasons. ; Note: Output buffering can also be controlled via Output Buffering Control ; functions. ; Possible Values: ; On = Enabled and buffer is unlimited. (Use with caution) ; Off = Disabled ; Integer = Enables the buffer and sets its maximum size in bytes. ; Note: This directive is hardcoded to Off for the CLI SAPI ; Default Value: Off ; Development Value: 4096 ; Production Value: 4096 ; http://php.net/output-buffering output_buffering = 4096 ; You can redirect all of the output of your scripts to a function. For ; example, if you set output_handler to "mb_output_handler", character ; encoding will be transparently converted to the specified encoding. ; Setting any output handler automatically turns on output buffering. ; Note: People who wrote portable scripts should not depend on this ini ; directive. Instead, explicitly set the output handler using ob_start(). ; Using this ini directive may cause problems unless you know what script ; is doing. ; Note: You cannot use both "mb_output_handler" with "ob_iconv_handler" ; and you cannot use both "ob_gzhandler" and "zlib.output_compression". ; Note: output_handler must be empty if this is set 'On' !!!! ; Instead you must use zlib.output_handler. ; http://php.net/output-handler ;output_handler = ; Transparent output compression using the zlib library ; Valid values for this option are 'off', 'on', or a specific buffer size ; to be used for compression (default is 4KB) ; Note: Resulting chunk size may vary due to nature of compression. PHP ; outputs chunks that are few hundreds bytes each as a result of ; compression. If you prefer a larger chunk size for better ; performance, enable output_buffering in addition. ; Note: You need to use zlib.output_handler instead of the standard ; output_handler, or otherwise the output will be corrupted. ; http://php.net/zlib.output-compression zlib.output_compression = Off ; http://php.net/zlib.output-compression-level ;zlib.output_compression_level = -1 ; You cannot specify additional output handlers if zlib.output_compression ; is activated here. This setting does the same as output_handler but in ; a different order. ; http://php.net/zlib.output-handler ;zlib.output_handler = ; Implicit flush tells PHP to tell the output layer to flush itself ; automatically after every output block. This is equivalent to calling the ; PHP function flush() after each and every call to print() or echo() and each ; and every HTML block. Turning this option on has serious performance ; implications and is generally recommended for debugging purposes only. ; http://php.net/implicit-flush ; Note: This directive is hardcoded to On for the CLI SAPI implicit_flush = Off ; The unserialize callback function will be called (with the undefined class' ; name as parameter), if the unserializer finds an undefined class ; which should be instantiated. A warning appears if the specified function is ; not defined, or if the function doesn't include/implement the missing class. ; So only set this entry, if you really want to implement such a ; callback-function. unserialize_callback_func = ; When floats & doubles are serialized store serialize_precision significant ; digits after the floating point. The default value ensures that when floats ; are decoded with unserialize, the data will remain the same. serialize_precision = 17 ; open_basedir, if set, limits all file operations to the defined directory ; and below. This directive makes most sense if used in a per-directory ; or per-virtualhost web server configuration file. This directive is ; *NOT* affected by whether Safe Mode is turned On or Off. ; http://php.net/open-basedir ;open_basedir = ; This directive allows you to disable certain functions for security reasons. ; It receives a comma-delimited list of function names. This directive is ; *NOT* affected by whether Safe Mode is turned On or Off. ; http://php.net/disable-functions disable_functions = ; This directive allows you to disable certain classes for security reasons. ; It receives a comma-delimited list of class names. This directive is ; *NOT* affected by whether Safe Mode is turned On or Off. ; http://php.net/disable-classes disable_classes = ; Colors for Syntax Highlighting mode. Anything that's acceptable in ; <span style="color: ???????"> would work. ; http://php.net/syntax-highlighting ;highlight.string = #DD0000 ;highlight.comment = #FF9900 ;highlight.keyword = #007700 ;highlight.default = #0000BB ;highlight.html = #000000 ; If enabled, the request will be allowed to complete even if the user aborts ; the request. Consider enabling it if executing long requests, which may end up ; being interrupted by the user or a browser timing out. PHP's default behavior ; is to disable this feature. ; http://php.net/ignore-user-abort ;ignore_user_abort = On ; Determines the size of the realpath cache to be used by PHP. This value should ; be increased on systems where PHP opens many files to reflect the quantity of ; the file operations performed. ; http://php.net/realpath-cache-size ;realpath_cache_size = 16k ; Duration of time, in seconds for which to cache realpath information for a given ; file or directory. For systems with rarely changing files, consider increasing this ; value. ; http://php.net/realpath-cache-ttl ;realpath_cache_ttl = 120 ; Enables or disables the circular reference collector. ; http://php.net/zend.enable-gc zend.enable_gc = On ; If enabled, scripts may be written in encodings that are incompatible with ; the scanner. CP936, Big5, CP949 and Shift_JIS are the examples of such ; encodings. To use this feature, mbstring extension must be enabled. ; Default: Off ;zend.multibyte = Off ; Allows to set the default encoding for the scripts. This value will be used ; unless "declare(encoding=...)" directive appears at the top of the script. ; Only affects if zend.multibyte is set. ; Default: "" ;zend.script_encoding = ;;;;;;;;;;;;;;;;; ; Miscellaneous ; ;;;;;;;;;;;;;;;;; ; Decides whether PHP may expose the fact that it is installed on the server ; (e.g. by adding its signature to the Web server header). It is no security ; threat in any way, but it makes it possible to determine whether you use PHP ; on your server or not. ; http://php.net/expose-php expose_php = On ;;;;;;;;;;;;;;;;;;; ; Resource Limits ; ;;;;;;;;;;;;;;;;;;; ; Maximum execution time of each script, in seconds ; http://php.net/max-execution-time ; Note: This directive is hardcoded to 0 for the CLI SAPI max_execution_time = 30 ; Maximum amount of time each script may spend parsing request data. It's a good ; idea to limit this time on productions servers in order to eliminate unexpectedly ; long running scripts. ; Note: This directive is hardcoded to -1 for the CLI SAPI ; Default Value: -1 (Unlimited) ; Development Value: 60 (60 seconds) ; Production Value: 60 (60 seconds) ; http://php.net/max-input-time max_input_time = 60 ; Maximum input variable nesting level ; http://php.net/max-input-nesting-level ;max_input_nesting_level = 64 ; How many GET/POST/COOKIE input variables may be accepted ; max_input_vars = 1000 ; Maximum amount of memory a script may consume (128MB) ; http://php.net/memory-limit memory_limit = 128M ;;;;;;;;;;;;;;;;;;;;;;;;;;;;;; ; Error handling and logging ; ;;;;;;;;;;;;;;;;;;;;;;;;;;;;;; ; This directive informs PHP of which errors, warnings and notices you would like ; it to take action for. The recommended way of setting values for this ; directive is through the use of the error level constants and bitwise ; operators. The error level constants are below here for convenience as well as ; some common settings and their meanings. ; By default, PHP is set to take action on all errors, notices and warnings EXCEPT ; those related to E_NOTICE and E_STRICT, which together cover best practices and ; recommended coding standards in PHP. For performance reasons, this is the ; recommend error reporting setting. Your production server shouldn't be wasting ; resources complaining about best practices and coding standards. That's what ; development servers and development settings are for. ; Note: The php.ini-development file has this setting as E_ALL. This ; means it pretty much reports everything which is exactly what you want during ; development and early testing. ; ; Error Level Constants: ; E_ALL - All errors and warnings (includes E_STRICT as of PHP 5.4.0) ; E_ERROR - fatal run-time errors ; E_RECOVERABLE_ERROR - almost fatal run-time errors ; E_WARNING - run-time warnings (non-fatal errors) ; E_PARSE - compile-time parse errors ; E_NOTICE - run-time notices (these are warnings which often result ; from a bug in your code, but it's possible that it was ; intentional (e.g., using an uninitialized variable and ; relying on the fact it's automatically initialized to an ; empty string) ; E_STRICT - run-time notices, enable to have PHP suggest changes ; to your code which will ensure the best interoperability ; and forward compatibility of your code ; E_CORE_ERROR - fatal errors that occur during PHP's initial startup ; E_CORE_WARNING - warnings (non-fatal errors) that occur during PHP's ; initial startup ; E_COMPILE_ERROR - fatal compile-time errors ; E_COMPILE_WARNING - compile-time warnings (non-fatal errors) ; E_USER_ERROR - user-generated error message ; E_USER_WARNING - user-generated warning message ; E_USER_NOTICE - user-generated notice message ; E_DEPRECATED - warn about code that will not work in future versions ; of PHP ; E_USER_DEPRECATED - user-generated deprecation warnings ; ; Common Values: ; E_ALL (Show all errors, warnings and notices including coding standards.) ; E_ALL & ~E_NOTICE (Show all errors, except for notices) ; E_ALL & ~E_NOTICE & ~E_STRICT (Show all errors, except for notices and coding standards warnings.) ; E_COMPILE_ERROR|E_RECOVERABLE_ERROR|E_ERROR|E_CORE_ERROR (Show only errors) ; Default Value: E_ALL & ~E_NOTICE & ~E_STRICT & ~E_DEPRECATED ; Development Value: E_ALL ; Production Value: E_ALL & ~E_DEPRECATED & ~E_STRICT ; http://php.net/error-reporting error_reporting = E_ALL ; This directive controls whether or not and where PHP will output errors, ; notices and warnings too. Error output is very useful during development, but ; it could be very dangerous in production environments. Depending on the code ; which is triggering the error, sensitive information could potentially leak ; out of your application such as database usernames and passwords or worse. ; It's recommended that errors be logged on production servers rather than ; having the errors sent to STDOUT. ; Possible Values: ; Off = Do not display any errors ; stderr = Display errors to STDERR (affects only CGI/CLI binaries!) ; On or stdout = Display errors to STDOUT ; Default Value: On ; Development Value: On ; Production Value: Off ; http://php.net/display-errors display_errors = On ; The display of errors which occur during PHP's startup sequence are handled ; separately from display_errors. PHP's default behavior is to suppress those ; errors from clients. Turning the display of startup errors on can be useful in ; debugging configuration problems. But, it's strongly recommended that you ; leave this setting off on production servers. ; Default Value: Off ; Development Value: On ; Production Value: Off ; http://php.net/display-startup-errors display_startup_errors = On ; Besides displaying errors, PHP can also log errors to locations such as a ; server-specific log, STDERR, or a location specified by the error_log ; directive found below. While errors should not be displayed on productions ; servers they should still be monitored and logging is a great way to do that. ; Default Value: Off ; Development Value: On ; Production Value: On ; http://php.net/log-errors log_errors = On ; Set maximum length of log_errors. In error_log information about the source is ; added. The default is 1024 and 0 allows to not apply any maximum length at all. ; http://php.net/log-errors-max-len log_errors_max_len = 1024 ; Do not log repeated messages. Repeated errors must occur in same file on same ; line unless ignore_repeated_source is set true. ; http://php.net/ignore-repeated-errors ignore_repeated_errors = Off ; Ignore source of message when ignoring repeated messages. When this setting ; is On you will not log errors with repeated messages from different files or ; source lines. ; http://php.net/ignore-repeated-source ignore_repeated_source = Off ; If this parameter is set to Off, then memory leaks will not be shown (on ; stdout or in the log). This has only effect in a debug compile, and if ; error reporting includes E_WARNING in the allowed list ; http://php.net/report-memleaks report_memleaks = On ; This setting is on by default. ;report_zend_debug = 0 ; Store the last error/warning message in $php_errormsg (boolean). Setting this value ; to On can assist in debugging and is appropriate for development servers. It should ; however be disabled on production servers. ; Default Value: Off ; Development Value: On ; Production Value: Off ; http://php.net/track-errors track_errors = On ; Turn off normal error reporting and emit XML-RPC error XML ; http://php.net/xmlrpc-errors ;xmlrpc_errors = 0 ; An XML-RPC faultCode ;xmlrpc_error_number = 0 ; When PHP displays or logs an error, it has the capability of formatting the ; error message as HTML for easier reading. This directive controls whether ; the error message is formatted as HTML or not. ; Note: This directive is hardcoded to Off for the CLI SAPI ; Default Value: On ; Development Value: On ; Production value: On ; http://php.net/html-errors html_errors = On ; If html_errors is set to On *and* docref_root is not empty, then PHP ; produces clickable error messages that direct to a page describing the error ; or function causing the error in detail. ; You can download a copy of the PHP manual from http://php.net/docs ; and change docref_root to the base URL of your local copy including the ; leading '/'. You must also specify the file extension being used including ; the dot. PHP's default behavior is to leave these settings empty, in which ; case no links to documentation are generated. ; Note: Never use this feature for production boxes. ; http://php.net/docref-root ; Examples ;docref_root = "/phpmanual/" ; http://php.net/docref-ext ;docref_ext = .html ; String to output before an error message. PHP's default behavior is to leave ; this setting blank. ; http://php.net/error-prepend-string ; Example: ;error_prepend_string = "<span style='color: #ff0000'>" ; String to output after an error message. PHP's default behavior is to leave ; this setting blank. ; http://php.net/error-append-string ; Example: ;error_append_string = "</span>" ; Log errors to specified file. PHP's default behavior is to leave this value ; empty. ; http://php.net/error-log ; Example: ;error_log = php_errors.log ; Log errors to syslog (Event Log on Windows). ;error_log = syslog ;windows.show_crt_warning ; Default value: 0 ; Development value: 0 ; Production value: 0 ;;;;;;;;;;;;;;;;; ; Data Handling ; ;;;;;;;;;;;;;;;;; ; The separator used in PHP generated URLs to separate arguments. ; PHP's default setting is "&". ; http://php.net/arg-separator.output ; Example: ;arg_separator.output = "&" ; List of separator(s) used by PHP to parse input URLs into variables. ; PHP's default setting is "&". ; NOTE: Every character in this directive is considered as separator! ; http://php.net/arg-separator.input ; Example: ;arg_separator.input = ";&" ; This directive determines which super global arrays are registered when PHP ; starts up. G,P,C,E & S are abbreviations for the following respective super ; globals: GET, POST, COOKIE, ENV and SERVER. There is a performance penalty ; paid for the registration of these arrays and because ENV is not as commonly ; used as the others, ENV is not recommended on productions servers. You ; can still get access to the environment variables through getenv() should you ; need to. ; Default Value: "EGPCS" ; Development Value: "GPCS" ; Production Value: "GPCS"; ; http://php.net/variables-order variables_order = "GPCS" ; This directive determines which super global data (G,P,C,E & S) should ; be registered into the super global array REQUEST. If so, it also determines ; the order in which that data is registered. The values for this directive are ; specified in the same manner as the variables_order directive, EXCEPT one. ; Leaving this value empty will cause PHP to use the value set in the ; variables_order directive. It does not mean it will leave the super globals ; array REQUEST empty. ; Default Value: None ; Development Value: "GP" ; Production Value: "GP" ; http://php.net/request-order request_order = "GP" ; This directive determines whether PHP registers $argv & $argc each time it ; runs. $argv contains an array of all the arguments passed to PHP when a script ; is invoked. $argc contains an integer representing the number of arguments ; that were passed when the script was invoked. These arrays are extremely ; useful when running scripts from the command line. When this directive is ; enabled, registering these variables consumes CPU cycles and memory each time ; a script is executed. For performance reasons, this feature should be disabled ; on production servers. ; Note: This directive is hardcoded to On for the CLI SAPI ; Default Value: On ; Development Value: Off ; Production Value: Off ; http://php.net/register-argc-argv register_argc_argv = Off ; When enabled, the ENV, REQUEST and SERVER variables are created when they're ; first used (Just In Time) instead of when the script starts. If these ; variables are not used within a script, having this directive on will result ; in a performance gain. The PHP directive register_argc_argv must be disabled ; for this directive to have any affect. ; http://php.net/auto-globals-jit auto_globals_jit = On ; Whether PHP will read the POST data. ; This option is enabled by default. ; Most likely, you won't want to disable this option globally. It causes $_POST ; and $_FILES to always be empty; the only way you will be able to read the ; POST data will be through the php://input stream wrapper. This can be useful ; to proxy requests or to process the POST data in a memory efficient fashion. ; http://php.net/enable-post-data-reading ;enable_post_data_reading = Off ; Maximum size of POST data that PHP will accept. ; Its value may be 0 to disable the limit. It is ignored if POST data reading ; is disabled through enable_post_data_reading. ; http://php.net/post-max-size post_max_size = 8M ; Automatically add files before PHP document. ; http://php.net/auto-prepend-file auto_prepend_file = ; Automatically add files after PHP document. ; http://php.net/auto-append-file auto_append_file = ; By default, PHP will output a character encoding using ; the Content-type: header. To disable sending of the charset, simply ; set it to be empty. ; ; PHP's built-in default is text/html ; http://php.net/default-mimetype default_mimetype = "text/html" ; PHP's default character set is set to empty. ; http://php.net/default-charset ;default_charset = "UTF-8" ; Always populate the $HTTP_RAW_POST_DATA variable. PHP's default behavior is ; to disable this feature. If post reading is disabled through ; enable_post_data_reading, $HTTP_RAW_POST_DATA is *NOT* populated. ; http://php.net/always-populate-raw-post-data ;always_populate_raw_post_data = On ;;;;;;;;;;;;;;;;;;;;;;;;; ; Paths and Directories ; ;;;;;;;;;;;;;;;;;;;;;;;;; ; UNIX: "/path1:/path2" ;include_path = ".:/php/includes" ; ; Windows: "\path1;\path2" ;include_path = ".;c:\php\includes" ; ; PHP's default setting for include_path is ".;/path/to/php/pear" ; http://php.net/include-path ; The root of the PHP pages, used only if nonempty. ; if PHP was not compiled with FORCE_REDIRECT, you SHOULD set doc_root ; if you are running php as a CGI under any web server (other than IIS) ; see documentation for security issues. The alternate is to use the ; cgi.force_redirect configuration below ; http://php.net/doc-root doc_root = ; The directory under which PHP opens the script using /~username used only ; if nonempty. ; http://php.net/user-dir user_dir = ; Directory in which the loadable extensions (modules) reside. ; http://php.net/extension-dir ; extension_dir = "./" ; On windows: ; extension_dir = "ext" ; Whether or not to enable the dl() function. The dl() function does NOT work ; properly in multithreaded servers, such as IIS or Zeus, and is automatically ; disabled on them. ; http://php.net/enable-dl enable_dl = Off ; cgi.force_redirect is necessary to provide security running PHP as a CGI under ; most web servers. Left undefined, PHP turns this on by default. You can ; turn it off here AT YOUR OWN RISK ; **You CAN safely turn this off for IIS, in fact, you MUST.** ; http://php.net/cgi.force-redirect ;cgi.force_redirect = 1 ; if cgi.nph is enabled it will force cgi to always sent Status: 200 with ; every request. PHP's default behavior is to disable this feature. ;cgi.nph = 1 ; if cgi.force_redirect is turned on, and you are not running under Apache or Netscape ; (iPlanet) web servers, you MAY need to set an environment variable name that PHP ; will look for to know it is OK to continue execution. Setting this variable MAY ; cause security issues, KNOW WHAT YOU ARE DOING FIRST. ; http://php.net/cgi.redirect-status-env ;cgi.redirect_status_env = ; cgi.fix_pathinfo provides *real* PATH_INFO/PATH_TRANSLATED support for CGI. PHP's ; previous behaviour was to set PATH_TRANSLATED to SCRIPT_FILENAME, and to not grok ; what PATH_INFO is. For more information on PATH_INFO, see the cgi specs. Setting ; this to 1 will cause PHP CGI to fix its paths to conform to the spec. A setting ; of zero causes PHP to behave as before. Default is 1. You should fix your scripts ; to use SCRIPT_FILENAME rather than PATH_TRANSLATED. ; http://php.net/cgi.fix-pathinfo ;cgi.fix_pathinfo=1 ; FastCGI under IIS (on WINNT based OS) supports the ability to impersonate ; security tokens of the calling client. This allows IIS to define the ; security context that the request runs under. mod_fastcgi under Apache ; does not currently support this feature (03/17/2002) ; Set to 1 if running under IIS. Default is zero. ; http://php.net/fastcgi.impersonate ;fastcgi.impersonate = 1 ; Disable logging through FastCGI connection. PHP's default behavior is to enable ; this feature. ;fastcgi.logging = 0 ; cgi.rfc2616_headers configuration option tells PHP what type of headers to ; use when sending HTTP response code. If it's set 0 PHP sends Status: header that ; is supported by Apache. When this option is set to 1 PHP will send ; RFC2616 compliant header. ; Default is zero. ; http://php.net/cgi.rfc2616-headers ;cgi.rfc2616_headers = 0 ;;;;;;;;;;;;;;;; ; File Uploads ; ;;;;;;;;;;;;;;;; ; Whether to allow HTTP file uploads. ; http://php.net/file-uploads file_uploads = On ; Temporary directory for HTTP uploaded files (will use system default if not ; specified). ; http://php.net/upload-tmp-dir ;upload_tmp_dir = ; Maximum allowed size for uploaded files. ; http://php.net/upload-max-filesize upload_max_filesize = 2M ; Maximum number of files that can be uploaded via a single request max_file_uploads = 20 ;;;;;;;;;;;;;;;;;; ; Fopen wrappers ; ;;;;;;;;;;;;;;;;;; ; Whether to allow the treatment of URLs (like http:// or ftp://) as files. ; http://php.net/allow-url-fopen allow_url_fopen = On ; Whether to allow include/require to open URLs (like http:// or ftp://) as files. ; http://php.net/allow-url-include allow_url_include = Off ; Define the anonymous ftp password (your email address). PHP's default setting ; for this is empty. ; http://php.net/from ;from="[email protected]" ; Define the User-Agent string. PHP's default setting for this is empty. ; http://php.net/user-agent ;user_agent="PHP" ; Default timeout for socket based streams (seconds) ; http://php.net/default-socket-timeout default_socket_timeout = 60 ; If your scripts have to deal with files from Macintosh systems, ; or you are running on a Mac and need to deal with files from ; unix or win32 systems, setting this flag will cause PHP to ; automatically detect the EOL character in those files so that ; fgets() and file() will work regardless of the source of the file. ; http://php.net/auto-detect-line-endings ;auto_detect_line_endings = Off ;;;;;;;;;;;;;;;;;;;;;; ; Dynamic Extensions ; ;;;;;;;;;;;;;;;;;;;;;; ; If you wish to have an extension loaded automatically, use the following ; syntax: ; ; extension=modulename.extension ; ; For example, on Windows: ; ; extension=msql.dll ; ; ... or under UNIX: ; ; extension=msql.so ; ; ... or with a path: ; ; extension=/path/to/extension/msql.so ; ; If you only provide the name of the extension, PHP will look for it in its ; default extension directory. ; ; Windows Extensions ; Note that ODBC support is built in, so no dll is needed for it. ; Note that many DLL files are located in the extensions/ (PHP 4) ext/ (PHP 5) ; extension folders as well as the separate PECL DLL download (PHP 5). ; Be sure to appropriately set the extension_dir directive. ; ;extension=php_bz2.dll ;extension=php_curl.dll ;extension=php_fileinfo.dll ;extension=php_gd2.dll ;extension=php_gettext.dll ;extension=php_gmp.dll ;extension=php_intl.dll ;extension=php_imap.dll ;extension=php_interbase.dll ;extension=php_ldap.dll ;extension=php_mbstring.dll ;extension=php_exif.dll ; Must be after mbstring as it depends on it ;extension=php_mysql.dll ;extension=php_mysqli.dll ;extension=php_oci8.dll ; Use with Oracle 10gR2 Instant Client ;extension=php_oci8_11g.dll ; Use with Oracle 11gR2 Instant Client ;extension=php_openssl.dll ;extension=php_pdo_firebird.dll ;extension=php_pdo_mysql.dll ;extension=php_pdo_oci.dll ;extension=php_pdo_odbc.dll ;extension=php_pdo_pgsql.dll ;extension=php_pdo_sqlite.dll ;extension=php_pgsql.dll ;extension=php_pspell.dll ;extension=php_shmop.dll ; The MIBS data available in the PHP distribution must be installed. ; See http://www.php.net/manual/en/snmp.installation.php ;extension=php_snmp.dll ;extension=php_soap.dll ;extension=php_sockets.dll ;extension=php_sqlite3.dll ;extension=php_sybase_ct.dll ;extension=php_tidy.dll ;extension=php_xmlrpc.dll ;extension=php_xsl.dll ;;;;;;;;;;;;;;;;;;; ; Module Settings ; ;;;;;;;;;;;;;;;;;;; [CLI Server] ; Whether the CLI web server uses ANSI color coding in its terminal output. cli_server.color = On [Date] ; Defines the default timezone used by the date functions ; http://php.net/date.timezone ;date.timezone = ; http://php.net/date.default-latitude ;date.default_latitude = 31.7667 ; http://php.net/date.default-longitude ;date.default_longitude = 35.2333 ; http://php.net/date.sunrise-zenith ;date.sunrise_zenith = 90.583333 ; http://php.net/date.sunset-zenith ;date.sunset_zenith = 90.583333 [filter] ; http://php.net/filter.default ;filter.default = unsafe_raw ; http://php.net/filter.default-flags ;filter.default_flags = [iconv] ;iconv.input_encoding = ISO-8859-1 ;iconv.internal_encoding = ISO-8859-1 ;iconv.output_encoding = ISO-8859-1 [intl] ;intl.default_locale = ; This directive allows you to produce PHP errors when some error ; happens within intl functions. The value is the level of the error produced. ; Default is 0, which does not produce any errors. ;intl.error_level = E_WARNING [sqlite] ; http://php.net/sqlite.assoc-case ;sqlite.assoc_case = 0 [sqlite3] ;sqlite3.extension_dir = [Pcre] ;PCRE library backtracking limit. ; http://php.net/pcre.backtrack-limit ;pcre.backtrack_limit=100000 ;PCRE library recursion limit. ;Please note that if you set this value to a high number you may consume all ;the available process stack and eventually crash PHP (due to reaching the ;stack size limit imposed by the Operating System). ; http://php.net/pcre.recursion-limit ;pcre.recursion_limit=100000 [Pdo] ; Whether to pool ODBC connections. Can be one of "strict", "relaxed" or "off" ; http://php.net/pdo-odbc.connection-pooling ;pdo_odbc.connection_pooling=strict ;pdo_odbc.db2_instance_name [Pdo_mysql] ; If mysqlnd is used: Number of cache slots for the internal result set cache ; http://php.net/pdo_mysql.cache_size pdo_mysql.cache_size = 2000 ; Default socket name for local MySQL connects. If empty, uses the built-in ; MySQL defaults. ; http://php.net/pdo_mysql.default-socket pdo_mysql.default_socket= [Phar] ; http://php.net/phar.readonly ;phar.readonly = On ; http://php.net/phar.require-hash ;phar.require_hash = On ;phar.cache_list = [mail function] ; For Win32 only. ; http://php.net/smtp SMTP = localhost ; http://php.net/smtp-port smtp_port = 25 ; For Win32 only. ; http://php.net/sendmail-from ;sendmail_from = [email protected] ; For Unix only. You may supply arguments as well (default: "sendmail -t -i"). ; http://php.net/sendmail-path ;sendmail_path = ; Force the addition of the specified parameters to be passed as extra parameters ; to the sendmail binary. These parameters will always replace the value of ; the 5th parameter to mail(), even in safe mode. ;mail.force_extra_parameters = ; Add X-PHP-Originating-Script: that will include uid of the script followed by the filename mail.add_x_header = On ; The path to a log file that will log all mail() calls. Log entries include ; the full path of the script, line number, To address and headers. ;mail.log = ; Log mail to syslog (Event Log on Windows). ;mail.log = syslog [SQL] ; http://php.net/sql.safe-mode sql.safe_mode = Off [ODBC] ; http://php.net/odbc.default-db ;odbc.default_db = Not yet implemented ; http://php.net/odbc.default-user ;odbc.default_user = Not yet implemented ; http://php.net/odbc.default-pw ;odbc.default_pw = Not yet implemented ; Controls the ODBC cursor model. ; Default: SQL_CURSOR_STATIC (default). ;odbc.default_cursortype ; Allow or prevent persistent links. ; http://php.net/odbc.allow-persistent odbc.allow_persistent = On ; Check that a connection is still valid before reuse. ; http://php.net/odbc.check-persistent odbc.check_persistent = On ; Maximum number of persistent links. -1 means no limit. ; http://php.net/odbc.max-persistent odbc.max_persistent = -1 ; Maximum number of links (persistent + non-persistent). -1 means no limit. ; http://php.net/odbc.max-links odbc.max_links = -1 ; Handling of LONG fields. Returns number of bytes to variables. 0 means ; passthru. ; http://php.net/odbc.defaultlrl odbc.defaultlrl = 4096 ; Handling of binary data. 0 means passthru, 1 return as is, 2 convert to char. ; See the documentation on odbc_binmode and odbc_longreadlen for an explanation ; of odbc.defaultlrl and odbc.defaultbinmode ; http://php.net/odbc.defaultbinmode odbc.defaultbinmode = 1 ;birdstep.max_links = -1 [Interbase] ; Allow or prevent persistent links. ibase.allow_persistent = 1 ; Maximum number of persistent links. -1 means no limit. ibase.max_persistent = -1 ; Maximum number of links (persistent + non-persistent). -1 means no limit. ibase.max_links = -1 ; Default database name for ibase_connect(). ;ibase.default_db = ; Default username for ibase_connect(). ;ibase.default_user = ; Default password for ibase_connect(). ;ibase.default_password = ; Default charset for ibase_connect(). ;ibase.default_charset = ; Default timestamp format. ibase.timestampformat = "%Y-%m-%d %H:%M:%S" ; Default date format. ibase.dateformat = "%Y-%m-%d" ; Default time format. ibase.timeformat = "%H:%M:%S" [MySQL] ; Allow accessing, from PHP's perspective, local files with LOAD DATA statements ; http://php.net/mysql.allow_local_infile mysql.allow_local_infile = On ; Allow or prevent persistent links. ; http://php.net/mysql.allow-persistent mysql.allow_persistent = On ; If mysqlnd is used: Number of cache slots for the internal result set cache ; http://php.net/mysql.cache_size mysql.cache_size = 2000 ; Maximum number of persistent links. -1 means no limit. ; http://php.net/mysql.max-persistent mysql.max_persistent = -1 ; Maximum number of links (persistent + non-persistent). -1 means no limit. ; http://php.net/mysql.max-links mysql.max_links = -1 ; Default port number for mysql_connect(). If unset, mysql_connect() will use ; the $MYSQL_TCP_PORT or the mysql-tcp entry in /etc/services or the ; compile-time value defined MYSQL_PORT (in that order). Win32 will only look ; at MYSQL_PORT. ; http://php.net/mysql.default-port mysql.default_port = ; Default socket name for local MySQL connects. If empty, uses the built-in ; MySQL defaults. ; http://php.net/mysql.default-socket mysql.default_socket = ; Default host for mysql_connect() (doesn't apply in safe mode). ; http://php.net/mysql.default-host mysql.default_host = ; Default user for mysql_connect() (doesn't apply in safe mode). ; http://php.net/mysql.default-user mysql.default_user = ; Default password for mysql_connect() (doesn't apply in safe mode). ; Note that this is generally a *bad* idea to store passwords in this file. ; *Any* user with PHP access can run 'echo get_cfg_var("mysql.default_password") ; and reveal this password! And of course, any users with read access to this ; file will be able to reveal the password as well. ; http://php.net/mysql.default-password mysql.default_password = ; Maximum time (in seconds) for connect timeout. -1 means no limit ; http://php.net/mysql.connect-timeout mysql.connect_timeout = 60 ; Trace mode. When trace_mode is active (=On), warnings for table/index scans and ; SQL-Errors will be displayed. ; http://php.net/mysql.trace-mode mysql.trace_mode = Off [MySQLi] ; Maximum number of persistent links. -1 means no limit. ; http://php.net/mysqli.max-persistent mysqli.max_persistent = -1 ; Allow accessing, from PHP's perspective, local files with LOAD DATA statements ; http://php.net/mysqli.allow_local_infile ;mysqli.allow_local_infile = On ; Allow or prevent persistent links. ; http://php.net/mysqli.allow-persistent mysqli.allow_persistent = On ; Maximum number of links. -1 means no limit. ; http://php.net/mysqli.max-links mysqli.max_links = -1 ; If mysqlnd is used: Number of cache slots for the internal result set cache ; http://php.net/mysqli.cache_size mysqli.cache_size = 2000 ; Default port number for mysqli_connect(). If unset, mysqli_connect() will use ; the $MYSQL_TCP_PORT or the mysql-tcp entry in /etc/services or the ; compile-time value defined MYSQL_PORT (in that order). Win32 will only look ; at MYSQL_PORT. ; http://php.net/mysqli.default-port mysqli.default_port = 3306 ; Default socket name for local MySQL connects. If empty, uses the built-in ; MySQL defaults. ; http://php.net/mysqli.default-socket mysqli.default_socket = ; Default host for mysql_connect() (doesn't apply in safe mode). ; http://php.net/mysqli.default-host mysqli.default_host = ; Default user for mysql_connect() (doesn't apply in safe mode). ; http://php.net/mysqli.default-user mysqli.default_user = ; Default password for mysqli_connect() (doesn't apply in safe mode). ; Note that this is generally a *bad* idea to store passwords in this file. ; *Any* user with PHP access can run 'echo get_cfg_var("mysqli.default_pw") ; and reveal this password! And of course, any users with read access to this ; file will be able to reveal the password as well. ; http://php.net/mysqli.default-pw mysqli.default_pw = ; Allow or prevent reconnect mysqli.reconnect = Off [mysqlnd] ; Enable / Disable collection of general statistics by mysqlnd which can be ; used to tune and monitor MySQL operations. ; http://php.net/mysqlnd.collect_statistics mysqlnd.collect_statistics = On ; Enable / Disable collection of memory usage statistics by mysqlnd which can be ; used to tune and monitor MySQL operations. ; http://php.net/mysqlnd.collect_memory_statistics mysqlnd.collect_memory_statistics = On ; Size of a pre-allocated buffer used when sending commands to MySQL in bytes. ; http://php.net/mysqlnd.net_cmd_buffer_size ;mysqlnd.net_cmd_buffer_size = 2048 ; Size of a pre-allocated buffer used for reading data sent by the server in ; bytes. ; http://php.net/mysqlnd.net_read_buffer_size ;mysqlnd.net_read_buffer_size = 32768 [OCI8] ; Connection: Enables privileged connections using external ; credentials (OCI_SYSOPER, OCI_SYSDBA) ; http://php.net/oci8.privileged-connect ;oci8.privileged_connect = Off ; Connection: The maximum number of persistent OCI8 connections per ; process. Using -1 means no limit. ; http://php.net/oci8.max-persistent ;oci8.max_persistent = -1 ; Connection: The maximum number of seconds a process is allowed to ; maintain an idle persistent connection. Using -1 means idle ; persistent connections will be maintained forever. ; http://php.net/oci8.persistent-timeout ;oci8.persistent_timeout = -1 ; Connection: The number of seconds that must pass before issuing a ; ping during oci_pconnect() to check the connection validity. When ; set to 0, each oci_pconnect() will cause a ping. Using -1 disables ; pings completely. ; http://php.net/oci8.ping-interval ;oci8.ping_interval = 60 ; Connection: Set this to a user chosen connection class to be used ; for all pooled server requests with Oracle 11g Database Resident ; Connection Pooling (DRCP). To use DRCP, this value should be set to ; the same string for all web servers running the same application, ; the database pool must be configured, and the connection string must ; specify to use a pooled server. ;oci8.connection_class = ; High Availability: Using On lets PHP receive Fast Application ; Notification (FAN) events generated when a database node fails. The ; database must also be configured to post FAN events. ;oci8.events = Off ; Tuning: This option enables statement caching, and specifies how ; many statements to cache. Using 0 disables statement caching. ; http://php.net/oci8.statement-cache-size ;oci8.statement_cache_size = 20 ; Tuning: Enables statement prefetching and sets the default number of ; rows that will be fetched automatically after statement execution. ; http://php.net/oci8.default-prefetch ;oci8.default_prefetch = 100 ; Compatibility. Using On means oci_close() will not close ; oci_connect() and oci_new_connect() connections. ; http://php.net/oci8.old-oci-close-semantics ;oci8.old_oci_close_semantics = Off [PostgreSQL] ; Allow or prevent persistent links. ; http://php.net/pgsql.allow-persistent pgsql.allow_persistent = On ; Detect broken persistent links always with pg_pconnect(). ; Auto reset feature requires a little overheads. ; http://php.net/pgsql.auto-reset-persistent pgsql.auto_reset_persistent = Off ; Maximum number of persistent links. -1 means no limit. ; http://php.net/pgsql.max-persistent pgsql.max_persistent = -1 ; Maximum number of links (persistent+non persistent). -1 means no limit. ; http://php.net/pgsql.max-links pgsql.max_links = -1 ; Ignore PostgreSQL backends Notice message or not. ; Notice message logging require a little overheads. ; http://php.net/pgsql.ignore-notice pgsql.ignore_notice = 0 ; Log PostgreSQL backends Notice message or not. ; Unless pgsql.ignore_notice=0, module cannot log notice message. ; http://php.net/pgsql.log-notice pgsql.log_notice = 0 [Sybase-CT] ; Allow or prevent persistent links. ; http://php.net/sybct.allow-persistent sybct.allow_persistent = On ; Maximum number of persistent links. -1 means no limit. ; http://php.net/sybct.max-persistent sybct.max_persistent = -1 ; Maximum number of links (persistent + non-persistent). -1 means no limit. ; http://php.net/sybct.max-links sybct.max_links = -1 ; Minimum server message severity to display. ; http://php.net/sybct.min-server-severity sybct.min_server_severity = 10 ; Minimum client message severity to display. ; http://php.net/sybct.min-client-severity sybct.min_client_severity = 10 ; Set per-context timeout ; http://php.net/sybct.timeout ;sybct.timeout= ;sybct.packet_size ; The maximum time in seconds to wait for a connection attempt to succeed before returning failure. ; Default: one minute ;sybct.login_timeout= ; The name of the host you claim to be connecting from, for display by sp_who. ; Default: none ;sybct.hostname= ; Allows you to define how often deadlocks are to be retried. -1 means "forever". ; Default: 0 ;sybct.deadlock_retry_count= [bcmath] ; Number of decimal digits for all bcmath functions. ; http://php.net/bcmath.scale bcmath.scale = 0 [browscap] ; http://php.net/browscap ;browscap = extra/browscap.ini [Session] ; Handler used to store/retrieve data. ; http://php.net/session.save-handler session.save_handler = files ; Argument passed to save_handler. In the case of files, this is the path ; where data files are stored. Note: Windows users have to change this ; variable in order to use PHP's session functions. ; ; The path can be defined as: ; ; session.save_path = "N;/path" ; ; where N is an integer. Instead of storing all the session files in ; /path, what this will do is use subdirectories N-levels deep, and ; store the session data in those directories. This is useful if you ; or your OS have problems with lots of files in one directory, and is ; a more efficient layout for servers that handle lots of sessions. ; ; NOTE 1: PHP will not create this directory structure automatically. ; You can use the script in the ext/session dir for that purpose. ; NOTE 2: See the section on garbage collection below if you choose to ; use subdirectories for session storage ; ; The file storage module creates files using mode 600 by default. ; You can change that by using ; ; session.save_path = "N;MODE;/path" ; ; where MODE is the octal representation of the mode. Note that this ; does not overwrite the process's umask. ; http://php.net/session.save-path ;session.save_path = "/tmp" ; Whether to use cookies. ; http://php.net/session.use-cookies session.use_cookies = 1 ; http://php.net/session.cookie-secure ;session.cookie_secure = ; This option forces PHP to fetch and use a cookie for storing and maintaining ; the session id. We encourage this operation as it's very helpful in combating ; session hijacking when not specifying and managing your own session id. It is ; not the end all be all of session hijacking defense, but it's a good start. ; http://php.net/session.use-only-cookies session.use_only_cookies = 1 ; Name of the session (used as cookie name). ; http://php.net/session.name session.name = PHPSESSID ; Initialize session on request startup. ; http://php.net/session.auto-start session.auto_start = 0 ; Lifetime in seconds of cookie or, if 0, until browser is restarted. ; http://php.net/session.cookie-lifetime session.cookie_lifetime = 0 ; The path for which the cookie is valid. ; http://php.net/session.cookie-path session.cookie_path = / ; The domain for which the cookie is valid. ; http://php.net/session.cookie-domain session.cookie_domain = ; Whether or not to add the httpOnly flag to the cookie, which makes it inaccessible to browser scripting languages such as JavaScript. ; http://php.net/session.cookie-httponly session.cookie_httponly = ; Handler used to serialize data. php is the standard serializer of PHP. ; http://php.net/session.serialize-handler session.serialize_handler = php ; Defines the probability that the 'garbage collection' process is started ; on every session initialization. The probability is calculated by using ; gc_probability/gc_divisor. Where session.gc_probability is the numerator ; and gc_divisor is the denominator in the equation. Setting this value to 1 ; when the session.gc_divisor value is 100 will give you approximately a 1% chance ; the gc will run on any give request. ; Default Value: 1 ; Development Value: 1 ; Production Value: 1 ; http://php.net/session.gc-probability session.gc_probability = 1 ; Defines the probability that the 'garbage collection' process is started on every ; session initialization. The probability is calculated by using the following equation: ; gc_probability/gc_divisor. Where session.gc_probability is the numerator and ; session.gc_divisor is the denominator in the equation. Setting this value to 1 ; when the session.gc_divisor value is 100 will give you approximately a 1% chance ; the gc will run on any give request. Increasing this value to 1000 will give you ; a 0.1% chance the gc will run on any give request. For high volume production servers, ; this is a more efficient approach. ; Default Value: 100 ; Development Value: 1000 ; Production Value: 1000 ; http://php.net/session.gc-divisor session.gc_divisor = 1000 ; After this number of seconds, stored data will be seen as 'garbage' and ; cleaned up by the garbage collection process. ; http://php.net/session.gc-maxlifetime session.gc_maxlifetime = 1440 ; NOTE: If you are using the subdirectory option for storing session files ; (see session.save_path above), then garbage collection does *not* ; happen automatically. You will need to do your own garbage ; collection through a shell script, cron entry, or some other method. ; For example, the following script would is the equivalent of ; setting session.gc_maxlifetime to 1440 (1440 seconds = 24 minutes): ; find /path/to/sessions -cmin +24 -type f | xargs rm ; Check HTTP Referer to invalidate externally stored URLs containing ids. ; HTTP_REFERER has to contain this substring for the session to be ; considered as valid. ; http://php.net/session.referer-check session.referer_check = ; How many bytes to read from the file. ; http://php.net/session.entropy-length ;session.entropy_length = 32 ; Specified here to create the session id. ; http://php.net/session.entropy-file ; Defaults to /dev/urandom ; On systems that don't have /dev/urandom but do have /dev/arandom, this will default to /dev/arandom ; If neither are found at compile time, the default is no entropy file. ; On windows, setting the entropy_length setting will activate the ; Windows random source (using the CryptoAPI) ;session.entropy_file = /dev/urandom ; Set to {nocache,private,public,} to determine HTTP caching aspects ; or leave this empty to avoid sending anti-caching headers. ; http://php.net/session.cache-limiter session.cache_limiter = nocache ; Document expires after n minutes. ; http://php.net/session.cache-expire session.cache_expire = 180 ; trans sid support is disabled by default. ; Use of trans sid may risk your users security. ; Use this option with caution. ; - User may send URL contains active session ID ; to other person via. email/irc/etc. ; - URL that contains active session ID may be stored ; in publicly accessible computer. ; - User may access your site with the same session ID ; always using URL stored in browser's history or bookmarks. ; http://php.net/session.use-trans-sid session.use_trans_sid = 0 ; Select a hash function for use in generating session ids. ; Possible Values ; 0 (MD5 128 bits) ; 1 (SHA-1 160 bits) ; This option may also be set to the name of any hash function supported by ; the hash extension. A list of available hashes is returned by the hash_algos() ; function. ; http://php.net/session.hash-function session.hash_function = 0 ; Define how many bits are stored in each character when converting ; the binary hash data to something readable. ; Possible values: ; 4 (4 bits: 0-9, a-f) ; 5 (5 bits: 0-9, a-v) ; 6 (6 bits: 0-9, a-z, A-Z, "-", ",") ; Default Value: 4 ; Development Value: 5 ; Production Value: 5 ; http://php.net/session.hash-bits-per-character session.hash_bits_per_character = 5 ; The URL rewriter will look for URLs in a defined set of HTML tags. ; form/fieldset are special; if you include them here, the rewriter will ; add a hidden <input> field with the info which is otherwise appended ; to URLs. If you want XHTML conformity, remove the form entry. ; Note that all valid entries require a "=", even if no value follows. ; Default Value: "a=href,area=href,frame=src,form=,fieldset=" ; Development Value: "a=href,area=href,frame=src,input=src,form=fakeentry" ; Production Value: "a=href,area=href,frame=src,input=src,form=fakeentry" ; http://php.net/url-rewriter.tags url_rewriter.tags = "a=href,area=href,frame=src,input=src,form=fakeentry" ; Enable upload progress tracking in $_SESSION ; Default Value: On ; Development Value: On ; Production Value: On ; http://php.net/session.upload-progress.enabled ;session.upload_progress.enabled = On ; Cleanup the progress information as soon as all POST data has been read ; (i.e. upload completed). ; Default Value: On ; Development Value: On ; Production Value: On ; http://php.net/session.upload-progress.cleanup ;session.upload_progress.cleanup = On ; A prefix used for the upload progress key in $_SESSION ; Default Value: "upload_progress_" ; Development Value: "upload_progress_" ; Production Value: "upload_progress_" ; http://php.net/session.upload-progress.prefix ;session.upload_progress.prefix = "upload_progress_" ; The index name (concatenated with the prefix) in $_SESSION ; containing the upload progress information ; Default Value: "PHP_SESSION_UPLOAD_PROGRESS" ; Development Value: "PHP_SESSION_UPLOAD_PROGRESS" ; Production Value: "PHP_SESSION_UPLOAD_PROGRESS" ; http://php.net/session.upload-progress.name ;session.upload_progress.name = "PHP_SESSION_UPLOAD_PROGRESS" ; How frequently the upload progress should be updated. ; Given either in percentages (per-file), or in bytes ; Default Value: "1%" ; Development Value: "1%" ; Production Value: "1%" ; http://php.net/session.upload-progress.freq ;session.upload_progress.freq = "1%" ; The minimum delay between updates, in seconds ; Default Value: 1 ; Development Value: 1 ; Production Value: 1 ; http://php.net/session.upload-progress.min-freq ;session.upload_progress.min_freq = "1" [MSSQL] ; Allow or prevent persistent links. mssql.allow_persistent = On ; Maximum number of persistent links. -1 means no limit. mssql.max_persistent = -1 ; Maximum number of links (persistent+non persistent). -1 means no limit. mssql.max_links = -1 ; Minimum error severity to display. mssql.min_error_severity = 10 ; Minimum message severity to display. mssql.min_message_severity = 10 ; Compatibility mode with old versions of PHP 3.0. mssql.compatability_mode = Off ; Connect timeout ;mssql.connect_timeout = 5 ; Query timeout ;mssql.timeout = 60 ; Valid range 0 - 2147483647. Default = 4096. ;mssql.textlimit = 4096 ; Valid range 0 - 2147483647. Default = 4096. ;mssql.textsize = 4096 ; Limits the number of records in each batch. 0 = all records in one batch. ;mssql.batchsize = 0 ; Specify how datetime and datetim4 columns are returned ; On => Returns data converted to SQL server settings ; Off => Returns values as YYYY-MM-DD hh:mm:ss ;mssql.datetimeconvert = On ; Use NT authentication when connecting to the server mssql.secure_connection = Off ; Specify max number of processes. -1 = library default ; msdlib defaults to 25 ; FreeTDS defaults to 4096 ;mssql.max_procs = -1 ; Specify client character set. ; If empty or not set the client charset from freetds.conf is used ; This is only used when compiled with FreeTDS ;mssql.charset = "ISO-8859-1" [Assertion] ; Assert(expr); active by default. ; http://php.net/assert.active ;assert.active = On ; Issue a PHP warning for each failed assertion. ; http://php.net/assert.warning ;assert.warning = On ; Don't bail out by default. ; http://php.net/assert.bail ;assert.bail = Off ; User-function to be called if an assertion fails. ; http://php.net/assert.callback ;assert.callback = 0 ; Eval the expression with current error_reporting(). Set to true if you want ; error_reporting(0) around the eval(). ; http://php.net/assert.quiet-eval ;assert.quiet_eval = 0 [COM] ; path to a file containing GUIDs, IIDs or filenames of files with TypeLibs ; http://php.net/com.typelib-file ;com.typelib_file = ; allow Distributed-COM calls ; http://php.net/com.allow-dcom ;com.allow_dcom = true ; autoregister constants of a components typlib on com_load() ; http://php.net/com.autoregister-typelib ;com.autoregister_typelib = true ; register constants casesensitive ; http://php.net/com.autoregister-casesensitive ;com.autoregister_casesensitive = false ; show warnings on duplicate constant registrations ; http://php.net/com.autoregister-verbose ;com.autoregister_verbose = true ; The default character set code-page to use when passing strings to and from COM objects. ; Default: system ANSI code page ;com.code_page= [mbstring] ; language for internal character representation. ; http://php.net/mbstring.language ;mbstring.language = Japanese ; internal/script encoding. ; Some encoding cannot work as internal encoding. ; (e.g. SJIS, BIG5, ISO-2022-*) ; http://php.net/mbstring.internal-encoding ;mbstring.internal_encoding = EUC-JP ; http input encoding. ; http://php.net/mbstring.http-input ;mbstring.http_input = auto ; http output encoding. mb_output_handler must be ; registered as output buffer to function ; http://php.net/mbstring.http-output ;mbstring.http_output = SJIS ; enable automatic encoding translation according to ; mbstring.internal_encoding setting. Input chars are ; converted to internal encoding by setting this to On. ; Note: Do _not_ use automatic encoding translation for ; portable libs/applications. ; http://php.net/mbstring.encoding-translation ;mbstring.encoding_translation = Off ; automatic encoding detection order. ; auto means ; http://php.net/mbstring.detect-order ;mbstring.detect_order = auto ; substitute_character used when character cannot be converted ; one from another ; http://php.net/mbstring.substitute-character ;mbstring.substitute_character = none; ; overload(replace) single byte functions by mbstring functions. ; mail(), ereg(), etc are overloaded by mb_send_mail(), mb_ereg(), ; etc. Possible values are 0,1,2,4 or combination of them. ; For example, 7 for overload everything. ; 0: No overload ; 1: Overload mail() function ; 2: Overload str*() functions ; 4: Overload ereg*() functions ; http://php.net/mbstring.func-overload ;mbstring.func_overload = 0 ; enable strict encoding detection. ;mbstring.strict_detection = Off ; This directive specifies the regex pattern of content types for which mb_output_handler() ; is activated. ; Default: mbstring.http_output_conv_mimetype=^(text/|application/xhtml\+xml) ;mbstring.http_output_conv_mimetype= [gd] ; Tell the jpeg decode to ignore warnings and try to create ; a gd image. The warning will then be displayed as notices ; disabled by default ; http://php.net/gd.jpeg-ignore-warning ;gd.jpeg_ignore_warning = 0 [exif] ; Exif UNICODE user comments are handled as UCS-2BE/UCS-2LE and JIS as JIS. ; With mbstring support this will automatically be converted into the encoding ; given by corresponding encode setting. When empty mbstring.internal_encoding ; is used. For the decode settings you can distinguish between motorola and ; intel byte order. A decode setting cannot be empty. ; http://php.net/exif.encode-unicode ;exif.encode_unicode = ISO-8859-15 ; http://php.net/exif.decode-unicode-motorola ;exif.decode_unicode_motorola = UCS-2BE ; http://php.net/exif.decode-unicode-intel ;exif.decode_unicode_intel = UCS-2LE ; http://php.net/exif.encode-jis ;exif.encode_jis = ; http://php.net/exif.decode-jis-motorola ;exif.decode_jis_motorola = JIS ; http://php.net/exif.decode-jis-intel ;exif.decode_jis_intel = JIS [Tidy] ; The path to a default tidy configuration file to use when using tidy ; http://php.net/tidy.default-config ;tidy.default_config = /usr/local/lib/php/default.tcfg ; Should tidy clean and repair output automatically? ; WARNING: Do not use this option if you are generating non-html content ; such as dynamic images ; http://php.net/tidy.clean-output tidy.clean_output = Off [soap] ; Enables or disables WSDL caching feature. ; http://php.net/soap.wsdl-cache-enabled soap.wsdl_cache_enabled=1 ; Sets the directory name where SOAP extension will put cache files. ; http://php.net/soap.wsdl-cache-dir soap.wsdl_cache_dir="/tmp" ; (time to live) Sets the number of second while cached file will be used ; instead of original one. ; http://php.net/soap.wsdl-cache-ttl soap.wsdl_cache_ttl=86400 ; Sets the size of the cache limit. (Max. number of WSDL files to cache) soap.wsdl_cache_limit = 5 [sysvshm] ; A default size of the shared memory segment ;sysvshm.init_mem = 10000 [ldap] ; Sets the maximum number of open links or -1 for unlimited. ldap.max_links = -1 [mcrypt] ; For more information about mcrypt settings see http://php.net/mcrypt-module-open ; Directory where to load mcrypt algorithms ; Default: Compiled in into libmcrypt (usually /usr/local/lib/libmcrypt) ;mcrypt.algorithms_dir= ; Directory where to load mcrypt modes ; Default: Compiled in into libmcrypt (usually /usr/local/lib/libmcrypt) ;mcrypt.modes_dir= [dba] ;dba.default_handler= [curl] ; A default value for the CURLOPT_CAINFO option. This is required to be an ; absolute path. ;curl.cainfo = ; Local Variables: ; tab-width: 4 ; End:
CHAPTER 1 Introduction to MySQL . . . . . . . . . . . . . . . . . . . . . . . . . . 3 1.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3 1.2 Database, Database Server, and Database Language. . . . . . . . . 4 1.3 The Relational Model . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6 1.4 What Is SQL? . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11 1.5 The History of SQL . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 16 1.6 From Monolithic via Client/Server to the Internet . . . . . . . . . . 18 1.7 Standardization of SQL . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 21 1.8 What Is Open Source Software?. . . . . . . . . . . . . . . . . . . . . . . . 25 1.9 The History of MySQL . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 26 1.10 The Structure of This Book . . . . . . . . . . . . . . . . . . . . . . . . . . . 27 CHAPTER 2 The Tennis Club Sample Database . . . . . . . . . . . . . . . . 29 2.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 29 2.2 Description of the Tennis Club . . . . . . . . . . . . . . . . . . . . . . . . . 29 2.3 The Contents of the Tables. . . . . . . . . . . . . . . . . . . . . . . . . . . . 33 2.4 Integrity Constraints. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 35 CHAPTER 3 Installing the Software . . . . . . . . . . . . . . . . . . . . . . . . . 37 3.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 37 3.2 Downloading MySQL . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 37 3.3 Installation of MySQL. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 38 3.4 Installing a Query Tool . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 38 3.5 Downloading SQL Statements from the Web Site . . . . . . . . . . 38 3.6 Ready? . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 39 CHAPTER 4 SQL in a Nutshell . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 41 4.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 41 4.2 Logging On to the MySQL Database Server . . . . . . . . . . . . . . . 41 4.3 Creating New SQL Users . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 43 4.4 Creating Databases . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 45 4.5 Selecting the Current Database . . . . . . . . . . . . . . . . . . . . . . . . 45 4.6 Creating Tables. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 46 4.7 Populating Tables with Data . . . . . . . . . . . . . . . . . . . . . . . . . . 48 4.8 Querying Tables . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 49 4.9 Updating and Deleting Rows . . . . . . . . . . . . . . . . . . . . . . . . . . 52 4.10 Optimizing Query Processing with Indexes. . . . . . . . . . . . . . . 54 4.11 Views. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 55 4.12 Users and Data Security . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 57 4.13 Deleting Database Objects . . . . . . . . . . . . . . . . . . . . . . . . . . . 57 4.14 System Variables. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 58 4.15 Grouping of SQL Statements . . . . . . . . . . . . . . . . . . . . . . . . . 59 4.16 The Catalog Tables . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 60 4.17 Retrieving Errors and Warnings . . . . . . . . . . . . . . . . . . . . . . . 68 4.18 Definitions of SQL Statements . . . . . . . . . . . . . . . . . . . . . . . . 69 PART II Querying and Updating Data . . . . . . . . . . . . . . . . 71 CHAPTER 5 SELECT Statement: Common Elements . . . . . . . . . . . . 73 5.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 73 5.2 Literals and Their Data Types . . . . . . . . . . . . . . . . . . . . . . . . . . 74 5.3 Expressions. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 88 5.4 Assigning Names to Result Columns . . . . . . . . . . . . . . . . . . . . 92 5.5 The Column Specification. . . . . . . . . . . . . . . . . . . . . . . . . . . . . 94 5.6 The User Variable and the SET Statement . . . . . . . . . . . . . . . . 95 5.7 The System Variable . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 97 5.8 The Case Expression . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 101 5.9 The Scalar Expression Between Brackets . . . . . . . . . . . . . . . . 106 viii Contents 5.10 The Scalar Function . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 107 5.11 Casting of Expressions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 111 5.12 The Null Value as an Expression . . . . . . . . . . . . . . . . . . . . . . 114 5.13 The Compound Scalar Expression . . . . . . . . . . . . . . . . . . . . 115 5.14 The Aggregation Function and the Scalar Subquery. . . . . . . 136 5.15 The Row Expression . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 137 5.16 The Table Expression . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 139 5.17 Answers . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 140 CHAPTER 6 SELECT Statements, Table Expressions, and Subqueries . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 145 6.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 145 6.2 The Definition of the SELECT Statement . . . . . . . . . . . . . . . . 145 6.3 Processing the Clauses in a Select Block. . . . . . . . . . . . . . . . . 150 6.4 Possible Forms of a Table Expression . . . . . . . . . . . . . . . . . . . 156 6.5 What Is a SELECT Statement? . . . . . . . . . . . . . . . . . . . . . . . . 159 6.6 What Is a Subquery?. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 160 6.7 Answers . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 166 CHAPTER 7 SELECT Statement:The FROM Clause. . . . . . . . . . . . . 171 7.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 171 7.2 Table Specifications in the FROM Clause . . . . . . . . . . . . . . . . 171 7.3 Again, the Column Specification. . . . . . . . . . . . . . . . . . . . . . . 173 7.4 Multiple Table Specifications in the FROM Clause . . . . . . . . . 174 7.5 Pseudonyms for Table Names. . . . . . . . . . . . . . . . . . . . . . . . . 178 7.6 Various Examples of Joins. . . . . . . . . . . . . . . . . . . . . . . . . . . . 179 7.7 Mandatory Use of Pseudonyms . . . . . . . . . . . . . . . . . . . . . . . 183 7.8 Tables of Different Databases . . . . . . . . . . . . . . . . . . . . . . . . 185 7.9 Explicit Joins in the FROM Clause. . . . . . . . . . . . . . . . . . . . . . 185 7.10 Outer Joins . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 189 7.11 The Natural Join . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 195 7.12 Additional Conditions in the Join Condition. . . . . . . . . . . . . 196 7.13 The Cross Join. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 199 7.14 Replacing Join Conditions with USING. . . . . . . . . . . . . . . . . 199 7.15 The FROM Clause with Table Expressions . . . . . . . . . . . . . . 200 7.16 Answers . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 208 Contents ix CHAPTER 8 SELECT Statement: The WHERE Clause . . . . . . . . . . . 213 8.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 213 8.2 Conditions Using Comparison Operators . . . . . . . . . . . . . . . 215 8.3 Comparison Operators with Subqueries . . . . . . . . . . . . . . . . 222 8.4 Comparison Operators with Correlated Subqueries. . . . . . . . 227 8.5 Conditions Without a Comparison Operator. . . . . . . . . . . . . 229 8.6 Conditions Coupled with AND, OR, XOR, and NOT . . . . . . . 231 8.7 The IN Operator with Expression List. . . . . . . . . . . . . . . . . . . 235 8.8 The IN Operator with Subquery . . . . . . . . . . . . . . . . . . . . . . . 241 8.9 The BETWEEN Operator . . . . . . . . . . . . . . . . . . . . . . . . . . . . 250 8.10 The LIKE Operator . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 252 8.11 The REGEXP Operator . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 255 8.12 The MATCH Operator . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 264 8.13 The IS NULL Operator . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 276 8.14 The EXISTS Operator . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 278 8.15 The ALL and ANY Operators . . . . . . . . . . . . . . . . . . . . . . . . 281 8.16 Scope of Columns in Subqueries . . . . . . . . . . . . . . . . . . . . . 289 8.17 More Examples with Correlated Subqueries . . . . . . . . . . . . . 294 8.18 Conditions with Negation. . . . . . . . . . . . . . . . . . . . . . . . . . . 299 8.19 Answers . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 302 CHAPTER 9 SELECT Statement: SELECT Clause and Aggregation Functions . . . . . . . . . . . . . . . . . . . . . . . . 315 9.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 315 9.2 Selecting All Columns (*) . . . . . . . . . . . . . . . . . . . . . . . . . . . . 316 9.3 Expressions in the SELECT Clause . . . . . . . . . . . . . . . . . . . . . 317 9.4 Removing Duplicate Rows with DISTINCT. . . . . . . . . . . . . . . 318 9.5 When Are Two Rows Equal?. . . . . . . . . . . . . . . . . . . . . . . . . . 321 9.6 More Select Options. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 323 9.7 An Introduction to Aggregation Functions. . . . . . . . . . . . . . . 324 9.8 COUNT Function . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 327 9.9 MAX and MIN Functions . . . . . . . . . . . . . . . . . . . . . . . . . . . . 331 9.10 The SUM and AVG Function. . . . . . . . . . . . . . . . . . . . . . . . . 336 9.11 The VARIANCE and STDDEV Functions. . . . . . . . . . . . . . . . 341 9.12 The VAR_SAMP and STDDEV_SAMP Functions . . . . . . . . . 343 9.13 The BIT_AND, BIT_OR, and BIT_XOR Functions . . . . . . . . 343 9.14 Answers . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 345 x Contents CHAPTER 10 SELECT Statement: The GROUP BY Clause . . . . . . . . 349 10.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 349 10.2 Grouping on One Column . . . . . . . . . . . . . . . . . . . . . . . . . . 350 10.3 Grouping on Two or More Columns . . . . . . . . . . . . . . . . . . 353 10.4 Grouping on Expressions . . . . . . . . . . . . . . . . . . . . . . . . . . . 356 10.5 Grouping of Null Values . . . . . . . . . . . . . . . . . . . . . . . . . . . 357 10.6 Grouping with Sorting . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 358 10.7 General Rules for the GROUP BY Clause . . . . . . . . . . . . . . . 359 10.8 The GROUP_CONCAT Function . . . . . . . . . . . . . . . . . . . . . 362 10.9 Complex Examples with GROUP BY. . . . . . . . . . . . . . . . . . . 363 10.10 Grouping with WITH ROLLUP . . . . . . . . . . . . . . . . . . . . . . 369 10.11 Answers . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 372 CHAPTER 11 SELECT Statement: The HAVING Clause . . . . . . . . . . 375 11.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 375 11.2 Examples of the HAVING Clause . . . . . . . . . . . . . . . . . . . . . 376 11.3 A HAVING Clause but not a GROUP BY Clause . . . . . . . . . 378 11.4 General Rule for the HAVING Clause . . . . . . . . . . . . . . . . . . 379 11.5 Answers . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 381 CHAPTER 12 SELECT Statement: The ORDER BY Clause . . . . . . . . 383 12.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 383 12.2 Sorting on Column Names . . . . . . . . . . . . . . . . . . . . . . . . . . 383 12.3 Sorting on Expressions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 385 12.4 Sorting with Sequence Numbers . . . . . . . . . . . . . . . . . . . . . 387 12.5 Sorting in Ascending and Descending Order . . . . . . . . . . . . 389 12.6 Sorting Null Values. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 392 12.7 Answers . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 393 CHAPTER 13 SELECT Statement: The LIMIT Clause. . . . . . . . . . . . . 395 13.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 395 13.2 Get the Top… . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 398 13.3 Subqueries with a LIMIT Clause . . . . . . . . . . . . . . . . . . . . . . 402 13.4 Limit with an Offset . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 404 13.5 The Select Option SQL_CALC_FOUND_ROWS . . . . . . . . . 405 13.6 Answers . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 406 Contents xi CHAPTER 14 Combining Table Expressions . . . . . . . . . . . . . . . . . . . 409 14.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 409 14.2 Combining with UNION. . . . . . . . . . . . . . . . . . . . . . . . . . . . 410 14.3 Rules for Using UNION . . . . . . . . . . . . . . . . . . . . . . . . . . . . 413 14.4 Keeping Duplicate Rows. . . . . . . . . . . . . . . . . . . . . . . . . . . . 416 14.5 Set Operators and the Null Value. . . . . . . . . . . . . . . . . . . . . 417 14.6 Answers . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 418 CHAPTER 15 The User Variable and the SET Statement . . . . . . . . . . 421 15.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 421 15.2 Defining Variables with the SET Statement . . . . . . . . . . . . . 421 15.3 Defining Variables with the SELECT Statement . . . . . . . . . . 423 15.4 Application Areas for User Variables . . . . . . . . . . . . . . . . . . 425 15.5 Life Span of User Variables . . . . . . . . . . . . . . . . . . . . . . . . . . 426 15.6 The DO Statement . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 428 15.7 Answers . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 428 CHAPTER 16 The HANDLER Statement. . . . . . . . . . . . . . . . . . . . . . 429 16.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 429 16.2 A Simple Example of the HANDLER Statement . . . . . . . . . . 429 16.3 Opening a Handler . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 430 16.4 Browsing the Rows of a Handler . . . . . . . . . . . . . . . . . . . . . 431 16.5 Closing a Handler. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 435 16.6 Answers . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 435 CHAPTER 17 Updating Tables . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 437 17.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 437 17.2 Inserting New Rows . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 437 17.3 Populating a Table with Rows from Another Table . . . . . . . 442 17.4 Updating Values in Rows . . . . . . . . . . . . . . . . . . . . . . . . . . . 444 17.5 Updating Values in Multiple Tables . . . . . . . . . . . . . . . . . . . 450 17.6 Substituting Existing Rows . . . . . . . . . . . . . . . . . . . . . . . . . . 452 17.7 Deleting Rows from a Table . . . . . . . . . . . . . . . . . . . . . . . . . 454 17.8 Deleting Rows from Multiple Tables. . . . . . . . . . . . . . . . . . . 456 17.9 The TRUNCATE Statement . . . . . . . . . . . . . . . . . . . . . . . . . 458 17.10 Answers . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 458 xii Contents CHAPTER 18 Loading and Unloading Data . . . . . . . . . . . . . . . . . . . 461 18.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 461 18.2 Unloading Data . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 461 18.3 Loading Data . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 465 CHAPTER 19 Working with XML Documents . . . . . . . . . . . . . . . . . . 471 19.1 XML in a Nutshell . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 471 19.2 Storing XML Documents . . . . . . . . . . . . . . . . . . . . . . . . . . . 473 19.3 Querying XML Documents . . . . . . . . . . . . . . . . . . . . . . . . . . 476 19.4 Querying Using Positions . . . . . . . . . . . . . . . . . . . . . . . . . . . 484 19.5 The Extended Notation of XPath . . . . . . . . . . . . . . . . . . . . . 486 19.6 XPath Expressions with Conditions . . . . . . . . . . . . . . . . . . . 488 19.7 Changing XML Documents. . . . . . . . . . . . . . . . . . . . . . . . . . 489 PART III Creating Database Objects. . . . . . . . . . . . . . . . . 491 CHAPTER 20 Creating Tables . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 493 20.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 493 20.2 Creating New Tables. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 493 20.3 Data Types of Columns . . . . . . . . . . . . . . . . . . . . . . . . . . . . 496 20.4 Adding Data Type Options. . . . . . . . . . . . . . . . . . . . . . . . . . 508 20.5 Creating Temporary Tables . . . . . . . . . . . . . . . . . . . . . . . . . 514 20.6 What If the Table Already Exists? . . . . . . . . . . . . . . . . . . . . . 515 20.7 Copying Tables . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 516 20.8 Naming Tables and Columns . . . . . . . . . . . . . . . . . . . . . . . . 521 20.9 Column Options: Default and Comment . . . . . . . . . . . . . . . 522 20.10 Table Options. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 524 20.11 The CSV Storage Engine . . . . . . . . . . . . . . . . . . . . . . . . . . . 532 20.12 Tables and the Catalog. . . . . . . . . . . . . . . . . . . . . . . . . . . . 534 20.13 Answers . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 537 CHAPTER 21 Specifying Integrity Constraints. . . . . . . . . . . . . . . . . . 539 21.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 539 21.2 Primary Keys. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 541 21.3 Alternate Keys. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 544 21.4 Foreign Keys . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 546 21.5 The Referencing Action. . . . . . . . . . . . . . . . . . . . . . . . . . . . . 550 21.6 Check Integrity Constraints . . . . . . . . . . . . . . . . . . . . . . . . . 553 Contents xiii 21.7 Naming Integrity Constraints . . . . . . . . . . . . . . . . . . . . . . . 556 21.8 Deleting Integrity Constraints. . . . . . . . . . . . . . . . . . . . . . . . 557 21.9 Integrity Constraints and the Catalog . . . . . . . . . . . . . . . . . 557 21.10 Answers . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 558 CHAPTER 22 Character Sets and Collations . . . . . . . . . . . . . . . . . . . 561 22.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 561 22.2 Available Character Sets and Collations. . . . . . . . . . . . . . . . 563 22.3 Assigning Character Sets to Columns. . . . . . . . . . . . . . . . . . 564 22.4 Assigning Collations to Columns . . . . . . . . . . . . . . . . . . . . . 566 22.5 Expressions with Character Sets and Collations. . . . . . . . . . 568 22.6 Sorting and Grouping with Collations . . . . . . . . . . . . . . . . . 571 22.7 The Coercibility of Expressions. . . . . . . . . . . . . . . . . . . . . . . 573 22.8 Related System Variables . . . . . . . . . . . . . . . . . . . . . . . . . . . 574 22.9 Character Sets and the Catalog . . . . . . . . . . . . . . . . . . . . . . 576 22.10 Answers . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 576 CHAPTER 23 The ENUM and SET Types . . . . . . . . . . . . . . . . . . . . . 577 23.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 577 23.2 The ENUM Data Type . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 578 23.3 The SET Data Type . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 582 23.4 Answers . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 589 CHAPTER 24 Changing and Dropping Tables . . . . . . . . . . . . . . . . . . 591 24.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 591 24.2 Deleting Entire Tables. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 591 24.3 Renaming Tables . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 593 24.4 Changing the Table Structure . . . . . . . . . . . . . . . . . . . . . . . . 593 24.5 Changing Columns . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 595 24.6 Changing Integrity Constraints. . . . . . . . . . . . . . . . . . . . . . . 599 24.7 Answers . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 602 CHAPTER 25 Using Indexes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 603 25.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 603 25.2 Rows, Tables, and Files. . . . . . . . . . . . . . . . . . . . . . . . . . . . . 604 25.3 How Does an Index Work?. . . . . . . . . . . . . . . . . . . . . . . . . . 605 25.4 Processing a SELECT Statement: The Steps . . . . . . . . . . . . . 610 25.5 Creating Indexes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 614 xiv Contents 25.6 Defining Indexes Together with the Tables . . . . . . . . . . . . . . 617 25.7 Dropping Indexes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 618 25.8 Indexes and Primary Keys . . . . . . . . . . . . . . . . . . . . . . . . . . . 619 25.9 The Big PLAYERS_XXL Table . . . . . . . . . . . . . . . . . . . . . . . . 620 25.10 Choosing Columns for Indexes. . . . . . . . . . . . . . . . . . . . . . 622 25.11 Indexes and the Catalog . . . . . . . . . . . . . . . . . . . . . . . . . . . 627 25.12 Answers . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 630 CHAPTER 26 Views . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 631 26.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 631 26.2 Creating Views . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 631 26.3 The Column Names of Views . . . . . . . . . . . . . . . . . . . . . . . . 635 26.4 Updating Views: WITH CHECK OPTION. . . . . . . . . . . . . . . 636 26.5 Options of Views . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 638 26.6 Deleting Views . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 639 26.7 Views and the Catalog . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 640 26.8 Restrictions on Updating Views . . . . . . . . . . . . . . . . . . . . . . 641 26.9 Processing View Statements . . . . . . . . . . . . . . . . . . . . . . . . . 642 26.10 Application Areas for Views . . . . . . . . . . . . . . . . . . . . . . . . 645 26.11 Answers . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 650 CHAPTER 27 Creating Databases. . . . . . . . . . . . . . . . . . . . . . . . . . . 653 27.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 653 27.2 Databases and the Catalog. . . . . . . . . . . . . . . . . . . . . . . . . . 653 27.3 Creating Databases . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 654 27.4 Changing Databases. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 655 27.5 Dropping Databases. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 656 CHAPTER 28 Users and Data Security . . . . . . . . . . . . . . . . . . . . . . . 659 28.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 659 28.2 Adding and Removing Users . . . . . . . . . . . . . . . . . . . . . . . . 660 28.3 Changing the Names of Users . . . . . . . . . . . . . . . . . . . . . . . 662 28.4 Changing Passwords . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 663 28.5 Granting Table and Column Privileges . . . . . . . . . . . . . . . . . 664 28.6 Granting Database Privileges . . . . . . . . . . . . . . . . . . . . . . . . 667 28.7 Granting User Privileges . . . . . . . . . . . . . . . . . . . . . . . . . . . . 670 28.8 Passing on Privileges: WITH GRANT OPTION . . . . . . . . . . 673 28.9 Restricting Privileges. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 674 Contents xv 28.10 Recording Privileges in the Catalog . . . . . . . . . . . . . . . . . . 675 28.11 Revoking Privileges . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 677 28.12 Security of and Through Views . . . . . . . . . . . . . . . . . . . . . . 680 28.13 Answers . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 682 CHAPTER 29 Statements for Table Maintenance . . . . . . . . . . . . . . . 683 29.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 683 29.2 The ANALYZE TABLE Statement . . . . . . . . . . . . . . . . . . . . . 684 29.3 The CHECKSUM TABLE Statement . . . . . . . . . . . . . . . . . . . 685 29.4 The OPTIMIZE TABLE Statement. . . . . . . . . . . . . . . . . . . . . 686 29.5 The CHECK TABLE Statement . . . . . . . . . . . . . . . . . . . . . . . 687 29.6 The REPAIR TABLE Statement . . . . . . . . . . . . . . . . . . . . . . . 689 29.7 The BACKUP TABLE Statement . . . . . . . . . . . . . . . . . . . . . . 690 29.8 The RESTORE TABLE Statement . . . . . . . . . . . . . . . . . . . . . 691 CHAPTER 30 The SHOW, DESCRIBE, and HELP Statements. . . . . . 693 30.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 693 30.2 Overview of SHOW Statements . . . . . . . . . . . . . . . . . . . . . . 693 30.3 Additional SHOW Statements . . . . . . . . . . . . . . . . . . . . . . . 698 30.4 The DESCRIBE Statement . . . . . . . . . . . . . . . . . . . . . . . . . . 699 30.5 The HELP Statement . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 699 PART IV Procedural Database Objects. . . . . . . . . . . . . . . 701 CHAPTER 31 Stored Procedures. . . . . . . . . . . . . . . . . . . . . . . . . . . . 703 31.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 703 31.2 An Example of a Stored Procedure . . . . . . . . . . . . . . . . . . . . 704 31.3 The Parameters of a Stored Procedure . . . . . . . . . . . . . . . . . 706 31.4 The Body of a Stored Procedure. . . . . . . . . . . . . . . . . . . . . . 707 31.5 Local Variables . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 709 31.6 The SET Statement . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 712 31.7 Flow-Control Statements . . . . . . . . . . . . . . . . . . . . . . . . . . . 712 31.8 Calling Stored Procedures . . . . . . . . . . . . . . . . . . . . . . . . . . 719 31.9 Querying Data with SELECT INTO. . . . . . . . . . . . . . . . . . . . 722 31.10 Error Messages, Handlers, and Conditions . . . . . . . . . . . . 726 31.11 Retrieving Data with a Cursor. . . . . . . . . . . . . . . . . . . . . . . 731 31.12 Including SELECT Statements Without Cursors . . . . . . . . . 736 31.13 Stored Procedures and User Variables . . . . . . . . . . . . . . . . 737 31.14 Characteristics of Stored Procedures . . . . . . . . . . . . . . . . . 737 xvi Contents 31.15 Stored Procedures and the Catalog . . . . . . . . . . . . . . . . . . 740 31.16 Removing Stored Procedures . . . . . . . . . . . . . . . . . . . . . . . 741 31.17 Security with Stored Procedures . . . . . . . . . . . . . . . . . . . . . 742 31.18 Advantages of Stored Procedures. . . . . . . . . . . . . . . . . . . . 743 CHAPTER 32 Stored Functions. . . . . . . . . . . . . . . . . . . . . . . . . . . . . 745 32.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 745 32.2 Examples of Stored Functions . . . . . . . . . . . . . . . . . . . . . . . 746 32.3 More on Stored Functions . . . . . . . . . . . . . . . . . . . . . . . . . . 752 32.4 Removing Stored Functions . . . . . . . . . . . . . . . . . . . . . . . . . 753 CHAPTER 33 Triggers. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 755 33.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 755 33.2 An Example of a Trigger . . . . . . . . . . . . . . . . . . . . . . . . . . . . 756 33.3 More Complex Examples . . . . . . . . . . . . . . . . . . . . . . . . . . . 759 33.4 Triggers as Integrity Constraints. . . . . . . . . . . . . . . . . . . . . . 763 33.5 Removing Triggers . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 765 33.6 Triggers and the Catalog . . . . . . . . . . . . . . . . . . . . . . . . . . . 765 33.7 Answers . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 765 CHAPTER 34 Events. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 767 34.1 What Is an Event?. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 767 34.2 Creating Events. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 768 34.3 Properties of Events . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 777 34.4 Changing Events. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 778 34.5 Removing Events . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 779 34.6 Events and Privileges . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 779 34.7 Events and the Catalog. . . . . . . . . . . . . . . . . . . . . . . . . . . . . 780 PART V Programming with SQL . . . . . . . . . . . . . . . . . . . 783 CHAPTER 35 MySQL and PHP. . . . . . . . . . . . . . . . . . . . . . . . . . . . . 785 35.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 785 35.2 Logging On to MySQL . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 786 35.3 Selecting a Database . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 787 35.4 Creating an Index . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 788 35.5 Retrieving Error Messages . . . . . . . . . . . . . . . . . . . . . . . . . . 790 35.6 Multiple Connections Within One Session . . . . . . . . . . . . . . 791 35.7 SQL Statements with Parameters . . . . . . . . . . . . . . . . . . . . . 793 35.8 SELECT Statement with One Row . . . . . . . . . . . . . . . . . . . . 794 Contents xvii 35.9 SELECT Statement with Multiple Rows . . . . . . . . . . . . . . . . 796 35.10 SELECT Statement with Null Values . . . . . . . . . . . . . . . . . . 800 35.11 Querying Data About Expressions . . . . . . . . . . . . . . . . . . . 801 35.12 Querying the Catalog . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 803 35.13 Remaining MYSQL Functions . . . . . . . . . . . . . . . . . . . . . . . 805 CHAPTER 36 Dynamic SQL with Prepared Statement. . . . . . . . . . . . 807 36.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 807 36.2 Working with Prepared SQL Statements . . . . . . . . . . . . . . . 807 36.3 Prepared Statements with User Variables. . . . . . . . . . . . . . . 810 36.4 Prepared Statements with Parameters . . . . . . . . . . . . . . . . . 810 36.5 Prepared Statements in Stored Procedures . . . . . . . . . . . . . 811 CHAPTER 37 Transactions and Multiuser Usage. . . . . . . . . . . . . . . . 815 37.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 815 37.2 What Is a Transaction? . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 815 37.3 Starting Transactions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 821 37.4 Savepoints . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 822 37.5 Stored Procedures and Transactions . . . . . . . . . . . . . . . . . . 824 37.6 Problems with Multiuser Usage . . . . . . . . . . . . . . . . . . . . . . 825 37.7 Locking . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 829 37.8 Deadlocks. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 830 37.9 The LOCK TABLE and UNLOCK TABLE Statements . . . . . . 830 37.10 The Isolation Level . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 832 37.11 Waiting for a Lock . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 834 37.12 Moment of Processing Statements . . . . . . . . . . . . . . . . . . . 834 37.13 Working with Application Locks . . . . . . . . . . . . . . . . . . . . . 835 37.14 Answers . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 837 APPENDIX A Syntax of SQL . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 839 A.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 839 A.2 The BNF Notation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 839 A.3 Reserved Words in SQL . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 843 A.4 Syntax Definitions of SQL Statements . . . . . . . . . . . . . . . . . . 845 APPENDIX B Scalar Functions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 903 APPENDIX C System Variables . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 953 APPENDIX D Bibliography . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 963 Index . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 967
Java 核心技术 卷1 Index Chapter 1: An Introduction to Java 1 Java As a Programming Platform 2 The Java “White Paper” Buzzwords 2 Java Applets and the Internet 7 A Short History of Java 9 Common Misconceptions about Java 11 Chapter 2: The Java Programming Environment 15 Installing the Java Development Kit 16 Choosing a Development Environment 21 Using the Command-Line Tools 22 Using an Integrated Development Environment 25 Running a Graphical Application 28 Building and Running Applets 31 Chapter 3: Fundamental Programming Structures in Java 35 A Simple Java Program 36 Comments 39 Data Types 40 Variables 44 Operators 46 Strings 53 Input and Output 63 Control Flow 71 Big Numbers 88 Arrays 90 Chapter 4: Objects and Classes 105 Introduction to Object-Oriented Programming 106 Using Predefined Classes 111 Defining Your Own Classes 122 Static Fields and Methods 132 Method Parameters 138 Object Construction 144 Packages 15 The Class Path 160 Documentation Comments 162 Class Design Hints 167 Chapter 5: Inheritance 171 Classes, Superclasses, and Subclasses 172 Object: The Cosmic Superclass 192 Generic Array Lists 204 Object Wrappers and Autoboxing 211 Methods with a Variable Number of Parameters 214 Enumeration Classes 215 Reflection 217 Design Hints for Inheritance 238 Chapter 6: Interfaces and Inner Classes 241 Interfaces 242 Object Cloning 249 Interfaces and Callbacks 255 Inner Classes 258 Proxies 275 Chapter 7: Graphics Programming 281 Introducing Swing 282 Creating a Frame 285 Positioning a Frame 288 Displaying Information in a Component 294 Working with 2D Shapes 299 Using Color 307 Using Special Fonts for Text 310 Displaying Images 318 Chapter 8: Event Handling 323 Basics of Event Handling 324 Actions 342 Mouse Events 349 The AWT Event Hierarchy 357 Chapter 9: User Interface Components with Swing 361 Swing and the Model-View-Controller Design Pattern 362 Introduction to Layout Management 368 Text Input 377 Choice Components 385 Menus 406 Sophisticated Layout Management 424 Dialog Boxes 452 Chapter 10: Deploying Applications and Applets 493 JAR Files 494 Java Web Start 501 Applets 516 Storage of Application Preferences 539 Chapter 11: Exceptions, Logging, Assertions, and Debugging 551 Dealing with Errors 552 Catching Exceptions 559 Tips for Using Exceptions 568 Using Assertions 571 Logging 575 Debugging Tips 591 Using a Debugger 607 Chapter 12: Generic Programming 613 Why Generic Programming? 614 Definition of a Simple Generic Class 616 Generic Methods 618 Bounds for Type Variables 619 Generic Code and the Virtual Machine 621 Restrictions and Limitations 626 Inheritance Rules for Generic Types 630 Wildcard Types 632 Reflection and Generics 640 Chapter 13: Collections 649 Collection Interfaces 650 Concrete Collections 658 The Collections Framework 689 Algorithms 700 Legacy Collections 707 Chapter 14: Multithreading 715 What Are Threads? 716 Interrupting Threads 728 Thread States 730 Thread Properties 733 Synchronization 736 Blocking Queues 764 Thread-Safe Collections 771 Callables and Futures 774 Executors 778 Synchronizers 785 Threads and Swing 794 Appendix 809 Index 813 Java 核心技术 卷2 Index Chapter 1: Streams and Files 1 Streams 2 Text Input and Output 11 Reading and Writing Binary Data 23 ZIP Archives 32 Object Streams and Serialization 39 File Management 59 New I/O 65 Regular Expressions 75 Chapter 2: XML 87 Introducing XML 88 Parsing an XML Document 93 Validating XML Documents 105 Locating Information with XPath 129 Using Namespaces 136 Streaming Parsers 138 Generating XML Documents 146 XSL Transformations 157 Chapter 3: Networking 169 Connecting to a Server 170 Implementing Servers 177 Interruptible Sockets 184 Sending E-Mail 191 Making URL Connections 196 Chapter 4: Database Programming 217 The Design of JDBC 218 The Structured Query Language 222 JDBC Configuration 227 Executing SQL Statements 232 Query Execution 242 Scrollable and Updatable Result Sets 254 Row Sets 260 Metadata 263 Transactions 273 Connection Management in Web and Enterprise Applications 278 Introduction to LDAP 279 Chapter 5: Internationalization 297 Locales 298 Number Formats 303 Date and Time 310 Collation 318 Message Formatting 324 Text Files and Character Sets 328 Resource Bundles 329 A Complete Example 333 Chapter 6: Advanced Swing 351 Lists 352 Tables 370 Trees 405 Text Components 442 Progress Indicators 479 Component Organizers 492 Chapter 7: Advanced AWT 521 The Rendering Pipeline 522 Shapes 524 Areas 540 Strokes 542 Paint 550 Coordinate Transformations 552 Clipping 557 Transparency and Composition 559 Rendering Hints 568 Readers and Writers for Images 575 Image Manipulation 585 Printing 601 The Clipboard 635 Drag and Drop 652 Platform Integration 668 Chapter 8: Javabeans Components 685 Why Beans? 686 The Bean-Writing Process 688 Using Beans to Build an Application 690 Naming Patterns for Bean Properties and Events 698 Bean Property Types 701 BeanInfo Classes 710 Property Editors 713 Customizers 723 JavaBeans Persistence 732 Chapter 9: Security 755 Class Loaders 756 Bytecode Verification 767 Security Managers and Permissions 771 User Authentication 790 Digital Signatures 805 Code Signing 822 Encryption 828 Chapter 10: Distributed Objects 841 The Roles of Client and Server 842 Remote Method Calls 845 The RMI Programming Model 846 Parameters and Return Values in Remote Methods 856 Remote Object Activation 865 Web Services and JAX-WS 871 Chapter 11: Scripting, Compiling, and Annotation Processing 883 Scripting for the Java Platform 884 The Compiler API 895 Using Annotations 905 Annotation Syntax 911 Standard Annotations 915 Source-Level Annotation Processing 919 Bytecode Engineering 926 Chapter 12: Native Methods 935 Calling a C Function from a Java Program 936 Numeric Parameters and Return Values 942 String Parameters 944 Accessing Fields 950 Encoding Signatures 954 Calling Java Methods 956 Accessing Array Elements 962 Handling Errors 966 Using the Invocation API 970 A Complete Example: Accessing the Windows Registry 975 Index 991
Contents Module Overview 1 Lesson 1: Memory 3 Lesson 2: I/O 73 Lesson 3: CPU 111 Module 3: Troubleshooting Server Performance Module Overview Troubleshooting server performance-based support calls requires product knowledge, good communication skills, and a proven troubleshooting methodology. In this module we will discuss Microsoft® SQL Server™ interaction with the operating system and methodology of troubleshooting server-based problems. At the end of this module, you will be able to:  Define the common terms associated the memory, I/O, and CPU subsystems.  Describe how SQL Server leverages the Microsoft Windows® operating system facilities including memory, I/O, and threading.  Define common SQL Server memory, I/O, and processor terms.  Generate a hypothesis based on performance counters captured by System Monitor.  For each hypothesis generated, identify at least two other non-System Monitor pieces of information that would help to confirm or reject your hypothesis.  Identify at least five counters for each subsystem that are key to understanding the performance of that subsystem.  Identify three common myths associated with the memory, I/O, or CPU subsystems. Lesson 1: Memory What You Will Learn After completing this lesson, you will be able to:  Define common terms used when describing memory.  Give examples of each memory concept and how it applies to SQL Server.  Describe how SQL Server user and manages its memory.  List the primary configuration options that affect memory.  Describe how configuration options affect memory usage.  Describe the effect on the I/O subsystem when memory runs low.  List at least two memory myths and why they are not true. Recommended Reading  SQL Server 7.0 Performance Tuning Technical Reference, Microsoft Press  Windows 2000 Resource Kit companion CD-ROM documentation. Chapter 15: Overview of Performance Monitoring  Inside Microsoft Windows 2000, Third Edition, David A. Solomon and Mark E. Russinovich  Windows 2000 Server Operations Guide, Storage, File Systems, and Printing; Chapters: Evaluating Memory and Cache Usage  Advanced Windows, 4th Edition, Jeffrey Richter, Microsoft Press Related Web Sites  http://ntperformance/ Memory Definitions Memory Definitions Before we look at how SQL Server uses and manages its memory, we need to ensure a full understanding of the more common memory related terms. The following definitions will help you understand how SQL Server interacts with the operating system when allocating and using memory. Virtual Address Space A set of memory addresses that are mapped to physical memory addresses by the system. In a 32-bit operation system, there is normally a linear array of 2^32 addresses representing 4,294,967,269 byte addresses. Physical Memory A series of physical locations, with unique addresses, that can be used to store instructions or data. AWE – Address Windowing Extensions A 32-bit process is normally limited to addressing 2 gigabytes (GB) of memory, or 3 GB if the system was booted using the /3G boot switch even if there is more physical memory available. By leveraging the Address Windowing Extensions API, an application can create a fixed-size window into the additional physical memory. This allows a process to access any portion of the physical memory by mapping it into the applications window. When used in combination with Intel’s Physical Addressing Extensions (PAE) on Windows 2000, an AWE enabled application can support up to 64 GB of memory Reserved Memory Pages in a processes address space are free, reserved or committed. Reserving memory address space is a way to reserve a range of virtual addresses for later use. If you attempt to access a reserved address that has not yet been committed (backed by memory or disk) you will cause an access violation. Committed Memory Committed pages are those pages that when accessed in the end translate to pages in memory. Those pages may however have to be faulted in from a page file or memory mapped file. Backing Store Backing store is the physical representation of a memory address. Page Fault (Soft/Hard) A reference to an invalid page (a page that is not in your working set) is referred to as a page fault. Assuming the page reference does not result in an access violation, a page fault can be either hard or soft. A hard page fault results in a read from disk, either a page file or memory-mapped file. A soft page fault is resolved from one of the modified, standby, free or zero page transition lists. Paging is represented by a number of counters including page faults/sec, page input/sec and page output/sec. Page faults/sec include soft and hard page faults where as the page input/output counters represent hard page faults. Unfortunately, all of these counters include file system cache activity. For more information, see also…Inside Windows 2000,Third Edition, pp. 443-451. Private Bytes Private non-shared committed address space Working Set The subset of processes virtual pages that is resident in physical memory. For more information, see also… Inside Windows 2000,Third Edition, p. 455. System Working Set Like a process, the system has a working set. Five different types of pages represent the system’s working set: system cache; paged pool; pageable code and data in the kernel; page-able code and data in device drivers; and system mapped views. The system working set is represented by the counter Memory: cache bytes. System working set paging activity can be viewed by monitoring the Memory: Cache Faults/sec counter. For more information, see also… Inside Windows 2000,Third Edition, p. 463. System Cache The Windows 2000 cache manager provides data caching for both local and network file system drivers. By caching virtual blocks, the cache manager can reduce disk I/O and provide intelligent read ahead. Represented by Memory:Cache Resident bytes. For more information, see also… Inside Windows 2000,Third Edition, pp. 654-659. Non Paged Pool Range of addresses guaranteed to be resident in physical memory. As such, non-paged pool can be accessed at any time without incurring a page fault. Because device drivers operate at DPC/dispatch level (covered in lesson 2), and page faults are not allowed at this level or above, most device drivers use non-paged pool to assure that they do not incur a page fault. Represented by Memory: Pool Nonpaged Bytes, typically between 3-30 megabytes (MB) in size. Note The pool is, in effect, a common area of memory shared by all processes. One of the most common uses of non-paged pool is the storage of object handles. For more information regarding “maximums,” see also… Inside Windows 2000,Third Edition, pp. 403-404 Paged Pool Range of address that can be paged in and out of physical memory. Typically used by drivers who need memory but do not need to access that memory from DPC/dispatch of above interrupt level. Represented by Memory: Pool Paged Bytes and Memory:Pool Paged Resident Bytes. Typically between 10-30MB + size of Registry. For more information regarding “limits,” see also… Inside Windows 2000,Third Edition, pp. 403-404. Stack Each thread has two stacks, one for kernel mode and one for user mode. A stack is an area of memory in which program procedure or function call addresses and parameters are temporarily stored. In Process To run in the same address space. In-process servers are loaded in the client’s address space because they are implemented as DLLs. The main advantage of running in-process is that the system usually does not need to perform a context switch. The disadvantage to running in-process is that DLL has access to the process address space and can potentially cause problems. Out of Process To run outside the calling processes address space. OLEDB providers can run in-process or out of process. When running out of process, they run under the context of DLLHOST.EXE. Memory Leak To reserve or commit memory and unintentionally not release it when it is no longer being used. A process can leak resources such as process memory, pool memory, user and GDI objects, handles, threads, and so on. Memory Concepts (X86 Address Space) Per Process Address Space Every process has its own private virtual address space. For 32-bit processes, that address space is 4 GB, based on a 32-bit pointer. Each process’s virtual address space is split into user and system partitions based on the underlying operating system. The diagram included at the top represents the address partitioning for the 32-bit version of Windows 2000. Typically, the process address space is evenly divided into two 2-GB regions. Each process has access to 2 GB of the 4 GB address space. The upper 2 GB of address space is reserved for the system. The user address space is where application code, global variables, per-thread stacks, and DLL code would reside. The system address space is where the kernel, executive, HAL, boot drivers, page tables, pool, and system cache reside. For specific information regarding address space layout, refer to Inside Microsoft Windows 2000 Third Edition pages 417-428 by Microsoft Press. Access Modes Each virtual memory address is tagged as to what access mode the processor must be running in. System space can only be accessed while in kernel mode, while user space is accessible in user mode. This protects system space from being tampered with by user mode code. Shared System Space Although every process has its own private memory space, kernel mode code and drivers share system space. Windows 2000 does not provide any protection to private memory being use by components running in kernel mode. As such, it is very important to ensure components running in kernel mode are thoroughly tested. 3-GB Address Space 3-GB Address Space Although 2 GB of address space may seem like a large amount of memory, application such as SQL Server could leverage more memory if it were available. The boot.ini option /3GB was created for those cases where systems actually support greater than 2 GB of physical memory and an application can make use of it This capability allows memory intensive applications running on Windows 2000 Advanced Server to use up to 50 percent more virtual memory on Intel-based computers. Application memory tuning provides more of the computer's virtual memory to applications by providing less virtual memory to the operating system. Although a system having less than 2 GB of physical memory can be booted using the /3G switch, in most cases this is ill-advised. If you restart with the 3 GB switch, also known as 4-Gig Tuning, the amount of non-paged pool is reduced to 128 MB from 256 MB. For a process to access 3 GB of address space, the executable image must have been linked with the /LARGEADDRESSAWARE flag or modified using Imagecfg.exe. It should be pointed out that SQL Server was linked using the /LAREGEADDRESSAWARE flag and can leverage 3 GB when enabled. Note Even though you can boot Windows 2000 Professional or Windows 2000 Server with the /3GB boot option, users processes are still limited to 2 GB of address space even if the IMAGE_FILE_LARGE_ADDRESS_AWARE flag is set in the image. The only thing accomplished by using the /3G option on these system is the reduction in the amount of address space available to the system (ISW2K Pg. 418). Important If you use /3GB in conjunction with AWE/PAE you are limited to 16 GB of memory. For more information, see the following Knowledge Base articles: Q171793 Information on Application Use of 4GT RAM Tuning Q126402 PagedPoolSize and NonPagedPoolSize Values in Windows NT Q247904 How to Configure Paged Pool and System PTE Memory Areas Q274598 W2K Does Not Enable Complete Memory Dumps Between 2 & 4 GB AWE Memory Layout AWE Memory Usually, the operation system is limited to 4 GB of physical memory. However, by leveraging PAE, Windows 2000 Advanced Server can support up to 8 GB of memory, and Data Center 64 GB of memory. However, as stated previously, each 32-bit process normally has access to only 2 GB of address space, or 3 GB if the system was booted with the /3-GB option. To allow processes to allocate more physical memory than can be represented in the 2GB of address space, Microsoft created the Address Windows Extensions (AWE). These extensions allow for the allocation and use of up to the amount of physical memory supported by the operating system. By leveraging the Address Windowing Extensions API, an application can create a fixed-size window into the physical memory. This allows a process to access any portion of the physical memory by mapping regions of physical memory in and out of the applications window. The allocation and use of AWE memory is accomplished by  Creating a window via VirtualAlloc using the MEM_PHYSICAL option  Allocating the physical pages through AllocateUserPhysicalPages  Mapping the RAM pages to the window using MapUserPhysicalPages Note SQL Server 7.0 supports a feature called extended memory in Windows NT® 4 Enterprise Edition by using a PSE36 driver. Currently there are no PSE drivers for Windows 2000. The preferred method of accessing extended memory is via the Physical Addressing Extensions using AWE. The AWE mapping feature is much more efficient than the older process of coping buffers from extended memory into the process address space. Unfortunately, SQL Server 7.0 cannot leverage PAE/AWE. Because there are currently no PSE36 drivers for Windows 2000 this means SQL Server 7.0 cannot support more than 3GB of memory on Windows 2000. Refer to KB article Q278466. AWE restrictions  The process must have Lock Pages In Memory user rights to use AWE Important It is important that you use Enterprise Manager or DMO to change the service account. Enterprise Manager and DMO will grant all of the privileges and Registry and file permissions needed for SQL Server. The Service Control Panel does NOT grant all the rights or permissions needed to run SQL Server.  Pages are not shareable or page-able  Page protection is limited to read/write  The same physical page cannot be mapped into two separate AWE regions, even within the same process.  The use of AWE/PAE in conjunction with /3GB will limit the maximum amount of supported memory to between 12-16 GB of memory.  Task manager does not show the correct amount of memory allocated to AWE-enabled applications. You must use Memory Manager: Total Server Memory. It should, however, be noted that this only shows memory in use by the buffer pool.  Machines that have PAE enabled will not dump user mode memory. If an event occurs in User Mode Memory that causes a blue screen and root cause determination is absolutely necessary, the machine must be booted with the /NOPAE switch, and with /MAXMEM set to a number appropriate for transferring dump files.  With AWE enabled, SQL Server will, by default, allocate almost all memory during startup, leaving 256 MB or less free. This memory is locked and cannot be paged out. Consuming all available memory may prevent other applications or SQL Server instances from starting. Note PAE is not required to leverage AWE. However, if you have more than 4GB of physical memory you will not be able to access it unless you enable PAE. Caution It is highly recommended that you use the “max server memory” option in combination with “awe enabled” to ensure some memory headroom exists for other applications or instances of SQL Server, because AWE memory cannot be shared or paged. For more information, see the following Knowledge Base articles: Q268363 Intel Physical Addressing Extensions (PAE) in Windows 2000 Q241046 Cannot Create a dump File on Computers with over 4 GB RAM Q255600 Windows 2000 utilities do not display physical memory above 4GB Q274750 How to configure SQL Server memory more than 2 GB (Idea) Q266251 Memory dump stalls when PAE option is enabled (Idea) Tip The KB will return more hits if you query on PAE rather than AWE. Virtual Address Space Mapping Virtual Address Space Mapping By default Windows 2000 (on an X86 platform) uses a two-level (three-level when PAE is enabled) page table structure to translate virtual addresses to physical addresses. Each 32-bit address has three components, as shown below. When a process accesses a virtual address the system must first locate the Page Directory for the current process via register CR3 (X86). The first 10 bits of the virtual address act as an index into the Page Directory. The Page Directory Entry then points to the Page Frame Number (PFN) of the appropriate Page Table. The next 10 bits of the virtual address act as an index into the Page Table to locate the appropriate page. If the page is valid, the PTE contains the PFN of the actual page in memory. If the page is not valid, the memory management fault handler locates the page and attempts to make it valid. The final 12 bits act as a byte offset into the page. Note This multi-step process is expensive. This is why systems have translation look aside buffers (TLB) to speed up the process. One of the reasons context switching is so expensive is the translation buffers must be dumped. Thus, the first few lookups are very expensive. Refer to ISW2K pages 439-440. Core System Memory Related Counters Core System Memory Related Counters When evaluating memory performance you are looking at a wide variety of counters. The counters listed here are a few of the core counters that give you quick overall view of the state of memory. The two key counters are Available Bytes and Committed Bytes. If Committed Bytes exceeds the amount of physical memory in the system, you can be assured that there is some level of hard page fault activity happening. The goal of a well-tuned system is to have as little hard paging as possible. If Available Bytes is below 5 MB, you should investigate why. If Available Bytes is below 4 MB, the Working Set Manager will start to aggressively trim the working sets of process including the system cache.  Committed Bytes Total memory, including physical and page file currently committed  Commit Limit • Physical memory + page file size • Represents the total amount of memory that can be committed without expanding the page file. (Assuming page file is allowed to grow)  Available Bytes Total physical memory currently available Note Available Bytes is a key indicator of the amount of memory pressure. Windows 2000 will attempt to keep this above approximately 4 MB by aggressively trimming the working sets including system cache. If this value is constantly between 3-4 MB, it is cause for investigation. One counter you might expect would be for total physical memory. Unfortunately, there is no specific counter for total physical memory. There are however many other ways to determine total physical memory. One of the most common is by viewing the Performance tab of Task Manager. Page File Usage The only counters that show current page file space usage are Page File:% Usage and Page File:% Peak Usage. These two counters will give you an indication of the amount of space currently used in the page file. Memory Performance Memory Counters There are a number of counters that you need to investigate when evaluating memory performance. As stated previously, no single counter provides the entire picture. You will need to consider many different counters to begin to understand the true state of memory. Note The counters listed are a subset of the counters you should capture. *Available Bytes In general, it is desirable to see Available Bytes above 5 MB. SQL Servers goal on Intel platforms, running Windows NT, is to assure there is approximately 5+ MB of free memory. After Available Bytes reaches 4 MB, the Working Set Manager will start to aggressively trim the working sets of process and, finally, the system cache. This is not to say that working set trimming does not happen before 4 MB, but it does become more pronounced as the number of available bytes decreases below 4 MB. Page Faults/sec Page Faults/sec represents the total number of hard and soft page faults. This value includes the System Working Set as well. Keep this in mind when evaluating the amount of paging activity in the system. Because this counter includes paging associated with the System Cache, a server acting as a file server may have a much higher value than a dedicated SQL Server may have. The System Working Set is covered in depth on the next slide. Because Page Faults/sec includes soft faults, this counter is not as useful as Pages/sec, which represents hard page faults. Because of the associated I/O, hard page faults tend to be much more expensive. *Pages/sec Pages/sec represent the number of pages written/read from disk because of hard page faults. It is the sum of Memory: Pages Input/sec and Memory: Pages Output/sec. Because it is counted in numbers of pages, it can be compared to other counts of pages, such as Memory: Page Faults/sec, without conversion. On a well-tuned system, this value should be consistently low. In and of itself, a high value for this counter does not necessarily indicate a problem. You will need to isolate the paging activity to determine if it is associated with in-paging, out-paging, memory mapped file activity or system cache. Any one of these activities will contribute to this counter. Note Paging in and of itself is not necessarily a bad thing. Paging is only “bad” when a critical process must wait for it’s pages to be in-paged, or when the amount of read/write paging is causing excessive kernel time or disk I/O, thus interfering with normal user mode processing. Tip (Memory: Pages/sec) / (PhysicalDisk: Disk Bytes/sec * 4096) yields the approximate percentage of paging to total disk I/O. Note, this is only relevant on X86 platforms with a 4 KB page size. Page Reads/sec (Hard Page Fault) Page Reads/sec is the number of times the disk was accessed to resolve hard page faults. It includes reads to satisfy faults in the file system cache (usually requested by applications) and in non-cached memory mapped files. This counter counts numbers of read operations, without regard to the numbers of pages retrieved by each operation. This counter displays the difference between the values observed in the last two samples, divided by the duration of the sample interval. Page Writes/sec (Hard Page Fault) Page Writes/sec is the number of times pages were written to disk to free up space in physical memory. Pages are written to disk only if they are changed while in physical memory, so they are likely to hold data, not code. This counter counts write operations, without regard to the number of pages written in each operation. This counter displays the difference between the values observed in the last two samples, divided by the duration of the sample interval. *Pages Input/sec (Hard Page Fault) Pages Input/sec is the number of pages read from disk to resolve hard page faults. It includes pages retrieved to satisfy faults in the file system cache and in non-cached memory mapped files. This counter counts numbers of pages, and can be compared to other counts of pages, such as Memory:Page Faults/sec, without conversion. This counter displays the difference between the values observed in the last two samples, divided by the duration of the sample interval. This is one of the key counters to monitor for potential performance complaints. Because a process must wait for a read page fault this counter, read page faults have a direct impact on the perceived performance of a process. *Pages Output/sec (Hard Page Fault) Pages Output/sec is the number of pages written to disk to free up space in physical memory. Pages are written back to disk only if they are changed in physical memory, so they are likely to hold data, not code. A high rate of pages output might indicate a memory shortage. Windows NT writes more pages back to disk to free up space when physical memory is in short supply. This counter counts numbers of pages, and can be compared to other counts of pages, without conversion. This counter displays the difference between the values observed in the last two samples, divided by the duration of the sample interval. Like Pages Input/sec, this is one of the key counters to monitor. Processes will generally not notice write page faults unless the disk I/O begins to interfere with normal data operations. Demand Zero Faults/Sec (Soft Page Fault) Demand Zero Faults/sec is the number of page faults that require a zeroed page to satisfy the fault. Zeroed pages, pages emptied of previously stored data and filled with zeros, are a security feature of Windows NT. Windows NT maintains a list of zeroed pages to accelerate this process. This counter counts numbers of faults, without regard to the numbers of pages retrieved to satisfy the fault. This counter displays the difference between the values observed in the last two samples, divided by the duration of the sample interval. Transition Faults/Sec (Soft Page Fault) Transition Faults/sec is the number of page faults resolved by recovering pages that were on the modified page list, on the standby list, or being written to disk at the time of the page fault. The pages were recovered without additional disk activity. Transition faults are counted in numbers of faults, without regard for the number of pages faulted in each operation. This counter displays the difference between the values observed in the last two samples, divided by the duration of the sample interval. System Working Set System Working Set Like processes, the system page-able code and data are managed by a working set. For the purpose of this course, that working set is referred to as the System Working Set. This is done to differentiate the system cache portion of the working set from the entire working set. There are five different types of pages that make up the System Working Set. They are: system cache; paged pool; page-able code and data in ntoskrnl.exe; page-able code, and data in device drivers and system-mapped views. Unfortunately, some of the counters that appear to represent the system cache actually represent the entire system working set. Where noted system cache actually represents the entire system working set. Note The counters listed are a subset of the counters you should capture. *Memory: Cache Bytes (Represents Total System Working Set) Represents the total size of the System Working Set including: system cache; paged pool; pageable code and data in ntoskrnl.exe; pageable code and data in device drivers; and system-mapped views. Cache Bytes is the sum of the following counters: System Cache Resident Bytes, System Driver Resident Bytes, System Code Resident Bytes, and Pool Paged Resident Bytes. Memory: System Cache Resident Bytes (System Cache) System Cache Resident Bytes is the number of bytes from the file system cache that are resident in physical memory. Windows 2000 Cache Manager works with the memory manager to provide virtual block stream and file data caching. For more information, see also…Inside Windows 2000,Third Edition, pp. 645-650 and p. 656. Memory: Pool Paged Resident Bytes Represents the physical memory consumed by Paged Pool. This counter should NOT be monitored by itself. You must also monitor Memory: Paged Pool. A leak in the pool may not show up in Pool paged Resident Bytes. Memory: System Driver Resident Bytes Represents the physical memory consumed by driver code and data. System Driver Resident Bytes and System Driver Total Bytes do not include code that must remain in physical memory and cannot be written to disk. Memory: System Code Resident Bytes Represents the physical memory consumed by page-able system code. System Code Resident Bytes and System Code Total Bytes do not include code that must remain in physical memory and cannot be written to disk. Working Set Performance Counter You can measure the number of page faults in the System Working Set by monitoring the Memory: Cache Faults/sec counter. Contrary to the “Explain” shown in System Monitor, this counter measures the total amount of page faults/sec in the System Working Set, not only the System Cache. You cannot measure the performance of the System Cache using this counter alone. For more information, see also…Inside Windows 2000,Third Edition, p. 656. Note You will find that in general the working set manager will usually trim the working sets of normal processes prior to trimming the system working set. System Cache System Cache The Windows 2000 cache manager provides a write-back cache with lazy writing and intelligent read-ahead. Files are not written to disk immediately but differed until the cache manager calls the memory manager to flush the cache. This helps to reduce the total number of I/Os. Once per second, the lazy writer thread queues one-eighth of the dirty pages in the system cache to be written to disk. If this is not sufficient to meet the needs, the lazy writer will calculate a larger value. If the dirty page threshold is exceeded prior to lazy writer waking, the cache manager will wake the lazy writer. Important It should be pointed out that mapped files or files opened with FILE_FLAG_NO_BUFFERING, do not participate in the System Cache. For more information regarding mapped views, see also…Inside Windows 2000,Third Edition, p. 669. For those applications that would like to leverage system cache but cannot tolerate write delays, the cache manager supports write through operations via the FILE_FLAG_WRITE_THROUGH. On the other hand, an application can disable lazy writing by using the FILE_ATTRIBUTE_TEMPORARY. If this flag is enabled, the lazy writer will not write the pages to disk unless there is a shortage of memory or the file is closed. Important Microsoft SQL Server uses both FILE_FLAG_NO_BUFFERING and FILE_FLAG_WRITE_THROUGH Tip The file system cache is not represented by a static amount of memory. The system cache can and will grow. It is not unusual to see the system cache consume a large amount of memory. Like other working sets, it is trimmed under pressure but is generally the last thing to be trimmed. System Cache Performance Counters The counters listed are a subset of the counters you should capture. Cache: Data Flushes/sec Data Flushes/sec is the rate at which the file system cache has flushed its contents to disk as the result of a request to flush or to satisfy a write-through file write request. More than one page can be transferred on each flush operation. Cache: Data Flush Pages/sec Data Flush Pages/sec is the number of pages the file system cache has flushed to disk as a result of a request to flush or to satisfy a write-through file write request. Cache: Lazy Write Flushes/sec Represents the rate of lazy writes to flush the system cache per second. More than one page can be transferred per second. Cache: Lazy Write Pages/sec Lazy Write Pages/sec is the rate at which the Lazy Writer thread has written to disk. Note When looking at Memory:Cache Faults/sec, you can remove cache write activity by subtracting (Cache: Data Flush Pages/sec + Cache: Lazy Write Pages/sec). This will give you a better idea of how much other page faulting activity is associated with the other components of the System Working Set. However, you should note that there is no easy way to remove the page faults associated with file cache read activity. For more information, see the following Knowledge Base articles: Q145952 (NT4) Event ID 26 Appears If Large File Transfer Fails Q163401 (NT4) How to Disable Network Redirector File Caching Q181073 (SQL 6.5) DUMP May Cause Access Violation on Win2000 System Pool System Pool As documented earlier, there are two types of shared pool memory: non-paged pool and paged pool. Like private memory, pool memory is susceptible to a leak. Nonpaged Pool Miscellaneous kernel code and structures, and drivers that need working memory while at or above DPC/dispatch level use non-paged pool. The primary counter for non-paged pool is Memory: Pool Nonpaged Bytes. This counter will usually between 3 and 30 MB. Paged Pool Drivers that do not need to access memory above DPC/Dispatch level are one of the primary users of paged pool, however any process can use paged pool by leveraging the ExAllocatePool calls. Paged pool also contains the Registry and file and printing structures. The primary counters for monitoring paged pool is Memory: Pool Paged Bytes. This counter will usually be between 10-30MB plus the size of the Registry. To determine how much of paged pool is currently resident in physical memory, monitor Memory: Pool Paged Resident Bytes. Note The paged and non-paged pools are two of the components of the System Working Set. If a suspected leak is clearly visible in the overview and not associated with a process, then it is most likely a pool leak. If the leak is not associated with SQL Server handles, OLDEB providers, XPROCS or SP_OA calls then most likely this call should be pushed to the Windows NT group. For more information, see the following Knowledge Base articles: Q265028 (MS) Pool Tags Q258793 (MS) How to Find Memory Leaks by Using Pool Bitmap Analysis Q115280 (MS) Finding Windows NT Kernel Mode Memory Leaks Q177415 (MS) How to Use Poolmon to Troubleshoot Kernel Mode Memory Leaks Q126402 PagedPoolSize and NonPagedPoolSize Values in Windows NT Q247904 How to Configure Paged Pool and System PTE Memory Areas Tip To isolate pool leaks you will need to isolate all drivers and third-party processes. This should be done by disabling each service or driver one at a time and monitoring the effect. You can also monitor paged and non-paged pool through poolmon. If pool tagging has been enabled via GFLAGS, you may be able to associate the leak to a particular tag. If you suspect a particular tag, you should involve the platform support group. Process Memory Counters Process _Total Limitations Although the rollup of _Total for Process: Private Bytes, Virtual Bytes, Handles and Threads, represent the key resources being used across all processes, they can be misleading when evaluating a memory leak. This is because a leak in one process may be masked by a decrease in another process. Note The counters listed are a subset of the counters you should capture. Tip When analyzing memory leaks, it is often easier to a build either a separate chart or report showing only one or two key counters for all process. The primary counter used for leak analysis is private bytes, but processes can leak handles and threads just as easily. After a suspect process is located, build a separate chart that includes all the counters for that process. Individual Process Counters When analyzing individual process for memory leaks you should include the counters listed.  Process: % Processor Time  Process: Working Set (includes shared pages)  Process: Virtual Bytes  Process: Private Bytes  Process: Page Faults/sec  Process: Handle Count  Process: Thread Count  Process: Pool Paged Bytes  Process: Pool Nonpaged Bytes Tip WINLOGON, SVCHOST, services, or SPOOLSV are referred to as HELPER processes. They provide core functionality for many operations and as such are often extended by the addition of third-party DLLs. Tlist –s may help identify what services are running under a particular helper. Helper Processes Helper Processes Winlogon, Services, and Spoolsv and Svchost are examples of what are referred to as HELPER processes. They provide core functionality for many operations and, as such, are often extended by the addition of third-party DLLs. Running every service in its own process can waste system resources. Consequently, some services run in their own processes while others share a process with other services. One problem with sharing a process is that a bug in one service may cause the entire process to fail. The resource kit tool, Tlist when used with the –s qualifier can help you identify what services are running in what processes. WINLOGON Used to support GINAs. SPOOLSV SPOOLSV is responsible for printing. You will need to investigate all added printing functionality. Services Service is responsible for system services. Svchost.exe Svchost.exe is a generic host process name for services that are run from dynamic-link libraries (DLLs). There can be multiple instances of Svchost.exe running at the same time. Each Svchost.exe session can contain a grouping of services, so that separate services can be run depending on how and where Svchost.exe is started. This allows for better control and debugging. The Effect of Memory on Other Components Memory Drives Overall Performance Processor, cache, bus speeds, I/O, all of these resources play a roll in overall perceived performance. Without minimizing the impact of these components, it is important to point out that a shortage of memory can often have a larger perceived impact on performance than a shortage of some other resource. On the other hand, an abundance of memory can often be leveraged to mask bottlenecks. For instance, in certain environments, file system cache can significantly reduce the amount of disk I/O, potentially masking a slow I/O subsystem. Effect on I/O I/O can be driven by a number of memory considerations. Page read/faults will cause a read I/O when a page is not in memory. If the modified page list becomes too long the Modified Page Writer and Mapped Page Writer will need to start flushing pages causing disk writes. However, the one event that can have the greatest impact is running low on available memory. In this case, all of the above events will become more pronounced and have a larger impact on disk activity. Effect on CPU The most effective use of a processor from a process perspective is to spend as much time possible executing user mode code. Kernel mode represents processor time associated with doing work, directly or indirectly, on behalf of a thread. This includes items such as synchronization, scheduling, I/O, memory management, and so on. Although this work is essential, it takes processor cycles and the cost, in cycles, to transition between user and kernel mode is expensive. Because all memory management and I/O functions must be done in kernel mode, it follows that the fewer the memory resources the more cycles are going to be spent managing those resources. A direct result of low memory is that the Working Set Manager, Modified Page Writer and Mapped Page Writer will have to use more cycles attempting to free memory. Analyzing Memory Look for Trends and Trend Relationships Troubleshooting performance is about analyzing trends and trend relationships. Establishing that some event happened is not enough. You must establish the effect of the event. For example, you note that paging activity is high at the same time that SQL Server becomes slow. These two individual facts may or may not be related. If the paging is not associated with SQL Servers working set, or the disks SQL is using there may be little or no cause/affect relationship. Look at Physical Memory First The first item to look at is physical memory. You need to know how much physical and page file space the system has to work with. You should then evaluate how much available memory there is. Just because the system has free memory does not mean that there is not any memory pressure. Available Bytes in combination with Pages Input/sec and Pages Output/sec can be a good indicator as to the amount of pressure. The goal in a perfect world is to have as little hard paging activity as possible with available memory greater than 5 MB. This is not to say that paging is bad. On the contrary, paging is a very effective way to manage a limited resource. Again, we are looking for trends that we can use to establish relationships. After evaluating physical memory, you should be able to answer the following questions:  How much physical memory do I have?  What is the commit limit?  Of that physical memory, how much has the operating system committed?  Is the operating system over committing physical memory?  What was the peak commit charge?  How much available physical memory is there?  What is the trend associated with committed and available? Review System Cache and Pool Contribution After you understand the individual process memory usage, you need to evaluate the System Cache and Pool usage. These can and often represent a significant portion of physical memory. Be aware that System Cache can grow significantly on a file server. This is usually normal. One thing to consider is that the file system cache tends to be the last thing trimmed when memory becomes low. If you see abrupt decreases in System Cache Resident Bytes when Available Bytes is below 5 MB you can be assured that the system is experiencing excessive memory pressure. Paged and non-paged pool size is also important to consider. An ever-increasing pool should be an indicator for further research. Non-paged pool growth is usually a driver issue, while paged pool could be driver-related or process-related. If paged pool is steadily growing, you should investigate each process to see if there is a specific process relationship. If not you will have to use tools such as poolmon to investigate further. Review Process Memory Usage After you understand the physical memory limitations and cache and pool contribution you need to determine what components or processes are creating the pressure on memory, if any. Be careful if you opt to chart the _Total Private Byte’s rollup for all processes. This value can be misleading in that it includes shared pages and can therefore exceed the actual amount of memory being used by the processes. The _Total rollup can also mask processes that are leaking memory because other processes may be freeing memory thus creating a balance between leaked and freed memory. Identify processes that expand their working set over time for further analysis. Also, review handles and threads because both use resources and potentially can be mismanaged. After evaluating the process resource usage, you should be able to answer the following:  Are any of the processes increasing their private bytes over time?  Are any processes growing their working set over time?  Are any processes increasing the number of threads or handles over time?  Are any processes increasing their use of pool over time?  Is there a direct relationship between the above named resources and total committed memory or available memory?  If there is a relationship, is this normal behavior for the process in question? For example, SQL does not commit ‘min memory’ on startup; these pages are faulted in into the working set as needed. This is not necessarily an indication of a memory leak.  If there is clearly a leak in the overview and is not identifiable in the process counters it is most likely in the pool.  If the leak in pool is not associated with SQL Server handles, then more often than not, it is not a SQL Server issue. There is however the possibility that the leak could be associated with third party XPROCS, SP_OA* calls or OLDB providers. Review Paging Activity and Its Impact on CPU and I/O As stated earlier, paging is not in and of itself a bad thing. When starting a process the system faults in the pages of an executable, as they are needed. This is preferable to loading the entire image at startup. The same can be said for memory mapped files and file system cache. All of these features leverage the ability of the system to fault in pages as needed The greatest impact of paging on a process is when the process must wait for an in-page fault or when page file activity represents a significant portion of the disk activity on the disk the application is actively using. After evaluating page fault activity, you should be able to answer the following questions:  What is the relationship between PageFaults/sec and Page Input/sec + Page Output/Sec?  What is the relationship if any between hard page faults and available memory?  Does paging activity represent a significant portion of processor or I/O resource usage? Don’t Prematurely Jump to Any Conclusions Analyzing memory pressure takes time and patience. An individual counter in and of it self means little. It is only when you start to explore relationships between cause and effect that you can begin to understand the impact of a particular counter. The key thoughts to remember are:  With the exception of a swap (when the entire process’s working set has been swapped out/in), hard page faults to resolve reads, are the most expensive in terms its effect on a processes perceived performance.  In general, page writes associated with page faults do not directly affect a process’s perceived performance, unless that process is waiting on a free page to be made available. Page file activity can become a problem if that activity competes for a significant percentage of the disk throughput in a heavy I/O orientated environment. That assumes of course that the page file resides on the same disk the application is using. Lab 3.1 System Memory Lab 3.1 Analyzing System Memory Using System Monitor Exercise 1 – Troubleshooting the Cardinal1.log File Students will evaluate an existing System Monitor log and determine if there is a problem and what the problem is. Students should be able to isolate the issue as a memory problem, locate the offending process, and determine whether or not this is a pool issue. Exercise 2 – Leakyapp Behavior Students will start leaky app and monitor memory, page file and cache counters to better understand the dynamics of these counters. Exercise 3 – Process Swap Due To Minimizing of the Cmd Window Students will start SQL from command line while viewing SQL process performance counters. Students will then minimize the window and note the effect on the working set. Overview What You Will Learn After completing this lab, you will be able to:  Use some of the basic functions within System Monitor.  Troubleshoot one or more common performance scenarios. Before You Begin Prerequisites To complete this lab, you need the following:  Windows 2000  SQL Server 2000  Lab Files Provided  LeakyApp.exe (Resource Kit) Estimated time to complete this lab: 45 minutes Exercise 1 Troubleshooting the Cardinal1.log File In this exercise, you will analyze a log file from an actual system that was having performance problems. Like an actual support engineer, you will not have much information from which to draw conclusions. The customer has sent you this log file and it is up to you to find the cause of the problem. However, unlike the real world, you have an instructor available to give you hints should you become stuck. Goal Review the Cardinal1.log file (this file is from Windows NT 4.0 Performance Monitor, which Windows 2000 can read). Chart the log file and begin to investigate the counters to determine what is causing the performance problems. Your goal should be to isolate the problem to a major area such as pool, virtual address space etc, and begin to isolate the problem to a specific process or thread. This lab requires access to the log file Cardinal1.log located in C:\LABS\M3\LAB1\EX1  To analyze the log file 1. Using the Performance MMC, select the System Monitor snap-in, and click the View Log File Data button (icon looks like a disk). 2. Under Files of type, choose PERFMON Log Files (*.log) 3. Navigate to the folder containing Cardinal1.log file and open it. 4. Begin examining counters to find what might be causing the performance problems. When examining some of these counters, you may notice that some of them go off the top of the chart. It may be necessary to adjust the scale on these. This can be done by right-clicking the rightmost pane and selecting Properties. Select the Data tab. Select the counter that you wish to modify. Under the Scale option, change the scale value, which makes the counter data visible on the chart. You may need to experiment with different scale values before finding the ideal value. Also, it may sometimes be beneficial to adjust the vertical scale for the entire chart. Selecting the Graph tab on the Properties page can do this. In the Vertical scale area, adjust the Maximum and Minimum values to best fit the data on the chart. Lab 3.1, Exercise 1: Results Exercise 2 LeakyApp Behavior In this lab, you will have an opportunity to work with a partner to monitor a live system, which is suffering from a simulated memory leak. Goal During this lab, your goal is to observe the system behavior when memory starts to become a limited resource. Specifically you will want to monitor committed memory, available memory, the system working set including the file system cache and each processes working set. At the end of the lab, you should be able to provide an answer to the listed questions.  To monitor a live system with a memory leak 1. Choose one of the two systems as a victim on which to run the leakyapp.exe program. It is recommended that you boot using the \MAXMEM=128 option so that this lab goes a little faster. You and your partner should decide which server will play the role of the problematic server and which server is to be used for monitoring purposes. 2. On the problematic server, start the leakyapp program. 3. On the monitoring system, create a counter that logs all necessary counters need to troubleshoot a memory problem. This should include physicaldisk counters if you think paging is a problem. Because it is likely that you will only need to capture less than five minutes of activity, the suggested interval for capturing is five seconds. 4. After the counters have been started, start the leaky application program 5. Click Start Leaking. The button will now change to Stop Leaking, which indicates that the system is now leaking memory. 6. After leakyapp shows the page file is 50 percent full, click Stop leaking. Note that the process has not given back its memory, yet. After approximately one minute, exit. Lab 3.1, Exercise 2: Questions After analyzing the counter logs you should be able to answer the following: 1. Under which system memory counter does the leak show up clearly? Memory:Committed Bytes 2. What process counter looked very similar to the overall system counter that showed the leak? Private Bytes 3. Is the leak in Paged Pool, Non-paged pool, or elsewhere? Elsewhere 4. At what point did Windows 2000 start to aggressively trim the working sets of all user processes? <5 MB Free 5. Was the System Working Set trimmed before or after the working sets of other processes? After 6. What counter showed this? Memory:Cache Bytes 7. At what point was the File System Cache trimmed? After the first pass through all other working sets 8. What was the effect on all the processes working set when the application quit leaking? None 9. What was the effect on all the working sets when the application exited? Nothing, initially; but all grew fairly quickly based on use 10. When the server was running low on memory, which was Windows spending more time doing, paging to disk or in-paging? Paging to disk, initially; however, as other applications began to run, in-paging increased Exercise 3 Minimizing a Command Window In this exercise, you will have an opportunity to observe the behavior of Windows 2000 when a command window is minimized. Goal During this lab, your goal is to observe the behavior of Windows 2000 when a command window becomes minimized. Specifically, you will want to monitor private bytes, virtual bytes, and working set of SQL Server when the command window is minimized. At the end of the lab, you should be able to provide an answer to the listed questions.  To monitor a command window’s working set as the window is minimized 1. Using System Monitor, create a counter list that logs all necessary counters needed to troubleshoot a memory problem. Because it is likely that you will only need to capture less than five minutes of activity, the suggested capturing interval is five seconds. 2. After the counters have been started, start a Command Prompt window on the target system. 3. In the command window, start SQL Server from the command line. Example: SQL Servr.exe –c –sINSTANCE1 4. After SQL Server has successfully started, Minimize the Command Prompt window. 5. Wait approximately two minutes, and then Restore the window. 6. Wait approximately two minutes, and then stop the counter log. Lab 3.1, Exercise 3: Questions After analyzing the counter logs you should be able to answer the following questions: 1. What was the effect on SQL Servers private bytes, virtual bytes, and working set when the window was minimized? Private Bytes and Virtual Bytes remained the same, while Working Set went to 0 2. What was the effect on SQL Servers private bytes, virtual bytes, and working set when the window was restored? None; the Working Set did not grow until SQL accessed the pages and faulted them back in on an as-needed basis SQL Server Memory Overview SQL Server Memory Overview Now that you have a better understanding of how Windows 2000 manages memory resources, you can take a closer look at how SQL Server 2000 manages its memory. During the course of the lecture and labs you will have the opportunity to monitor SQL Servers use of memory under varying conditions using both System Monitor counters and SQL Server tools. SQL Server Memory Management Goals Because SQL Server has in-depth knowledge about the relationships between data and the pages they reside on, it is in a better position to judge when and what pages should be brought into memory, how many pages should be brought in at a time, and how long they should be resident. SQL Servers primary goals for management of its memory are the following:  Be able to dynamically adjust for varying amounts of available memory.  Be able to respond to outside memory pressure from other applications.  Be able to adjust memory dynamically for internal components. Items Covered  SQL Server Memory Definitions  SQL Server Memory Layout  SQL Server Memory Counters  Memory Configurations Options  Buffer Pool Performance and Counters  Set Aside Memory and Counters  General Troubleshooting Process  Memory Myths and Tips SQL Server Memory Definitions SQL Server Memory Definitions Pool A group of resources, objects, or logical components that can service a resource allocation request Cache The management of a pool or resource, the primary goal of which is to increase performance. Bpool The Bpool (Buffer Pool) is a single static class instance. The Bpool is made up of 8-KB buffers and can be used to handle data pages or external memory requests. There are three basic types or categories of committed memory in the Bpool.  Hashed Data Pages  Committed Buffers on the Free List  Buffers known by their owners (Refer to definition of Stolen) Consumer A consumer is a subsystem that uses the Bpool. A consumer can also be a provider to other consumers. There are five consumers and two advanced consumers who are responsible for the different categories of memory. The following list represents the consumers and a partial list of their categories  Connection – Responsible for PSS and ODS memory allocations  General – Resource structures, parse headers, lock manager objects  Utilities – Recovery, Log Manager  Optimizer – Query Optimization  Query Plan – Query Plan Storage Advanced Consumer Along with the five consumers, there are two advanced consumers. They are  Ccache – Procedure cache. Accepts plans from the Optimizer and Query Plan consumers. Is responsible for managing that memory and determines when to release the memory back to the Bpool.  Log Cache – Managed by the LogMgr, which uses the Utility consumer to coordinate memory requests with the Bpool. Reservation Requesting the future use of a resource. A reservation is a reasonable guarantee that the resource will be available in the future. Committed Producing the physical resource Allocation The act of providing the resource to a consumer Stolen The act of getting a buffer from the Bpool is referred to as stealing a buffer. If the buffer is stolen and hashed for a data page, it is referred to as, and counted as, a Hashed buffer, not a stolen buffer. Stolen buffers on the other hand are buffers used for things such as procedure cache and SRV_PROC structures. Target Target memory is the amount of memory SQL Server would like to maintain as committed memory. Target memory is based on the min and max server configuration values and current available memory as reported by the operating system. Actual target calculation is operating system specific. Memory to Leave (Set Aside) The virtual address space set aside to ensure there is sufficient address space for thread stacks, XPROCS, COM objects etc. Hashed Page A page in pool that represents a database page. SQL Server Memory Layout Virtual Address Space When SQL Server is started the minimum of physical ram or virtual address space supported by the OS is evaluated. There are many possible combinations of OS versions and memory configurations. For example: you could be running Microsoft Windows 2000 Advanced Server with 2 GB or possibly 4 GB of memory. To avoid page file use, the appropriate memory level is evaluated for each configuration. Important Utilities can inject a DLL into the process address space by using HKEY_LOCAL_MACHINE\Software\Microsoft\Windows NT\CurrentVersion\Windows\AppInit_DLLs When the USER32.dll library is mapped into the process space, so, too, are the DLLs listed in the Registry key. To determine what DLL’s are running in SQL Server address space you can use tlist.exe. You can also use a tool such as Depends from Microsoft or HandelEx from http://ww.sysinternals.com. Memory to Leave As stated earlier there are many possible configurations of physical memory and address space. It is possible for physical memory to be greater than virtual address space. To ensure that some virtual address space is always available for things such as thread stacks and external needs such as XPROCS, SQL Server reserves a small portion of virtual address space prior to determining the size of the buffer pool. This address space is referred to as Memory To Leave. Its size is based on the number of anticipated tread stacks and a default value for external needs referred to as cmbAddressSave. After reserving the buffer pool space, the Memory To Leave reservation is released. Buffer Pool Space During Startup, SQL Server must determine the maximum size of the buffer pool so that the BUF, BUFHASH and COMMIT BITMAP structures that are used to manage the Bpool can be created. It is important to understand that SQL Server does not take ‘max memory’ or existing memory pressure into consideration. The reserved address space of the buffer pool remains static for the life of SQL Server process. However, the committed space varies as necessary to provide dynamic scaling. Remember only the committed memory effects the overall memory usage on the machine. This ensures that the max memory configuration setting can be dynamically changed with minimal changes needed to the Bpool. The reserved space does not need to be adjusted and is maximized for the current machine configuration. Only the committed buffers need to be limited to maintain a specified max server memory (MB) setting. SQL Server Startup Pseudo Code The following pseudo code represents the process SQL Server goes through on startup. Warning This example does not represent a completely accurate portrayal of the steps SQL Server takes when initializing the buffer pool. Several details have been left out or glossed over. The intent of this example is to help you understand the general process, not the specific details.  Determine the size of cmbAddressSave (-g)  Determine Total Physical Memory  Determine Available Physical Memory  Determine Total Virtual Memory  Calculate MemToLeave maxworkterthreads * (stacksize=512 KB) + (cmbAddressSave = 256 MB)  Reserve MemToLeave and set PAGE_NOACCESS  Check for AWE, test to see if it makes sense to use it and log the results • Min(Available Memory, Max Server Memory) > Virtual Memory • Supports Read Scatter • SQL Server not started with -f • AWE Enabled via sp_configure • Enterprise Edition • Lock Pages In Memory user right enabled  Calculate Virtual Address Limit VA Limit = Min(Physical Memory, Virtual Memory – MemtoLeave)  Calculate the number of physical and virtual buffers that can be supported AWE Present Physical Buffers = (RAM / (PAGESIZE + Physical Overhead)) Virtual Buffers = (VA Limit / (PAGESIZE + Virtual Overhead)) AWE Not Present Physical Buffers = Virtual Buffers = VA Limit / (PAGESIZE + Physical Overhead + Virtual Overhead)  Make sure we have the minimum number of buffers Physical Buffers = Max(Physical Buffers, MIN_BUFFERS)  Allocate and commit the buffer management structures  Reserve the address space required to support the Bpool buffers  Release the MemToLeave SQL Server Startup Pseudo Code Example The following is an example based on the pseudo code represented on the previous page. This example is based on a machine with 384 MB of physical memory, not using AWE or /3GB. Note CmbAddressSave was changed between SQL Server 7.0 and SQL Server 2000. For SQL Server 7.0, cmbAddressSave was 128. Warning This example does not represent a completely accurate portrayal of the steps SQL Server takes when initializing the buffer pool. Several details have been left out or glossed over. The intent of this example is to help you understand the general process, not the specific details.  Determine the size of cmbAddressSave (No –g so 256MB)  Determine Total Physical Memory (384)  Determine Available Physical Memory (384)  Determine Total Virtual Memory (2GB)  Calculate MemToLeave maxworkterthreads * (stacksize=512 KB) + (cmbAddressSave = 256 MB) (255 * .5MB + 256MB = 384MB)  Reserve MemToLeave and set PAGE_NOACCESS  Check for AWE, test to see if it makes sense to use it and log the results (AWE Not Enabled)  Calculate Virtual Address Limit VA Limit = Min(Physical Memory, Virtual Memory – MemtoLeave) 384MB = Min(384MB, 2GB – 384MB)  Calculate the number of physical and virtual buffers that can be supported AWE Not Present 48664 (approx) = 384 MB / (8 KB + Overhead)  Make sure we have the minimum number of buffers Physical Buffers = Max(Physical Buffers, MIN_BUFFERS) 48664 = Max(48664,1024)  Allocate and commit the buffer management structures  Reserve the address space required to support the Bpool buffers  Release the MemToLeave Tip Trace Flag 1604 can be used to view memory allocations on startup. The cmbAddressSave can be adjusted using the –g XXX startup parameter. SQL Server Memory Counters SQL Server Memory Counters The two primary tools for monitoring and analyzing SQL Server memory usage are System Monitor and DBCC MEMORYSTATUS. For detailed information on DBCC MEMORYSTATUS refer to Q271624 Interpreting the Output of the DBCC MEMORYSTAUS Command. Important Represents SQL Server 2000 Counters. The counters presented are not the same as the counters for SQL Server 7.0. The SQL Server 7.0 counters are listed in the appendix. Determining Memory Usage for OS and BPOOL Memory Manager: Total Server memory (KB) - Represents all of SQL usage Buffer Manager: Total Pages - Represents total bpool usage To determine how much of Total Server Memory (KB) represents MemToLeave space; subtract Buffer Manager: Total Pages. The result can be verified against DBCC MEMORYSTATUS, specifically Dynamic Memory Manager: OS In Use. It should however be noted that this value only represents requests that went thru the bpool. Memory reserved outside of the bpool by components such as COM objects will not show up here, although they will count against SQL Server private byte count. Buffer Counts: Target (Buffer Manager: Target Pages) The size the buffer pool would like to be. If this value is larger than committed, the buffer pool is growing. Buffer Counts: Committed (Buffer Manager: Total Pages) The total number of buffers committed in the OS. This is the current size of the buffer pool. Buffer Counts: Min Free This is the number of pages that the buffer pool tries to keep on the free list. If the free list falls below this value, the buffer pool will attempt to populate it by discarding old pages from the data or procedure cache. Buffer Distribution: Free (Buffer Manager / Buffer Partition: Free Pages) This value represents the buffers currently not in use. These are available for data or may be requested by other components and mar

“相关推荐”对你有帮助么?

  • 非常没帮助
  • 没帮助
  • 一般
  • 有帮助
  • 非常有帮助
提交
评论
添加红包

请填写红包祝福语或标题

红包个数最小为10个

红包金额最低5元

当前余额3.43前往充值 >
需支付:10.00
成就一亿技术人!
领取后你会自动成为博主和红包主的粉丝 规则
hope_wisdom
发出的红包
实付
使用余额支付
点击重新获取
扫码支付
钱包余额 0

抵扣说明:

1.余额是钱包充值的虚拟货币,按照1:1的比例进行支付金额的抵扣。
2.余额无法直接购买下载,可以购买VIP、付费专栏及课程。

余额充值