sql oltp_SQL Server中的内存中OLTP的快速概述

sql oltp

This is in continuation of the previous articles How to monitor internal data structures of SQL Server In-Memory database objects and SQL Server In-Memory database internal memory structure monitoring.

这是前面文章如何监视SQL Server内存数据库对象的内部数据结构和SQL Server内存数据库内部内存结构监视的延续。

This article describes the concepts of In-memory-optimization and design principles.

本文介绍了内存中优化和设计原理的概念。

In this article, we will cover:

在本文中，我们将介绍：

1. Details of the In-Memory OLTP evolution
   内存中OLTP演进的详细信息
2. Provides samples to demonstrate In-Memory optimized tables
   提供样本以演示内存中优化的表
3. Create In-Memory Optimized tables using templates
   使用模板创建内存优化表
4. Explain In-Memory memory usage considerations
   解释内存中内存使用注意事项
5. And more…
   和更多…

In-Memory technologies in SQL Database can improve the performance of the application and it is the right choice for the workloads such as data ingestion, data load, and analytical queries.

SQL数据库中的内存中技术可以提高应用程序的性能，它是诸如数据摄取，数据加载和分析查询之类的工作负载的正确选择。

The business’s ability to adapt quickly driven innovation and meet new competition is a challenge. The challenging task of DBA always tends to adopt the changing needs and formulate the right design strategy to meet the needs of your services, and enterprise. Many of us will have a question about the factors for tradeoffs for seamless integration of scale-out options and to design High-Performance Computing System is massively rely on the better hardware configuration. The era of hardware, surging the hardware trends such as declining memory costs, multi-core processors, and stalling CPU clock rate increase—prompted the architectural design of in-memory computing.

企业适应快速驱动的创新并应对新竞争的能力是一项挑战。 DBA具有挑战性的任务总是倾向于适应不断变化的需求，并制定正确的设计策略来满足您的服务和企业的需求。 我们中的许多人都会对横向扩展选项的无缝集成以及设计高性能计算系统的权衡因素产生疑问，而高性能计算系统在很大程度上依赖更好的硬件配置。 硬件时代的来临，激增的硬件趋势，例如存储器成本的下降，多核处理器以及CPU时钟速率的增长停滞不前，这促进了内存计算的体系结构设计。

The system where performance is the key and the system should work on the near real-time data then the In-Memory technology solution is a choice. The way technology is trending leads to the evolution of such new features.

系统是性能的关键，并且系统应该在接近实时的数据上工作，然后选择内存技术解决方案。 技术的发展趋势导致了这些新功能的发展。

介绍 (Introduction)

In-Memory OLTP is a specialized, memory-optimized relational data management engine and native stored procedure compiler, integrated into SQL Server. Microsoft designed In-Memory OLTP to handle the most demanding OLTP workloads. In many cases, the memory-optimized tables can be created with DURABILITY = SCHEMA_ONLY option to avoid all logging and I/O.

内存中OLTP是专用的，内存优化的关系数据管理引擎和本机存储过程编译器，已集成到SQL Server中。 Microsoft设计了内存中OLTP以处理最苛刻的OLTP工作负载。 在许多情况下，可以使用DURABILITY = SCHEMA_ONLY选项创建内存优化表，以避免所有日志记录和I / O。

In-Memory OLTP introduces the following concepts:

内存中OLTP引入以下概念：

• In-Memory optimized tables and indexes
  内存中优化的表和索引
• Non-durable tables, traditional temp tables
  非耐用表，传统临时表
• Natively compiled stored procedures and UDF’s
  本机编译的存储过程和UDF
• Memory-optimized table type for table variable – This can be used as a replacement for temporary objects
  表变量的内存优化表类型–可以替代临时对象
• And more…
  和更多…

内存中OLTP系统的含义 (Implications of In-Memory OLTP systems)

• Low latency, high throughput, faster response time
  低延迟，高吞吐量，更快的响应时间
• High efficiency
  高效率
• High performance
  高性能
• Zero or no lock escalation management is through an optimistic concurrency model, better concurrency management
  零或无锁升级管理是通过乐观并发模型，更好的并发管理

建议使用内存中OLTP？ (In-Memory OLTP recommended?)

If the system with one or more of the following condition is “Yes” then seriously consider the potential benefits of migrating to In-Memory OLTP:

如果具有以下一个或多个以下条件的系统为“是”，请认真考虑迁移到内存中OLTP的潜在好处：

• Existing SQL Server (or other relational databases) applications that require performance and scalability gains
  需要性能和可伸缩性获得的现有SQL Server（或其他关系数据库）应用程序
• RDBMS that is experiencing database bottlenecks – most prevalently around locking/latching or code execution
  遇到数据库瓶颈的RDBMS –最普遍是围绕锁定/锁存或代码执行
• Environments that do not use a relational database in the critical performance path due to the perceived performance overhead
  由于感知到的性能开销而在关键性能路径中不使用关系数据库的环境

好处 (Benefits)

• Eliminate contention
  消除竞争
• Minimize I/O logging
  最小化I / O日志记录
• Efficient data retrieval
  高效的数据检索
• Minimize code execution time
  减少代码执行时间
• CPU efficiency
  CPU效率
• I/O reduction/removal
  I / O减少/删除

局限性 (Limitation)

• The table should have at least one index
  该表应至少有一个索引
• No concept of HEAP
  没有HEAP的概念
• Except for an application lock, In-Memory OLTP does not offer the ability to lock records like standard SQL Server queries
  除应用程序锁定外，In-Memory OLTP不提供锁定记录的功能，例如标准SQL Server查询
• Memory Limitations
  内存限制

As mentioned earlier, data structures that make up memory-optimized tables are all stored in memory, and unlike traditional B-tree objects are not backed by durable storage. Scenarios, where sufficient memory is not available to store the memory-optimized rows, can be problematic. When evaluating migration, determine the size of the memory required. It is also critical to consider the workload that may produce multiple versions of rows, which requires additional memory allocations.

如前所述，构成内存优化表的数据结构都存储在内存中，并且与传统的B树对象不同，它不由持久存储支持。 在没有足够的内存来存储经过内存优化的行的情况下，可能会出现问题。 在评估迁移时，请确定所需的内存大小。 考虑可能会产生多个版本的行的工作负载也很重要，这需要额外的内存分配。

内存中OLTP设计注意事项 (In-Memory OLTP Design Considerations)

Reducing the time for each business transaction can be an important goal in terms of overall performance. Migrating the Transact-SQL code into natively compiled stored procedures and reducing the latency of transaction executions are critical factors in improving the overall user experience.

就整体绩效而言，减少每次业务交易的时间可能是一个重要目标。 将Transact-SQL代码迁移到本地编译的存储过程中并减少事务执行的延迟是改善整体用户体验的关键因素。

入门 (Getting started)

1. Create a filegroup with memory_optimimized_data option
   使用memory_optimimized_data选项创建文件组
2. Implement logical file to the group
   将逻辑文件实施到组
3. Create a table with memory_optimization options
   创建带有memory_optimization选项的表

• Note: In Azure SQL Database, in-memory technologies are only available in the Premium service tier. I will discuss more about it in the next article.
• 注意：在Azure SQL数据库中，内存技术仅在高级服务层中可用。 我将在下一篇文章中讨论更多有关它的内容。

创建文件组 (Create filegroup)

ALTER DATABASE AdventureWorks2014 
ADD FILEGROUP InMemAdventureWorks2014FG CONTAINS MEMORY_OPTIMIZED_DATA ;
     
ALTER DATABASE AdventureWorks2014 
ADD FILE (
    NAME=InMemAdventureWorks2014File, 
    FILENAME='f:\PowerSQL\InMemAdventureWorks2014File') 
TO FILEGROUP InMemAdventureWorks2014FG;

Now, let us verify MEMORY_OPTIMIZED_DATA option is enabled for the database.

现在，让我们确认已为数据库启用MEMORY_OPTIMIZED_DATA选项。

SELECT 
    sfg.name Name, 
    sdf.name logicalFileName,
    sfg.type_desc, 
    sdf.physical_name
FROM sys.filegroups sfg JOIN sys.database_files sdf ON sfg.data_space_id = sdf.data_space_id 
WHERE sfg.type = 'FX' AND sdf.type = 2

OR

要么

Browse the database properties, select Filegroups option; in the right pane, you can see the memory_optimized_data.

浏览数据库属性，选择“ 文件组”选项； 在右窗格中，您可以看到memory_optimized_data。

This section deals with creating a table and use of various parameters required for in-line memory optimization techniques

本节介绍如何创建表以及如何使用在线内存优化技术所需的各种参数

1. MEMORY_OPTIMIZED=ON : The table is memory-optimized
   MEMORY_OPTIMIZED = ON：该表是内存优化的
2. 1. Maintain session state management for an application
      维护应用程序的会话状态管理
   2. Commonly used as staging tables in an ETL scenario
      通常在ETL场景中用作登台表
   3. Temp table
      临时表
3. DURABILITY = SCHEMA_AND_DATA: Schema and Data available all the time in-Memory. The data is persistent in the memory and it is the default setting when creating memory-optimized tables
   DURABILITY = SCHEMA_AND_DATA：架构和数据始终在内存中可用。 数据在内存中持久保存，并且是创建内存优化表时的默认设置

SCHEMA_ONLY (SCHEMA_ONLY)

The following example creates an in-memory optimized table named InsertInMemDemo with Durability option SCHEMA_ONLY.

以下示例使用持久性选项SCHEMA_ONLY创建一个名为InsertInMemDemo的内存优化表。

CREATE TABLE InsertInMemDemo
(
Id INT NOT NULL,
data varchar(25)
constraint pk_id_1 primary key nonclustered(id))
WITH 
(MEMORY_OPTIMIZED=ON, DURABILITY=SCHEMA_ONLY)

Next, insert some dummy data from the InsertDemo table into the In-Memory optimized table and validate the number of records between the tables. This is just an example. You can insert the records in many ways.

接下来，将一些InsertDemo表中的伪数据插入内存中优化的表中，并验证表之间的记录数。 这只是一个例子。 您可以通过多种方式插入记录。

Now, let us restart the SQL instance. This is to test the persistence of the newly created in-memory table. You can see from the below image, the newly created in-memory optimized table persistence is temporary and it’s bound to memory. Whenever the instance restarts, the data is flushed out of the memory.

现在，让我们重新启动SQL实例。 这是为了测试新创建的内存表的持久性。 您可以从下图看到，新创建的内存中优化表持久性是临时的，并且已绑定到内存。 每当实例重新启动时，数据就会从内存中清除。

SCHEMA_AND_DATA (SCHEMA_AND_DATA )

The following example created in-memory optimized table with data persistence and high performance:

以下示例创建了具有数据持久性和高性能的内存优化表：

CREATE TABLE InsertInMemDemo
(Id   INT NOT NULL, 
 data VARCHAR(25)
 CONSTRAINT pk_id_1 PRIMARY KEY NONCLUSTERED (id)
) WITH(MEMORY_OPTIMIZED = ON, DURABILITY = SCHEMA_AND_DATA);

The following T-SQL statement is an example to demonstrate the performance implication of traditional objects against the in-memory optimized table.

以下T-SQL语句是一个示例，用于说明传统对象对内存优化表的性能影响。

SET NOCOUNT ON;
DECLARE @start DATETIME= GETDATE();
DECLARE @id INT= 1;
WHILE @id < 10000
    BEGIN
        INSERT INTO InsertInMemDemo
        (id, 
         data
        )
        VALUES
        (@id, 
         'SQLShackDemo'
        );
        SET @id = @id + 1;
    END;
SELECT DATEDIFF(s, @start, GETDATE()) AS [MemInsert];
GO
DECLARE @start DATETIME= GETDATE();
DECLARE @id INT= 1;
WHILE @id < 10000
    BEGIN
        INSERT INTO InsertDemo
        (id, 
         data
        )
        VALUES
        (@id, 
         'SQLShackDemo'
        );
        SET @id = @id + 1;
    END;
     DATEDIFF(s, @start, GETDATE()) AS [Insert];
DROP TABLE InsertInMemDemo;
DROP TABLE InsertDemo;

The sample output proves that it’s 37 times faster than traditional non-memory optimized objects.

样本输出证明它比传统的非内存优化对象快37倍。

内存大小注意事项 (Memory Sizing Consideration)

Each delta file is sized approximately to 16MB for computers with memory greater than 16GB, and 1MB for computers with less than or equal to 16GB. Starting SQL Server 2016, SQL Server can use large checkpoint mode if it deems the storage subsystem is fast enough. In large checkpoint mode, delta files are sized at 128MB.

对于内存大于16GB的计算机，每个增量文件的大小大约为16MB，对于小于或等于16GB的计算机，每个增量文件的大小为1MB。 从SQL Server 2016开始，如果SQL Server认为存储子系统足够快，则可以使用大型检查点模式。 在大型检查点模式下，增量文件的大小为128MB。

In-memory optimized table—data is stored in the data and delta file pairs. It is also called a checkpoint-file-pair (CFP). The data file is used to store DML commands and the delta file is used for deleted rows. During DML operations many CFPs will be created, this causes increased recovery time and disk space usage.

内存中优化表-数据存储在数据和增量文件对中。 也称为检查点文件对（CFP）。 数据文件用于存储DML命令，而增量文件用于已删除的行。 在DML操作期间，将创建许多CFP，这会导致恢复时间和磁盘空间使用量增加。

In the following example, the sample table dbo.InMemDemoOrderTBL is created and loaded with some dummy values using a piece of T-SQL code and then followed with the memory calculation.

在以下示例中，示例表dbo。 创建InMemDemoOrderTBL并使用一段T-SQL代码加载一些虚拟值，然后进行内存计算。

CREATE TABLE dbo.InMemDemoOrderTBL
(Order_ID     INT NOT NULL, 
 CustomerName VARCHAR(25), 
 Order_Date   DATETIME DEFAULT GETDATE(), 
 Description NVARCHAR(100),
 CONSTRAINT PK_ID PRIMARY KEY NONCLUSTERED(Order_ID)
) WITH(MEMORY_OPTIMIZED = ON, DURABILITY = SCHEMA_AND_DATA);

The following T-SQL is code is used to generate sample rows.

下面的T-SQL是用于生成示例行的代码。

SET NOCOUNT ON;
DECLARE @start DATETIME= GETDATE();
DECLARE @id INT= 1;
WHILE @id < 500
BEGIN
INSERT INTO InMemDemoOrderTBL(Order_ID, 
CustomerName, 
Description)
VALUES(@id, 
'SQLShackDemo', 
'Table and Row Size Computation example');
SET @id = @id + 1;
END;
    
SELECT *
FROM InMemDemoOrderTBL;

The string function DATALENGTH is used to generate the size of the description column.

字符串函数DATALENGTH用于生成描述列的大小。

SELECT AVG(DATALENGTH(Description)) AS TEXTFieldSize FROM InMemDemoOrderTBL

Size is calculated as SUM ([size of data types]).

大小以SUM（[数据类型的大小]）计算。

• Bit: 1
  位：1
• Tinyint: 1
  Tinyint：1
• Smallint: 2
  Smallint：2
• Int: 4
  整数：4
• Real: 4
  实数：4
• Smalldatetime: 4
  小日期时间：4
• Smallmoney: 4
  小钱：4
• Bigint: 8
  Bigint：8
• Datetime: 8
  日期时间：8
• Datetime2: 8
  日期时间2：8
• Float: 8
  浮点数：8
• Money: 8
  金钱：8
• Numeric (precision <=18): 8
  数值（精度<= 18）：8
• Time: 8
  时间：8
• Numeric(precision>18): 16
  数值（精度> 18）：16
• Uniqueidentifier: 16
  唯一标识符：16

The following table defines the key metrics that are needed to estimate the size of data and indexes.

下表定义了估计数据和索引大小所需的关键指标。

Header Type	Data Structure	Bytes
RowHeader		32
	Begin TS	8
	End TS	8
	StmtID	4
	IdxLinCount	2
	IndexPointerArray	8
Row Data	Payload(=4(Order_ID)+24(CustomerName)+8(Order_Date)+76( average length of Description))	106

标头类型	数据结构	字节数
行标题		32
	开始TS	8
	结束TS	8
	身份识别码	4
	IdxLinCount	2
	IndexPointerArray	8
行数据	有效载荷（= 4（订单ID）+24（客户名称）+8（订单日期）+76（描述的平均长度））	106

To estimate the size data and index size, use the following formula.

要估计大小数据和索引大小，请使用以下公式。

Data Size	[RowHeaderBytes+Index*(8 bytes Per Index)+RowData]*No_Of_Rows
	{[(32+(1*8)+113]*499}/1024	74.55762
Index Size	[PointerSize(idxLinCount+IndexPointArray)+sum(keyColumnDataTypes)]*No_of_Rows
	((2+8+4)*499)/1024	6.822266
Table Size	DataSize+IndexSize	81.39355

资料大小	[RowHeaderBytes + Index *（每个索引8个字节）+ RowData] * No_Of_Rows
	{[（32+（1 * 8）+113] * 499} / 1024	74.55762
索引大小	[PointerSize（idxLinCount + IndexPointArray）+ sum（keyColumnDataTypes）] * No_of_Rows
	（（2 + 8 + 4）* 499）/ 1024	6.822266
桌子尺寸	数据大小+索引大小	81.39355

• Note: The calculated data and index sizes are near exact values. For row version, we need an extra 75 kb and for additional growth its better to reserve 30%(~25 kb). In total, we need a memory of 180 kb to process the in-memory optimized tables efficiently.
• 注意：计算的数据和索引大小接近精确值。 对于行版本，我们需要额外的75 kb，并且为了进一步增长，最好保留30％（〜25 kb）。 总体而言，我们需要180 kb的内存才能有效地处理内存中的优化表。

SELECT *
FROM sys.dm_db_xtp_table_memory_stats
WHERE object_id = OBJECT_ID('dbo.InMemDemoOrderTBL');

摘要： (Summary:)

In this article, we covered various concepts of the In-Memory optimized table and its features. I have included the important piece to understand the implication of memory consideration and requirements. I would recommend you to know the limitation. I will discuss this in more detail in the upcoming article. In today’s world, the techniques matter a lot and are measured in terms of performance. The In-Memory OLTP is a highly memory-centric feature introduced in SQL 2014. It’s a high performance, memory-optimized engine integrated into the SQL Server engine and designed for modern hardware.

在本文中，我们介绍了内存优化表及其功能的各种概念。 我已经包括了重要的部分，以了解内存注意事项和要求的含义。 我建议您了解限制。 我将在下一篇文章中对此进行更详细的讨论。 在当今世界，这些技术至关重要，并以性能来衡量。 内存中OLTP是SQL 2014中引入的高度以内存为中心的功能。它是一种高性能，内存优化引擎，已集成到SQL Server引擎中，并为现代硬件设计。

The traditional OLTP systems with an overhead of huge transactions lead to the evaluation of managing the transactions in a volatile space and it gives a great performance boost to the application execution. The right usage of this feature could see a great improvement in the performance.

具有大量事务开销的传统OLTP系统导致评估了在易变空间中管理事务的能力，并且极大地提高了应用程序的性能。 正确使用此功能可能会改善性能。

Let me know what you think…

让我知道你的想法…

翻译自: https://www.sqlshack.com/a-quick-overview-of-in-memory-oltp-in-sql-server/

sql oltp