如何收集HANA 内存itab trace

For a temporary workaround, recommend to perform the following statements

ALTER SYSTEM CLEAR SQL PLAN CACHE;

ALTER SYSTEM CLEAR COLUMN RESULT CACHE;

Here are the steps to collect the itab traces: (*Please be aware the collection of traces will reduce the DB performance for while the traces are on)

(Need to capture the time when the itab allocator increases)

1. With the database in a low memory pressure state (for example, after restarting or clearing the sql plan cache), take note of the current time and enable the itab trace leak:

SELECT CURRENT_TIMESTAMP as ITAB_START_TIME from dummy;

ALTER SYSTEM ALTER CONFIGURATION ('indexserver.ini', 'SYSTEM') SET
('itab', 'trace_leaks') = 'yes',
('itab', 'trace_leakcallstack') = 'yes',
('itab', 'trace_addrefcallstack') = 'no',
('itab', 'leaktrace_max_callstacks') = '20000'
with reconfigure;

Important: Enabling this trace may have a performance impact, proceed with caution, avoid running this in production unless instructed by HANA Development Support.

2. [Optional] Clear SQL Plan Cache immediately after activating itab leak tracing:

ALTER SYSTEM CLEAR SQL PLAN CACHE;

This step is only necessary if the memory pressure is already high as clearing the sql plan cache can help reducing itab memory consumption.

3. [Optional] Preserve the current state of M_TEMPORARY_TABLES before the itab memory consumption increases:

CREATE COLUMN TABLE TEMPORARY_TABLES_BEFORE_MEM_GROWTH AS (SELECT * FROM M_TEMPORARY_TABLES);

4. Wait for memory consumption to noticeably increase and retain (without decreasing). The memory pressure should be representative, and depending on the scenario for which the trace is being activated, it may need to be active for a couple of days or more. Use the query below or other variations, such as query HANA_Memory_TopConsumers_History from SAP Note 1969700, to analyze the patterns of memory consumption to decide when to stop tracing:

SELECT * FROM
(SELECT TO_VARCHAR(CURRENT_TIMESTAMP) AS THETIME, HOST,PORT, CATEGORY,
ROUND((EXCLUSIVE_SIZE_IN_USE) / 1024 / 1024, 2) AS USED_MEMORY_SIZE_IN_MB
FROM M_HEAP_MEMORY
WHERE CATEGORY IN ('<allocator>')
ORDER BY HOST,PORT,USED_MEMORY_SIZE_IN_MB DESC)
UNION ALL
(SELECT THETIME, HOST, PORT, CATEGORY, MAX(USED_MEMORY_SIZE_IN_MB) AS USED_MEMORY_SIZE_IN_MB FROM
(SELECT TO_VARCHAR(SERVER_TIMESTAMP) AS THETIME, HOST, PORT, CATEGORY, ROUND((EXCLUSIVE_SIZE_IN_USE) / 1024 / 1024, 2) AS USED_MEMORY_SIZE_IN_MB FROM "_SYS_STATISTICS"."HOST_HEAP_ALLOCATORS"
WHERE CATEGORY IN ('<allocator>'))
GROUP BY HOST, PORT, CATEGORY, THETIME)
ORDER BY THETIME DESC;

5. Record temp tables and itab leak traces in separate persistent tables:

CREATE COLUMN TABLE TEMPORARY_TABLES_AFTER_MEM_GROWTH AS (SELECT * FROM M_TEMPORARY_TABLES);
CREATE COLUMN TABLE ITAB_LEAKS_AFTER_MEM_GROWTH AS (SELECT * FROM SYS.M_DEV_ITAB_LEAK ORDER BY CREATION_TIME ASC);

6. Attempt to reduce memory utilization. This is normally done by clearing the SQL Plan Cache, as many of the itabs are related to entries in the SQL Plan cache or the cache itself. In order to do that, use the commands below:

ALTER SYSTEM CLEAR SQL PLAN CACHE;
ALTER SYSTEM CLEAR COLUMN RESULT CACHE;

7. Record temp tables and itab leak traces in separate persistent tables again, to keep track of the involved objects created since the start of the trace:

CREATE COLUMN TABLE TEMPORARY_TABLES_AFTER_MEM_GROWTH_CACHE_CLEAR AS (SELECT * FROM M_TEMPORARY_TABLES);
CREATE COLUMN TABLE ITAB_LEAKS_AFTER_MEM_GROWTH_CACHE_CLEAR AS (SELECT * FROM SYS.M_DEV_ITAB_LEAK ORDER BY CREATION_TIME ASC);

8. Disable traces:

ALTER SYSTEM ALTER CONFIGURATION ('indexserver.ini', 'SYSTEM') UNSET
('itab', 'trace_leaks'),
('itab', 'trace_leakcallstack'),
('itab', 'trace_addrefcallstack'),
('itab', 'leaktrace_max_callstacks')
with reconfigure;

9. Wait for the Statistics server to collect fresh information from M_HEAP_MEMORY, by default this is done every 15 minutes. Create a copy of _SYS_STATISTICS.HOST_HEAP_ALLOCATORS_BASE based on the trace timeframe:

CREATE COLUMN TABLE HOST_HEAP_ALLOCATORS_BASE_COPY AS 
(SELECT * FROM _SYS_STATISTICS.HOST_HEAP_ALLOCATORS_BASE 
WHERE server_timestamp between 
<time where the itab_leak traces started from step 1 - ITAB_START_TIME> 
and current_timestamp );

10. Export the tables created in previous steps:

EXPORT
TEMPORARY_TABLES_BEFORE_MEM_GROWTH,
TEMPORARY_TABLES_AFTER_MEM_GROWTH,
ITAB_LEAKS_AFTER_MEM_GROWTH,
TEMPORARY_TABLES_AFTER_MEM_GROWTH_CACHE_CLEAR,
ITAB_LEAKS_AFTER_MEM_GROWTH_CACHE_CLEAR,
HOST_HEAP_ALLOCATORS_BASE_COPY AS BINARY INTO '/target/linux/folder' WITH THREADS 6;

Remarks: As it is costly to keep all call stacks for all itab leaks, parameter leaktrace_max_callstacks (set to 20000 on this document) should have a sufficiently large number in order to capture the callstack associated with the suspicious statements.