If you seeing this message “A significant part of sql server process memory has been paged out. This may result in a performance degradation. Duration: 1851 seconds. Working set (KB): #####, committed (KB): #####, memory utilization: ##%. ”
What does it mean: System is running low on memory basically windows is moving the allocated objects from memory into a paged file to deal with memory pressure.
Is SQL Server taking way too much memory? : May be not this is known behavior of windows to trim down the memory used by the current process so it can allot memory to new requests.
SQL Server has mechanism in place to use low resource memory notification it uses a function called CreateMemoryResourceNotification to create a system wide memory notification if windows detects low memory it sends out a notification with a signal 0 (available physical memory is running low ) or 1 (available physical memory is high) based on the result SQL Server reacts accordingly by performing the necessary action increasing the buffer pool or decreasing it. Note: if neither of these are signaled SQL Server maintains a constant memory foot print.
If SQL Sever has a mechanism to deal with low memory why is windows moving the allocated in memory objects to page file?
Can be many reasons for it
- Windows might not be sending the notifications in appropriate instances or SQL Sever might not be receiving those events.
- SQL Server might not be responding in timely manner may be too slow
When SQL Server receives low physical memory notification it will scale down its memory usage by trimming internal caches. This effect is applied for 5 seconds and then paused for 1 minute. This is to avoid any hungry/faulty application to consume all the memory making SQL Server to scale its usage continuously. If low memory conditions still exist after 1 minute pause, the effect is applied again. So if there is physical memory pressure even after SQL Server scaling its usage for 5 seconds windows will still page out SQL Server’s working set.
We can query these memory notifications in ring buffer
CONVERT (varchar(30), GETDATE(), 121) as runtime,
DATEADD (ms, a.[Record Time] – sys.ms_ticks, GETDATE()) AS Notification_time,
sys.ms_ticks AS [Current Time]
(SELECT x.value(‘(//Record/ResourceMonitor/Notification)’, ‘varchar(30)’) AS [Notification_type],
x.value(‘(//Record/MemoryRecord/MemoryUtilization)’, ‘int’) AS [MemoryUtilization %],
x.value(‘(//Record/MemoryRecord/TotalPhysicalMemory)’, ‘bigint’) AS [TotalPhysicalMemory_KB],
x.value(‘(//Record/MemoryRecord/AvailablePhysicalMemory)’, ‘bigint’) AS [AvailablePhysicalMemory_KB],
x.value(‘(//Record/MemoryRecord/TotalPageFile)’, ‘bigint’) AS [TotalPageFile_KB],
x.value(‘(//Record/MemoryRecord/AvailablePageFile)’, ‘bigint’) AS [AvailablePageFile_KB],
x.value(‘(//Record/MemoryRecord/TotalVirtualAddressSpace)’, ‘bigint’) AS [TotalVirtualAddressSpace_KB],
x.value(‘(//Record/MemoryRecord/AvailableVirtualAddressSpace)’, ‘bigint’) AS [AvailableVirtualAddressSpace_KB],
x.value(‘(//Record/MemoryNode/@id)’, ‘int’) AS [Node Id],
x.value(‘(//Record/MemoryNode/ReservedMemory)’, ‘bigint’) AS [SQL_ReservedMemory_KB],
x.value(‘(//Record/MemoryNode/CommittedMemory)’, ‘bigint’) AS [SQL_CommittedMemory_KB],
x.value(‘(//Record/@id)’, ‘bigint’) AS [Record Id],
x.value(‘(//Record/@type)’, ‘varchar(30)’) AS [Type],
x.value(‘(//Record/ResourceMonitor/Indicators)’, ‘int’) AS [Indicators],
x.value(‘(//Record/@time)’, ‘bigint’) AS [Record Time]
FROM (SELECT CAST (record as xml) FROM sys.dm_os_ring_buffers
WHERE ring_buffer_type = ‘RING_BUFFER_RESOURCE_MONITOR’) AS R(x)) a
CROSS JOIN sys.dm_os_sys_info sys
ORDER BY DATEADD (ms, a.[Record Time] – sys.ms_ticks, GETDATE())
We can also increase the LowMemoryThreshold value so the OS will notify applications such as SQL on low memory conditions much earlier and SQL Server can respond to memory pressure much early before the system is starving for memory.
How to set the LowMemoryThreshold value (in MB)?
In Regedit -> go to
Right click on the right pane,
Select New -> select click DWORD Value -> enter LowMemoryThreshold
Double Click LowMemoryThreshold -> value (choose decimal) -> 512
System Reboot is required to take effect.
While I was trying to poke around in hadoop environment I noticed that namenode wouldn’t start up. After doing some research I found that its the IPv6 issue. So I summarized steps I took to resolve the issue
We are starting the hadoop stack
we can see the name node seems like it did not start as we can see in jps list
first step is make sure you can ssh local machine (127.0.0.1)
I tried tee see port activity looks like entire hadoop stack is using Ip6. we can change this behavior by changing hadoop-env.sh file in conf directory
stop all hadoop services and start hadoop stak
we spend lot of time trying to look at log files when we run any particular SQl files have a standard frame work to collect these errors and store in table, amkes the application management simple at same time provides some insights into exceptions over the time
Above scripts creates two objects a table and procedure
Here is how we should plan to use
SELECT * FROM wm_errorlog
by default a remote query (Query using Linked Server) expires in 10 mins. We can find this value in SQL Server using below queries
We can find more details in MS doc
in order to change it we have to sp_configure and we don’t need to restart the database engine
This will set the time out to 3600 seconds/ 1 hr
it would be tiresome to enter environment variables each time you login to sqlplus, good thing is sqlplus allows us to use a script (login.sql) which has the environment variables defined each time we login (user profile) https://docs.oracle.com/cd/E11882_01/server.112/e16604/ch_two.htm#i1133044 . Once you define the sqlpath sqlplus looks for login.sql in the path directory and executes it once you login successful. This can be also configured at global level called site profile it allows the DBA to set default env for all the users. so when a user connects through sqlplus first the global site profile (glogin.sql) is executed and then the user profile (login.sql) is executed which is defined in SQLPATH environment variable in Windows
here %USERPROFILE% is an environment variable has default location of the user who is logged into the machine C:\USERS\Username
we have to make sure we have login.sql available in that location
you can see below sqlprompt has changed to what we defined in login.sql (‘&_user.@&_connect_identifier.>’)