Setup SQL Agent alerts for User Error Messages

Setup a simple alert with SQL agent

1

When I test the alert it does not trigger the alert

Capture 2

 

Since error is not being logged, we can verify in the sys.messages

Capture 3

Lets alter the message to log

Capture 4

 

SELECT * FROM sys.messages WHERE
language_id = 1033 AND severity = 16
and message_id = 2812
EXEC sp_altermessage @message_id = 2812, @parameter = ‘WITH_LOG’ ,@parameter_value = ‘True’

Test the alert again, now we can see number of occurrences increases to 1

Capture 5

Advertisement

Log SQL Errors using Extended Events

First create the extended event session to monitor for any errors with severity grater than equal to 11

CREATE EVENT SESSION [ErrorCapture] ON SERVER ADD EVENT sqlserver.error_reported (ACTION(sqlserver.client_app_name, sqlserver.client_hostname, sqlserver.database_name, sqlserver.sql_text, sqlserver.username) WHERE ([severity] >= (11))) ADD TARGET package0.event_file (
	SET filename = N'ErrorCapture'
	,max_file_size = (100)
	,max_rollover_files = (5)
	)
	WITH (
MAX_MEMORY = 4096 KB
,EVENT_RETENTION_MODE = ALLOW_SINGLE_EVENT_LOSS
,MAX_DISPATCH_LATENCY = 30 SECONDS
,MAX_EVENT_SIZE = 0 KB
,MEMORY_PARTITION_MODE = NONE
,TRACK_CAUSALITY = OFF
,STARTUP_STATE = ON
)
GO

 

How to view them in TSQL

DECLARE @error_log VARCHAR(max) = convert(VARCHAR(max), SERVERPROPERTY('ErrorLogFileName'))
DECLARE @error_log_dir VARCHAR(max) = reverse(substring(reverse(@error_log), charindex('\', reverse(@error_log)), len(@error_log)))

SELECT *
FROM sys.fn_xe_file_target_read_file(@error_log_dir + 'errorcap*.xel', NULL, NULL, NULL)

 

Tips and Tricks to avoid common pit falls with SQL CLR deployment

  1. If you are using Visual Studio to build and deploy the SQL CLR solution ensure you have below settings

SQLCLR

SQL CLR2

SQLCLR3.jpg

2) First things first, Ensure CLR Integration is enabled at instance level.

EXEC sp_configure 'show advanced options' , '1';
reconfigure;

EXEC sp_configure 'clr enabled' , '1' ;
reconfigure;

EXEC sp_configure 'show advanced options' , '0';
reconfigure;

 

3) Ensure Appropriate CLR Integration Code Access Security levels are set when creating assembly’s in SQL Server

CREATE ASSEMBLY [SOAPSQLSPCLR]
AUTHORIZATION [dbo]
FROM 'C:\Program Files\Microsoft SQL Server\MSSQL10_50.WMVDSQL01\MSSQL\Binn\SQLCLRSoapConsumer.dll'
WITH PERMISSION_SET = [EXTERNAL_ACCESS,SAFE,UNSAFE]
GO

 

  • SAFE : Only internal computation and local data access are allowed. SAFE is the most restrictive permission set. Code executed by an assembly with SAFE permissions cannot access external system resources such as files, the network, environment variables, or the registry
  • EXTERNAL_ACCESS: These assemblies have the same permissions as SAFE assemblies, with the additional ability to access external system resources such as files, networks, environmental variables, and the registry
  • UNSAFE:  unrestricted access to resources, both within and outside SQL Server. Code executing from within an UNSAFE assembly can also call unmanaged code.

Please refer to msdn documentation here https://msdn.microsoft.com/en-us/library/ms345101.aspx

4) Ensure the database is setup correctly

If you are using assemblies with External access and Unsafe the database property TRUSTWORTHY needs to be turned on.

USE database_name
GO
EXEC sp_changedbowner 'sa'
ALTER DATABASE database_name SET TRUSTWORTHY ON 

 

Refer to MSDN documentation on TRUSTWORTHY https://msdn.microsoft.com/en-us/library/ms187861.aspx

Clone your SQL Database instantly with new DBCC command CLONEDATABASE

 

Based on MS documentation this is a quick way to create a copy of database (only Schema) including statistics and Indexes of source database this was released in SQL 14 service pack 2.

When this command is issued SQL Server creates an internal snapshot of source database just like how it creates for checkdb and drops this snapshot when the cloning process is done but during the cloning process it holds a shared lock on source database and X lock on target database and it leaves target database in read only mode although you can change the state of the target database if you intent to add data or modify.

So why do you need to clone database.

According to MS DBCC CLONEDATABASE should be used to create a schema and statistics only copy of a production database in order to investigate query performance issues.”. The original intention of the feature is to diagnose any performance issues of a production database with out needing to effect the production database. Although this is so late in the game its never to late for new feature.

Don’t confuse with Database Snapshots that’s totally different concept.
dbcc clonedatabase([AdventureWorks2014],[AdventureWorks2014_Clone])

 

dbcc clone 1

 

dbcc clone 2

https://support.microsoft.com/en-us/kb/3177838

Bottom line: Saves a lot of time when you are debugging execution plans and performance related issues.

High CPU Usage SQL Server (One Bad Query)

My colleague reported to me that one of our database server is reporting consistent high CPU usage so I looked at it I noticed CPU was at 100% from last one week when I contacted the application owner and I foundthat they implemented a new feature that polls the database for every second to ensure the data collection process is running properly as it was necessary to ensure that we are under compliance in terms of reporting and auditing. So I ran a query to pull the queries with high cpu utilization with execution count. I certainly noticed a query running more often with high cpu usage.

 

Exec stats

I know that above highlighted query is causing the high cpu usage, next I looked at query stats and noticed this query is running twice every second, so I looked at the plan

Select top 1 col1 from table order by 1

Table is clustered and col1 is not part of clustered index and does not have an index. simple enough SQL server decides to do Clustered index scan and sorts(fully blocking) col1 and selects 1 row with no predicate SQL server doesn’t think its missing an index.

PLan 1

PLan 1 properties .jpg
SQL Server Execution Times: with out non clustered index
CPU time = 2296 ms, elapsed time = 658 ms.

So I created a non clustered index on col1 in desc on the table

plan 2.jpg
SQL Server Execution Times: with non clustered index
CPU time = 0 ms, elapsed time = 0 ms.

Cpu Usage dramatically reduced

Cpu usage.jpg

Bottom line:

Its very important to understand no matter how much physical resources you might have on a  server its very important understand that one bad query can literally bring the server down to it knees.

 

Comparing sys.dm_db_index_physical_stats mode levels with Heap and Clustered Index

Create a table with some data

drop table t3
CREATE TABLE t3 (col1 int PRIMARY KEY, col2 varchar(500)) WITH(DATA_COMPRESSION = PAGE);
GO
--BEGIN TRAN
DECLARE @idx int = (select max(isnuLL(col1,0))+1 from t3);
WHILE @idx < 1000000
BEGIN
BEGIN TRAN
    INSERT INTO t3 (col1, col2) 
    VALUES (@idx+convert(int,rand() * 100), 
    REPLICATE ('a', 100) + CAST (@idx as varchar(10)) + REPLICATE ('a', 380))
    SET @idx = @idx + 1
	COMMIT
END
--COMMIT;
GO

 

Check Fragmentation using sys.dm_db_index_physical_stats

 

SELECT 'Limited' as Mode,page_count, compressed_page_count as cm_page_count,index_type_desc
,index_depth,index_level,avg_fragmentation_in_percent,record_count
,fragment_count,object_name(object_id) as Object_name
FROM sys.dm_db_index_physical_stats (db_id(),
object_id ('t3'), null, null, 'LIMITED');

SELECT 'Sampled' as Mode,page_count, compressed_page_count as cm_page_count
,index_type_desc,index_depth,index_level,avg_fragmentation_in_percent,record_count
,fragment_count,object_name(object_id) as Object_name
FROM sys.dm_db_index_physical_stats (db_id(),
object_id ('t3'), null, null, 'SAMPLED');

SELECT 'Detailed' as Mode,page_count, compressed_page_count as cm_page_count
,index_type_desc,index_depth,index_level,avg_fragmentation_in_percent,record_count
,fragment_count,object_name(object_id) as Object_name
FROM sys.dm_db_index_physical_stats (db_id(),
object_id ('t3'), null, null, 'DETAILED');

Fragmentation output

Mode: Limited

  • Only looks at leaf level pages
  • Checks the logical fragmentation at leaf level (Out of Order pages)/ out-of-order extents in the leaf pages of a heap
  • Doesn’t look into the actual pages.

Mode: Sampled

  • Only looks at leaf level pages
  • Checks the logical fragmentation at leaf level (Out of Order pages) out-of-order extents in the leaf pages of a heap
  • Scans a sample of leaf pages (10 % approximately)  it may randomly these pages.

Mode: Detailed

  • Does what limited mode normally do but scans every page in the index including the intermediate and root pages.
  • Check the logical fragmentation at every level of Index (Out of Order pages)/ out-of-order extents in the leaf pages of a heap
  • Scans the entire index.

Detailed mode is more accurate than any other mode because it scans every page of the index but this  might create some performance bottle neck. It depends on where each of this mode can be used if an index with high fanout (index key length) which is directly proportional to the number of levels , it makes sense to check the fragmentation at intermediate levels using detailed mode at least once a week as part of Index Maint plan.

 

Terms : 

  •  Out of Order pages : An out-of-order page is a page for which the next physical page allocated to the index is not the page pointed to by the next-page pointer in the current leaf page.
  • Out-of-order extent: An out-of-order extent is one for which the extent that contains the current page for a heap is not physically the next extent after the extent that contains the previous page.

 

B-Tree Index Structure:

Heap:

Heap

Non Clustered or Clustered Index

Non Clustered or Clustered Index

 

 

FIX[solved]:Unable to communicate with the runtime for ‘R’ script. Please check the requirements of ‘R’ runtime, Fatal error: cannot create ‘R_TempDir’

I was trying out new R feature in SQL 2016 and encountered this error so I thought I would document this issue.

Resolution steps:

  1. Locate rlauncher config file: (this is used by the MSSQL Launch pad service)
    1. Its normally located in the Binn folder which is available in instance root directory in my case D:\Program Files\Microsoft SQL Server\MSSQL13.MSSQLSERVER\MSSQL\Binn
  2. View the contents of rlauncher and check the working directory which is where we are having issues creating temp dir.
    1. R does not like spaces in the working directory

R launcher 1

 

 

 

 

 

3. Create a folder in C:\ called temp “C:\temp”

4. Change the working directory to c:\temp

 

R launcher 2

 

 

 

 

 
5. Grant permissions for the service MSSQLaunchpad service account in my case its NT Service\MSSQLaunchpad

R launcher 3

 

 

 

 

 

 

 

 

 

 

 

 

Restart the SQL server services and SQL Launch pad services.

 

We can also fix this by using short paths by replacing the working directory with a short path

Eample below cmd will give us the short path for the location

cmd /c for %A in (“D:\Program Files\Microsoft SQL Server\MSSQL13.MSSQLSERVER\MSSQL\ExtensibilityData”) do @echo %~sA

 

SQL 2016 DB engine features Compress and Decompress functions

SQL server added new function Compress and Decompress scalar valued function that take char/varchar as input and output a GZIP compressed varbinary data. This feature allows us to use column level compression of char/varchar data and reduces the cost of IO.

sql2016 compress
 

we can see we achieved almost 99% please note these results varies when data is stored in a table (row in flow)

Lets look at this in a detail example.

sql2016 compress2

 

sql2016 compress3

 

 

IO and time test

sql2016 compress4

 

Summary of comparision matrix between compress function and regular table we achieve 99% savings on storage on certain situations and achieve even more space when used in conjunction with row and page compression.

sql2016 compress5

Standard SQL Server Error log procedure

we spend lot of time trying to look at log files when we run any particular SQl files have a standard frame work to collect these errors and store in table, amkes the application management simple at same time provides some insights into exceptions over the time

IF EXISTS (SELECT 1 FROM sys.tables WHERE name = 'wm_ErrorLog')
DROP TABLE [wm_ErrorLog]
IF EXISTS (SELECT * FROM sys.objects WHERE name = 'WMSp_ErrorLog_Ins_Error' AND type ='P')
DROP PROCEDURE [WMSp_ErrorLog_Ins_Error]
CREATE TABLE [wm_ErrorLog] (
    [Error_id] INT IDENTITY(1,1)
	,[Error_Number] [INT] NOT NULL
	,[Error_procedure] VARCHAR(max) NOT NULL
	,[Error_Message] [VARCHAR](MAX) NULL
	,[Error_Line] INT
	,[Error_State] INT 
	,[Error_Severity] INT 
	,[SPID] [INT] NULL
	,[Program_Name] [VARCHAR](255) NULL
	,[Client_Address] [VARCHAR](255) NULL
	,[Authentication] [VARCHAR](50) NULL
	,[Error_User_Application] [VARCHAR](100) NULL
	,[Error_Date] [DATETIME] NULL
	,[Error_User_System] [sysname] NOT NULL
	) ON [PRIMARY] TEXTIMAGE_ON [PRIMARY]
GO

SET ANSI_PADDING OFF
GO

ALTER TABLE wm_ErrorLog ADD CONSTRAINT [dfltErrorLog_error_date] DEFAULT(GETDATE())
FOR [Error_Date]
GO

ALTER TABLE wm_ErrorLog ADD CONSTRAINT [dfltErrorLog_error_user_system] DEFAULT(SUSER_SNAME())
FOR [Error_User_System]
GO

ALTER TABLE [wm_ErrorLog] ADD CONSTRAINT wmpk_wm_errorlog_id PRIMARY KEY (error_id)

GO


Create  PROCEDURE [WMSp_ErrorLog_Ins_Error] (
	 @Error_Number INT = NULL
	,@Error_procedure SYSNAME = 'Ad hoc sql'
	,@Error_Message VARCHAR(4000) = NULL
	,@UserID INT = NULL
	)
AS
BEGIN
	BEGIN TRY
		INSERT INTO wm_ErrorLog (
			[Error_Number]
			,[Error_procedure]
			,[Error_Message]
			,[Error_line]
			,[Error_state]
			,[Error_severity]
			,[SPID]
			,[Program_Name]
			,[Client_Address]
			,[Authentication]
			,[Error_User_System]
			,[Error_User_Application]
			)
		SELECT [Error_Number] = ISNULL(@Error_Number, ERROR_NUMBER())
			,[Error_procedure] = COALESCE(ERROR_PROCEDURE(),@Error_procedure)
			,[Error_Message] = ISNULL(@Error_Message, ERROR_MESSAGE())
			,[Error_line] = ERROR_LINE()
			,[Error_state] =ERROR_STATE()
			,[Error_severity] = ERROR_SEVERITY()
			,[SPID] = @@SPID
			,[Program_Name] = ses.program_name
			,[Client_Address] = con.client_net_address
			,[Authentication] = con.auth_scheme
			,[Error_User_System] = SUSER_SNAME()
			,[Error_User_Application] = @UserID
		FROM sys.dm_exec_sessions ses 
		LEFT JOIN sys.dm_exec_connections con ON con.session_id = ses.session_id
		WHERE ses.session_id= @@SPID
	END TRY

	BEGIN CATCH
		-- We even failed at the log entry so let's get basic  
		INSERT INTO wm_ErrorLog (
			ERROR_NUMBER
			,ERROR_procedure
			,ERROR_MESSAGE
			)
		VALUES (
			- 100
			,OBJECT_NAME(@@PROCID)
			,ERROR_MESSAGE()
			)
	END CATCH
END
GO

Above scripts creates two objects a table and procedure

Here is how we should plan to use

BEGIN TRY
RAISERROR(‘error’,16,1)
END TRY
BEGIN CATCH
EXECUTE [WMSp_ErrorLog_Ins_Error]
END CATCH

SELECT * FROM wm_errorlog

Sql error log