Data Compression

Looping through each table, to see what type of data compression to use.

drop table if exists #tmpObjectList,#tmpFinalResults;
go

declare @i_i int,
@i_maxID int,
@nvc_SQL nvarchar(2000) ;

declare @tmpResults table
(
idResults int identity(1,1) not null primary key clustered,
objName varchar(128) not null,
schName varchar(128) not null,
index_id int not null,
partition_number int not null,
current_compression_size bigint null, /kb/
requested_compression_size bigint null, /kb/
current_compression_size_sample bigint null, /kb/
requested_compression_size_sample bigint null/kb/
);

create table #tmpFinalResults
(
idFinalResults int identity(1,1) not null primary key clustered,
objName varchar(128) not null,
schName varchar(128) not null,
index_id int not null,
partition_number int not null,
current_compression_size bigint null, /kb/
requested_compression_size bigint null, /kb/
CompressionType varchar(50) not null
);

create table #tmpObjectList
(
idObject int identity(1,1) not null primary key clustered,
objName varchar(128) not null,
schName varchar(128) not null
);

INSERT INTO #tmpObjectList(objName,schName)
SELECT o.name, OBJECT_SCHEMA_NAME(o.object_id) as schName
FROM sys.objects as o
INNER JOIN sys.indexes AS i ON i.object_id = o.object_id
INNER JOIN sys.dm_db_partition_stats as ps ON ps.object_id = i.object_id and ps.index_id = i.index_id
INNER JOIN sys.partitions as p ON ps.object_id = p.object_id and ps.partition_id = p.partition_id and ps.index_id = p.index_id
WHERE
o.is_ms_shipped =0
and o.type =’U’
GROUP BY
o.name,OBJECT_SCHEMA_NAME(o.object_id)
HAVING
CONVERT(decimal(18,2),SUM(reserved_page_count) * 8/1024.0) > 10.0 /* sa fie mai mare ca 10 MB/ /
SELECT o.name, OBJECT_SCHEMA_NAME(o.object_id) as schName
FROM sys.objects as o
WHERE
o.is_ms_shipped =0
and o.type =’U’
*/

/preluam limitele min , max/
SELECT @i_maxID = MAX(idObject) ,
@i_i = MIN(idObject)
FROM #tmpObjectList;

WHILE (@i_i<= @i_maxID)
/WHILE (@i_i<= 2)/
BEGIN
/* row compression*/
SELECT @nvc_SQL =N’EXEC sp_estimate_data_compression_savings ‘ + tol.schName + N’, ‘ + tol.objName + N’, NULL, NULL, ”ROW”’
FROM #tmpObjectList as tol
WHERE idObject = @i_i;

/*print @nvc_SQL;*/

DELETE FROM @tmpResults;
INSERT INTO @tmpResults(objName,schName,index_id,partition_number,
                current_compression_size ,requested_compression_size,
                current_compression_size_sample ,requested_compression_size_sample)
EXEC(@nvc_SQL);

INSERT INTO #tmpFinalResults(objName,schName,index_id,partition_number,current_compression_size ,requested_compression_size,CompressionType)
SELECT t.objName,t.schName,t.index_id,t.partition_number,t.current_compression_size,t.requested_compression_size,'ROW'
FROM @tmpResults as t

/*page compression*/

SELECT @nvc_SQL =N'EXEC sp_estimate_data_compression_savings ' + tol.schName + N', ' + tol.objName + N', NULL, NULL, ''PAGE'''
FROM #tmpObjectList as tol
WHERE idObject = @i_i;

/*print @nvc_SQL;*/

DELETE FROM @tmpResults;
INSERT INTO @tmpResults(objName,schName,index_id,partition_number,
                current_compression_size ,requested_compression_size,
                current_compression_size_sample ,requested_compression_size_sample)
EXEC(@nvc_SQL);

INSERT INTO #tmpFinalResults(objName,schName,index_id,partition_number,current_compression_size ,requested_compression_size,CompressionType)
SELECT t.objName,t.schName,t.index_id,t.partition_number,t.current_compression_size,t.requested_compression_size,'PAGE'
FROM @tmpResults as t


SET @i_i = @i_i +1;
SET @nvc_SQL =N'';

END;

/*
SELECT * FROM #tmpFinalResults;
*/

SELECT ISNULL(r.objName,p.objName) as tblName,ISNULL(r.schName,p.schName) as tblSchName,ISNULL(r.index_id,p.index_id)as index_ID,
r.current_compression_size , p.current_compression_size,
r.requested_compression_size as row_compression_size,p.requested_compression_size as page_compression_size,
CAST(r.requested_compression_size*1.0 / (r.current_compression_size ) *100.0 as tinyint) as ROW_Procent,
CAST(p.requested_compression_size*1.0 / (p.current_compression_size ) *100.0 as tinyint) as PAGE_Procent
FROM
(
SELECT r.objName,r.schName,r.index_id,r.current_compression_size,r.requested_compression_size
FROM #tmpFinalResults as r
WHERE r.CompressionType =’ROW’
) as r
FULL JOIN
(
SELECT p.objName,p.schName,p.index_id,p.current_compression_size,p.requested_compression_size
FROM #tmpFinalResults as p
WHERE p.CompressionType =’PAGE’
) as p
ON r.objName = p.objName AND r.schName = p.schName AND r.index_id = p.index_id

Conversation

Today I had a conversation with a person from a partner company related to a piece of code.
About 1 year ago, each company had to prepare/write some piece of code to take some information from a server and send/move to a different server. So, around this period, a year ago, we gather a 3 person team and developed our part of code/server.

Returning to present, the partner company, did not do anything about their code, so they use our code.
Today, I had a discussion about why a particular field is not populated.
Looking into it, we discovered and on their site there were info that were not entered/completed.

And before de conversation ended, was a comment: “it was easier if I received instructions
I don’t know if it’s about the code we wrote… but it left me with a bitter taste…

S

Home Office

I started to like it.

And not just because the commute time you save , but also because you have the history of conversations . For example, one year ago, we developed an object (a view) and now some one is asking a question about it… I have comments in it, but for this particular question , no , nada, null.

So, I returned to mails and conversations and found that a particular field was put on NULL because was splitting the group by and because some information were not provided and needed clarifies from another department.

If we were in office, probably, this conversation would have been taken in live speech and no history remaining only notes if available.

S

Select or insert

Imagine you have a dimension table and at some point you receive a value and don’t know if exists and in this case to return the @id of it or don’t exists and need to insert it .

Usually, this table should performed more selects then inserts.
So, the first statement will be to get the @id of it.

If the @id is null , meaning , this value don’t exists, then we need to insert it. So, the next step, is to insert it, and get the @id. This is the ideal case.

But what, another session/transaction/client is doing the same , with same value. And is a little faster, meaning that when you are still in “between” step1 and step2, he is already inserted. Then , a error is risen . See catch block for handle it.

In catch block, for particulars error numbers, the record exists now, and we are getting the @id back to client.

There are a few others way to deal with it, like table hint

CREATE PROCEDURE [dbo].[usp_dimMode_GetAndInsert]
(
@mddMode varchar(100)
)
AS
BEGIN
SET NOCOUNT ON;
SET XACT_ABORT ON;
DECLARE @mddID as tinyint;


BEGIN TRY
    SELECT @mddID = mddID FROM dbo.dimMode WHERE mddMode = @mddMode;

    /* @mddID is null,  insert*/
    IF @mddID IS NULL
    BEGIN
        INSERT INTO dbo.dimMode(mddMode)
        VALUES(@mddMode);

        SELECT @mddID = SCOPE_IDENTITY();
    END

    RETURN @mddID;

END TRY
BEGIN CATCH

  -- ignore duplicate key errors, throw the rest.
  IF ERROR_NUMBER() IN (2601, 2627)
  BEGIN
        SELECT @mddID = mddID FROM dbo.dimMode WHERE mddMode = @mddMode;

        RETURN @mddID;
  END

END CATCH
END

S

NULL DFLT ON/OFF

There are 2 interesting settings :

ANSI_NULL_DFLT_ON

ANSI_NULL_DFLT_OFF

These only affect the nullability of a new column , when nullability is not specified.

When SET ANSI_NULL_DFLT_OFF is ON – new columns are NOT NULL
When SET ANSI_NULL_DFLT_ON is ON – new columns allow null values if the nullability status of the column is not explicitly specified

Both SET ANSI_NULL_DFLT_OFF and SET ANSI_NULL_DFLT_ON cannot be set ON at the same time.
either ANSI_NULL_DFLT_OFF or SET ANSI_NULL_DFLT_ON can be set ON
or both can be set OFF

From BOL :

https://docs.microsoft.com/en-us/sql/t-sql/statements/set-ansi-null-dflt-off-transact-sql

https://docs.microsoft.com/en-us/sql/t-sql/statements/set-ansi-null-dflt-on-transact-sql

S

IO block Size

I have read an interested blog post about the IO block Size.

Here it is .

Also, important information can be found here (please keep in mind it’s a sql server 2008 version).

The conclusions are:

  • transaction log flushes – 512 Bytes UP TO 60KB
  • data file IO
    • checkpoint – 8KB to 1MB
    • lazy writer -8KB to 1MB
    • reading – 8*8KB to 1MB
    • read ahead – 128KB to 512KB
    • bulk loads – 256KB
  • Backup/restore – 1MB

S

Random password

How to generate a six character password.

You have to choose from a list of allowed characters (

(SELECT ‘abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ0123456789’ AS srcText , 62 as srcLen) AS S

)

select
empID as empID
,p.psw as psw1
,ca2.psw1+ ” +ca2.psw2+ ” +ca2.psw3+ ” +ca2.psw4+ ” +ca2.psw5+ ” +ca2.psw6 as psw2
–,len(ca2.psw1+ ” +ca2.psw2+ ” +ca2.psw3+ ” +ca2.psw4+ ” +ca2.psw5+ ” +ca2.psw6) as psw2Len
–,ca1.pozChar1,ca1.pozChar2,ca1.pozChar3,ca1.pozChar4,ca1.pozChar5,ca1.pozChar6
from dbo.Employees
CROSS APPLY
(SELECT ‘abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ0123456789’ AS srcText , 62 as srcLen) AS S
CROSS APPLY
(SELECT CAST(ROUND( (s.srcLen-1 – 0 ) *RAND(CHECKSUM(NEWID())) + 1,0) as int) as pozChar1,
CAST(ROUND( (s.srcLen-1 – 0 ) *RAND(CHECKSUM(NEWID())) + 1,0) as int) as pozChar2,
CAST(ROUND( (s.srcLen-1 – 0 ) *RAND(CHECKSUM(NEWID())) + 1,0) as int) as pozChar3,
CAST(ROUND( (s.srcLen-1 – 0 ) *RAND(CHECKSUM(NEWID())) + 1,0) as int) as pozChar4,
CAST(ROUND( (s.srcLen-1 – 0 ) *RAND(CHECKSUM(NEWID())) + 1,0) as int) as pozChar5,
CAST(ROUND( (s.srcLen-1 – 0 ) *RAND(CHECKSUM(NEWID())) + 1,0) as int) as pozChar6) as ca1
CROSS APPLY
(SELECT SUBSTRING(s.srcText,ca1.pozChar1,1) as psw1,
SUBSTRING(s.srcText,ca1.pozChar2,1) as psw2,
SUBSTRING(s.srcText,ca1.pozChar3,1) as psw3,
SUBSTRING(s.srcText,ca1.pozChar4,1) as psw4,
SUBSTRING(s.srcText,ca1.pozChar5,1) as psw5,
SUBSTRING(s.srcText,ca1.pozChar6,1) as psw6) as ca2

CROSS APPLY
(
    SELECT CHAR(ROUND(   (122-49 - 1) *RAND(CHECKSUM(NEWID())) + 49,0)) +   '' + 
            CHAR(ROUND(  (122-49 - 1) *RAND(CHECKSUM(NEWID())) + 49,0)) +   '' +
            CHAR(ROUND(  (122-49 - 1) *RAND(CHECKSUM(NEWID())) + 49,0)) +   '' +
            CHAR(ROUND(  (122-49 - 1) *RAND(CHECKSUM(NEWID())) + 49,0)) +   '' +
            CHAR(ROUND(  (122-49 - 1) *RAND(CHECKSUM(NEWID())) + 49,0)) +   '' +
            CHAR(ROUND(  (122-49 - 1) *RAND(CHECKSUM(NEWID())) + 49,0)) +   '' +
            CHAR(ROUND(  ( 57-49 - 1) *RAND(CHECKSUM(NEWID())) + 49,0)) as psw
)p

Some very interesting script is here : https://stackoverflow.com/a/15038572/3992746

S

SET NOCOUNT ON

SET NOCOUNT ON and ASYNC_NETWORK_IO

SET NOCOUNT ON

From BOL , SET NOCOUNT ON the count is not returned. When is OFF, the count is returned.

Additional, prevents the sending of DONE_IN_PROC messages to the client . This is used within stored procedures and triggers to avoid showing the affected rows message. (XXX rows affected)

So , after every statement , in a stored procedure or trigger, sql server sends a message DONE_IN_PROC (rows affected) .With SET NOCOUNT ON, you can suppress these messages.

If you are not in a stored procedure , if you are in a batch , SQL Server sends DONE message/token for each statement.  This can lead (in case of client not consume it fast) to waits s ASYNC_NETWORK_IO.  To reduce it , you can use a temp stored procedure or sp_executeSQL .

When a statement is executed, the result ends with a DONE, DONEPROC, or DONEINPROC, depending of the where this statement is: stored procedure, batch…

  • DONE_IN_PROC, DONE_PROC are sent for statements executed in stored procedures , triggers
  • DONE are sent for statements executed in batch

Conclusion:

SET NOCOUNT ON , suppress only DONE_IN_PROC and not DONE messages.

S

Simple parameterization

I was checking my cache plan , and founded that there was a database with more then 44k+ plans , that are only one time used.

Digging up, I discovered a query and asked my self , way it wasn’t parameterized.

Here is a sample:

select *
from sales.SalesOrderHeader as soh
where soh.SalesOrderID = 43675

 

select TOP(1) *
from sales.SalesOrderHeader as soh
where soh.SalesOrderID = 43675

 

Here is the execution plan(part of it)

 

and xml representation:

 

SimpleParam_notParamXML

SimpleParam_ParamXML.jpg
Simple parameterization can happens , in the following situation :
– must be a trivial plan . if it’s not trivial, no simple parameterization
in xml plan is : StatementOptmLevel=”TRIVIAL”

– the query must not have JOIN, IN, TOP,GROUP BY, HAVING, Distinct , Subqueries

 

Conclusion:

QO decide to parameterize a query when it consider it safe to param , in a trivial plan , and not have : joins, In, Top, subqueries….

 

S

 

 

Check DB

How to find the last good known checkDB date?

Starting with SQL Server 2016 SP2, we have DATABASEPROPERTYEX (‘dbname’ , ‘LastGoodCheckDbTime’ )

In SQL Server 2017, CU14 ( I tested it) , this is also working.

Like so:

SELECT DATABASEPROPERTYEX (‘db_workspace’ , ‘LastGoodCheckDbTime’ )

or

SELECT DATABASEPROPERTYEX (d.name , ‘LastGoodCheckDbTime’ )

FROM sys.databases as d

 

Value of  `1900-01-01 00:00:00.000` means – no check db was  run.
Please, see here more info
S