Stock Aging part 2

Now , that we have the raw table, we can go next.
We have to prepare the source so that we have how we desired, so that we have to show , on years the stock aging, starting with year 2017 till 2022.

For this , we create the list of years, with date start been January 1, and end date been December 31.

select
number,
DATEADD(Year,number,’20170101′) as dataStart,
DATEADD(DAY,-1,DATEADD(Year,number + 1 ,’20170101′)) as dataEnd,
DATEADD(Year,number + 1 ,’20170101′) as dataEndNextYear
into #tmpInterval
from master.dbo.spt_values as v
where v.type =’p’
and number <= 5


NumberDataStartDataEndDataEndNextYear
02017-01-012017-12-312018-01-01
12018-01-012018-12-312019-01-01
22019-01-012019-12-312020-01-01
32020-01-012020-12-312021-01-01
42021-01-012021-12-312022-01-01
52022-01-012022-12-312023-01-01

Next , we prepare the raw table for better use, and here is the point, to split the supply by it’s consumptions +

in case of remaining stock available .

The output will look like this:

The final select, that will generate the desired output, will be in the next post.

S

Stock Aging part 1

If we have some supplies and consumptions than what is the stock aging
So we have to raw table with all the tradings.

+Qty is the supply
-Qty is the consumption
IdRowMySelf is the idRow of the supply row, in case of the consumption
it is not null only in case of the consumption
For the moment, the value it’s not taken in account just for the simplicity.

idRowDATE TRANSACTIONITEMIDQTYIDROWMYSELF
|12018-02-02a125NULL
22019-02-03a1101
32020-02-03a131

CREATE TABLE dbo.rawMaterials
(
idRow int not null primary key clustered,
dateTransaction date not null,
itemID varchar(50) not null,
Qty int not null,
idRowMySelf int null
);

INSERT INTO dbo.rawMaterials(idRow,dateTransaction,itemID,Qty,idRowMySelf)
VALUES(1,’20180202′,’a1′,25,NULL),
(2,’20190203′,’a1′,10,1),
(3,’20200203′,’a1′,3,1);

SELECT *
FROM dbo.rawMaterials;

Unpivot

How to …

See here one way to unpivot, thanks to Michael J Swart, let say, a generic way to unpivot.

And in comments, it’s a way that I like, with the help of a JSON.

It’s a combination between FOR JSON cu OPENJSON.

FOR JSON will give, let say, a string/array/combination between columns and values;
and OPENJSON will do the magic, will return rows having the desired output, having the unpivot result set.


FOR JSON AUTO , WITHOUT_ARRAY_WRAPPER , INCLUDE_NULL_VALUES
+ OPENJSON

S

AKA sababa

Search

Usually, in our system, we use , for our primary key some kind of notation/coding for it, for example : xxxID.
So, given 3 liters + ‘ID’ , return the name of the table(s) , from each database existing in the instance.

For this, I have to search in each database, and I use some code from Aaron Bertrand , credit to him for this . link1 , link2.

CREATE
OR
ALTER PROCEDURE [dbo].[sp_PrimaryKeySearch] @pkColName nvarchar(128) 
AS 
BEGIN 
DECLARE @nvc_SQLTEXT nvarchar(2000) =N'
		IF EXISTS(SELECT *
					FROM sys.key_constraints as kc 
					INNER JOIN sys.columns as c
					ON c.object_id = kc.parent_object_id
					WHERE kc.type =''PK'' AND c.name = '''+ @pkColName +''')
		BEGIN
			SELECT DB_NAME() as dbName, STRING_AGG(PKnUTable.name,'','') as lstTabele
			FROM sys.key_constraints as PKnUKEY
				INNER JOIN sys.tables as PKnUTable
					ON PKnUTable.object_id = PKnUKEY.parent_object_id
				INNER JOIN sys.index_columns as PKnUColIdx
					ON PKnUColIdx.object_id = PKnUTable.object_id
					AND PKnUColIdx.index_id = PKnUKEY.unique_index_id
				INNER JOIN sys.columns as PKnUKEYCol
					ON PKnUKEYCol.object_id = PKnUTable.object_id
					AND PKnUKEYCol.column_id = PKnUColIdx.column_id
			WHERE PKnUKEY.type = ''PK''
				AND PKnUKEYCol.name = '''+ @pkColName +'''
		END
		'; /*PRINT @nvc_SQLText;*/ 
                EXEC sp_inEachDB @nvc_SQLTEXT
 END

And add this code to the SSMS , to query shortcuts…and done.

S

Parameter embedding Optimization

Parameter embedding Optimization PEO

A very interesting subject and here it is post about it very good
https://sqlperformance.com/2013/08/t-sql-queries/parameter-sniffing-embedding-and-the-recompile-options

When PEO is present,

parameters are replaced with literal constant values during query parsing.

https://sqlperformance.com/2013/08/t-sql-queries/parameter-sniffing-embedding-and-the-recompile-options

A point to remember , between others , if you are using OPTION(RECOMPILE) for a statement and expect PEO to happens and don’t , it’s because you assigns to variables and PEO will not be present.

S

Puzzle CE 2

There is this query in sql server 2019:

select

rezProcesat

from dbo.tblRezervari

where 1=1

and rezProcesat is null

and rezUserName= CAST(SUSER_SNAME() as varchar(256))

What will be the estimate for it?

Let’s take each predicate and find it estimate.

For rezProcesat is null, the estimate is 19140. For this, the is a filtered index on this predicate that have rows matching predicate=18204,unfiltered_rows=17593170

select rows,unfiltered_rows,rows*1.0/unfiltered_rows as Selectivity1 ,(rows*1.0/unfiltered_rows)*18498100 as EstimatePredicate1 from sys.dm_db_stats_properties_internal(object_id(‘dbo.tblFluxRezervari’),13);

rows unfiltered_rows Selectivity1 EstimatePredicate1
18204 17593170 0.00103471972362001 19140.34891950

For the second predicate : rezUserName= CAST(SUSER_SNAME() as varchar(256)), we have to look at the statistics information.

DBCC SHOW_STATISTICS (‘dbo.tblRezervari’,rezUserName) wITH DENSITY_VECTOR;

All density Average Length Columns
0.009708738 9.014213 rezUserName

select 18498100 * 0.009708738 as EstimatePredicate2

EstimatePredicate2 : 179593.206397800

And putting all together, we get:

EstimatePredicate1 = 19140; SelectivityPredicate1 = 0.00103472

EstimatePredicate2 = 179594; SelectivityPredicate2 = 0.009708738

select 18498100 * (0.00103472*0.009708738)

185.82868252393161600

S

Puzzle CE

Sql server 2019, I have this query, and I was wondered from where is coming the estimate of 19140:

select
rezProcesat
from tblRezervari
where 1=1
and rezProcesat is null
OPTION
(
QUERYTRACEON 3604, QUERYTRACEON 2363 –New CE Stats Report
)

And the estimate for it is: 19140


the output from TF2363 :

Begin selectivity computation

Input tree:

LogOp_Select

  CStCollBaseTable(ID=1, CARD=1.84981e+07 TBL: tblRezervari)

  ScaOp_Comp x_cmpIs

      ScaOp_Identifier QCOL: [Stefan_2019].[dbo].[tblRezervari].rezProcesat

      ScaOp_Const TI(datetime,Null,ML=8) XVAR(datetime,Not Owned,Value=NULL)

Plan for computation:

CSelCalcColumnInInterval

  Column: QCOL: [Stefan_2019].[dbo].[tblRezervari].rezProcesat

Loaded histogram for column COL: KeyCo2 from stats with id 13

Selectivity: 0.00103472

Stats collection generated:

CStCollFilter(ID=2, CARD=19140.4)

  CStCollBaseTable(ID=1, CARD=1.84981e+07 TBL: tblRezervari)

End selectivity computation


So, the value 19140 is base on 0.00103472 * 1.84981e+07


But , from where did we get the Selectivity: 0.00103472 ?
I have a filtered index on that column, with filtered clause on exact that predicate.



Now, the next value : CARD=1.84981e+07, translated to 18498100

This should be the cardinality of the table… but I don’t from where is that value.


I don’t exactly know from where the exact value 18498100 come from , but this probably would be a future post after more research

S

Powershell run sql export csv

Image that you have a folder with some sql scripts and you need to run them. And the result to be saved , to a csv file.
and if a script is marked somehow, in this case with "_1" like xxxx_1, this script should be executed for each database in that server…

<#
run scripts
to a server
to a database
from a folder

To DO
pentru anumite scripturi, care au , de exemplu, in numele lor, la final ceva , de ex: _1 $string.SubString($string.Length-2)
pt ele ar trebuie sa trecem prin toate bazele de date existente pe server
$string.SubString($string.Length-2)
#>

Add-PSSnapin SqlServerCmdletSnapin100 # here live Invoke-SqlCmd
[System.Reflection.Assembly]::LoadWithPartialName(‘Microsoft.SqlServer.SMO’) | out-null

clear

$server = ‘SQLSERVERDEV2019′
$db =’master’
$filename = ”
#unde sunt scripturile care treb rulate
$SourceLocation = ‘E:\Documents\SQL Server Management Studio\Projects\Work\scripts_DBA\scripts2′

#unde exportam rezultatele
$FolderPathExport =’E:\Documents\SQL Server Management Studio\Projects\Work\scripts_DBA\output’
$NewOutputFolder = $FolderPathExport + "\" + $Server.replace(‘\’,’_’) + "_" + $(get-date -f yyyy_MM_dd)

#daca folderul exista sau nu, atunci il cream
#si daca exista ii punem si ora_minutele_sec
IF(!(Test-Path -path $NewOutputFolder))
{
New-Item -ItemType Directory -Force -Path $NewOutputFolder
}
ELSE
{
$NewOutputFolder =$NewOutputFolder +’__’ + $(get-date -f hh_mm_ss)
New-Item -ItemType Directory -Force -Path $NewOutputFolder
}

<# vedem ce baze de date sunt pe server #>

$s = New-Object (‘Microsoft.SqlServer.Management.Smo.Server’) $server
$dbs=$s.Databases
#####

$listOfFiles = Get-ChildItem $SourceLocation

$listOfFiles | forEach-object {
#setam numele fisierul care reprezinta sursa pt rulare script
$fileNameInput = $_.FullName

#setam numele fisierului pt export
$fileName =”
$fileName = $NewOutputFolder+ "\" + $_.BaseName.replace(‘ ‘,’_’)

IF( $_.BaseName.SubString($_.BaseName.Length-2) -eq’_1′)
{
#daca avem -1 in coada, atunci trecem prin toate bazele de date
$dbs | ForEach-Object {
$fileNameInterior = $fileName + "_" + $_.Name + "_" + $(get-date -f yyyy_MM_dd) + ".csv"
Invoke-sqlcmd -ServerInstance $server -Querytimeout 600 -Database $db -InputFile $fileNameInput| Export-Csv -Path $fileNameInterior -NoTypeInformation
}
}
ELSE
{

#buna , dar este inlocuita cu $fileName = $($FolderPathExport + "\" + $Server.replace(‘\’,’_’) ) + "_" + $_.BaseName.replace(‘ ‘,’_’) + "_" + $(get-date -f yyyy_MM_dd) + ".csv"
$fileName = $fileName + "_" + $(get-date -f yyyy_MM_dd) + ".csv"
#$fileName
Invoke-sqlcmd -ServerInstance $server -Querytimeout 600 -Database $db -InputFile $_.FullName| Export-Csv -Path $fileName -NoTypeInformation
}
}

Data Compression

Looping through each table, to see what type of data compression to use.

drop table if exists #tmpObjectList,#tmpFinalResults;
go

declare @i_i int,
@i_maxID int,
@nvc_SQL nvarchar(2000) ;

declare @tmpResults table
(
idResults int identity(1,1) not null primary key clustered,
objName varchar(128) not null,
schName varchar(128) not null,
index_id int not null,
partition_number int not null,
current_compression_size bigint null, /kb/
requested_compression_size bigint null, /kb/
current_compression_size_sample bigint null, /kb/
requested_compression_size_sample bigint null/kb/
);

create table #tmpFinalResults
(
idFinalResults int identity(1,1) not null primary key clustered,
objName varchar(128) not null,
schName varchar(128) not null,
index_id int not null,
partition_number int not null,
current_compression_size bigint null, /kb/
requested_compression_size bigint null, /kb/
CompressionType varchar(50) not null
);

create table #tmpObjectList
(
idObject int identity(1,1) not null primary key clustered,
objName varchar(128) not null,
schName varchar(128) not null
);

INSERT INTO #tmpObjectList(objName,schName)
SELECT o.name, OBJECT_SCHEMA_NAME(o.object_id) as schName
FROM sys.objects as o
INNER JOIN sys.indexes AS i ON i.object_id = o.object_id
INNER JOIN sys.dm_db_partition_stats as ps ON ps.object_id = i.object_id and ps.index_id = i.index_id
INNER JOIN sys.partitions as p ON ps.object_id = p.object_id and ps.partition_id = p.partition_id and ps.index_id = p.index_id
WHERE
o.is_ms_shipped =0
and o.type =’U’
GROUP BY
o.name,OBJECT_SCHEMA_NAME(o.object_id)
HAVING
CONVERT(decimal(18,2),SUM(reserved_page_count) * 8/1024.0) > 10.0 /* sa fie mai mare ca 10 MB/ /
SELECT o.name, OBJECT_SCHEMA_NAME(o.object_id) as schName
FROM sys.objects as o
WHERE
o.is_ms_shipped =0
and o.type =’U’
*/

/preluam limitele min , max/
SELECT @i_maxID = MAX(idObject) ,
@i_i = MIN(idObject)
FROM #tmpObjectList;

WHILE (@i_i<= @i_maxID)
/WHILE (@i_i<= 2)/
BEGIN
/* row compression*/
SELECT @nvc_SQL =N’EXEC sp_estimate_data_compression_savings ‘ + tol.schName + N’, ‘ + tol.objName + N’, NULL, NULL, ”ROW”’
FROM #tmpObjectList as tol
WHERE idObject = @i_i;

/*print @nvc_SQL;*/

DELETE FROM @tmpResults;
INSERT INTO @tmpResults(objName,schName,index_id,partition_number,
                current_compression_size ,requested_compression_size,
                current_compression_size_sample ,requested_compression_size_sample)
EXEC(@nvc_SQL);

INSERT INTO #tmpFinalResults(objName,schName,index_id,partition_number,current_compression_size ,requested_compression_size,CompressionType)
SELECT t.objName,t.schName,t.index_id,t.partition_number,t.current_compression_size,t.requested_compression_size,'ROW'
FROM @tmpResults as t

/*page compression*/

SELECT @nvc_SQL =N'EXEC sp_estimate_data_compression_savings ' + tol.schName + N', ' + tol.objName + N', NULL, NULL, ''PAGE'''
FROM #tmpObjectList as tol
WHERE idObject = @i_i;

/*print @nvc_SQL;*/

DELETE FROM @tmpResults;
INSERT INTO @tmpResults(objName,schName,index_id,partition_number,
                current_compression_size ,requested_compression_size,
                current_compression_size_sample ,requested_compression_size_sample)
EXEC(@nvc_SQL);

INSERT INTO #tmpFinalResults(objName,schName,index_id,partition_number,current_compression_size ,requested_compression_size,CompressionType)
SELECT t.objName,t.schName,t.index_id,t.partition_number,t.current_compression_size,t.requested_compression_size,'PAGE'
FROM @tmpResults as t


SET @i_i = @i_i +1;
SET @nvc_SQL =N'';

END;

/*
SELECT * FROM #tmpFinalResults;
*/

SELECT ISNULL(r.objName,p.objName) as tblName,ISNULL(r.schName,p.schName) as tblSchName,ISNULL(r.index_id,p.index_id)as index_ID,
r.current_compression_size , p.current_compression_size,
r.requested_compression_size as row_compression_size,p.requested_compression_size as page_compression_size,
CAST(r.requested_compression_size*1.0 / (r.current_compression_size ) *100.0 as tinyint) as ROW_Procent,
CAST(p.requested_compression_size*1.0 / (p.current_compression_size ) *100.0 as tinyint) as PAGE_Procent
FROM
(
SELECT r.objName,r.schName,r.index_id,r.current_compression_size,r.requested_compression_size
FROM #tmpFinalResults as r
WHERE r.CompressionType =’ROW’
) as r
FULL JOIN
(
SELECT p.objName,p.schName,p.index_id,p.current_compression_size,p.requested_compression_size
FROM #tmpFinalResults as p
WHERE p.CompressionType =’PAGE’
) as p
ON r.objName = p.objName AND r.schName = p.schName AND r.index_id = p.index_id