Simple powershell script to Bulk Load csv into a SQL Server table.

#Read csv file 
#open connection to database using bulkcopy
#convert array to data table
#bulkload data into table
#note: column sequence and data types should match

function Get-Type 
{ 
    param($type) 
 
$types = @( 
'System.Boolean', 
'System.Byte[]', 
'System.Byte', 
'System.Char', 
'System.Datetime', 
'System.Decimal', 
'System.Double', 
'System.Guid', 
'System.Int16', 
'System.Int32', 
'System.Int64', 
'System.Single', 
'System.UInt16', 
'System.UInt32', 
'System.UInt64') 
 
    if ( $types -contains $type ) { 
        Write-Output "$type" 
    } 
    else { 
        Write-Output 'System.String' 
         
    } 
}


function Out-DataTable 
{ 
    [CmdletBinding()] 
    param([Parameter(Position=0, Mandatory=$true, ValueFromPipeline = $true)] [PSObject[]]$InputObject) 
 
    Begin 
    { 
        $dt = new-object Data.datatable   
        $First = $true  
    } 
    Process 
    { 
        foreach ($object in $InputObject) 
        { 
            $DR = $DT.NewRow()   
            foreach($property in $object.PsObject.get_properties()) 
            {   
                if ($first) 
                {   
                    $Col =  new-object Data.DataColumn   
                    $Col.ColumnName = $property.Name.ToString()   
                    if ($property.value) 
                    { 
                        if ($property.value -isnot [System.DBNull]) { 
                            $Col.DataType = [System.Type]::GetType("$(Get-Type $property.TypeNameOfValue)") 
                         } 
                    } 
                    $DT.Columns.Add($Col) 
                }   
                if ($property.Gettype().IsArray) { 
                    $DR.Item($property.Name) =$property.value | ConvertTo-XML -AS String -NoTypeInformation -Depth 1 
                }   
               else { 
                    $DR.Item($property.Name) = $property.value 
                } 
            }   
            $DT.Rows.Add($DR)   
            $First = $false 
        } 
    }  
      
    End 
    { 
        Write-Output @(,($dt)) 
    } 
 
} #Out-DataTable

$file="C:\samplecsv.csv"
$dbserver="ENTER DB SERVER NAME"
$database="ENTER DB NAME"
$cn = new-object System.Data.SqlClient.SqlConnection("Data Source=$dbserver;Integrated Security=SSPI;Initial Catalog=$database");
$cn.Open()
$bc = new-object ("System.Data.SqlClient.SqlBulkCopy") $cn
$bc.BatchSize = 10000;
$bc.BulkCopyTimeout = 1000
$bc.DestinationTableName = "ENTER TABLE NAME WITH SCHEMA NAME"

$data = Import-Csv $file | Out-DataTable

$bc.WriteToServer($data)

[solved] Docker and Windows 10 hyper-v Containers, byLinuxVM’ failed to start.. . Virtual machine ‘MobyLinuxVM’ could not be started because the hypervisor is not running

If you are like me trying to run windows Docker containers in a hyper-v vm windows 10 guest on a windows 10 host, you might have come across above issue, we can fix this issue in two steps.

  • First make sure you are running on most up to date windows 10, prerequisite for OS requirements is to have Windows 10 Anniversary Update.
  • Second enable nested virtualization, why do we need to do that ?

nesteddiagram2

In this case, Hyper-V has been configured to expose virtualization extensions to its guest VM. A guest VM can take advantage of this, and install its own hypervisor. It can then run its own guest VMs, more info visit msdn blog

So how to enable nested virtualization,

first turn off the vm, run below command on the host OS, I would say the root OS.

Set-VMProcessor -VMName <VMName> -ExposeVirtualizationExtensions $true

Enable networking for nested virtual machine, can be done in two ways Mac spoofing or NAT mode.

will go with Mac Spoofing, to enable MAC spoofing, run below command on host os.

Get-VMNetworkAdapter -VMName <VMName> | Set-VMNetworkAdapter -MacAddressSpoofing On

Make sure the guest OS has enough memory to run MobyLinuxVm docker container to initialize, I would say at least 4 gigs.

List installed SQL Server instances with version on local or remote computer [Powershell]

 

 

 

$MachineName = . # Replace . with server name for a remote computer

$reg = [Microsoft.Win32.RegistryKey]::OpenRemoteBaseKey(LocalMachine, $MachineName)

$regKey= $reg.OpenSubKey("SOFTWARE\\Microsoft\\Microsoft SQL Server\\Instance Names\\SQL" )

$values = $regkey.GetValueNames()

$values | ForEach-Object {$value = $_ ; $inst = $regKey.GetValue($value); 
			  $path = "SOFTWARE\\Microsoft\\Microsoft SQL Server\\"+$inst+"\\MSSQLServer\\"+"CurrentVersion";
			  #write-host $path; 
			  $version = $reg.OpenSubKey($path).GetValue("CurrentVersion");
                          write-host "Instance" $value;
			  write-host  "Version" $version}

Compress Database backups on SQL Express Edition

As we know Microsoft does not support compression for SQL Server Express but time to time we use express edition database engine when all you need is small database that might not even grow more than 1 gig or 5 gigs.

Prep:

we need 7z zip program its an opensource software will help us compress the files, download the latest version 64 bit (http://www.7-zip.org/)

Enable xp_cmdShell ( we are going to use xp_cmdshell to execute the batch command to compress or delete files)

-- To allow advanced options to be changed.  
EXEC sp_configure 'show advanced options', 1;  
GO  
-- To update the currently configured value for advanced options.  
RECONFIGURE;  
GO  
-- To enable the feature.  
EXEC sp_configure 'xp_cmdshell', 1;  
GO  
-- To update the currently configured value for this feature.  
RECONFIGURE;  
GO  

TSQL Script( Where Magic happens)

How does script work:  it first looks at mypath and lists all .bak files and folders it loops each of these files using a cursor and execute the 7zip command to compress the .bak to .zip and cleanuphrs parameter will clean the backup zip files older than specified number of hours.

/*
##Script to Compress the backups in sql express using 7 zip 
##Author: Akhil 
##Date:09/19/2016
##Rev:1 initial Setup
##Rev:2 fix the date add to negitive 
##Rev:3 Remove xp_delete_file instead use batch command to remove old files.
*/

--parameter
declare @myPath varchar(4000) =  'R:\mssql\userdb';
declare @apath varchar(1000) = '';
declare @aextn varchar(10)= 'zip';
declare @7zexe varchar(1000)= 'C:\Program Files\7-Zip\7z.exe'
declare @cleanupdays int = 4;
declare @cleanupdate datetime;

IF OBJECT_ID('tempdb..#DirectoryTree') IS NOT NULL
DROP TABLE #DirectoryTree;

CREATE TABLE #DirectoryTree (
	id int IDENTITY(1,1)
	,subdirectory nvarchar(512)
	--,filename 
	,depth int
	,isfile bit
	, ParentDirectory int
	,flag tinyint default(0));

-- top level directory
INSERT #DirectoryTree (subdirectory,depth,isfile)
	VALUES (@myPath,0,0);
-- all the rest under top level
INSERT #DirectoryTree (subdirectory,depth,isfile)
	EXEC master.sys.xp_dirtree @myPath,0,1;


UPDATE #DirectoryTree
	SET ParentDirectory = (
		SELECT MAX(Id) FROM #DirectoryTree
		WHERE Depth = d.Depth - 1 AND Id < d.Id	)
FROM #DirectoryTree d;

-- SEE all with full paths
declare @archivefiles cursor ;
declare @bfile sysname;
declare @afile sysname;
declare @conataner sysname;
declare @result int;
declare @cmd varchar(8000);

--Get all the files 

set @archivefiles = cursor for
WITH dirs AS (
	 SELECT
		 Id,subdirectory,depth,isfile,ParentDirectory,flag
		 , CAST (null AS NVARCHAR(MAX)) AS container
		 , CAST([subdirectory] AS NVARCHAR(MAX)) AS dpath
		 FROM #DirectoryTree
		 WHERE ParentDirectory IS NULL 
	 UNION ALL
	 SELECT
		 d.Id,d.subdirectory,d.depth,d.isfile,d.ParentDirectory,d.flag
		 , dpath as container
		 , dpath +'\'+d.[subdirectory]  
	 FROM #DirectoryTree AS d
	 INNER JOIN dirs ON  d.ParentDirectory = dirs.id
)
SELECT dpath,case when @apath is null or len(@apath) = 0 then replace(dpath,'.bak','.'+@aextn) 
 else @apath+replace(subdirectory,'.bak','.'+@aextn)  end as afile
    FROM dirs 
-- Dir style ordering
where isfile = 1 and subdirectory like '%.bak'


print('Archive Process')

open @archivefiles	

fetch next from @archivefiles into @bfile,@afile
while @@FETCH_STATUS = 0
begin 
print (@bfile)
print (@afile)
set @cmd = '""'+@7zexe+'" a "'+@afile+'" "'+@bfile+'"'+' -sdel'+'"'
print('Archive Command'+@cmd)
exec @result =  xp_cmdshell @cmd
print(@result)
fetch next from @archivefiles into @bfile,@afile
end

print('Clean up')

set @cmd = '"forfiles -p "'+@myPath+'" -s -m *.'+@aextn+' /D -'+convert(varchar,@cleanupdays)+' /C "cmd /c del @path"'
print('Delete Command '+@cmd)
exec @result =  xp_cmdshell @cmd
print(@result)