1. Computer problem? Tech Support Guy is completely free -- paid for by advertisers and donations. Click here to join today! If you're new to Tech Support Guy, we highly recommend that you visit our Guide for New Members.

[Self-Heal] Disk Usage Analysis and Delete:Please respond for the two items under "St

Discussion in 'Software Development' started by Phanikiran, May 8, 2015.

Thread Status:
Not open for further replies.
  1. Phanikiran

    Phanikiran Thread Starter

    Joined:
    Jul 28, 2014
    Messages:
    39
    We have work on a script that would help reducing tickets raised by disk space issues.We just finished to test the script,
    Deliverables:

    • Action Report: A report for each server of first-level subfolders of a given folder (e.g.: C: or C:\Users), mentioning whether said sub-folder has been “accessed” in the last 60 days and the size of the folder.
    The idea here is to provide Infra with a list of folders that are not used anymore (e.g.: backup folders, deployment temp folders etc.)

    • Action Delete: For folders that are known to be “deletable” (e.g.: temp folder), we go in and delete any subfolder where all files are older than 30 days.
    This should provide immediate relief for our disk space alerts.

    • IIS Log Cleanup: The script automatically finds out where IIS stores its logs and deletes all logs older than 365 days and compresses the whole folder.
    Immediate relief for disk space alerts

    Usage:
    The idea is not to set this up as a pro-active measure (i.e.: in response of a SCOM alert) but as a scheduled task running from the SMA server (every week?).
    The script took 210 minutes to run on the MIMOS UAT 10 servers.

    Parameters and compatibility:
    We have tried to write this script as modular and parametrised as possible so that it could be easily reused by other teams. Currently we target only the MIMOS UAT group but the target group (provided by SCOM) could be added as a parameter.
    The minimum age of the file and all switches to say whether files should be deletes or reported and whether folders should be compressed, deleted or reported can all be specified when you call the function.
    The folders to be analysed (report) or deleted can also be parametrised. Currently I target:
    ReportDrives = @("C:", "C:\Users\", "D:", "E:")
    DeleteDrives = @("C:\Windows\SoftwareDistribution\ScanFile\", "C:\Windows\Temp\")

    Still to do:

    • the size expressed in the report is currently in GB (plus 2 digits). I should at least provide one more digit as most records (folders) now appear as 0.00.
    • Parametrise group and drives to analyse (the problem here is that the parameter string could become very long.

    As usual, none of this is carved in stone and we are very happy to receive actionable feedback.

    Below is the log of the script CheckServersDiskUsage.ps1
    and the report (which is in an Excel sheet that retrieves data dynamically from the SQL Server DB where we stored it.)
    ----------------------------------------------------------------CheckServersDiskUsage.ps1------------------------------------------------------

    function fn_CheckFolderUsage {
    param (
    [string] $Computer,
    [string] $folder,
    [int] $filemaxage=30
    )
    Write-Output "Starting"

    if ($folder.EndsWith("\") -eq $false) {$folder = $folder + "\"}

    $output = Invoke-Command -ComputerName $Computer -ArgumentList $folder, $filemaxage -ScriptBlock {
    param(
    [string] $folder,
    [int] $filemaxage
    )

    $RootDirectories = Get-ChildItem $USING:folder -ErrorAction SilentlyContinue | Where-object {$_.psIsContainer -eq $true -and $_.lastaccesstime -le (get-date).AddDays(-($USING:filemaxage)) } | Select fullname, lastaccesstime

    $results = @()

    foreach($dir in $RootDirectories)
    {
    $size = $(Get-ChildItem $dir.fullname -Recurse -Force -ErrorAction SilentlyContinue | Measure-Object -property length -sum -ErrorAction SilentlyContinue).Sum /1MB
    $size = "{0:F2}" -f $size

    $serverresult = New-Object System.Object

    $serverresult | Add-Member -type NoteProperty -name ServerName -value $USING:Computer
    $serverresult | Add-Member -type NoteProperty -name FolderName -value $dir.fullname
    $serverresult | Add-Member -type NoteProperty -name AccessTime -value $dir.lastaccesstime
    $serverresult | Add-Member -type NoteProperty -name Size -value $size
    $results += $serverresult

    }

    $results
    }
    $output
    }
    function fn_CheckIISFolderUsage {
    param([string] $ComputerName)

    $Logs = Invoke-Command -ComputerName $ComputerName -ArgumentList $ComputerName -ScriptBlock {
    param(
    [string] $computer
    )
    Import-Module WebAdministration -ErrorAction SilentlyContinue
    $WebLogs = $(Get-ChildItem IIS:\Sites -ErrorAction SilentlyContinue).logFile.directory | Get-Unique
    $results = @()

    foreach($Log in $WebLogs)
    {
    $Log = $Log -replace "%SystemDrive%", "C:"
    Write-Output "Analizing $Log"
    $return = ([wmi]"\\$computer\root\cimv2:win32_directory.name='$Log'").compress()

    Get-ChildItem $Log -Recurse -Force -ErrorAction SilentlyContinue | Where-object {!$_.psIsContainer -eq $true -and $_.CreationTime -le (get-date).AddDays(-365) } | Remove-Item

    $serverresult = New-Object System.Object

    $serverresult | Add-Member -type NoteProperty -name ServerName -value $computer
    $serverresult | Add-Member -type NoteProperty -name FolderName -value $Log
    $serverresult | Add-Member -type NoteProperty -name AccessTime -value ""
    $serverresult | Add-Member -type NoteProperty -name Size -value 0
    $results += $serverresult

    }
    $results
    }
    $Logs
    }
    function fn_deleteOldFiles {
    param (
    [string] $Computer,
    [string] $folder,
    [int] $filemaxage=30
    )

    if ($folder.EndsWith("\") -eq $false) {$folder = $folder + "\"}

    $output = Invoke-Command -ComputerName $Computer -ArgumentList $folder, $filemaxage -ScriptBlock {
    param(
    [string] $folder,
    [int] $filemaxage
    )
    Get-ChildItem $folder -ErrorAction SilentlyContinue | Where-object {$_.psIsContainer -eq $true -and $_.lastaccesstime -le (get-date).AddDays(-($filemaxage)) } | Remove-Item -Recurse -Force -Confirm:$false
    }
    }
    function CheckServersDiskUsage{
    Import-Module SQLPS -ErrorAction SilentlyContinue -WarningAction SilentlyContinue
    "Querying DB for Servers...$(Get-Date -format ""MM/dd/yyyy HH:mm:ss"")" | Out-File "c:\temp\CheckDrives.log" -Append
    $SQL = "select distinct servernamefqdn from vServerList where applicationgroup = 'CCRM MIMOS UAT'"
    # $SQL = "$SQL AND (servernamefqdn = 'CO1MIMOSUATWEB4.partners.extranet.microsoft.com')" # OR servernamefqdn = 'CO1MIMOSUATWEB3.partners.extranet.microsoft.com'
    $servers = Invoke-Sqlcmd -Query $SQL -ServerInstance "tk5ccrmmonprod7.redmond.corp.microsoft.com" -Database "NGReporting"

    $drives = @("C:", "C:\Users\", "D:", "E:")
    $deldrives = @("C:\Windows\SoftwareDistribution\ScanFile\", "C:\Windows\Temp\", "C:\Temp\")
    "Starting...$(Get-Date -format ""MM/dd/yyyy HH:mm:ss"")" | Out-File "c:\temp\CheckDrives.log" -Append
    foreach ($server in $servers)
    {
    if(Test-Connection -ComputerName $($Server.servernamefqdn) -ErrorAction SilentlyContinue -Count 4) {

    "Starting report for $($Server.servernamefqdn)" | Out-File "c:\temp\CheckDrives.log" -Append
    #Initialize Server (Enable Powershell remoting)
    #fn_InitializeServer -ComputerName $($Server.servernamefqdn) -AdminUser "REDMOND\\dynscom"
    #Generate report for drives specified in the drives array
    foreach ($drive in $drives){
    "Server $($Server.servernamefqdn) on drive: $drive" | Out-File "c:\temp\CheckDrives.log" -Append
    $results = fn_CheckFolderUsage -Computer $($Server.servernamefqdn) -folder $drive -filemaxage 60
    foreach ($result in $results)
    {
    if ($result.ServerName) {
    $SQL1 = "EXEC [dbo].[sp_shDiskSpaceMgmtUpdate] @ServerName = '$($result.ServerName)',@Directory = '$($result.FolderName)',@Size = $($result.Size),@Action='Report',@TimeStamp = '$(Get-Date -format ""MM/dd/yyyy HH:mm:ss"")'"
    $sqlout = Invoke-Sqlcmd -Query $SQL1 -ServerInstance "tk5ccrmmonprod7.redmond.corp.microsoft.com" -Database "NGReporting"
    }
    }
    }

    "Starting IIS Check for $($Server.servernamefqdn)" | Out-File "c:\temp\CheckDrives.log" -Append
    Write-Verbose "IIS Server $($Server.servernamefqdn) Log Analysis"
    #delete old IIS logs and compress the IISLOGS folder
    $results = fn_CheckIISFolderUsage -ComputerName $($Server.servernamefqdn)
    foreach ($result in $results)
    {
    if ($result.ServerName) {
    $SQL1 = "EXEC [dbo].[sp_shDiskSpaceMgmtUpdate] @ServerName = '$($result.ServerName)',@Directory = '$($result.FolderName)',@Size = $($result.Size),@Action='IIS Cleanup',@TimeStamp = '$(Get-Date -format ""MM/dd/yyyy HH:mm:ss"")'"
    $sqlout = Invoke-Sqlcmd -Query $SQL1 -ServerInstance "tk5ccrmmonprod7.redmond.corp.microsoft.com" -Database "NGReporting"
    }
    }

    "Starting File Deletion for $($Server.servernamefqdn)" | Out-File "c:\temp\CheckDrives.log" -Append
    #Delete files older than 30 days in the temp folders specified in the deldrives array
    foreach ($drive in $deldrives){
    Write-Verbose "Server $($Server.servernamefqdn) on drive: $drive"

    $results = fn_deleteOldFiles -Computer $($Server.servernamefqdn) -folder $drive -filemaxage 30
    $SQL1 = "EXEC [dbo].[sp_shDiskSpaceMgmtUpdate] @ServerName = '$($Server.servernamefqdn)',@Directory = '$drive',@Size = 0,@Action='Delete',@TimeStamp = '$(Get-Date -format ""MM/dd/yyyy HH:mm:ss"")'"
    $sqlout = Invoke-Sqlcmd -Query $SQL1 -ServerInstance "tk5ccrmmonprod7.redmond.corp.microsoft.com" -Database "NGReporting"
    }
    }
    else {
    $SQL1 = "EXEC [dbo].[sp_shDiskSpaceMgmtUpdate] @ServerName = '$($Server.servernamefqdn)',@Directory = 'Unreachable',@Size = 0,@Action='Unreachable',@TimeStamp = '$(Get-Date -format ""MM/dd/yyyy HH:mm:ss"")'"
    Write-Verbose $SQL1
    #$SQL = "INSERT INTO [dbo].[ServerDiskUsage] ([ServerName],[FolderName],) VALUES ('','','')"
    $sqlout = Invoke-Sqlcmd -Query $SQL1 -ServerInstance "tk5ccrmmonprod7.redmond.corp.microsoft.com" -Database "NGReporting"
    Write-Verbose $sqlout
    }
    }
    "Ending...$(Get-Date -format ""MM/dd/yyyy HH:mm:ss"")" | Out-File "c:\temp\CheckDrives.log" -Append
    }
    CheckServersDiskUsage



    Phanikiran Pisipati
     

    Attached Files:

  2. draceplace

    draceplace

    Joined:
    Jun 8, 2001
    Messages:
    2,583
    Looks reasonable but no action report attached. The text file is the code. In general its a good idea, the SCOM alert is usually set to happen when its critical. Your trying to prevent that alert, a good idea.
     
As Seen On
As Seen On...

Welcome to Tech Support Guy!

Are you looking for the solution to your computer problem? Join our site today to ask your question. This site is completely free -- paid for by advertisers and donations.

If you're not already familiar with forums, watch our Welcome Guide to get started.

Join over 733,556 other people just like you!

Thread Status:
Not open for further replies.

Short URL to this thread: https://techguy.org/1147905

  1. This site uses cookies to help personalise content, tailor your experience and to keep you logged in if you register.
    By continuing to use this site, you are consenting to our use of cookies.
    Dismiss Notice