This article is part of the series "Practical PowerShell for IT Security". Check out the rest:
A few months ago, I began a mission to prove that PowerShell can be used as a security monitoring tool. I left off with this post, which had PowerShell code to collect file system events, perform some basic analysis, and then present the results in graphical format. My Security Scripting Platform (SSP) may not be a minimally viable product, but it was, I think, useful as simple monitoring tool for a single file directory.
After finishing the project, I knew there were areas for improvement. The event handling was clunky, the passing of information between various parts of the SSP platform was anything but straightforward, and the information being displayed using the very primitive Out-GridViewwas really a glorified table.
Get the Free PowerShell and Active Directory Essentials Video Course
New and Improved
I took up the challenge of making SSP a bit more viable. My first task was to streamline event handling. I had initially worked it out so that file event messages were picked up by a handler in my Register-EngineEvent scriptblock and sent to an internal queue and then finally forwarded to the main piece of code, the classification software.
I regained my sanity, and realized I could just directly forward the messages with Register-EngineEvent -forward from within the event handling scriptblock, removing an unnecessary layer of queuing craziness.
You can see the meaner, leaner version below.
- #Count events, detect bursts, forward to main interface
- $cur = Get-Date
- $Global:Count=0
- $Global:baseline = @{"Monday" = @(1,1,1); "Tuesday" = @(1,.5,1);"Wednesday" = @(4,4,4);"Thursday" = @(7,12,4); "Friday" = @(5,4,6); "Saturday"=@(2,1,1); "Sunday"= @(2,4,2)}
- $Global:cnts = @(0,0,0)
- $Global:burst = $false
- $Global:evarray = New-Object System.Collections.ArrayList
- $action = {
- $Global:Count++
- $d=(Get-Date).DayofWeek
- $i= [math]::floor((Get-Date).Hour/8)
- $Global:cnts[$i]++
- #event auditing!
- $rawtime = $EventArgs.NewEvent.TargetInstance.LastAccessed.Substring(8,6)
- $filename = $EventArgs.NewEvent.TargetInstance.Name
- $etime= [datetime]::ParseExact($rawtime,"HHmmss",$null)
- $msg="$($etime)): Access of file $($filename)"
- $msg|Out-File C:\Users\Administrator\Documents\events.log -Append
- New-Event -SourceIdentifier Delta -MessageData "Access" -EventArguments $filename #notify
- $Global:evarray.Add(@($filename,$etime))
- if(!$Global:burst) {
- $Global:start=$etime
- $Global:burst=$true
- }
- else {
- if($Global:start.AddMinutes(15) -gt $etime ) {
- $Global:Count++
- #File behavior analytics
- $sfactor=2*[math]::sqrt( $Global:baseline["$($d)"][$i])
- if ($Global:Count -gt $Global:baseline["$($d)"][$i] + 2*$sfactor) { #at 95% level of poisson
- "$($etime): Burst of $($Global:Count) accesses"| Out-File C:\Users\Administrator\Documents\events.log -Append
- $Global:Count=0
- $Global:burst =$false
- New-Event -SourceIdentifier Delta -MessageData "Burst" -EventArguments $Global:evarray #notify on burst
- $Global:evarray= [System.Collections.ArrayList] @()
- }
- }
- else { $Global:burst =$false; $Global:Count=0; $Global:evarray= [System.Collections.ArrayList] @()}
- }
- }
- Register-EngineEvent -SourceIdentifier Delta -Forward
- Register-WmiEvent -Query "SELECT * FROM __InstanceModificationEvent WITHIN 5 WHERE TargetInstance ISA 'CIM_DataFile' and TargetInstance.Path = '\\Users\\Administrator\\' and targetInstance.Drive = 'C:' and (targetInstance.Extension = 'txt' or targetInstance.Extension = 'doc' or targetInstance.Extension = 'rtf') and targetInstance.LastAccessed > '$($cur)' " -sourceIdentifier "Accessor" -Action $action
- Write-Host "starting engine ..."
- while ($true) {
- Wait-Event -SourceIdentifier Access # just hang on this so I don't exit
- }
Then I took on the main piece of code, where I classify files based on whether they have social security numbers and other sensitive keywords. As events come in from the handler, the file reclassification is triggered. This code then periodically displays some of the results of the classification.
In this latest version, I removed the “real-time” classification and focused on cleaning up the PowerShell code and improving the graphics — more on that below.
I took a wrong turn in the original version by relying on a PowerShell data locking module to synchronize data access from concurrent tasks, which I used for some of the grunt work. On further testing, the freebie module that implements the Lock-Object cmdlet didn’t seem to work.
As every junior system programmer knows, it’s easier to synchronize with messages than with low-level locks. I reworked the code to take the messages from the event handler above, and send them directly to a main message processing loop. In short: I was able to deal with asynchronous events in a synchronous manner.
.Net Framework Charts and PowerShell
My great discovery in the last month was that I could embed Microsoft-style charts inside PowerShell. In other word, the bar, line, scatter and other charts that are available in Excel and Word can be controlled programmatically in PowerShell. As a newbie PowerShell programmer, this was exciting to me. You can read more about .Net Framework Controls in this post.
It’s a great idea, and it meant I could also replace the messy Out-GridViewcode.
But the problem, I quickly learned, is that you also have to deal with some of the interactive programming involved with Microsoft forms. I just wanted to display my .Net charts while not having to code the low-level details. Is there a lazy way out?
After much struggle, I came to see that the easiest way to do this is to launch each chart in its own runspace as a separate task. (Nerd Note: this is how I avoided coding message handling for all the charts since each runs separately as modal dialogs.)
I also benefited from this freebie PowerShell module that wraps the messy .Net chart controls. Thanks Marius!
I already had set up a tasking system earlier to scan and classify each file in the directory I was monitoring, so it was just a matter of reusing this tasking code to launch graphs.
I created a pie chart for showing relative concentration of sensitive data, a bar chart for a breakdown of files by sensitive data types, and, the one I’m most proud is a classic event stair-step chart for file access burst conditions — a possible sign of an attack.
For those who are curious about the main chunk of code doing all the work of my SSP, here it is for your entertainment:
- $scan = { #file content scanner
- $name=$args[0]
- function scan {
- Param (
- [parameter(position=1)]
- [string] $Name
- )
- $classify =@{"Top Secret"=[regex]'[tT]op [sS]ecret'; "Sensitive"=[regex]'([Cc]onfidential)|([sS]nowflake)'; "Numbers"=[regex]'[0-9]{3}-[0-9]{2}-[0-9]{3}' }
- $data = Get-Content $Name
- $cnts= @()
- if($data.Length -eq 0) { return $cnts}
- foreach ($key in $classify.Keys) {
- $m=$classify[$key].matches($data)
- if($m.Count -gt 0) {
- $cnts+= @($key,$m.Count)
- }
- }
- $cnts
- }
- scan $name
- }
- #launch a .net chart
- function nchart ($r, $d, $t,$g,$a) {
- $task= {
- Param($d,$t,$g,$a)
- Import-Module C:\Users\Administrator\Documents\charts.psm1
- $chart = New-Chart -Dataset $d -Title $t -Type $g -Axis $a
- Show-Chart $chart
- }
- $Task = [powershell]::Create().AddScript($task).AddArgument($d).AddArgument($t).AddArgument($g).AddArgument($a)
- $Task.RunspacePool = $r
- $Task.BeginInvoke()
- }
- Register-EngineEvent -SourceIdentifier Delta -Action {
- if($event.MessageData -eq "Burst") { #just look at bursts
- New-Event -SourceIdentifier File -MessageData $event.MessageData -EventArguments $event.SourceArgs
- }
- Remove-Event -SourceIdentifier Delta
- }
- $list=Get-WmiObject -Query "SELECT * From CIM_DataFile where Path = '\\Users\\Administrator\\' and Drive = 'C:' and (Extension = 'txt' or Extension = 'doc' or Extension = 'rtf')"
- #long list --let's multithread
- #runspace
- $RunspacePool = [RunspaceFactory]::CreateRunspacePool(1,5)
- $RunspacePool.Open()
- $Tasks = @()
- foreach ($item in $list) {
- $Task = [powershell]::Create().AddScript($scan).AddArgument($item.Name)
- $Task.RunspacePool = $RunspacePool
- $status= $Task.BeginInvoke()
- $Tasks += @($status,$Task,$item.Name)
- }
- #wait
- while ($Tasks.isCompleted -contains $false){
- }
- #Analytics, count number of sensitive content for each file
- $obj = @{}
- $tdcnt=0
- $sfcnt=0
- $nfcnt=0
- for ($i=0; $i -lt $Tasks.Count; $i=$i+3) {
- $match=$Tasks[$i+1].EndInvoke($Tasks[$i])
- if ($match.Count -gt 0) {
- $s = ([string]$Tasks[$i+2]).LastIndexOf("\")+1
- $obj.Add($Tasks[$i+2].Substring($s),$match)
- for( $j=0; $j -lt $match.Count; $j=$j+2) {
- switch -wildcard ($match[$j]) {
- 'Top*' { $tdcnt+= 1 }
- 'Sens*' { $sfcnt+= 1}
- 'Numb*' { $nfcnt+=1}
- }
- }
- }
- $Tasks[$i+1].Dispose()
- }
- #Display Initial Dashboard
- #Pie chart of sensitive files based on total counts of senstive dat
- $piedata= @{}
- foreach ( $key in $obj.Keys) {
- $senscnt =0
- for($k=1; $k -lt $obj[$key].Count;$k=$k+2) {
- $senscnt+= $obj[$key][$k]
- }
- $piedata.Add($key, $senscnt)
- }
- nchart $RunspacePool $piedata "Files with Sensitive Content" "Pie" $false
- #Bar Chart of Total Files, Sensitive vs Total
- $bardata = @{"Total Files" = $Tasks.Count}
- $bardata.Add("Files w. Top Secret",$tdcnt)
- $bardata.Add("Files w. Sensitive", $sfcnt)
- $bardata.Add("Files w. SS Numbers",$nfcnt)
- nchart $RunspacePool $bardata "Sensitive Files" "Bar" $false
- #run event handler as a seperate job
- Start-Job -Name EventHandler -ScriptBlock({C:\Users\Administrator\Documents\evhandler.ps1})
- while ($true) { #main message handling loop
- [System.Management.Automation.PSEventArgs] $args = Wait-Event -SourceIdentifier File # wait on event
- Remove-Event -SourceIdentifier File
- #Write-Host $args.SourceArgs
- if ($args.MessageData -eq "Burst") {
- #Display Bursty event
- $dt=$args.SourceArgs
- #time in seconds
- [datetime]$sevent =$dt[0][1]
- $xyarray = [ordered]@{}
- $xyarray.Add(0,1)
- for($j=1;$j -lt $dt.Count;$j=$j+1) {
- [timespan]$diff = $dt[$j][1] - $sevent
- $xyarray.Add($diff.Seconds,$j+1)
- }
- nchart $RunspacePool $xyarray "Burst Event" "StepLine" $true
- }
- }#while
- Write-Host "Done!"
Lessons Learned
Of course, with any mission the point is the journey not the actual goal, right? The key thing I learned is that you can use PowerShell to do security monitoring. For a single directory, on a small system. And only using it sparingly.
While I plan on improving what I just presented by adding real-time graphics, I’m under no illusion that my final software would be anything more than a toy project.
File event monitoring, analysis, and graphical display of information for an entire system is very, very hard to do on your own. You can, perhaps, recode my solution using C++, but you’ll still have to deal with the lags and hiccups of processing low-level events in the application space. To do this right, you need to have hooks deep in the OS — for starters — and then do far more serious analysis of file events than is performed in my primitive analytics code. That ain’t easy!
I usually end up these DIY posts by saying “you know where this is going.” I won’t disappoint you.
You know where this is going. Our own enterprise-class solution is a true data security platform or DSP – it handles classification, analytics, threat detection, and more for entire IT systems.
By all means, try to roll your own, perhaps based on this project, to learn the difficulties and appreciate what a DSP is actually doing.
Have questions? Feel free to contact us!
Next Steps
If you’re interested in learning more practical, security focused PowerShell, you can unlock the full 3 hour video course on PowerShell and Active Directory Essentials with the code cmdlet.
