10 May

Decoding SAML responses in form post data using PowerShell

If you are ever inspecting a network trace when diagnosing SAML issues and the computer you are using is restricted enough that you cannot install useful SAML specific tools, you can always use PowerShell to help you demystify the network trace.

The techniques in this post can be used for decoding any application/x-www-form-urlencoded data however I will describe fully the procedure for decoding the SAMLResponse HTTP POST data and displaying the resultant XML in a nicely indented format.

In your network trace you will see HTTP POST data similar to the following:


SAMLResponse=PHNhbWxwOlJlc3BvbnNlIHhtbG5zOnNhbWxwPSJ1cm46b2FzaXM6bmFtZXM6dGM6U0FNTDoyLjA6cHJvdG9jb2
wiIHhtbG5zOnNhbWw9InVybjpvYXNpczpuYW1lczp0YzpTQU1MOjIuMDphc3NlcnRpb24iIElEPSJfOGU4ZGM1ZjY5YTk4Y2M0Y
zFmZjM0MjdlNWNlMzQ2MDZmZDY3MmY5MWU2IiBWZXJzaW9uPSIyLjAiIElzc3VlSW5zdGFudD0iMjAxNC0wNy0xN1QwMTowMTo0
OFoiIERlc3RpbmF0aW9uPSJodHRwOi8vc3AuZXhhbXBsZS5jb20vZGVtbzEvaW5kZXgucGhwP2FjcyIgSW5SZXNwb25zZVRvPSJ
PTkVMT0dJTl80ZmVlM2IwNDYzOTVjNGU3NTEwMTFlOTdmODkwMGI1MjczZDU2Njg1Ij48c2FtbDpJc3N1ZXI%2baHR0cDovL2lk
cC5leGFtcGxlLmNvbS9tZXRhZGF0YS5waHA8L3NhbWw6SXNzdWVyPjxzYW1scDpTdGF0dXM%2bPHNhbWxwOlN0YXR1c0NvZGUgV
mFsdWU9InVybjpvYXNpczpuYW1lczp0YzpTQU1MOjIuMDpzdGF0dXM6U3VjY2VzcyIgLz48L3NhbWxwOlN0YXR1cz48c2FtbDpB
c3NlcnRpb24geG1sbnM6eHNpPSJodHRwOi8vd3d3LnczLm9yZy8yMDAxL1hNTFNjaGVtYS1pbnN0YW5jZSIgeG1sbnM6eHM9Imh
0dHA6Ly93d3cudzMub3JnLzIwMDEvWE1MU2NoZW1hIiBJRD0iX2Q3MWEzYThlOWZjYzQ1YzllOWQyNDhlZjcwNDkzOTNmYzhmMD
RlNWY3NSIgVmVyc2lvbj0iMi4wIiBJc3N1ZUluc3RhbnQ9IjIwMTQtMDctMTdUMDE6MDE6NDhaIj48c2FtbDpJc3N1ZXI%2baHR
0cDovL2lkcC5leGFtcGxlLmNvbS9tZXRhZGF0YS5waHA8L3NhbWw6SXNzdWVyPjxzYW1sOlN1YmplY3Q%2bPHNhbWw6TmFtZUlE
IFNQTmFtZVF1YWxpZmllcj0iaHR0cDovL3NwLmV4YW1wbGUuY29tL2RlbW8xL21ldGFkYXRhLnBocCIgRm9ybWF0PSJ1cm46b2F
zaXM6bmFtZXM6dGM6U0FNTDoyLjA6bmFtZWlkLWZvcm1hdDp0cmFuc2llbnQiPl9jZTNkMjk0OGI0Y2YyMDE0NmRlZTBhMGIzZG
Q2ZjY5YjZjZjg2ZjYyZDc8L3NhbWw6TmFtZUlEPjxzYW1sOlN1YmplY3RDb25maXJtYXRpb24gTWV0aG9kPSJ1cm46b2FzaXM6b
mFtZXM6dGM6U0FNTDoyLjA6Y206YmVhcmVyIj48c2FtbDpTdWJqZWN0Q29uZmlybWF0aW9uRGF0YSBOb3RPbk9yQWZ0ZXI9IjIw
MjQtMDEtMThUMDY6MjE6NDhaIiBSZWNpcGllbnQ9Imh0dHA6Ly9zcC5leGFtcGxlLmNvbS9kZW1vMS9pbmRleC5waHA%2fYWNzI
iBJblJlc3BvbnNlVG89Ik9ORUxPR0lOXzRmZWUzYjA0NjM5NWM0ZTc1MTAxMWU5N2Y4OTAwYjUyNzNkNTY2ODUiIC8%2bPC9zYW
1sOlN1YmplY3RDb25maXJtYXRpb24%2bPC9zYW1sOlN1YmplY3Q%2bPHNhbWw6Q29uZGl0aW9ucyBOb3RCZWZvcmU9IjIwMTQtM
DctMTdUMDE6MDE6MThaIiBOb3RPbk9yQWZ0ZXI9IjIwMjQtMDEtMThUMDY6MjE6NDhaIj48c2FtbDpBdWRpZW5jZVJlc3RyaWN0
aW9uPjxzYW1sOkF1ZGllbmNlPmh0dHA6Ly9zcC5leGFtcGxlLmNvbS9kZW1vMS9tZXRhZGF0YS5waHA8L3NhbWw6QXVkaWVuY2U
%2bPC9zYW1sOkF1ZGllbmNlUmVzdHJpY3Rpb24%2bPC9zYW1sOkNvbmRpdGlvbnM%2bPHNhbWw6QXV0aG5TdGF0ZW1lbnQgQXV0
aG5JbnN0YW50PSIyMDE0LTA3LTE3VDAxOjAxOjQ4WiIgU2Vzc2lvbk5vdE9uT3JBZnRlcj0iMjAyNC0wNy0xN1QwOTowMTo0OFo
iIFNlc3Npb25JbmRleD0iX2JlOTk2N2FiZDkwNGRkY2FlM2MwZWI0MTg5YWRiZTNmNzFlMzI3Y2Y5MyI%2bPHNhbWw6QXV0aG5D
b250ZXh0PjxzYW1sOkF1dGhuQ29udGV4dENsYXNzUmVmPnVybjpvYXNpczpuYW1lczp0YzpTQU1MOjIuMDphYzpjbGFzc2VzOlB
hc3N3b3JkPC9zYW1sOkF1dGhuQ29udGV4dENsYXNzUmVmPjwvc2FtbDpBdXRobkNvbnRleHQ%2bPC9zYW1sOkF1dGhuU3RhdGVt
ZW50PjxzYW1sOkF0dHJpYnV0ZVN0YXRlbWVudD48c2FtbDpBdHRyaWJ1dGUgTmFtZT0idWlkIiBOYW1lRm9ybWF0PSJ1cm46b2F
zaXM6bmFtZXM6dGM6U0FNTDoyLjA6YXR0cm5hbWUtZm9ybWF0OmJhc2ljIj48c2FtbDpBdHRyaWJ1dGVWYWx1ZSB4c2k6dHlwZT
0ieHM6c3RyaW5nIj50ZXN0PC9zYW1sOkF0dHJpYnV0ZVZhbHVlPjwvc2FtbDpBdHRyaWJ1dGU%2bPHNhbWw6QXR0cmlidXRlIE5
hbWU9Im1haWwiIE5hbWVGb3JtYXQ9InVybjpvYXNpczpuYW1lczp0YzpTQU1MOjIuMDphdHRybmFtZS1mb3JtYXQ6YmFzaWMiPj
xzYW1sOkF0dHJpYnV0ZVZhbHVlIHhzaTp0eXBlPSJ4czpzdHJpbmciPnRlc3RAZXhhbXBsZS5jb208L3NhbWw6QXR0cmlidXRlV
mFsdWU%2bPC9zYW1sOkF0dHJpYnV0ZT48c2FtbDpBdHRyaWJ1dGUgTmFtZT0iZWR1UGVyc29uQWZmaWxpYXRpb24iIE5hbWVGb3
JtYXQ9InVybjpvYXNpczpuYW1lczp0YzpTQU1MOjIuMDphdHRybmFtZS1mb3JtYXQ6YmFzaWMiPjxzYW1sOkF0dHJpYnV0ZVZhb
HVlIHhzaTp0eXBlPSJ4czpzdHJpbmciPnVzZXJzPC9zYW1sOkF0dHJpYnV0ZVZhbHVlPjxzYW1sOkF0dHJpYnV0ZVZhbHVlIHhz
aTp0eXBlPSJ4czpzdHJpbmciPmV4YW1wbGVyb2xlMTwvc2FtbDpBdHRyaWJ1dGVWYWx1ZT48L3NhbWw6QXR0cmlidXRlPjwvc2F
tbDpBdHRyaWJ1dGVTdGF0ZW1lbnQ%2bPC9zYW1sOkFzc2VydGlvbj48L3NhbWxwOlJlc3BvbnNlPg%3d%3d

To decode this, we need to initially interpret this data as application/x-www-form-urlencoded data and for this we will use the HttpUtility.ParseQueryString available in System.Web.dll.

First off we need to load the System.Web assembly:

[System.Reflection.Assembly]::LoadWithPartialName("System.Web")

If it is successfull we will see output similar to the following:


GAC Version Location
--- ------- --------
True v4.0.30319 C:\WINDOWS\Microsoft.Net\assembly\GAC_64\System.Web\v4.0_4.0.0.0__b03f5f7f11d50a3a\System.Web.dll

Now let’s assign our form data to a variable and parse it:

$urlencodedformdata = @"
SAMLResponse=PHNhbWxwOlJlc3BvbnNlIHhtbG5zOnNhbWxwPSJ1cm46b2FzaXM6bmFtZXM6dGM6U0FNTDoyLjA6cHJvdG9jb2
wiIHhtbG5zOnNhbWw9InVybjpvYXNpczpuYW1lczp0YzpTQU1MOjIuMDphc3NlcnRpb24iIElEPSJfOGU4ZGM1ZjY5YTk4Y2M0Y
zFmZjM0MjdlNWNlMzQ2MDZmZDY3MmY5MWU2IiBWZXJzaW9uPSIyLjAiIElzc3VlSW5zdGFudD0iMjAxNC0wNy0xN1QwMTowMTo0
...................................................................................................
...................................................................................................
...................................................................................................
JtYXQ9InVybjpvYXNpczpuYW1lczp0YzpTQU1MOjIuMDphdHRybmFtZS1mb3JtYXQ6YmFzaWMiPjxzYW1sOkF0dHJpYnV0ZVZhb
HVlIHhzaTp0eXBlPSJ4czpzdHJpbmciPnVzZXJzPC9zYW1sOkF0dHJpYnV0ZVZhbHVlPjxzYW1sOkF0dHJpYnV0ZVZhbHVlIHhz
aTp0eXBlPSJ4czpzdHJpbmciPmV4YW1wbGVyb2xlMTwvc2FtbDpBdHRyaWJ1dGVWYWx1ZT48L3NhbWw6QXR0cmlidXRlPjwvc2F
tbDpBdHRyaWJ1dGVTdGF0ZW1lbnQ%2bPC9zYW1sOkFzc2VydGlvbj48L3NhbWxwOlJlc3BvbnNlPg%3d%3d
"@
$parsedformdata = [System.Web.HttpUtility]::ParseQueryString($urlencodedformdata)

If everything was successful we should get pure base64 encoded data as the value of the SAMLResponse entry in $parsedformdata

$parsedformdata["SAMLResponse"]
PHNhbWxwOlJlc3BvbnNlIHhtbG5zOnNhbWxwPSJ1cm46b2FzaXM6bmFtZXM6dGM6U0FNTDoyLjA6cHJvdG9jb2
......................................................................................
......................................................................................
......................................................................................
tbDpBdHRyaWJ1dGVTdGF0ZW1lbnQ+PC9zYW1sOkFzc2VydGlvbj48L3NhbWxwOlJlc3BvbnNlPg==

At this point we can get the XML by base64 decoding the data and interpreting it as a UTF8 string:

$base64encodedxml = $parsedformdata["SAMLResponse"]
$xml = [System.Text.Encoding]::UTF8.GetString([Convert]::FromBase64String($base64encodedxml))

At this point $xml should look like the following:

<samlp:Response xmlns:samlp="urn:oasis:names:tc:SAML:2.0:protocol" xmlns:saml="urn:oasis:names:tc:SAML:2.0:assertion" ID="_8e8dc5f69a98cc4c1ff3427e5ce34606fd672f91e6" Version="2.0" IssueInstant="2014-07-17T01:01:48Z" Destination="http://sp.example.com/demo1/index.php?acs" InResponseTo="ONELOGIN_4fee3b046395c4e751011e97f8900b5273d56685"><saml:Issuer>http://idp.example.com/metadata.php</saml:Issuer><samlp:Status><samlp:StatusCode Value="urn:oasis:names:tc:SAML:2.0:status:Success" /></samlp:Status><saml:Assertion xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:xs="http://www.w3.org/2001/XMLSchema" ID="_d71a3a8e9fcc45c9e9d248ef7049393fc8f04e5f75" Version="2.0" IssueInstant="2014-07-17T01:01:48Z"><saml:Issuer>http://idp.example.com/metadata.php</saml:Issuer><saml:Subject><saml:NameID SPNameQualifier="http://sp.example.com/demo1/metadata.php" Format="urn:oasis:names:tc:SAML:2.0:nameid-format:transient">_ce3d2948b4cf20146dee0a0b3dd6f69b6cf86f62d7</saml:NameID><saml:SubjectConfirmation Method="urn:oasis:names:tc:SAML:2.0:cm:bearer"><saml:SubjectConfirmationData NotOnOrAfter="2024-01-18T06:21:48Z" Recipient="http://sp.example.com/demo1/index.php?acs" InResponseTo="ONELOGIN_4fee3b046395c4e751011e97f8900b5273d56685" /></saml:SubjectConfirmation></saml:Subject><saml:Conditions NotBefore="2014-07-17T01:01:18Z" NotOnOrAfter="2024-01-18T06:21:48Z"><saml:AudienceRestriction><saml:Audience>http://sp.example.com/demo1/metadata.php</saml:Audience></saml:AudienceRestriction></saml:Conditions><saml:AuthnStatement AuthnInstant="2014-07-17T01:01:48Z" SessionNotOnOrAfter="2024-07-17T09:01:48Z" SessionIndex="_be9967abd904ddcae3c0eb4189adbe3f71e327cf93"><saml:AuthnContext><saml:AuthnContextClassRef>urn:oasis:names:tc:SAML:2.0:ac:classes:Password</saml:AuthnContextClassRef></saml:AuthnContext></saml:AuthnStatement><saml:AttributeStatement><saml:Attribute Name="uid" NameFormat="urn:oasis:names:tc:SAML:2.0:attrname-format:basic"><saml:AttributeValue xsi:type="xs:string">test</saml:AttributeValue></saml:Attribute><saml:Attribute Name="mail" NameFormat="urn:oasis:names:tc:SAML:2.0:attrname-format:basic"><saml:AttributeValue xsi:type="xs:string">[email protected]</saml:AttributeValue></saml:Attribute><saml:Attribute Name="eduPersonAffiliation" NameFormat="urn:oasis:names:tc:SAML:2.0:attrname-format:basic"><saml:AttributeValue xsi:type="xs:string">users</saml:AttributeValue><saml:AttributeValue xsi:type="xs:string">examplerole1</saml:AttributeValue></saml:Attribute></saml:AttributeStatement></saml:Assertion></samlp:Response>

Now we can use the XmlTextWriter class to indent and format the XML for us.

Jeffrey Snover has already given a sample snippet on how this can be done over at the PowerShell Team Blog. I have repeated their code here:

function Format-XML ($xml, $indent=2) 
{ 
    $StringWriter = New-Object System.IO.StringWriter 
    $XmlWriter = New-Object System.XMl.XmlTextWriter $StringWriter 
    $xmlWriter.Formatting = “indented” 
    $xmlWriter.Indentation = $Indent 
    $xml.WriteContentTo($XmlWriter) 
    $XmlWriter.Flush() 
    $StringWriter.Flush() 
    Write-Output $StringWriter.ToString() 
}

We finally use this function to format our XML data nicely on screen:

Format-Xml -xml $xml -indent 4

Our final output should look like this:

<samlp:Response xmlns:samlp="urn:oasis:names:tc:SAML:2.0:protocol" xmlns:saml="urn:oasis:names:tc:SAML:2.0:assertion" ID="_8e8dc5f69a98cc4c1ff3427e5ce34606fd672f91e6" Version="2.0" IssueInstant="2014-07-17T01:01:48Z" Destination="http://sp.example.com/demo1/index.php?acs" InResponseTo="ONELOGIN_4fee3b046395c4e751011e97f8900b5273d56685">
    <saml:Issuer>http://idp.example.com/metadata.php</saml:Issuer>
    <samlp:Status>
        <samlp:StatusCode Value="urn:oasis:names:tc:SAML:2.0:status:Success" />
    </samlp:Status>
    <saml:Assertion xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:xs="http://www.w3.org/2001/XMLSchema" ID="_d71a3a8e9fcc45c9e9d248ef7049393fc8f04e5f75" Version="2.0" IssueInstant="2014-07-17T01:01:48Z">
        <saml:Issuer>http://idp.example.com/metadata.php</saml:Issuer>
        <saml:Subject>
            <saml:NameID SPNameQualifier="http://sp.example.com/demo1/metadata.php" Format="urn:oasis:names:tc:SAML:2.0:nameid-format:transient">_ce3d2948b4cf20146dee0a0b3dd6f69b6cf86f62d7</saml:NameID>
            <saml:SubjectConfirmation Method="urn:oasis:names:tc:SAML:2.0:cm:bearer">
                <saml:SubjectConfirmationData NotOnOrAfter="2024-01-18T06:21:48Z" Recipient="http://sp.example.com/demo1/index.php?acs" InResponseTo="ONELOGIN_4fee3b046395c4e751011e97f8900b5273d56685" />
            </saml:SubjectConfirmation>
        </saml:Subject>
        <saml:Conditions NotBefore="2014-07-17T01:01:18Z" NotOnOrAfter="2024-01-18T06:21:48Z">
            <saml:AudienceRestriction>
                <saml:Audience>http://sp.example.com/demo1/metadata.php</saml:Audience>
            </saml:AudienceRestriction>
        </saml:Conditions>
        <saml:AuthnStatement AuthnInstant="2014-07-17T01:01:48Z" SessionNotOnOrAfter="2024-07-17T09:01:48Z" SessionIndex="_be9967abd904ddcae3c0eb4189adbe3f71e327cf93">
            <saml:AuthnContext>
                <saml:AuthnContextClassRef>urn:oasis:names:tc:SAML:2.0:ac:classes:Password</saml:AuthnContextClassRef>
            </saml:AuthnContext>
        </saml:AuthnStatement>
        <saml:AttributeStatement>
            <saml:Attribute Name="uid" NameFormat="urn:oasis:names:tc:SAML:2.0:attrname-format:basic">
                <saml:AttributeValue xsi:type="xs:string">test</saml:AttributeValue>
            </saml:Attribute>
            <saml:Attribute Name="mail" NameFormat="urn:oasis:names:tc:SAML:2.0:attrname-format:basic">
                <saml:AttributeValue xsi:type="xs:string">[email protected]</saml:AttributeValue>
            </saml:Attribute>
            <saml:Attribute Name="eduPersonAffiliation" NameFormat="urn:oasis:names:tc:SAML:2.0:attrname-format:basic">
                <saml:AttributeValue xsi:type="xs:string">users</saml:AttributeValue>
                <saml:AttributeValue xsi:type="xs:string">examplerole1</saml:AttributeValue>
            </saml:Attribute>
        </saml:AttributeStatement>
    </saml:Assertion>
</samlp:Response>

This is a lot more readable than what we had to begin with. Hopefully if you ever find yourself in this situation you will remember that PowerShell can come to your rescue.

10 Nov

Resize the OS Disk in Resource Manager Azure Deployments using PowerShell

Recently I’ve had to increase the OS Disk size of about 10 Ubuntu virtual machines running on Microsoft Azure. Looking to the future I can also foresee this happening quite a few times so I thought I would write a script that could do this for me. The script essentially shuts the virtual machine down (after prompting you for confirmation), checks the new size is larger than the old size (an Azure requirement), resizes the disk to the new size and then starts the virtual machine back up.

There is a small quirk in the process that you will sometimes notice if you ever use the web UI to resize the OS disk. Azure sometimes doesn’t report the size of the disk correctly and so you don’t always know what the new size should be. To get around this I previously wrote a script (see here) which can get the disk size in an indirect way. Make sure you grab the retrieve size function from the linked post.

An example usage is the following:

Set-AzureRmOSDiskSize -VirtualMachineName "APP01" -ResourceGroupName "Platform-Dev" -SizeInGB 40

Here is the code below:

function Ask-ToContinue {

    param(
        [string]$message = $(throw "specify message"),
        [string]$prompt = $(throw "specify prompt")
    )

    $choices = New-Object Collections.ObjectModel.Collection[Management.Automation.Host.ChoiceDescription]
    $choices.Add((New-Object Management.Automation.Host.ChoiceDescription -ArgumentList '&Yes'))
    $choices.Add((New-Object Management.Automation.Host.ChoiceDescription -ArgumentList '&No'))

    $decision = $Host.UI.PromptForChoice($message, $prompt, $choices, 1)
    
    return $decision -eq 0

}

function Set-AzureRmOSDiskSize () {

    param(
        [string]$VirtualMachineName = $(throw "Specify virtual machine name"),
        [string]$ResourceGroupName = $(throw "Specify resource group name"),
        [int]$SizeInGB = (throw "Specify Size in GB")
    )

    $currentSize = Get-AzureRmOSDiskSize -VirtualMachineName $VirtualMachineName -ResourceGroupName $ResourceGroupName

    if ($currentSize -ne $null -and $SizeInGB -le $currentSize) {

        throw "Specified Disk Size is not larger than current size"

    }

    $VM = Get-AzureRmVM -ResourceGroupName $ResourceGroupName -Name $VirtualMachineName -Status

    if ($VM -eq $null) {

        throw "Virtual Machine not found"

    }

    $VMRunning = ($VM.Statuses | ? { $_.Code -eq "PowerState/running" } | Measure-Object | Select -ExpandProperty Count) -eq 1

    if ($VMRunning) {

        Write-Host "The VM is currently running." -ForegroundColor Magenta

        $stopTheVM = Ask-ToContinue -message "The VM must be stopped" -prompt "Would you like to stop the VM now?"

        if ($stopTheVM) {

            Write-Host -ForegroundColor Yellow "Stopping the VM"
            Stop-AzureRmVM -Name $VirtualMachineName -ResourceGroupName $ResourceGroupName -Force

        } else {

            Write-Host -ForegroundColor Cyan "Not stopping the VM."
            return

        }

    }

    $VM = Get-AzureRmVM -ResourceGroupName $ResourceGroupName -Name $VirtualMachineName
    $VM.StorageProfile.OsDisk.DiskSizeGB = $SizeInGB

    $result = Update-AzureRmVM -VM $VM -ResourceGroupName $ResourceGroupName

    if ($result.IsSuccessStatusCode) {
        
        Write-Host -ForegroundColor Green "Updated VM Successfully"

    }

    $startTheVM = Ask-ToContinue -message "The VM is currently stopped" -prompt "Would you like to start the VM now?"

    if ($startTheVM) {

        Write-Host -ForegroundColor Yellow "Starting the VM"
        Start-AzureRmVM -Name $VirtualMachineName -ResourceGroupName $ResourceGroupName

    }

}
20 Sep

Testing TCP connections with PowerShell

I’ve been in the situation where I have needed to test whether I could make a TCP connection from one Windows host to another Windows host to verify that a network team had indeed opened firewall ports. It seems like a trivial thing to do. Just connect from host A to host B on the specified port. What programs can we use to do this? It’s a bit overkill to install a whole piece of server and client software to test this. Let alone read any necessary documentation to configure the correct port. It gets worse if things don’t work as you still don’t know whether it’s the firewall or your configuration!

Linux users could just install Netcat on both hosts and check this in a few seconds. Window users still have the ability to install some networking utilities similar to Netcat but I find them overly complicated considering 99% of the time I just want to know whether an intermediate firewall is blocking a connection.

PowerShell is so useful and gives you the full power of the .NET framework. This means we can create these utilities ourselves natively without installing any third party libraries.

I’ve created two functions, Listen-Tcp and Connect-Tcp, which have code listsings at the bottom of the post. The following is an example use of the utilities:

Listen-Tcp -port <Int32>
Connect-Tcp -hostname <string> -port <Int32>

Running the corresponding scripts on two hosts, hosta and hostb will give you the following outputs:

hosta > Listen-Tcp -port 3000
Listening on port 3000
Stopped Listening
hostb > Connect-Tcp -hostname "hosta" -port 3000
Data sent to and received from target successfully

For convenience I have added these functions to my PowerShell profile so that they are available on all servers I log into within the domain.

Below is the code listing

function Listen-Tcp()
{
	param(
		[Int32] $port
	)
	
	$server = New-Object -TypeName System.Net.Sockets.TcpListener -ArgumentList @([System.Net.IPAddress]::Any, $port)
	$server.Start()
	
	Write-Host ("Listening on port {0}" -f $port)
	$clientSocket = $server.AcceptSocket()
	
	$buffer = New-Object -TypeName byte[] -ArgumentList 4
	$clientSocket.Receive($buffer) | Out-Null
	
	$clientSocket.Send($buffer) | Out-Null
	$clientSocket.Close()
	
	$server.Stop()
	
	Write-Host "Stopped Listening"
}
function Connect-Tcp()
{
	param(
		[string]$hostname,
		[Int32]$port
	)
	
	try
	{
		$client = New-Object -TypeName System.Net.Sockets.TcpClient -ArgumentList $hostname,$port
		$stream = $client.GetStream()
		
		$buffer = [System.Text.Encoding]::ASCII.GetBytes("EHLO")
		$stream.Write($buffer, 0, $buffer.Length)
		
		$receiveBuffer = New-Object -TypeName byte[] -ArgumentList $buffer.Length
		$stream.Read($receiveBuffer, 0, $receiveBuffer.Length) | Out-Null
		
		$receivedText = [System.Text.Encoding]::ASCII.GetString($receiveBuffer)
		
		$stream.Close()
		$client.Close()
		
		if ($receivedText -eq "EHLO") {
			Write-Host "Data sent to and received from target successfully"
		} else {
			Write-Host "Data receieved was not as expected"
		}
	} Catch [Exception]
	{
		Write-Host "Could not connect to target machine"
	}
}
22 Aug

Splitting and Joining Files with PowerShell

Sometimes it is useful to be able to split large files into smaller chunks. This can be because the file is bigger than a file limit size for a particular communication or storage medium. There is plenty of software that will do just that. To name a few 7-zip, WinZip and WinRAR.

However as I usually have my PowerShell profile synced to all my machines it is an easy task to do in PowerShell. I wrote some PowerShell functions a while ago that split and joined files. Here are a few examples of how they should be used and then the code follows at the bottom:

Split-File -filename .\fileToSplit.dat -outprefix splitFilePrefix -splitSize 2M
Join-File -filename .\splitFilePrefix.001 -outfile CopyOfFileToSplit.dat

You can specify the split size using the suffixes K, M and G for kilobytes, megabytes and gigabytes respectively.

Note that the file locations are relative to the processes current working directory and not PowerShell’s current location. To avoid confusion and strange behaviour use absolute paths. If you want to understand more about the difference then I recommend you check out this blog which came out near the top when googling for an insightful link, http://www.beefycode.com/post/The-Difference-between-your-Current-Directory-and-your-Current-Location.aspx

Here are the functions below:

function Split-File()
{
	param
	(
		[string] $filename = $(throw "file required"),
		[string] $outprefix = $(throw "outprefix required"),
		[string] $splitSize = "50M",
		[switch] $Quiet
	)
	
	$match = [System.Text.RegularExpressions.Regex]::Match($splitSize, "^(\d+)([BKMGbkmg]?)$")
	[int64]$size = $match.Groups[1].Value
	$sizeUnit = $match.Groups[2].Value.ToUpper()
	$sizeUnitValue = 0
	switch($sizeUnit)
	{
		"K" { $sizeUnitValue = 1024 }
		"M" { $sizeUnitValue = 1048576 }
		"G" { $sizeUnitValue = 1073741824 }
		default { $sizeUnitValue = 1 }
	}
	
	$size = $sizeUnitValue * $size
	
	Write-Host ("Size Split is {0}" -f $size) -ForegroundColor Magenta
	
	$outFilePrefix = [System.IO.Path]::Combine((Get-Location).Path, $outprefix)
	
	$inFileName = [IO.Path]::Combine((Get-Location).Path,$filename)
	
	Write-Host ("Input File full path is {0}" -f $inFileName)
	
	if ([IO.File]::Exists($inFileName) -ne $true)
	{
		Write-Host ("{0} does not exist" -f $inFileName) -ForegroundColor Red
		return
	}
	
	$bufferSize = 1048576
	
	$ifs = [IO.File]::OpenRead($inFileName)
	$ofs = $null
	$buffer = New-Object -typeName byte[] -ArgumentList $bufferSize
	$outFileCounter = 0
	$bytesReadTotal = 0
	
	$bytesRead = 1 #Non zero starting number to ensure loop entry
	while ($bytesRead -gt 0)
	{
		$bytesToRead = [Math]::Min($size-$bytesReadTotal, $bufferSize)
		$bytesRead = $ifs.Read($buffer, 0, $bytesToRead)
		
		if ($bytesRead -ne 0)
		{		
			if ($ofs -eq $null)
			{
				$outFileCounter++
				$ofsName = ("{0}.{1:D3}" -f $outFilePrefix,$outFileCounter)
				$ofs = [IO.File]::OpenWrite($ofsName)
				if ($Quiet -ne $true)
				{
					Write-Host ("Created file {0}" -f $ofsName) -ForegroundColor Yellow
				}
			}
			
			$ofs.Write($buffer, 0, $bytesRead)
			$bytesReadTotal += $bytesRead
			
			if ($bytesReadTotal -ge $size)
			{
				$ofs.Close()
				$ofs.Dispose()
				$ofs = $null
				$bytesReadTotal = 0
			}
		}
	}
	
	if ($ofs -ne $null)
	{
		$ofs.Close()
		$ofs.Dispose()
	}
	
	Write-Host "Finished"
	
	$ifs.Close()
	$ifs.Dispose()
}

function Join-File()
{
	param
	(
		[string] $filename = $(throw "filename required"),
		[string] $outfile	= $(throw "out filename required")
	)
	
	$outfilename = [IO.Path]::Combine((Get-Location).Path, $outfile)
	$ofs = [IO.File]::OpenWrite($outfilename)
	
	$match = [System.text.RegularExpressions.Regex]::Match([IO.Path]::Combine((Get-Location).Path,$filename), "(.+)\.\d+$")
	if ($match.Success -ne $true)
	{
		Write-Host "Unrecognised filename format" -FroegroundColor Red
	}
	$fileprefix = $match.Groups[1].Value
	$filecounter = 1
	$bufferSize = 1048576
	$buffer = New-Object -TypeName byte[] -ArgumentList $bufferSize
	
	while ([IO.File]::Exists(("{0}.{1:D3}" -f $fileprefix,$filecounter)))
	{
		$ifs = [IO.File]::OpenRead(("{0}.{1:D3}" -f $fileprefix,$filecounter))
		
		$bytesRead = $ifs.Read($buffer, 0, $bufferSize)
		while ($bytesRead -gt 0)
		{
			$ofs.Write($buffer,0,$bytesRead)
			$bytesRead = $ifs.Read($buffer, 0, $bufferSize)
		}		
		
		$ifs.Close()
		$ifs.Dispose()
	
		$filecounter++
	}
	
	$ofs.Close()
	$ofs.Dispose()

	Write-Host ("{0} created" -f $outfilename) -ForegroundColor Yellow
}
07 Feb

Signing your PowerShell Scripts

At some point or another, we have all been in the situation, usually on a new machine, we get the familiar PSSecurityException.

.\xxx.ps1 : File C:\xxx.ps1 cannot be loaded. The file C:\xxx.ps1 is not digitally signed. You cannot run this
script on the current system. For more information about running scripts and setting execution policy, see
about_Execution_Policies at http://go.microsoft.com/fwlink/?LinkID=135170.
At line:1 char:1
+ .\xxx.ps1
+ ~~~~~~~~~~
    + CategoryInfo          : SecurityError: (:) [], PSSecurityException
    + FullyQualifiedErrorId : UnauthorizedAccess

Of course we know this is down to PowerShell Policy. The behaviour of PowerShell when it executes a script can be configured to one of the following:

  • Restricted – This is the most secure but the least versatile. It does not allow any scripts to be run.
  • AllSigned – This allows scripts that have been digitally signed by a certificate that the computer trusts to run.
  • Unrestricted – This is the least secure. It allows all scripts regardless of origin to run.
  • RemoteSigned – This is a step up from Unrestricted. Scripts that have been downloaded are not allowed to run unless they are digitally signed by a certificate that the computer trusts.

The default policy depends on a number of things including group policy and which operating system you are using. A lot of people get around this by setting the execution policy to Unrestricted.

Set-ExecutionPolicy Unrestricted

I however do not like this as it leaves your system open to abuse.

RemoteSigned should be the minimum you ever set this policy to. Even with RemoteSigned you may find you still get the above error when your script is located on a network drive. In this case rather than changing the execution policy to Unrestricted you should change your internet options so the server that the file share is hosted on is trusted. To do this you should go to Control Panel and Internet Options to open the Internet Options dialog:

eecef1e1-3f9f-47fa-b0d5-c61b61b910cf_01

Under Internet Options select the Security tab and then the Local Intranet zone. Once selected click Sites then Advanced. Add the server to the list in the form file://<servername> (untick Require server verification). Once this has been updated restart any PowerShell windows and try executing your script again on a network share and you should find they run.

However I think the best approach is to sign your scripts. The argument against this is that it’s a “complicated” process involving certificates which people seem to find hard anyway.

It is true that you will need to get a code signing certificate and if your IT Support has set things up correctly you should be able to get a code signing certificate in about 30 seconds (see Create a Code Signing Certificate in 30 Seconds). Sometimes your IT Support will not have configured your infrastructure for maximum usability and security and so you may need to expend a bit more effort, however this is an infrequent process with big advantages for security.

If you have a code signing certificate then signing your scripts is incredibly easy. Just substitute the certificate thumbprint with your own:

$cert = Get-ChildItem cert:\CurrentUser\My\FACE9812CAFE7634BABE54561A2B3C4D5E6FDEAD
Set-AuthenticodeSignature -Certificate $cert -FilePath X:\path_to_script.ps1

This will add a block like the following to the end of your script:

# SIG # Begin signature block
# MIIInAYJKoZIhvcNAQcCoIIIjTCCCIkCAQExCzAJBgUrDgMCGgUAMGkGCisGAQQB
......................
# NNDblcNY9HDwe+tXuDRtu3YLoBWBq4xasQTml46HHtEd1z6L+qTDi1gwbv/dFmIB
# SIG # End signature block

This is the digital signature and will be what PowerShell verifies when trying to run a script. See how easy that was?

Top tip

My home directory is hosted on a network server and so my PowerShell profile file is hosted on the network share. This means by default, without signing my PowerShell profile, every time I load PowerShell I will get a message:

. : File \\domain.com\FILES\USERS\my.username\My Documents\WindowsPowerShell\Microsoft.PowerShell_profile.ps1
cannot be loaded. The file \\domain.com\FILES\USERS\my.username\My
Documents\WindowsPowerShell\Microsoft.PowerShell_profile.ps1 is not digitally signed. You cannot run this script on
the current system. For more information about running scripts and setting execution policy, see
about_Execution_Policies at http://go.microsoft.com/fwlink/?LinkID=135170.
At line:1 char:3
+ . '\\domain.com\FILES\USERS\my.username\My Documents\WindowsPowerShell\Micr ...
+ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
    + CategoryInfo          : SecurityError: (:) [], PSSecurityException
    + FullyQualifiedErrorId : UnauthorizedAccess

It was easy enough to sign my profile file:

$cert = Get-ChildItem cert:\CurrentUser\My\FACE9812CAFE7634BABE54561A2B3C4D5E6FDEAD
Set-AuthenticodeSignature -Certificate $cert -FilePath $profile

The problem is that I frequently add, remove or edit things in my profile which usually leads to the following error message:

. : File \\domain.com\FILES\USERS\my.username\My Documents\WindowsPowerShell\Microsoft.PowerShell_profile.ps1
cannot be loaded. The contents of file \\domain.com\FILES\USERS\my.username\My
Documents\WindowsPowerShell\Microsoft.PowerShell_profile.ps1 might have been changed by an unauthorized user or
process, because the hash of the file does not match the hash stored in the digital signature. The script cannot run
on the specified system. For more information, run Get-Help about_Signing..
At line:1 char:3
+ . '\\domain.com\FILES\USERS\my.username\My Documents\WindowsPowerShell\Micr ...
+ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
    + CategoryInfo          : SecurityError: (:) [], PSSecurityException
    + FullyQualifiedErrorId : UnauthorizedAccess

This is due to the file being changed so the digital signature is no longer valid. It’s easy enough to correct by signing the file again:

$cert = Get-ChildItem cert:\CurrentUser\My\FACE9812CAFE7634BABE54561A2B3C4D5E6FDEAD
Set-AuthenticodeSignature -Certificate $cert -FilePath $profile

but I find this tedious as looking up my thumbprint and then typing the path is more effort than I’m willing to spend. I always usually have a PowerShell window open even when I am editing my profile and so to save me some time I have defined the following function in my profile:

function Update-ProfileSignature()
{
	$signingCertificateThumbprint = (Get-AuthenticodeSignature $profile).SignerCertificate.Thumbprint	
	$codeSigningCertificate = Get-ChildItem -Recurse Cert:\ | ? { $_.Thumbprint -eq $signingCertificateThumbprint -and $_.HasPrivateKey } | Select -First 1
	Set-AuthenticodeSignature -FilePath $profile -Certificate $codeSigningCertificate | Out-Null

	Write-Host "Updated signature on profile" -ForegroundColor Green
}

Everytime I edit my profile, in an already open PowerShell window, I execute the function Update-ProfileSignature which re-signs my profile file.

It’s important that a PowerShell window is already open (and therefore has the Update-ProfileSignature function loaded and defined) as once you edit your profile you cannot load your profile without error into any new PowerShell windows until it is re-signed and therefore you cannot call Update-ProfileSignature until it is re-signed, which would make it useless.