A high level overview #Meltdown

Leaking data from the core of the operating system to an unprivileged program

The system running on your computer consists of two parts, the operating system (Windows, Linux, Android, MacOS) and the programs you run (Internet Explorer, Firefox, Photoshop, Warcraft, Chrome, Safari, Steam)

Although it’s not easy for the end user to tell the difference apart from many of the built in applications from the core of the operating system but if you interact with it IS an application and this is known as USER SPACE.

That underlying core, known as THE KERNEL, acts as a broker between USER SPACE (the applications) and the actual hardware of the computer. That’s the DISK, NETWORK, WiFi, SCREEN, KEYBOARD, MOUSE etc

In order to keep the KERNEL separate from USER SPACE computer processors (this is the CPU and is a big piece of electronics in the heart of your computer) are designed to have several levels of operation called RINGs. The most important of these are RING0 which is where the KERNEL is supposed to run and RING3 which is where USER SPACE is supposed to run.

In order to improve the speed of your home computer the manufactures have used various methods of trying to predict what program code is going to run next. Unfortunately the techniques used have flaws and it is possible for someone to write a simple program which runs in USER SPACE, this can even run in your web browser, which would allow the owner of a website to access passwords, cryptographic keys (PGP etc), bitcoin, your thumb print or face ID, your bank authentication, etc in the KERNEL.

The software updates being issued by Microsoft, Apple, Linux, etc are effectively working around a fundamental problem with the design of the electronics in computer chips and until the manufacturers resolve these they will exist for years  if not decades to come.


Enumerating NTFS ACE/ACL settings

When dealing with large file stores you occasionally have to review and audit what has been configured (in the future GDPR has the potential to make this compulsory).

There’s some lovely commercial tools out there to do this but if you don’t have access to these then you have to rely on the native Microsoft toolkit.

Most competent Microsoft sysadmins will be aware of cacls.exe and the many wrappers MS provide for it, the most recent being icacls. This is an excellent tool for quickly enumerating, backing up, and changing ACLs but when you hit an audit request to identify the individuals who can access data then cacls will only give you the directly assigned members and the groups.

This is where a little bit of Powershell comes into play as Microsoft have the wonderful Get-Acl, Get-ADGroupMember, and Get-ADUser commandlets.

I’ve taken the framework from this TechNet article to construct (with some suggestions and collaborations from colleagues and Google) the following script which can be run by anyone with read access to a folder structure to enumerate who else has access and what type they have. It’s nowhere near perfect as it doesn’t recurse down through multiple groups and the output isn’t pretty but it works, and it gets an auditor off your back quickly!

#ACE/ACL Enumerator
Param (
#acl_enum.ps1 Folderpath -detailed -recurse
if ($recurse)
       $tree = Get-ChildItem $folderpath -Directory -Recurse
       $tree = Get-ChildItem $folderpath -Directory
foreach ($dir in $tree)
Write-Output "****************************"
Write-Output $dir.FullName
Write-Output "****************************"
       $acls = (Get-ACL $dir.FullName).Access
       foreach ($ace in $acls)
                 if ($detailed) {
                           Write-Output "++++++++++++++++++++++++++++"
                           Get-ADGroupMember $ace.identityreference.tostring().split("\")[1]
                           Get-ADUser $ace.identityreference.tostring().split("\")[1]
                           Write-Output "++++++++++++++++++++++++++++"
                    else {
                           $ace.IdentityReference, $ace.FileSystemRights



ACE is an Access Control Entry, each individual user or group assigned to a folder is an Access Control Entry

ACL is an Access Control List, a list of ACE’s



Draw AD – Visualizing Active Directory Groups

Graph AD Groups \

A bended knee to the author of this script. This is an excellent work but should not be needed!

If only Microsoft had thought to provide a tool set to visualise how Active Directory groups interact. Using ADUC is painful with nested groups and why should we have to create scripts in the ever changing Powershell to track how user accounts arrive at their priviledges.

Tech fails in action – Rsync misuse

Take a scenario of having to synchronise several webservers with identical content, rsync would seem a reasonable choice.It just comes down to how you implement it.

What you don’t do is run rsync pull via cron, with two similar jobs, both writing to the same log file (overwring the same log file whilst dutifully sending stderr there as well), running under a normal user whilst trying to use the root only -o option.

Other things you don’t do is to modify a production web server and the let the replicas PULL the configuration to them using the aforementioned rsync/crontab. This needs a Picard facepalm!

“Well it used to work before!”, was that before you weren’t allowed to run it as root maybe? Guess the answer. You ran it as ROOT??!!! So any compromise would have been replicated as root and you’d have no log files to trace what might have been.

“So what’s the solution?” Well getting a proper deployment system in place that pushes out configurations from a seperate deployment server.

To spell it out:

  • Legacy use of root needs to be weeded out and exterminated
  • Configuration servers should ALLWAYS exist outside of end user access
  • Always PUSH configuration from the configuration server
  • All logs go to Syslog, never ever write your own
  • Any requirements to vary the deployment from this get rejected

Virtualised MSCS (VMWare)

This relates to vSphere 5.5

Below vSphere 6 MSCS isn’t supported as a guest but even with vS6 it’s only supported with RDMs (Raw Device Mappings).

However you can build a cluster providing you remember the following rules:

  • You CANNOT use vMotion
    • this will create duplicate blanks of the shared disks
    • recovering from this requires deleting the duplicates and then manually re adding the shared disks from within the Virtual Machine configuration
  • the boot and cluster disks MUST be on different SCSI controllers
    • The scsi controller must be ParaVirtual
    • the shared disks MUST be thick provisioned (eager zeroed thick)
  • The cluster validation tool will always show warnings about disk and network but as long as the ERROR messages are cleared it will work
  • Resizing a cluster shared disk reverts it to lazy zeroed thick and you have to ‘eager’ it from the ESX cli as it’s not possible via the GUI.
    • From the correct foler in the correct datastore run
      • vmkfstools –eagerzero nameofvmdk.vmdk

Windows Server Failover Clusters – Removing Mount Point Disks

Really why, in all that is within the realms of common sense, does MS CS default to removing all disk resources from a group if you do not offline the resource beforehand?

Surely just blocking the option would be easier?

The technet article is not the first that appears when you research removing disks from a cluster or even the second  or third or fourth (you have to search on removing disks from the group) … although it may now be moving it up the search ratings.

The crux of the article is:

“Now, we get to the main point of this post. The above process is fine for deleting resources from a cluster group unless the resource you are deleting is configured as a “Physical Disk’ resource, and it a mount point disk. The process differs slightly and you must follow this process or you could find yourself unintentionally moving every resource in the group into ‘Available Storage’.”

Yes the online storage is linked to the cluster, yes the cluster will fail if an online resource is removed, but why remove all the mountpoints from the group rather than just denying the action!

And there’s an added gotcha, occasionally it will not just move the disks out of the group, it will online them on the other cluster node. Which leaves you in the state of having the mountpoints on one node and the named disks and cluster IP(s) on the other.

In summary and to avoid that classic head meets desk moment, in order to remove a mount point volume from an MSCS group you have to:

  • offline the disk
  • remove it from the group
  • delete the resource
  • delete the SAN storage volume
  • rescan both nodes