Convert Little-endian UTF-16 to ASCII

hacker screen
Photo by Markus Spiske on

I generated some text files working with Get-Acl Powershell, but I did not know how to get Powershell to do some advanced features. (Basically, I wanted to the Select-String to include the next 2 lines and see whether a specific group was in that list. And maybe some exclusions.) So, I copied the files over to my Linux home to check there.

The basic most grep? Nothing.

I used ls -l and confirmed they have data. I used less to confirm I can see it.

I copied a string and did a grep for it. Nothing.

I did a dos2unix. That didn’t fix it. Finally, I did:

file filename.txt

That revealed the files had types of:

  1. Original: Little-endian UTF-16 Unicode text, with CRLF line terminators
  2. dos2unix converted: Little-endian UTF-16 Unicode text

Basically, this told me that the dos2unix fixed one problem but not both. The “with CRLF line terminators” means that Windows and Unix have philosophical differences in how to format text lines.

Little-endian is a geeky homage to Gulliver’s travels. It has to do with which direction one encodes the bits. But, it isn’t really the big problem here. UTF-16 is the problem because apparently, I need it to be UTF-8 for grep to read it. So, the fix is to use an encoding converting:

iconv -f utf-16 -t utf-8 filename.txt > filename_new.txt

List AD Group Members

Team lead of a group was curious which of his people have access to a certain system. I know the information is in an Active Directory Group. I knew where to find the group.

So, I pulled up the AD Users and Computer, found the group and was dismayed because I was looking at three screenshots to get the information because the Properties box is not expandable by size. That bothered me quite a bit. See, I would live with one

Enough so, that I decided it would be worth writing a Powershell script to dump the data. I am sure other coworkers have one. But, I did a quick Google and found something that looked way too easy. The last time I tried to do this, I had to script connecting to the AD server, searching through the forest, etc. This was one command. I added a couple options to make it better presentable.

Get-ADGroupMember <group_name> | Select-Object -exp name | Sort-Object


Search Within Files

I keep some logs in a directory just in case I need to reference them later. The kind of data that has saved my bacon on a handful to times. Of course, it has over 3,000 files (16GB) in the directory. Of those less than a hundred were potentially relevant. And in the end only a couple dozen had the data I sought.

Windows Explorer used to make this easy to search. I could put in the pattern I wanted and tell it to search the text within them. It would give me the file name for each containing the search string. For whatever reason Windows 7 had to make it more difficult.

So, I wrote the easiest of Powershell scripts:

Get-Content $filelist | Select-String -pattern “Search String

Good thing too because apparently I need to to go through my Indexing Options and identify every file extension I want to search to index file contents. What a royal pain. My guess is doing so would also blow up the EDB data file from its currently 2GB to something way larger. 10GB? 50? 100? Yuck.


Counting Folders

Back in June, after a whole seven months in our system, one of our clients hit Microsoft’s limit in the max number of folders within a folder. They are on track to do it again a year later.

I lost the first script I wrote to figure out which directories were affected, so I wrote a new one yesterday. The problem directories are all in the same path with the schoolcode\courses as the name. A configuration for the name of the folder to use for the school has the application create new folders for each new course at this location. When this folder fills up, the solution is to add a number to that name so a new folder is created and subsequent courses put there. So I looked for all directories fitting this pattern. Then I loop through the list and count the number of objects in each directory and post to the output this count.

$logdate = Get-Date -uformat "%Y%m%d%H%M"
$sites = 'nas1 nas2'
# Loop through each NAS
foreach ($site in $sites)

# Compile the list of directories for NAS
$dirlist = Get-ChildItem \\$site\share\path\to\client_stuff\*\courses*
# Loop through found directories
foreach ($diritem in $dirlist)

# Count
$getcount = (Get-ChildItem $diritem).Count
# Post to output
Add-Content \\nas1\share\to\scripts\LOGS\count_directories_$logdate.log "$diritem  ==  $getcount"



Obviously, I do not really want to run this script every term or even six months. So I really need to design a check for our monitor service. It might report any who cross over 50,000. That would give us a 15,533 cushion to have configuration changed to start using a new folder. The correct threshold is hard to choose. As is, it is slightly bigger than the largest term for our largest client. It would ensure no surprises. With something like 5,000 it is possible for us to get no warning one day, then the start of a new term hit the limit when the client sent 14,795 courses for the Fall 2013 term. But really only that one school has this problem. The next two largest clients had 8,246 and 6,364 courses for Fall 2013. (Combined almost the same as the biggest.)

Really, though, the problem with a check is Powershell takes 2 hours to count this amount of stuff. So probably I need to record this data to a database and have the monitor check this database.

BOF Windows Powershell #USGRockEagle13

Bird of a Feather so open discussion. Intro question:

  • How are you using it?
  • How do you like it?
  • Good sources of documentation? Information?
  • Tools/Modules used? e.g. editors, PowerTab module, ISE ScriptingGuy, Codeplex, etc/
  • Using Powershell: User-written functions, cmdlets, modules
  • Version 3? Version 4?


  • Passing around various books.
  • Manning occasionally has half off deals on ebooks. (True. Got my Powershell 2 book on such a deal.)
  • Listserv: ga-powershell at
  • Scripting works great for managing large number of virtual machines.

Detecting .NET Version

Background: We got a notice a few minutes before last week’s maintenance window that side work to configure a tool ran into a snap. Today we found out that the server intended to be used did not have the right .NET version. Why did they not just install it? Good question.

This made me curious about which of the 114 servers have which version. So I found a Stackflow post on identifying .NET version. I then wrote this Powershell wrapper script around one of the solutions. The one I picked looks at the registry keys.

# Get my list of servers
$computers = Get-Content ‘server_list.txt’

# Contact each server in the list in order before moving on to next
foreach ($computer in $computers)
# Remote connect to current server
Invoke-Command -computername $computer -scriptblock {
# ID host name
$remotehostname = c:\windows\system32\hostname.exe
# Find largest .NET version
$dotnet = Get-ChildItem ‘HKLM:\SOFTWARE\Microsoft\NET Framework Setup\NDP’ | sort pschildname -des | select -fi 1 -exp pschildname
# Write hostname and .NET version
Write-Host “$remotehostname == $dotnet”

The foreach loop essentially forces the Invoke-Command to do the test in order. This is slower as it contacts the remote server, does the commands, then closes the connection. Without it, servers would respond at about the same time and the output reside on the same line and out of order.

Probably there are better ways to do this, but I am just a beginner.

Also on the Stackflow page there are better approaches. Because uninstalling .NET does not remove the registry keys, this method is vulnerable to be incorrect. I guess that is a possibility.

Oh, on which have what: 22 have .NET2, 48 have .NET3.5, and 44 have .NET4. Not consistently obvious factors like version of the application, life cycle tiers, or anything obvious explain why which have what. Though, they do tend to be the same version within the same cluster of servers.