{"id":9154,"date":"2019-02-27T17:23:26","date_gmt":"2019-02-27T22:23:26","guid":{"rendered":"https:\/\/www.ezrasf.com\/wplog\/?p=9154"},"modified":"2019-02-27T16:29:29","modified_gmt":"2019-02-27T21:29:29","slug":"convert-little-endian-utf-16-to-ascii","status":"publish","type":"post","link":"https:\/\/www.ezrasf.com\/wplog\/2019\/02\/27\/convert-little-endian-utf-16-to-ascii\/","title":{"rendered":"Convert Little-endian UTF-16 to ASCII"},"content":{"rendered":"
\"hacker
Photo by Markus Spiske temporausch.com on Pexels.com<\/a><\/figcaption><\/figure>\n

I generated some text files working with Get-Acl Powershell, but I did not know how to get Powershell to do some advanced features. (Basically, I wanted to the Select-String to include the next 2 lines and see whether a specific group was in that list. And maybe some exclusions.) So, I copied the files over to my Linux home to check there.<\/p>\n

The basic most grep? Nothing.<\/p>\n

I used ls -l and confirmed they have data. I used less to confirm I can see it.<\/p>\n

I copied a string and did a grep for it. Nothing.<\/p>\n

I did a dos2unix. That didn’t fix it. Finally, I did:<\/p>\n

file filename.txt<\/pre>\n
That revealed the files had types of:<\/p>\n
    \n
  1. Original: Little-endian UTF-16 Unicode text, with CRLF line terminators<\/li>\n
  2. dos2unix converted: Little-endian UTF-16 Unicode text<\/li>\n<\/ol>\n
    Basically, this told me that the dos2unix fixed one problem but not both. The “with CRLF line terminators” means that Windows and Unix have philosophical differences in how to format text lines.<\/p>\n
    Little-endian is a geeky homage to Gulliver’s travels. It has to do with which direction one encodes the bits. But, it isn’t really the big problem here. UTF-16 is the problem because apparently, I need it to be UTF-8 for grep to read it. So, the fix is to use an encoding converting:<\/p>\n
    iconv -f utf-16 -t utf-8 filename.txt > filename_new.txt<\/pre>\n","protected":false},"excerpt":{"rendered":"
    I generated some text files working with Get-Acl Powershell, but I did not know how to get Powershell to do some advanced features. (Basically, I wanted to the Select-String to include the next 2 lines and see whether a specific group was in that list. And maybe some exclusions.) So, I copied the files over […]<\/p>\n","protected":false},"author":2,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"jetpack_post_was_ever_published":false,"_jetpack_newsletter_access":"","_jetpack_dont_email_post_to_subs":false,"_jetpack_newsletter_tier_id":0,"_jetpack_memberships_contains_paywalled_content":false,"_jetpack_memberships_contains_paid_content":false,"activitypub_content_warning":"","activitypub_content_visibility":"","activitypub_max_image_attachments":4,"activitypub_interaction_policy_quote":"anyone","activitypub_status":"","footnotes":"","jetpack_publicize_message":"","jetpack_publicize_feature_enabled":true,"jetpack_social_post_already_shared":true,"jetpack_social_options":{"image_generator_settings":{"template":"highway","default_image_id":0,"font":"","enabled":false},"version":2}},"categories":[621,109],"tags":[3441,3444,3442,3443,3445,839,3439,2784,3440],"class_list":["post-9154","post","type-post","status-publish","format-standard","hentry","category-microsoft","category-unix","tag-ascii","tag-dos2unix","tag-get-acl","tag-grep","tag-iconv","tag-linux","tag-little-endian","tag-powershell","tag-utf-16"],"jetpack_publicize_connections":[],"jetpack_featured_media_url":"","jetpack_sharing_enabled":true,"jetpack_shortlink":"https:\/\/wp.me\/p1rUBW-2nE","jetpack-related-posts":[],"jetpack_likes_enabled":true,"_links":{"self":[{"href":"https:\/\/www.ezrasf.com\/wplog\/wp-json\/wp\/v2\/posts\/9154","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.ezrasf.com\/wplog\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.ezrasf.com\/wplog\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.ezrasf.com\/wplog\/wp-json\/wp\/v2\/users\/2"}],"replies":[{"embeddable":true,"href":"https:\/\/www.ezrasf.com\/wplog\/wp-json\/wp\/v2\/comments?post=9154"}],"version-history":[{"count":0,"href":"https:\/\/www.ezrasf.com\/wplog\/wp-json\/wp\/v2\/posts\/9154\/revisions"}],"wp:attachment":[{"href":"https:\/\/www.ezrasf.com\/wplog\/wp-json\/wp\/v2\/media?parent=9154"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.ezrasf.com\/wplog\/wp-json\/wp\/v2\/categories?post=9154"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.ezrasf.com\/wplog\/wp-json\/wp\/v2\/tags?post=9154"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}