Dataclysm: Who We Are
Dataclysm: Who We Are by Christian Rudder
My rating: 3 of 5 stars

Maybe really 2.5 stars, but I rounded up.

I have read the OkTrends blog since its inception. Human behavior fascinates me, so I take any opportunity to read on it. The We Experiment On Human Beings post ensnared my attention since it flubs its nose at academic sensibilities at what is ethical experimentation. But, this review is not about Rudder’s ethics, so I will move on to the book.

The writing engaged a technologist interested in Big Data, interesting links, and how data can be used in interesting ways. (Hardly surprising.) Many references made me laugh out loud. I highlighted 32 places according to my Kindle stats. Much more were worthy. The writing alone would make me give it 5 stars.

My first problem manifested in the lack of details in the main text. Where I expected to read about how conclusions were reached, the details were light. Where it all fell apart for me fell in the Coda section where he delved further into the methods used. Suddenly the assumptions, based on nothing but super wild ass guesses (SWAGs) came into complete view. For example, his conservative estimate is that active OkCupid users go on at least one date every two months and uses this with active users/month to arrive at 30,000 dates will happen tonight because of OkCupid. This number is used for other calculations. I would give this aspect no stars.

So an average of 2.5 stars rounded up is the reviewed 3.

View all my reviews

I was challenged by a friend to list 10 books that have stayed with me in some way. Don’t take more than a few minutes and do not overthink it. Not necessarily classics, but ones that have affected you in some way.

There are so many other books I ought to have included on this list. Oh, well. What is done, is done.

I think for next year’s books I am going to try to read some of the books friends included on their lists. (The collection.)


On the Internet, nobody knows if you’re a dog… but everybody knows if you’re a jackass. — stevemb



Nor has Youtube.

I have several problems with Amazon Isn’t Killing Writing, The Market Is. The article is about the Amazon Kindle Unlimited subscription for $9.95/month for unlimited access to the Kindle eBook library.

First, Amazon already offers borrowing from some of the Kindle library as part of their Prime service. I borrowed a book from it which I probably was not going to buy otherwise. (That was a good move because the book kind of sucked.) Only being able to read the book on a couple of my Kindle devices was really offputting, so unless there is a very compelling reason to do so, I probably will not borrow another book.

For authors, my library aficionado patron friends are terrible. They check out books from a library instead of buying them. These friends also tend to rent videos instead of buying Blurays or DVDs. They prefer music streaming services or listening to the radio over buying music. They are not into building collections of media. All of these are terrible for content creators. A library buying a physical book means the author gets royalties for the single purchase not each person who checks out the book. Under newer library eBook subscription model, I suspect a book will need to be checked out many times for a book to equal one physical book’s purchase. So, bestsellers may do better and everyone else do worse.

Back on Memorial Day, I had a momentary bout of guilt. The owner of a Five Guys restaurant and I talked about the newspapers having trouble figuring out online distribution. He made the comment that we as consumers should pay the people who create the content we like. Mom spent a lot of time shopping in and selling to the used book store back where I grew up, so I have no fear of buying used. (I’ve bought 22 used books since Memorial Day from betterworldbooks alone. Another 11 used and 2 new on Amazon.) Buying a used book earns authors no royalties. So, authors should hate people like me. Before shifting to buying so much online, most in person purchases were off sales racks. The only books I bought new were novel series where I waited for a long time for it to drop.

The other piece of guilt was a shift in my reading to public domain books. Edgar Rice Burroughs, L. Frank Baum, and Alexandre Dumas are all fantastic for publishing so much years and years ago. I can read them all for free through the Kindle because they are dead and volunteer efforts like Project Gutenberg.

Just like Netflix did not kill cable or movie theaters, Amazon Kindle Unlimited will not kill book publishing. Sure, some small number of people will shift from buying books to subscribing, but these are likely people who were not going to be be buying many books anyway.

Flash Boys: A Wall Street Revolt
Flash Boys: A Wall Street Revolt by Michael Lewis
My rating: 5 of 5 stars

This book is really the story of why and how IEX was created. Humans made bad decisions. So, to protect people from other people, we moved the operation of the stock market to being run by computers. The natural consequence was for people to game the system with computer code. Rather than stay vigilant against new exploitations, we just redefined fair. The team behind IEX created it to eliminate these problems and establish a fair place for trading to occur.

About halfway through the book, I watched a commercial where an investment company touted their guaranteed one second trades. To the average person, this probably sounds amazing. The thing is that companies like this operate in milliseconds (1/1,000) and nanoseconds (1/1,000,000). Plus, they operate Dark Pools where the trade is obfuscated from independent review. Your trade could get executed where it benefits them and not you.

The overarching theme is that complexity and obfuscation created an environment where bad things can happen. As a technologist in education, I fight against this every day. We desire simplicity. Yet every change and especially those we execute without a good understanding of the business case creates complexity which will result in a failure. When no one fully understands how all the components work together, it exists to fail. Funnily enough, my team, the database administrators (really application administrators) sit at the intersection of the analysts, vendors, operating system admins, storage admins, network admins, and others. So this is familiar territory.

Zoran Perkov and Sergey Aleynikov are unsung heroes I am sure about whom I will spend more time reading.

View all my reviews

Tags: , , , , , ,

Had a conversation with a restaurant manager when he said he hates computers. His life has gone from 90% working with food to maybe 60%. Naturally he did not get into this kind of work to spend so much time dealing with computers.

GeorgiaVIEW Admin Retreat

At first I thought the issue was empowerment. A few decades ago, important people had assistants to do all their minutia. They did not write letters so much as quickly express what it should say, someone else wrote, and had it approved before going off to send it. Now, important people write an email themselves. Well, more so than they used to do. Technology has made minutia easier and changed assistant jobs into accomplishing more complex tasks.

As it turns out the issue was more organizational complexity. The manager’s accountant found a mistake and told him to talk to another department who sent him to third who sent him to a fourth. Each admitted the mistake should be fixed, but none could correct it.

Sound familiar? You might have encountered it dealing with customer support with a utility or government agency. The organization is so big and so complex individuals within it are not capable of knowing where to direct customers to have the problem solved. Only the most tenacious can force the difficult issues. When employees are empowered with autonomy to make decisions and solve problems, they make things move along and keep customers happy.

Still sad computers take the blame for people designing organizations.

Tags: , , , , , ,

When Amazon bought Goodreads, the main hope for me was a tighter integration.

Many of the books I read have a variety of editions, so I have to figure out which one to select on Goodreads. Different editions might even have the same cover, so it can be a challenge. If the book is an ebook, then the ASIN is definitive.

Importing my purchases into Goodreads would be easier on me. So I do love the new Add Your Amazon Books to Goodreads. It only shows those books I’ve purchased on Amazon but not added to Goodreads, so I do not have to scroll through hundreds of books to find the new purchases. They are also organized so the newest purchase sits first. Finally, it does not automatically add books which means I have to mark them as to-read, currently-reading, or read one-by-one. That is fine, as I have some sets I bought on Amazon that I list individually on Goodreads.

Overall, I am pretty pleased and caught up on some listings I missed.

A listing of oddities…

  • If Goodreads librarians did not add books correctly, then the there might be a mismatch in editions.

Tags: ,

Not even sure how this came up, but someone explained to me how it infuriated him that because the Boy Scouts of America decided not to accept homosexual scout leaders Walt Disney stopped sponsoring them. It was not right for Disney to do that.

First, I asked if it would be right for some other company to stop giving money to BSA if they took the opposite stance and did accept homosexual scout leaders. He realized that was a trick question and refused to answer it. I was not going to get him to see it from the other side.

Next, I approached it from a publicity standpoint. The BSA does not really care where they get sponsors, just that they make enough to thrive as an organization and achieve their mission. When a sponsor pulls their money, demonize them as far and wide as you can. People get so upset the “victim” makes up for it and more. He brought up the Chic-fil-A boycott where they had record day, maybe weeks, because supporters came out to help.

The scary part of the Internet is people getting caught up in the emotionally manipulating stories not realizing it. Some are obvious. Others are very subtle. It is not just one demographic, but all of us. Even those of us who consider ourselves rational. Well, especially us. Everyone else are in the wrong.

Divisiveness in the end tears us apart. It keeps us from seeing things from the perspective of others.

Tuesday afternoon a bit of snow hit Georgia. We get them occasionally even down here in the southern United States. Usually they hit the mountains. Everywhere else, we just shut down for a few days. Unfortunately, we do not have all of our Atlanta schools on one system or all of our south Georgia schools on one system. Each system has some of each.

These graphs for current connections (application servers to database so an inference to end users) to Desire2Learn do tell these stories:

  1. Some Atlanta universities shut down around noon. Just before noon and moving quickly down, we had fewer and fewer users.
  2. While campuses were closed, some faculty kept their due dates and had students turn in work via Desire2Learn.
  3. Wednesday was a delayed opening for several campuses, so the first peak of the day was around 4pm instead of the normal 11am.
  4. Based on Twitter rumblings a bit of our traffic were students checking to see whether or not class was canceled.

My conclusion is we kept 60-70% of our normal traffic while around 75% of our campus user base was closed.

We will have fully made it when these closing cause our numbers to go higher than normal because when they get home they come into our system to keep working.

Snowstorm Q Site

Snowstorm Q Site

Snowstorm X Site

Snowstorm X Site

P.S. If we had gone with the desired technical architecture of a database/application stack dedicated to each school, then I could show these numbers per school not just per site.

As “plumbers” for 17 databases and over a hundred application servers, we really do not have the time to sit there and watch them all. We design things so problems are pushed up to our attention. We are still getting point of predictive alerts to failures like we were on RHEL/Weblogic/Oracle, so usually we only get an alert after the failure.

The lack of a log for a component means when it fails, we know nothing of why. In this case the vendor setup the component and wants to know why it failed even though they did not set it up to collect any information. Playing with this component manually I noticed it sends information to the screen, so I wrote my own wrapper script to capture this information.

First stab at it used the vendor’s timestamp method:

SET mydate=%date:~10%_%date:~4,2%_%date:~7,2%
SET mytime=%time:~0,2%_%time:~3,2%

These values were plugged into other variables for the log names so I get a log for each run. (standard out log and standard error log)

The afternoon I worked on this script appeared to have no problems. The script was scheduled for the morning which is when the vendor had it run. Review of those logs showed instead of being “name_2014_01_23_04_51.log,” I got “name_2014_01_23_.” It was clear it broke on the hour. So I ran:

echo %time%

This returned a time with a leading space instead of a leading zero. That seemed strange. So I started Googling. Turns out this is a very common problem. The solution I chose came from Need leading zero for batch script using %time% variable.

:prepare time stamp
set year=%date:~10,4%
set month=%date:~4,2%
set day=%date:~7,2%
set hour=%time:~0,2%
:replace leading space with 0 for hours < 10
if “%hour:~0,1%” == ” ” set hour=0%hour:~1,1%
set minute=%time:~3,2%
set second=%time:~6,2%
set timeStamp=%year%_%month%_%day%_%hour%_%minute%_%second%

Uglier. Though, it did remind me to use the call command to put this logic into a another script file so I can easily re-use it for another script I will inevitably need to write.

And how I much prefer Powershell or batch scripts.

P.S. I left out the first solution was to put the log file names in double quotes. This is a common way to deal spaces in Windows file names. That was in no way satisfactory to me. It more just confirmed the problem. So I went Googling.

Tags: , , ,

« Older entries