LMS Brackets

I don’t really follow basketball. So, it is odd that I even registered this tweet from Dennis Kavelman, @dkavelman:

Dennis Kavelman  Team D2L has 4 schools in sweet sixteen! Go #marquette, #arizona, #msu, #osu! @desire2learn

He is excited because four universities who are clients of his company are in the major NCAA basketball championship contest. If only four are Deseire2Learn customers, then that means the other twelve are not. That made me wonder what are the Learning Management Systems used by these sixteen schools?

    1. Louisville : Blackboard
    2. Oregon : Blackboard
    3. Michigan State : D2L
    4. Duke : Sakai
    5. Wichita State : Blackboard
    6. La Salle : Blackboard
    7. Arizona : D2L
    8. Ohio State : D2L
    9. Kansas : Blackboard
    10. Michigan : Sakai
    11. Florida : Sakai
    12. Florida Gulf Coast : Angel (Blackboard) and Canvas in 39 days
    13. Indiana : Sakai
    14. Syracuse : Blackboard
    15. Marquette : D2L
    16. Miami (FL) : Blackboard

Looks like the breakdown is:

Blackboard 8
Desire2Learn: 4
Sakai: 4

This is an interesting grouping. I kind of knew Sakai tended to be the product of choice for well off schools with the money to spend on customization. So, schools with strong athletics probably are more likely to have something like Sakai. Of course, I expect Canvas to be better represented too as it is hot of late. While Moodle tends to be favored by really small schools without a budget, I still figured it would have some representation (really just FGCU).

Which LMSes will be involved with those in the Final Four?

Big Bad Blip

I was at lunch last week when I saw pages about a failed monitoring checks on one of our sites. My coworkers were working on CE/Vista SP6 upgrades. Though it was one upgraded yesterday. When I returned to the office, I asked about it. Exactly 24 hours to the second after checking the license in yesterday’s final start, the JMS node failed a license check four times about a minute apart. On the fourth failure, it started a shutdown of the node. Others in the cluster did as well.

Fortunately, a coworker caught it soon enough to start them again so not enough were shut down the load balancer would stop sending us traffic. Also, this was between terms so we did not have a normal work load.

Still, JMS migrated. That made Weblogic edit the config.xml and probably left the cluster in a weird state. So I set cron to shutdown the cluster at 4am, copy a known good config.xml into place, check the config with our monitor script (pages if bad), and start the cluster. That was a disaster. Various nodes failed their early The startup started the admin node, but the JMS failed to start. So I was paged about it still being down when it ought to have been running.

My 6:30 am starts failed for the same reason: bad encrypted password in boot.properties. My only idea how to fix this was a coworker had mentioned having to re-install an admin node for a security error. So I called the coworker. I explained the problem and the solution I really did not want to take. She looked at the error and thought about it some. She decided it might work to replace the boot.properties with an unencrypted version because Weblogic would encrypted it when discovered. She also suggested removing the servers directory and placing a REFRESH file which would prompt the node to download a new copy of the files it needs from the admin node.

That worked to getting the nodes to start correctly. It was fine during the normal maintenance on Friday. Looks like we are in the clear.

That afternoon I brought it up on our normal check-in call with Blackboard. An unable to find license file issue was why Blackboard pulled CE/Vista SP4. It also was a Weblogic upgrade.

Pick Up Line

(I will never use.)

My name’s Vista. Can I crash at your place tonight?

Noticed at geekpickuplines.

Especially funny for me because the product I run is the Blackboard Learning Management System Vista Enterprise. We just call it “Vista”. (Yes, very confusing when Windows Vista users want to know the compatibility of Vista with Vista. The answer: barely.)

The Cause

Found I develop free software because of CUNY and Blackboard following the Blackboard security issues story. It is a really good blog post. This conclusion made me smile. I am certain there are plenty of people in the system I support who strongly agree. I just wish there was an easier way of finding and applauding them.

As long as our IT departments are dominated by Microsoft-trained technicians and corporate-owned CIOs, perhaps the best way to advance the cause – the cause of justice in the way that student money is spent – is to create viable alternatives to Blackboard and its ilk, alternatives that are free (as in speech) and cheap (as in beer). This, more than anything else, is why I develop free software, the idea that I might play a role in creating the viable alternatives. In the end, it’s not just about Blackboard, of course. The case of Blackboard and CUNY is a particularly problematic example of a broader phenomenon, where vulnerable populations are controlled through proprietary software. Examples abound: Facebook, Apple, Google. (See also my Project Reclaim.) The case of Blackboard and its contracts with public institutions like CUNY is just one instance of these exploitative relationships, but it’s the instance that hits home the most for me, because CUNY is such a part of me, and because the exploitation is, in this case, so severe and so terrible.

The training plan is to make me one of those “Microsoft-trained technicians”. It makes me feel stupider just thinking about it.

Blackboard Security

Interesting articles accusing Blackboard of being lax about security. A Black Eye for Blackboard Over Its Response to Major Security Flaws which is about Millions of student exams, tests and data exposed. I saw the security bulletins, but I was not aware of the back story leading to why it was announced. We run an unaffected product, so I mostly ignored it. After reading the stories a couple times and the security bulletins again, my general read is still: overblown.

Blackboard’s practice is to work with the reporting client to determine the nature of the issue, whether it is being exploited, and test the fix. On the occasion where I was the reporting client, I was asked not to publish information about it as that would allow malicious individuals to exploit it before other clients implemented the fix. As I recall, the time from my reporting it to getting a patch was about a month. Plus, what I reported was pretty specific, Blackboard took that and looked more broadly and fixed everything they found. Then again, I reported a single issue not 16. Also, I tend to report such things to John Porter directly as I trust him to seriously address them. Someone opening a low priority ticket to the Tier I helpdesk, not providing the data Bb requests, or even worse incomprehensible data can get stuck in the Blackhole (where support tickets go to die). Every client needs to read Blackboard’s information on how to report security issues.

A problem with Blackboard only talking to the reporting client(s) is other individuals might already be aware of the exploit. The idea of keeping mum will prevent others from finding out fails to consider Newton invented Calculus at the same time as Gottfried Leibniz. Security by hoping no one else finds out… isn’t secure. Clients not provided ways of detecting whether the exploit is being used cannot report to Blackboard that their systems were compromised.

“We are not aware of any institution’s academic or student data having been compromised in any way by these issues,” Tan said.

In this statement, “any institution” means the clients who discovered this vulnerability not all clients. Blackboard is reassuring that the problem is minor and clients applying the patches quickly will keep it minor. Calling this a zero-day security vulnerability implies attack code is out there available to be used. So attackers potentially have information while defenders do not? Unfair. Epic fail. But only when it leaks to the attackers or they independently figure it out.

More interesting is the vulnerability claims Blackboard considered invalid because they “were due to misconfigured security settings.” So if an administrator sets an incorrect configuration the problem does not exist? For example, an administrator does not set Secure HTTP on the login, so a malicious person in a coffee shop snatches passwords and uses it to alter grades. (Or worse a 9 year-old compromises his teacher’s password.) Yes, it is the administrator’s negligence, but as a partner Blackboard should be helping administrators not be negligent. Keep this in mind: When a Blackboard system is compromised, only Blackboard cares whether it was administrator negligence or Blackboard code.

As a defender, I want all the information I can to protect my users from attackers. Whenever I talk about this with other clients, I hear the same thing. Instead I am left with fear, uncertainty, and doubt. Not that I expect any other vendor to provide me more information than Blackboard. This is why I like the idea of open source.

Bb Mobile Learn Phones Home

We acquired Blackboard Mobile Learn Powerlink version 1.1.9 to solve a problem. Something was done on the Blackboard side before our institution administrators could successfully get to the registration page. This version fixed that issue by doing a registration behind the scenes. Naturally the information it has is wrong, but at least now the institution administrators can change it to the correct information.

Looking to gather logs on an issue, I stumbled across a new-to-me log: vistaMobile.log.

There was a call to a strange URL:


What shocked me most was this was an institution where Mobile was not ready for use. By going to the URL, I figured out it was the advertisement image on the course list after Mobile Learn is set up. (I would have the image show up on the blog, but they seem to prevent it from working on my blog. Direct access works. For now? :))

Every user logging into our system by a non-mobile web browser unwittingly connects to Blackboard Mobile Learn servers to download this image. This Powerlink sets this channel on by default. It is a wealth of information on users we were not informed Blackboard would acquire. So we will likely turn off this channel and stress pushing the advocacy for Mobile by other means.

If a user were to click the link on the advertisement, then they would get this URL:


This has not one or two but THREE marketing campaign products tracking the users. Not intrinsically a bad thing, but it makes me nervous to unknowingly contribute to a marketing campaign operating through my system.

Browser Checker

Mark Wescott wrote to a Blackboard Learn 9 list,

Imagine a world where Bb Inc. provides browser checkers for each Rev/SP combo, and all we (aka “The Customers”) have to do is place a link on our Bb logon page to the browser checker that matches our production environment…. mmmm serenity.

Said browser checkers would:

  • be GREAT customer service; EVERY institution that uses Bb would benefit
  • be relatively simple for Bb Inc’s staff to create
  • be branded with Bb’s logos and marketing
  • be up to date
  • be released commensurately with each SP
  • have a static, publicly available URL
  • be found on the Course Sites logon page
  • eliminate a topic that appears about every 6 months on this board

Sadly, we Blackboard Vista clients have Mark’s “imagine a world”. Yet still the topic appears in our email lists every time a new web browser or version of Vista was released. Why?

  • Telling a user their web browser may have problems is not a deterrent. Their (correct) opinion is Blackboard should fix the product so the browser they use every day will work. Students and instructors should not have to become a computer geek to take or teach a class.
  • Blackboard only checks a small amount of browser and operating system combinations, so potentially fine web browsers were marked “not tested”. Blackboard has better things to do than test the long tail of browsers. So users have stopped trusting the browser checker because untested browsers often do work. The browser checker has cried “Wolf!” too many times for people to believe it.
  • The browser checker is not consistent with the official supported browser list. Oh, and that is intentional. Browsers which used to work are nor removed from the list unless clients make a stink about them not working. Blackboard stopped testing them, so they are clueless whether it continues to work, but users are not alerted to the problems.

Imagine a world where Blackboard products work in every web browser because it does not reply on coding for specific browsers. You know… Web Standards.


Blackboard Mobile Learn Support

Last week about this time both Blackboard Mobile Learn and SafeAssign were experiencing an outage. Both were resolved by the afternoon. However, Support Bulletins, how I have come to expect to receive notifications about Blackboard issues came only for SafeAssign. I complained about this to my support representative by CC’ing him on an internal message. Wednesday afternoon we had our normal conference call where I went into more detail. Thursday morning he wanted more details. I probably went too far when I wrote, “I am just looking for Mobile to put on its big boy pants and alert us through the appropriate support channels.”

Before I continue, here is what I understood about Blackboard Mobile Learn. Mobile Learn is a new acquisition of Blackboard. According to TechCrunch’s CrunchBase…

In July of 2009, Blackboard Inc. acquired Terriblyclever for $4mil, at which time TC had 5 full time employees, most of whom were close friends and current students at Stanford University.

Rather than being completely rolled up into the Learn division of Blackboard, Mobile is a separate division. Well, even if it has been added to Learn, there is the possibility like with Angel it would have been allowed to do things their way. I heard Ray Henderson say a mistake Blackboard made in buying WebCT was to try and integrate the support structures too soon. With 5 employees there was not so much a support structure at TC as maybe a half to full person? Plus supporting a couple dozen clients is far different than opening up to thousands of clients who have tens of millions of users. So the big boy pants comment was about integrating with the rest of the company rather than sitting off to the side doing their own thing.

Anyway, I got an email from Francois Hedouin asking to pick my brain about planned improvements to Mobile Learn support. I will not go into the specifics of what he and Mobile Learn are planning. I liked what I heard. He seemed to like my input. He had done his homework and knew about me and my organization. I probably drank the Kool-Aid, but I came away feeling like any client should: The vendor understands my needs. A conference call a couple weeks ago about Mobile Learn left me feeling like we got a sales guy who was on his first day at the job because he did not say anything about the product that was not already in the advertisements we had already read.